GenAI and Democracy Project

Authors: Tianhao Gu, Ayoub El Hassouni, Pauline Jakob (group of 3) as part of the GenerativeAI and Democracy seminar at HfP / TUM.

Overview

This project retrieves news articles matching a user's query (English, German, French) and ranks them by a composite score that balances topical relevance and bias mitigation. The goal is to present users with results that are both relevant and less opinionated.

---

How articles are ranked (the score)

Each candidate article receives a composite score made of two main components:

1. Relevance (semantic similarity)

   • Generated by embedding the user query and each article using a SentenceTransformer model.
   • Semantic similarity (e.g., dot product or cosine similarity) between the query and article embeddings yields the relevance value. Higher means more topically related.

2. Bias (fairness & safety signal)

   • Computed from automated checks for toxicity, polarity/sentiment, hurtfulness, and other fairness signals.
   • The bias value reflects how opinionated, harmful, or one-sided an article appears to be.

Conceptual combination

A tunable combination produces the final combined_score used for ranking. For example, conceptually:

combined_score = w_rel * normalize(relevance) - w_bias * normalize(bias)

• w_rel and w_bias are weights that control the trade-off between relevance and bias mitigation.
• Normalization scales components to comparable ranges (e.g., 0–1).
• Articles are sorted by combined_score in descending order; higher scores appear first.

This favors articles that are both relevant and measured as less biased. When relevance is similar, lower-bias articles are promoted.

---

What the bias component evaluates

Typical automated checks feeding the bias value include:

• Toxicity: presence of abusive or harmful language.
• Polarity / Sentiment: strong positive/negative slant that may indicate opinionated coverage.
• Hurtfulness: content likely to harm specific groups.
• Group fairness checks: probing for gender, racial, occupational biases.
• Quantitative ratios & error analysis: e.g., toxicity ratio across groups.
• Benchmarking & manual review: periodic checks using standard datasets and human inspection to catch issues automated tests miss.

---

User-facing behavior

• The system returns the top-N articles with their combined_score and short metadata (language, source id, snippet).
• Showing numeric scores increases transparency so users can see why a result ranked highly.
• Snippets and source links are still provided so users can judge content directly.

Goals

• Prioritize relevance while reducing opinionated or harmful content in top results.
• Provide transparency through numeric scores and short explanations.
• Allow maintainers to tune weights and metrics based on evaluation and user feedback.

---

Data & robustness (brief)

• Articles are fetched and parsed into JSON files containing id, language, content, and source URL.
• Faulty or unparsable articles are skipped to keep processing robust.

---

For maintainers

• Adjust w_rel / w_bias to change the emphasis on relevance vs. bias mitigation.
• Re-benchmark periodically and include human review to ensure fairness objectives are met.
• Consider exposing score components (relevance vs. bias) in the UI for added transparency.

---

This README focuses on article ranking and scoring, and identifies project authors.