50k Poorly Geocoded News

This is a dataset consisting of 50k full text news that contain some kind of geocoded location entity. An LLM extracted geospatial entites and added lat lon coordinates. This repo is more of a proof of concept or an example of what does not work rather than an actual dataset you should use for any downstream task.

Live Demo here: https://do-me.github.io/geoparquet-visualizer/?url=https://huggingface.co/datasets/do-me/50k_poorly_geocoded_news/resolve/main/geocoded_news.parquet?download=true

Why?

The idea was: LLMs probably have some geocoding ability. How bad can it possibly be?

Results

The answer is: yes it's bad and you should not use it for anything, but generally it's better than expected!

A first overview doesn't look that bad on low zoom levels...

... but zooming closer reveals systemic errors.

How?

I used an mlx-optimized, quantized version of liquid's LFM2 model: mlx-community/LFM2-8B-A1B-3bit-MLX. It's fairly small and fast but large enough to contain enough information about locations to guess where a place might be. Note that this model performs generally really well for its size which is the reason I went for it!

Prompt:

PROMPT_TEMPLATE = """Extract up to 3 place names from the news article. For each, provide latitude and longitude (EPSG:4326),place confidence (0-1), and a sentiment score (-1 to 1) based on the article’s tone toward that place. Return ONLY valid JSON, following this schema:

[
  {
    "place_name": "string",
    "lat": float,
    "lon": float,
    "place_confidence": float,
    "sentiment_score": float
  }
]

If no places are identifiable, return [].

Article:
"""

I ran this prompt on more than 1M full texts and afterwards filtered out nonsense (broken json, null values etc.). I should have used structured output to get better results.

What?

I used https://huggingface.co/datasets/BAAI/IndustryCorpus_news but note that there is a cleaned version available here: https://huggingface.co/datasets/BAAI/IndustryCorpus2.

Questions?

Open a discussion or reach out to me, you find me here: geo.rocks/about