Few-Shot Clustering Reproduction & Extension

Overview

This project presents a reproduction and critical evaluation of the paper:

Large Language Models Enable Few-Shot Clustering (Viswanathan et al., 2023)

In addition to reproducing the original experiments, this work proposes complementary evaluations and improvements, including:

• Extended experimentation on additional datasets
• A combined LLM-based clustering pipeline
• Analysis of performance, robustness, and limitations

The full methodology, experiments, and analysis are detailed in the project report.

---

Project Objectives

• Reproduce the main results of LLM-enhanced clustering methods
• Evaluate the role of LLMs at different stages of clustering:
  • Before clustering: keyphrase enrichment
  • During clustering: pairwise constraint generation
  • After clustering: cluster refinement
• Analyze strengths, limitations, and real-world applicability
• Propose and evaluate a combined pipeline approach

---

Installation

Option 1 — Install via pip

pip install few-shot-clustering

Option 2 — Install from source

git submodule update --init
pip install -e .

Install dependencies

pip install -r requirements.txt

---

Usage

• Reproduction of original results:

  reproduced_results.ipynb
  

• Additional experiments and evaluations:

  Added_approaches.ipynb
  

---

Experimental Setup

Datasets

The project evaluates clustering on multiple datasets:

• CLINC — intent classification (clean, structured)
• Bank77 — banking-related intents
• Tweet (custom) — noisy, short-text dataset
• 20News (added) — complex, real-world dataset with overlapping topics

Due to missing resources in the original repository, some datasets (e.g., Tweet) were reconstructed manually.

---

Key Contributions

1. Reproduction of LLM-based Clustering

We reproduced three main strategies:

• Keyphrase Augmentation (Before clustering)
• Pairwise Constraints (During clustering)
• Post-clustering Correction (After clustering)

2. Pipeline Integration

We proposed a combined pipeline:

1. Keyphrase enrichment
2. LLM-based constraint generation
3. Cluster refinement

3. Additional Evaluation

• Evaluation across different dataset sizes (200 → 1000 samples)
• Introduction of 20News dataset for realistic complexity
• Comparative analysis of Accuracy and NMI metrics

---

Main Findings

• Keyphrase-based clustering performs best overall
• Combining methods yields limited improvements and sometimes degrades performance
• Data quality is the main bottleneck:
  • Strong performance on clean datasets (CLINC, Bank77)
  • Poor performance on noisy/complex data (Tweet, 20News)
• LLMs provide cost-effective supervision, replacing large amounts of human labeling

---

Limitations

• Limited computational resources:
  • 8GB RAM
  • Free-tier API usage
• Maximum dataset size: 1000 samples
• Missing elements from original work:
  • Some datasets
  • Prompts and hyperparameters
• Only text clustering reproduced (no entity canonicalization)
• API constraints required workarounds (multiple API keys)

---

Data & Cache Files

To reduce computation time and API usage, the project includes a cache_files/ directory.

Cache Files

This folder contains precomputed LLM outputs to avoid re-running expensive operations:

• *_keyphrase.json → cached keyphrase generations
• *_pairwise.json → cached pairwise constraints

Example structure:

cache_files/
├── 20news_cache/
├── bank_cache/
├── clinc_cache/
├── tweet_cache/

These caches allow faster experimentation and reproducibility without repeatedly querying the LLM.

Results File

• accuracies_methods.json → contains the results of the added approaches and experiments (Accuracy & evaluation metrics)

This file summarizes the performance of different methods and is used for analysis and comparisons.

---

Implementation Notes

• Code adapted and corrected from the original repository
• Bugs fixed (e.g., constraint loading issues)
• Custom prompts designed for multiple datasets
• Caching enabled (read_only=True) to accelerate execution

---

Future Work

• Improve performance on noisy datasets (Tweet, 20News)
• Explore hybrid approaches combining LLMs and local models
• Evaluate open-source LLM alternatives
• Add preprocessing pipelines (noise reduction, normalization)

---

---

Authors

• Meriem BAHA
• Maya BEN ABDELATIF

Master Machine Learning for Data Science — 2025–2026