Resume-to-Score pipeline that extracts structured data from PDFs, enriches with GitHub signals, and outputs a fair, explainable evaluation.
This project got a lot of attention recently, and some of the discussion surfaced misconceptions worth addressing directly.
What this is not:
What it actually is:
Every year HackerRank receives 50,000–60,000 intern applications. No human can read that many resumes well. This tool was built to rank them — helping decide which resumes to read first. Resumes scoring below the cutoff are filtered out, but the cutoff is intentionally set very low so only candidates at the very bottom of the distribution are removed. The vast majority pass through to human review, where the real decisions are made.
Since this was built, HackerRank has also shipped AI Interviewer (Chakra) to automate the first round of interviews — so candidates are no longer assessed on their resume alone.
On the default model:
The repo ships with gemma4:latest as the default because it runs locally on most laptops without any cloud API key. Actual intern resumes at HackerRank are evaluated using a top-tier Gemini model. The repo ships with a demo config, not the production one.
Articles and discussions that have shaped how we think about improving this project:
| Article | Key takeaway |
|---|---|
| HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74/100. No — 88/100. Actually 83/100. — Dan Kinsky | Deep statistical analysis of score variance across 100 runs of the same resume. Isolates which categories are stable (technical skills) vs. noisy (project quality judgments). Points to LLM non-determinism as the root cause. |
| The Score Depends on the Roll of the Dice — Pinggy Blog | Reproduces the variance findings and surfaces a security issue: invisible text embedded in PDFs can inflate scores significantly. |
| The Hiring Rubric Inside — ByteIota | Breaks down the scoring weights and argues that a GitHub-centric rubric disadvantages engineers whose work is in private enterprise repos. Also notes the signal degradation risk as candidates optimize for the now-public rubric. |
| Analyzing resume scoring consistency — Mariano Gobea Alcoba, DEV Community | Proposes concrete fixes: standardized data formats, versioned evaluation models, ensemble scoring, and explainability layers to reduce variance and make the system more robust. |
| AI-Powered Pipeline for Explainable Resume Scoring — AIToolly | Covers the launch and highlights the transparency argument — making scoring logic public allows scrutiny that proprietary ATS systems never face. |
| Hacker News discussion | 200+ comment thread covering LLM determinism, GDPR Article 22 implications, and the broader ethics of automated resume filtering. |
Video coverage
Community tools built on this repo
Hiring Agent parses a resume PDF to Markdown, extracts sectioned JSON using a local or hosted LLM, augments the data with GitHub profile and repository signals, then produces an objective evaluation with category scores, evidence, bonus points, and deductions. You can run fully local with Ollama or use Google Gemini.
|
Flow
|
Key modules
|
Python 3.11+
The repository pins .python-version to 3.11.13.
One LLM backend (either of them)
ollama serve.$ git clone https://github.com/interviewstreet/hiring-agent
$ cd hiring-agent
$ python -m venv .venv
# Linux or macOS
$ source .venv/bin/activate
# Windows
# .venv\Scripts\activate
$ pip install -r requirements.txt
Pull the model you want to use. For example:
$ ollama pull gemma4:latest
If you want different results, you can pull other models such as:
# For higher system configuration
$ ollama pull gemma3:12b
# For lower system configuration
$ ollama pull gemma3:1b
Copy the template and set your environment variables.
$ cp .env.example .env
Environment variables
| Variable | Values | Description |
|---|---|---|
DEFAULT_MODEL | for example gemma4:latest, gemini-2.5-pro, or gpt-6-luna | Model to use; must exist in providers.json — the provider is inferred from which provider lists it. Defaults to default_model in providers.json. |
GEMINI_API_KEY | string | Required when using a Gemini model. |
OPENAI_API_KEY | string | Required when using an OpenAI model. |
GITHUB_TOKEN | optional | Inherits from your shell environment, improves GitHub API rate limits. |
Provider mapping lives in providers.json — each provider declares its base_url, an optional API-key env var, and per-model parameters; config.py loads it and resolves the provider for a model. config.py also has a flag:
# config.py
DEVELOPMENT_MODE = True # enables caching and CSV export
You can leave it on during iteration. See the next section for details.
pymupdf_rag.py and pdf.py read the PDF using PyMuPDF and convert pages to Markdown-like text.to_markdown routine handles headings, links, tables, and basic formatting.prompts/templates/*.jinja define strict instructions for each section
Basics, Work, Education, Skills, Projects, Awards.pdf.PDFHandler calls the LLM per section and assembles a JSONResume object (see models.py).github.py extracts a username from the resume profiles, fetches profile and repos, and classifies each project.evaluator.py scores the resume against the role selected on the command line.roles/<role_name>/ and defines its own scoring categories and weights in role.json, plus its own criteria.jinja and system_message.jinja prompts (encoding fairness and scoring rules).software_engineering_intern role scores open_source, self_projects, production, technical_skills, and ai_fluency, plus bonus and deductions, with evidence for each. Other roles can define entirely different categories.bonus_rules in role.json. The LLM returns evidence and allowed point values for each named bonus; Python calculates the total and itemized explanation. For HackerRank Orchestrate, the model supplies the best rank from awards and its points; Python validates +4 for ranks 1–100, +3 for 101–200, +1 for 201–500, and 0 otherwise, once per candidate. All bonuses share the 20-point cap. Roles without bonus_rules keep the original total-and-breakdown response format. Evidence interpretation still depends on the LLM.score.py prints a readable summary to stdout.DEVELOPMENT_MODE=True it creates or appends a per-role resume_evaluations_<role>.csv with key fields (columns follow the role's categories), and caches intermediate JSON under cache/.Provide a path to a resume PDF and the role to score against. --role is the
name of a directory under roles/ and is required.
$ python score.py ./resume/sample.pdf --role software_engineering_intern
What happens:
cache/resumecache_<basename>.json.cache/githubcache_<basename>.json.resume_evaluations_<role>.csv.A role bundles its rubric in roles/<role_name>/:
roles/software_engineering_intern/
├── role.json # categories, weights (max), bonus_max, score bounds, position_title
├── criteria.jinja # evaluation criteria prompt (receives {{ text_content }})
└── system_message.jinja
role.json drives the scoring schema, the printed report, the CSV columns, and
the score caps — so each role can score against its own categories and weights.
To add a role, scaffold one with basic template files and then edit them:
$ python score.py --init-role backend_engineer
# edit roles/backend_engineer/{role.json,criteria.jinja,system_message.jinja}
$ python score.py ./resume/sample.pdf --role backend_engineer
--init-role creates the role directory with placeholder categories and prompts
(it only scaffolds; it does not score a resume). You can also copy an existing
role directory instead.
.
├── .env.example
├── .python-version
├── config.py
├── evaluator.py
├── github.py
├── llm_utils.py
├── models.py
├── pdf.py
├── prompt.py
├── prompts/
│ ├── template_manager.py
│ └── templates/
│ ├── awards.jinja
│ ├── basics.jinja
│ ├── education.jinja
│ ├── github_project_selection.jinja
│ ├── projects.jinja
│ ├── skills.jinja
│ ├── system_message.jinja
│ └── work.jinja
├── providers.json
├── pymupdf_rag.py
├── requirements.txt
├── roles.py
├── roles/
│ └── software_engineering_intern/
│ ├── role.json
│ ├── criteria.jinja
│ └── system_message.jinja
├── score.py
└── transform.py
DEFAULT_MODEL to any pulled model listed in providers.json, for example gemma4:latestmodels.OpenAICompatibleProvider against Ollama's OpenAI-compatible endpoint (http://localhost:11434/v1)DEFAULT_MODEL to a Gemini model listed in providers.json, for example gemini-3.8-flashGEMINI_API_KEYmodels.OpenAICompatibleProvider wrapper is used, pointed at Gemini's OpenAI-compatible endpointDEFAULT_MODEL to an OpenAI model listed in providers.json, for example gpt-6-lunaOPENAI_API_KEYPlease read the CONTRIBUTING.md for detailed guidelines on filing issues, proposing changes, and submitting pull requests. Key principles include:
MIT © HackerRank
(top 24 of 36)
400 followers · starred Jun 2026
4,438 followers · starred Jul 2026
492 followers · starred Jun 2026
947 followers · starred Jul 2026
Resume-to-Score pipeline that extracts structured data from PDFs, enriches with GitHub signals, and outputs a fair, explainable evaluation.
This project got a lot of attention recently, and some of the discussion surfaced misconceptions worth addressing directly.
What this is not:
What it actually is:
Every year HackerRank receives 50,000–60,000 intern applications. No human can read that many resumes well. This tool was built to rank them — helping decide which resumes to read first. Resumes scoring below the cutoff are filtered out, but the cutoff is intentionally set very low so only candidates at the very bottom of the distribution are removed. The vast majority pass through to human review, where the real decisions are made.
Since this was built, HackerRank has also shipped AI Interviewer (Chakra) to automate the first round of interviews — so candidates are no longer assessed on their resume alone.
On the default model:
The repo ships with gemma4:latest as the default because it runs locally on most laptops without any cloud API key. Actual intern resumes at HackerRank are evaluated using a top-tier Gemini model. The repo ships with a demo config, not the production one.
Articles and discussions that have shaped how we think about improving this project:
| Article | Key takeaway |
|---|---|
| HackerRank open sourced its ATS. My resume scored 90/100. Oh wait 74/100. No — 88/100. Actually 83/100. — Dan Kinsky | Deep statistical analysis of score variance across 100 runs of the same resume. Isolates which categories are stable (technical skills) vs. noisy (project quality judgments). Points to LLM non-determinism as the root cause. |
| The Score Depends on the Roll of the Dice — Pinggy Blog | Reproduces the variance findings and surfaces a security issue: invisible text embedded in PDFs can inflate scores significantly. |
| The Hiring Rubric Inside — ByteIota | Breaks down the scoring weights and argues that a GitHub-centric rubric disadvantages engineers whose work is in private enterprise repos. Also notes the signal degradation risk as candidates optimize for the now-public rubric. |
| Analyzing resume scoring consistency — Mariano Gobea Alcoba, DEV Community | Proposes concrete fixes: standardized data formats, versioned evaluation models, ensemble scoring, and explainability layers to reduce variance and make the system more robust. |
| AI-Powered Pipeline for Explainable Resume Scoring — AIToolly | Covers the launch and highlights the transparency argument — making scoring logic public allows scrutiny that proprietary ATS systems never face. |
| Hacker News discussion | 200+ comment thread covering LLM determinism, GDPR Article 22 implications, and the broader ethics of automated resume filtering. |
Video coverage
Community tools built on this repo
Hiring Agent parses a resume PDF to Markdown, extracts sectioned JSON using a local or hosted LLM, augments the data with GitHub profile and repository signals, then produces an objective evaluation with category scores, evidence, bonus points, and deductions. You can run fully local with Ollama or use Google Gemini.
|
Flow
|
Key modules
|
Python 3.11+
The repository pins .python-version to 3.11.13.
One LLM backend (either of them)
ollama serve.$ git clone https://github.com/interviewstreet/hiring-agent
$ cd hiring-agent
$ python -m venv .venv
# Linux or macOS
$ source .venv/bin/activate
# Windows
# .venv\Scripts\activate
$ pip install -r requirements.txt
Pull the model you want to use. For example:
$ ollama pull gemma4:latest
If you want different results, you can pull other models such as:
# For higher system configuration
$ ollama pull gemma3:12b
# For lower system configuration
$ ollama pull gemma3:1b
Copy the template and set your environment variables.
$ cp .env.example .env
Environment variables
| Variable | Values | Description |
|---|---|---|
DEFAULT_MODEL | for example gemma4:latest, gemini-2.5-pro, or gpt-6-luna | Model to use; must exist in providers.json — the provider is inferred from which provider lists it. Defaults to default_model in providers.json. |
GEMINI_API_KEY | string | Required when using a Gemini model. |
OPENAI_API_KEY | string | Required when using an OpenAI model. |
GITHUB_TOKEN | optional | Inherits from your shell environment, improves GitHub API rate limits. |
Provider mapping lives in providers.json — each provider declares its base_url, an optional API-key env var, and per-model parameters; config.py loads it and resolves the provider for a model. config.py also has a flag:
# config.py
DEVELOPMENT_MODE = True # enables caching and CSV export
You can leave it on during iteration. See the next section for details.
pymupdf_rag.py and pdf.py read the PDF using PyMuPDF and convert pages to Markdown-like text.to_markdown routine handles headings, links, tables, and basic formatting.prompts/templates/*.jinja define strict instructions for each section
Basics, Work, Education, Skills, Projects, Awards.pdf.PDFHandler calls the LLM per section and assembles a JSONResume object (see models.py).github.py extracts a username from the resume profiles, fetches profile and repos, and classifies each project.evaluator.py scores the resume against the role selected on the command line.roles/<role_name>/ and defines its own scoring categories and weights in role.json, plus its own criteria.jinja and system_message.jinja prompts (encoding fairness and scoring rules).software_engineering_intern role scores open_source, self_projects, production, technical_skills, and ai_fluency, plus bonus and deductions, with evidence for each. Other roles can define entirely different categories.bonus_rules in role.json. The LLM returns evidence and allowed point values for each named bonus; Python calculates the total and itemized explanation. For HackerRank Orchestrate, the model supplies the best rank from awards and its points; Python validates +4 for ranks 1–100, +3 for 101–200, +1 for 201–500, and 0 otherwise, once per candidate. All bonuses share the 20-point cap. Roles without bonus_rules keep the original total-and-breakdown response format. Evidence interpretation still depends on the LLM.score.py prints a readable summary to stdout.DEVELOPMENT_MODE=True it creates or appends a per-role resume_evaluations_<role>.csv with key fields (columns follow the role's categories), and caches intermediate JSON under cache/.Provide a path to a resume PDF and the role to score against. --role is the
name of a directory under roles/ and is required.
$ python score.py ./resume/sample.pdf --role software_engineering_intern
What happens:
cache/resumecache_<basename>.json.cache/githubcache_<basename>.json.resume_evaluations_<role>.csv.A role bundles its rubric in roles/<role_name>/:
roles/software_engineering_intern/
├── role.json # categories, weights (max), bonus_max, score bounds, position_title
├── criteria.jinja # evaluation criteria prompt (receives {{ text_content }})
└── system_message.jinja
role.json drives the scoring schema, the printed report, the CSV columns, and
the score caps — so each role can score against its own categories and weights.
To add a role, scaffold one with basic template files and then edit them:
$ python score.py --init-role backend_engineer
# edit roles/backend_engineer/{role.json,criteria.jinja,system_message.jinja}
$ python score.py ./resume/sample.pdf --role backend_engineer
--init-role creates the role directory with placeholder categories and prompts
(it only scaffolds; it does not score a resume). You can also copy an existing
role directory instead.
.
├── .env.example
├── .python-version
├── config.py
├── evaluator.py
├── github.py
├── llm_utils.py
├── models.py
├── pdf.py
├── prompt.py
├── prompts/
│ ├── template_manager.py
│ └── templates/
│ ├── awards.jinja
│ ├── basics.jinja
│ ├── education.jinja
│ ├── github_project_selection.jinja
│ ├── projects.jinja
│ ├── skills.jinja
│ ├── system_message.jinja
│ └── work.jinja
├── providers.json
├── pymupdf_rag.py
├── requirements.txt
├── roles.py
├── roles/
│ └── software_engineering_intern/
│ ├── role.json
│ ├── criteria.jinja
│ └── system_message.jinja
├── score.py
└── transform.py
DEFAULT_MODEL to any pulled model listed in providers.json, for example gemma4:latestmodels.OpenAICompatibleProvider against Ollama's OpenAI-compatible endpoint (http://localhost:11434/v1)DEFAULT_MODEL to a Gemini model listed in providers.json, for example gemini-3.8-flashGEMINI_API_KEYmodels.OpenAICompatibleProvider wrapper is used, pointed at Gemini's OpenAI-compatible endpointDEFAULT_MODEL to an OpenAI model listed in providers.json, for example gpt-6-lunaOPENAI_API_KEYPlease read the CONTRIBUTING.md for detailed guidelines on filing issues, proposing changes, and submitting pull requests. Key principles include:
MIT © HackerRank
(top 24 of 36)
400 followers · starred Jun 2026
4,438 followers · starred Jul 2026
492 followers · starred Jun 2026
947 followers · starred Jul 2026