SkillRet benchmark studies, pilots, and reproducible evidence.
See the codeCode, recorded results, and documentation for studying how natural-language requests retrieve and select agent skills. Start with the 16-study benchmark catalog or the published EDA report.
| Directory | What you will find |
|---|---|
| experiments/ | Self-contained studies grouped into benchmarks, pilots, and scaling |
| EDA/ | Dataset analysis, golden-skill distributions, and the GitHub Pages homepage |
| paper/ | Editable proposal, bibliography, NeurIPS style, and template example |
| demos/ | The separate proposal demonstration notebook and its validation limits |
| docs/ | Study catalog, course template, paper guides, publication audit, and supplied-input inventory |
| scripts/ | Repository maintenance and local paper build commands |
.
├── README.md
├── AGENTS.md
├── EXPERIMENTS.md
├── experiments/
│ ├── benchmarks/
│ ├── pilots/
│ └── scaling/
├── EDA/
├── paper/
├── demos/
├── docs/
│ ├── course/
│ ├── inputs/
│ ├── paper/
│ └── publication/
└── scripts/
Each study keeps its source, results, manifests, and reproduction instructions together. The experiment index records completion limits; partial, failed, and unrun arms must not be treated as completed benchmarks. Scores from different query panels, document views, and candidate pools are not directly comparable.
The golden-label analysis can be checked without downloads or inference:
python3 EDA/golden_skills.py --check
python3 -m unittest discover -s EDA -p 'test_*.py'
GitHub Pages publishes EDA/eda_report.html automatically from main.
Paper sources also build automatically through
Build LaTeX PDFs.
Keep all code and documentation on main. Follow AGENTS.md and the
publication/privacy guide.
Routine updates scan changed files only. Large model downloads, environments,
private credentials, and original private execution files stay outside this
public repository.
Team aliases: deepujoy88, speccy88, vsimard.
SkillRet benchmark studies, pilots, and reproducible evidence.
See the codeCode, recorded results, and documentation for studying how natural-language requests retrieve and select agent skills. Start with the 16-study benchmark catalog or the published EDA report.
| Directory | What you will find |
|---|---|
| experiments/ | Self-contained studies grouped into benchmarks, pilots, and scaling |
| EDA/ | Dataset analysis, golden-skill distributions, and the GitHub Pages homepage |
| paper/ | Editable proposal, bibliography, NeurIPS style, and template example |
| demos/ | The separate proposal demonstration notebook and its validation limits |
| docs/ | Study catalog, course template, paper guides, publication audit, and supplied-input inventory |
| scripts/ | Repository maintenance and local paper build commands |
.
├── README.md
├── AGENTS.md
├── EXPERIMENTS.md
├── experiments/
│ ├── benchmarks/
│ ├── pilots/
│ └── scaling/
├── EDA/
├── paper/
├── demos/
├── docs/
│ ├── course/
│ ├── inputs/
│ ├── paper/
│ └── publication/
└── scripts/
Each study keeps its source, results, manifests, and reproduction instructions together. The experiment index records completion limits; partial, failed, and unrun arms must not be treated as completed benchmarks. Scores from different query panels, document views, and candidate pools are not directly comparable.
The golden-label analysis can be checked without downloads or inference:
python3 EDA/golden_skills.py --check
python3 -m unittest discover -s EDA -p 'test_*.py'
GitHub Pages publishes EDA/eda_report.html automatically from main.
Paper sources also build automatically through
Build LaTeX PDFs.
Keep all code and documentation on main. Follow AGENTS.md and the
publication/privacy guide.
Routine updates scan changed files only. Large model downloads, environments,
private credentials, and original private execution files stay outside this
public repository.
Team aliases: deepujoy88, speccy88, vsimard.