HF ML Tasksmith

Fifty PR-derived Harbor tasks from Accelerate, Diffusers, PEFT, Transformers and TRL, including CPU and GPU tasks.

Contains 50 Harbor tasks generated with the owned tasksmith recipe in Repo2RLEnv. Browse the complete task bundles in Harbor Visualiser or open the task folders. Each folder is a runnable Harbor task:

tasks/<task_id>/
├── task.toml                 # Harbor configuration and provenance
├── instruction.md            # Task shown to the coding agent
├── environment/Dockerfile    # Learner sandbox and its build context
├── solution/solve.sh         # Reference solution entry point
└── tests/
    ├── test.sh               # Verifier entry point; writes the reward
    └── Dockerfile            # Separate verifier sandbox, when configured

Example: task.toml · instruction · verifier · oracle.

data/tasks.jsonl is an auxiliary metadata index. Download the task folders or archive below to run the environments. manifest.json records source identity, evidence, diagnostics and measured costs. The generic tabular Hub viewer is disabled so it does not present the index as the task dataset.

Generation

Tasksmith explores a merged PR, bootstraps the repository, designs an instruction and deterministic verifier, packages offline assets and a reference solution, then applies bounded review, repair, baseline/oracle checks, probes and blind Sonnet rollouts. This release is an assisted practical-acceptance campaign; source PRs, execution controls and independent audit hashes are bound per task.

Implementation revision: a8ea8fa644ec4c3980291f00d7f8190cdfb45a45. The exact recipe, source revision, reward kinds and quality status remain in each original task.toml.

Validation and limitations

Quality label counts: {"verified": 50}. An exported task is a generation artifact. Baseline/reference controls establish only the behavior recorded in that task's evidence. They do not establish blind solver success, difficulty, verifier completeness or resistance to reward hacking. An LLM consistency review is separate from the deterministic task reward.

• 50 assisted practical-acceptance tasks: this is not an unattended yield estimate.
• Sonnet fully solved 19 tasks; 31 did not receive full reward. Solver success was not required for task acceptance.
• 39 tasks request CPU only, 6 request one GPU and 5 request two GPUs; select a provider with matching resources.
• Existing task metadata may reference historical local evidence paths; use the published manifest for portable acceptance summaries and evidence hashes.
• Source files retain upstream repository licenses.

Download and run

tasks.tar.gz preserves executable file modes and the original bundle identities:

hf download HuggingEnvs/HF_ML_Tasksmith tasks.tar.gz --repo-type dataset --local-dir ./dataset
tar -xzf ./dataset/tasks.tar.gz -C ./dataset
harbor run --path ./dataset/tasks --agent oracle --env daytona

Configure Daytona credentials and Harbor's provider dependencies before execution. The archive includes references and private tests for the harness; the solver should receive only the instruction and learner environment. registry.json pins the unpacked task paths to the immutable upload commit.

Economics

manifest.json includes generation costs, failed-attempt costs and outstanding reservations when available. Cloud lifetime estimates are labeled separately from provider invoices. Retained task costs and new-generation costs use separate scopes.

Credits and licensing

• https://github.com/huggingface/Repo2RLEnv/blob/a8ea8fa644ec4c3980291f00d7f8190cdfb45a45/docs/rfcs/0028-tasksmith-pr-pilot.md
• https://github.com/huggingface/Repo2RLEnv/blob/a8ea8fa644ec4c3980291f00d7f8190cdfb45a45/docs/pipelines/tasksmith_campaign_retrospective.md
• https://www.harborframework.com/docs/tasks
• https://github.com/huggingface/transformers
• https://github.com/huggingface/trl
• https://github.com/huggingface/peft
• https://github.com/huggingface/diffusers
• https://github.com/huggingface/accelerate

See LICENSES.md, bundled notices and per-task provenance. These are owned adaptations inspired by the credited methods, not an upstream benchmark release or a claim of exact reproduction of its published results.

Citation

@misc{fineenvs,
  author = {Kolavi, Adithya S},
  title  = {FineEnvs: Open Source RL Environments for LLM Agents},
  year   = {2026},
  url    = {https://github.com/adithya-s-k/FineEnvs}
}