JANUS Counterfactual Decision-Stability ASR Workspace

Generated: 2026-05-18T15:17:05+08:00

This repository is a local research workspace for JANUS high-stakes call-center ASR data.

The single short-term paper axis is Counterfactual Decision-Stability ASR (CDS-ASR).

Whisper and Breeze-ASR baselines provide ASR hypotheses. The main contribution is a way to test whether downstream decisions remain stable under plausible ASR alternatives.

The paper-facing frame is decision stability under plausible transcript alternatives in high-stakes conversational decision systems. Repo documents use the confident, evidence-led voice defined in docs/research_writing_style_policy.md: start from a citation-backed real-world problem, summarize current solutions fairly, identify the remaining decision-stability gap, and present CDS-ASR as the new viewpoint.

audio
-> ASR transcript + confidence / n-best / timestamps
-> risk atom extraction
-> plausible counterfactual transcript variants
-> downstream decision stability / CEIS
-> automatic constrained recovery or conservative machine action

The original downloaded zip files were kept, and the extracted folders were moved into stable part-### names. Large audio/transcript assets remain local.

Layout

Writing Style

Use Taiwan Traditional Chinese for Chinese-language project notes. Human-facing docs should use confident, generous, positive-scope phrasing: lead with the contribution and evidence, then state boundaries as scope controls and next validation steps. The repo policy is docs/research_writing_style_policy.md.

Inventory files

Notes

• Missing expected part: 004.
• The 2026-05-18 archive organization pass performed no data deletion.
• The 2026-05-25 janus_old_train import is local-only under 90_legacy_imports/; non-selected LoRA and partial-encoder parameter files were pruned from the repo copy while experiment metadata and analysis records were retained.
• The 2026-05-25 canonical 258-row test split comparison now has aggregate six-model evidence under 70_experiments/runs/janus_258_test_split_asr_cds_proxy/: legacy partial encoder, legacy LoRA, Breeze-ASR-25 base, Breeze-ASR-26, Whisper large-v2, and Whisper small. The partial encoder remains the current ASR hypothesis generator candidate.
• The expanded ASR candidate matrix is recorded in docs/asr_candidate_expansion_2026_05_25.md and 60_whisper_asr_finetuning/configs/janus-15-asr-model-candidates.yaml. New candidates must pass smoke, 15-row hypothesis contract, runtime logging, and Taiwan Traditional Chinese locale gates before any full split run. The 2026-05-25 runtime gate is recorded in 70_experiments/runs/asr_candidate_runtime_gate_2026_05_25/: Whisper large-v3 and large-v3-turbo completed the 15-row gate but had locale violations. The 2026-05-26 extension in 70_experiments/runs/asr_candidate_15_row_extension_2026_05_26/ promoted SenseVoiceSmall and Qwen3-ASR-0.6B to the fixed 15-row gate; both passed the field contract but failed the strict zh-TW locale gate (14/15 and 15/15 locale-violation rows). Qwen3-ASR-1.7B still times out before inference at fetch/load, and Gemma 4 E2B/E4B remain blocked until an isolated multimodal runtime exposes AutoModelForMultimodalLM. A 2026-05-26 query-time verification rechecked the registry, reran SenseVoice/Qwen 0.6B hypothesis validators, and confirmed the local Transformers runtime still lacks the Gemma 4 multimodal model classes. Later 2026-05-26 02:03 and 02:44 CST live checks also confirmed all requested model pages remain public and ungated, SenseVoice/Qwen3-ASR-0.6B 15-row validators still pass, and the local Gemma 4 multimodal classes are still absent. The current 2026-05-26 bounded recheck in 70_experiments/runs/asr_candidate_current_recheck_2026_05_26/ validated Whisper large-v3, Whisper large-v3-turbo, SenseVoiceSmall, and Qwen3-ASR-0.6B against the fixed 15-row contract, reran a 60-second Qwen3-ASR-1.7B load gate, and repeated the Gemma 4 class probe. Decision: no remaining requested candidate should move to full-split runtime before locale/runtime policy changes. A follow-up live check at 2026-05-26 03:43 CST confirmed the same decision: public model metadata is still available, the four 15-row candidates still validate, Qwen3-ASR-1.7B still times out at fetch/load, and Gemma 4 still needs an isolated runtime because local Transformers does not recognize model_type=gemma4. The 2026-05-26 05:19 CST live recheck repeated the same bounded gate: four 15-row candidate files still pass field-contract validation, Qwen3-ASR-1.7B still times out at fetch/load after 60.07s, and Gemma 4 remains runtime-blocked. No full-split promotion is justified until locale/runtime policy changes. A response-time validation at 2026-05-26 06:45 CST revalidated the four existing 15-row hypothesis files and the local Gemma class probe without starting a new full inference run; the decision remains unchanged. A 2026-05-26 07:24 CST bounded recheck again validated the four 15-row files, confirmed the seven model pages remain public/ungated, confirmed local Gemma multimodal classes are still absent, and intentionally did not rerun Qwen3-ASR-1.7B because repeated tracked bounded gates already timed out before inference. A 2026-05-26 07:45 CST response-time verification repeated the same safe gate: four 15-row files still pass contract validation, locale blockers remain unchanged, all seven model pages are still public/ungated, and the local Gemma 4 multimodal classes are still absent. This is a no-promotion decision record, not a new full-split model experiment. A 2026-05-26 08:25 CST bounded verification reran the safe gate, including a 60-second Qwen3-ASR-1.7B load attempt; the four 15-row files still pass, locale blockers are unchanged, Qwen3-ASR-1.7B still timed out before inference at 60.07s, and Gemma 4 E2B/E4B remain blocked because local Transformers cannot recognize model_type=gemma4.
• The postdoc-level roadmap after the 258-row gate is recorded in docs/postdoc_next_steps_2026_05_25.md. It defines the next sequence: complete comparable 258-row baselines, add split-aware metric inputs, run the 300-row high-stakes main experiment, then evaluate recovery policies.
• The machine-checkable evidence-chain readiness gate is 80_semantic_risk_asr/scoring/check_evidence_chain_readiness.py, with the current aggregate output under 70_experiments/runs/postdoc_evidence_chain_2026_05_25/. Current status: ok=true, paper_ready=true, with 10/10 readiness gates completed. The selected-300 human review is complete for the non-transcript risk, decision, safe-action, confidence, per-model assessment, and per-row timing fields: 30/30 risk/decision rows and 90/90 model assessments are reviewed, with 0 pending rows and 0 pending model assessments. Transcript ground truth is already accepted for WER/CER scoring and was not reopened. The reviewer action gate now reports response_complete_ready_to_write, and the post-review sequence reports post_review_sequence_complete. The current paper-facing scope is explicit: the 258-row evidence is used as scope-controlled split/model-comparison evidence, selected-300 proxy outputs are treated as input provenance, and selected-300 human-reviewed predictor and recovery outputs carry the paper-grade risk/recovery claims. The normal refresh_human_audit_evidence.py --require-complete path now reproduces this state and records publishable_ready=true, consequence_paper_claims_ready=true, roadmap_complete=true, objective_requirements_ready=true, and consistency_audit_ok=true. Tracked reviewer workflow files record only row numbers, commands, field names, counts, status, privacy boundaries, and runtime; they do not track audio IDs, transcripts, hypotheses, selected sample IDs, local row content, or reviewer notes. The aggregate consistency audit 80_semantic_risk_asr/scoring/audit_evidence_chain_consistency.py now checks these summaries together, including reviewer handoff freshness and timing awareness, per-row timing-helper command coverage, the response gap/action TSVs, the aggregate review work order, the post-review sequence gate, and the post-review command plan. Current status is ok=true with 26/26 checks passing: transcript ground truth is not reopened, the selected-300 review scope is fully covered, scoped proxy/provenance boundaries are preserved, and expanded ASR/Gemma candidates remain behind locale/runtime gates. The post-review evidence checklist 80_semantic_risk_asr/annotation/build_post_review_evidence_checklist.py records post_review_evidence_ready with paper_ready=true, publishable_ready=true, and consequence_ready=true. A stricter objective-by-objective publication audit is 80_semantic_risk_asr/scoring/audit_publishable_evidence_chain.py; its current output records publishable_ready=true, with all 7/7 objectives completed under the scoped paper-claim interpretation. The explicit original-objective requirements audit 80_semantic_risk_asr/scoring/audit_postdoc_objective_requirements.py verifies the named 0-6 requirements directly. Current status: objective_requirements_ready=true, with 15/15 requirements satisfied. audit_postdoc_roadmap_completion.py also reports roadmap_complete=true and blocking_gate=none.
• Split-aware metric-input generation now lives at 80_semantic_risk_asr/scoring/build_janus_metric_inputs.py, with validation recorded in 70_experiments/runs/janus_split_aware_metric_inputs_2026_05_25/. Manifest transcripts used as WER/CER scoring references are treated as already human-reviewed ground truth. CDS/risk-atom labels are a separate review surface; do not reopen transcript review unless the requested human-review fields differ from the provided ground-truth transcript fields.
• The first automatic recovery policy gate now lives at 80_semantic_risk_asr/recovery/evaluate_recovery_policies.py, with the six-model 258-row proxy result recorded in 70_experiments/runs/janus_258_recovery_policy_proxy_2026_05_25/. Treat it as engineering evidence only until the selected 300-row high-stakes and human risk-atom audit gates run.
• The selected-300 human-reviewed recovery rerun path now lives at 80_semantic_risk_asr/recovery/evaluate_human_reviewed_recovery_policies.py, with the completed aggregate summary recorded in 70_experiments/runs/janus_300_high_stakes_recovery_human_reviewed_2026_05_26/. Normal human-audit refresh updates this summary before the post-review checklist; current status is human_reviewed_complete, with 30/30 reviewed rows and 90/90 reviewed model assessments.
• The selected-300 metric-predictor proxy gate now lives at 80_semantic_risk_asr/scoring/analyze_metric_predictors.py, with aggregate output recorded in 70_experiments/runs/janus_300_high_stakes_metric_predictor_proxy_2026_05_25/. It compares WER/CER/SRES/CEIS against downstream label flips, unsafe downrouting, high-risk misses, and low-WER danger counts without tracking transcript or sample-level rows.
• The selected-300 human risk-atom audit queue now lives at 70_experiments/runs/janus_300_high_stakes_human_audit_selection_2026_05_25/. The tracked protocol is 80_semantic_risk_asr/annotation/selected_300_human_risk_atom_audit_protocol_2026_05_25.md. The transcript-bearing audit sheet remains local-only under ignored artifacts/; review is still pending. The local sheet is validated by 80_semantic_risk_asr/annotation/validate_human_risk_atom_audit.py, whose strict --require-complete mode currently fails as expected until all 30 risk/decision row-review fields and 90 model assessments are reviewed; this is not a duplicate transcript-review requirement. Aggregate review status is produced by 80_semantic_risk_asr/annotation/summarize_human_risk_atom_audit.py, and model-level reviewer assessments are required before making model-comparison claims from the human audit. Human-reviewed WER/CER/SRES/CEIS predictor tables are produced by 80_semantic_risk_asr/annotation/analyze_human_audit_predictors.py after review. Local row-by-row review support lives in 80_semantic_risk_asr/annotation/review_human_risk_atom_audit.py; its --show-row output is transcript-bearing and must stay local-only. The next local review batch is prepared by 80_semantic_risk_asr/annotation/prepare_human_audit_review_batch.py; current tracked batch records point to the ignored clean_control packet for rows 23-26 and 12 model assessments. Current batch completion is audited by 80_semantic_risk_asr/annotation/audit_human_review_batch_status.py and remains batch_pending: 0/4 rows and 0/12 model assessments reviewed for the current packet. A local-only reviewer handoff package for this packet was prepared at /home/jnln3799/Downloads/cib_asr_human_review_packet_2026-05-26_batch6_clean_control and zipped beside it; the package includes transcript-bearing material and must not be committed. Local TSV response entry is handled by 80_semantic_risk_asr/annotation/apply_human_audit_batch_response.py; the current ignored response template has 12 rows and its blank dry-run status is response_pending. The template now has optional review-timing columns, and the tracked apply summary records aggregate timing coverage without exposing row content. The response workflow also appends a repo-safe apply log row for every dry-run/write attempt and writes a machine-readable apply-log summary. A one-file reviewer handoff is available from 80_semantic_risk_asr/annotation/build_human_audit_reviewer_handoff.py; current handoff status is reviewer_input_pending and freshness_status=fresh. The handoff records source-summary SHA-256 digests; run the same tool with --check-existing before reviewer work and require handoff_fresh. Before opening local review files, run 80_semantic_risk_asr/annotation/preflight_human_audit_review_session.py; current preflight status is review_session_ready, with both local paths present. The reviewer value contract is generated by 80_semantic_risk_asr/annotation/build_human_audit_reviewer_rubric.py; current rubric status is rubric_ready, validator constants match the strict audit validator, and the contract records that transcript ground truth is not re-reviewed for WER/CER. The aggregate reviewer action checklist is generated by 80_semantic_risk_asr/annotation/build_human_audit_reviewer_action_checklist.py; current checklist status is reviewer_action_ready with rubric_status=rubric_ready, 4/4 packet rows, and 12/12 model assessments still pending in the ignored local response TSV. The normal reviewer-session start command is 80_semantic_risk_asr/annotation/start_human_audit_review_session.py; current start status is reviewer_session_started, with a tracked aggregate session-start summary/log and no change to human-review completion. The strict dry-run/write commands generated from the handoff now require --require-session-start-gate against that session-start summary; the live strict dry-run records session_start_gate.ok=true and still fails only because reviewer response content is incomplete. Response closeout is tracked by 80_semantic_risk_asr/annotation/build_human_audit_response_closeout_checklist.py; current closeout status is response_closeout_blocked because 0/4 row decisions and 0/12 model assessments are filled; the closeout summary now names the per-row aggregate gaps by row number only. The strict --require-complete dry-run currently exits nonzero as expected with ok=false and incomplete_response=1; this is the completion gate before any --write. After strict dry-run passes, use --write --refresh-after-write to update the ignored local sheet, audit the current batch, and refresh aggregate readiness/publishable status in one pass. Add --prepare-next-after-write when the same command should also prepare the next local review packet and response TSV template. After review edits, use 80_semantic_risk_asr/annotation/refresh_human_audit_evidence.py to rerun validation, aggregate review-progress counts, aggregate review summaries, human-reviewed predictor outputs, the evidence-chain readiness gate, and the objective-level publishable completion audit, and the roadmap completion audit in one recorded pass. Current refresh status is review_complete: 30/30 risk/decision row reviews and 90/90 model assessments reviewed, with paper_ready=true, publishable_ready=true, and roadmap_complete=true.
• The roadmap-level completion audit 80_semantic_risk_asr/scoring/audit_postdoc_roadmap_completion.py maps the original postdoc objective steps 0-6 plus the human-review/publishable gate to tracked aggregate evidence. Current output: roadmap_complete=true, publishable_ready=true, paper_ready=true, post_review_evidence_ready=true, and blocking gate none. It also records that the expanded ASR/Gemma candidates should not be promoted while strict zh-TW locale or multimodal runtime gates remain unresolved.
• The 2026-05-25 WER audit is recorded in 70_experiments/runs/wer_metric_audit_2026_05_25/. The latest audit checks legacy 15-row, six 258-row, and high-stakes 300-row hypothesis files against canonical manifests, records package versions, fails paper-facing summaries on zero-reference metric units, and cross-checks zh-jieba corpus WER against jiwer. Pre-audit WER fields are legacy raw whitespace-token values; paper-facing ASR tables should use the cer_zh_micro aggregate column as the primary surface metric and wer_zh_jieba_micro only as a supplemental segmented word metric. audit_wer_journal_compliance.py records the current journal-compliance verdict: paper reporting is compliant under that policy, while not all stored legacy wer fields are journal-compliant evidence.
• Treat audio/call data and filenames as sensitive.
• If storage cleanup is needed later, review 30_review_flags/REVIEW.md and 20_inventory/largest_files.tsv first.

2026-05-22 Whisper ASR Workspace Update

• Top-level .venv/ is treated as disposable and should be rebuilt from requirements-whisper.txt.
• All old symlinks that pointed at /home/jnln3799/Downloads/JANUS_ubuntu24/... were rewritten to repo-relative targets.
• The training entry point is now 60_whisper_asr_finetuning/datasets/janus_165_v1/hf_audiofolder.
• Experiment records should be registered in 70_experiments/registry.tsv before long training runs.

Purpose-Oriented Library

A complete purpose/type overlay is available at 50_janus_data_library/.

Use it to navigate the archive by goal:

• source archives
• raw audio
• segmented audio
• labels and transcripts
• Breeze-ASR-25 fine-tune-ready dataset
• models and checkpoints
• code and pipelines
• runtime environments
• evaluation and reports
• inventory and audit

For Whisper-specific work, start with docs/REPO_MAP.md and 60_whisper_asr_finetuning/README.md.

For the paper-facing research frame, start with 80_semantic_risk_asr/README.md and 80_semantic_risk_asr/paper/story_outline.md. For the current postdoc-level execution sequence, start with docs/postdoc_next_steps_2026_05_25.md.

Automated Version Control

This repo uses SemVer-style automated versioning. Current version:

v2.5.9

Source of truth:

• VERSION
• version_manifest.json
• CHANGELOG.md
• version_history.jsonl
• VERSIONING.md

Install the local git hook once per checkout:

python3 scripts/install_version_hooks.py

After installation, every commit that stages versioned repo content runs scripts/auto_version.py --stage, bumps the version, updates the manifest, and records a human-readable plus JSONL version log.