📝 Automatic Meeting Minutes & Action Item Extraction System

Turn a meeting recording (audio) or a transcript into structured minutes: an abstractive summary, key decisions, and a list of action items {owner, task, due}. A pipeline of a pretrained Whisper ASR front-end → a trainable abstractive summarizer (map-reduce for long meetings) → a trainable action-item extractor, wrapped in an agentic finite-state machine with input routing, long-vs-short routing, a quality gate, an action-item confidence gate, and a summary-faithfulness gate.

NLP in Industry — Final Assignment. Author: Le Dinh Minh Quan (Student 23127460). Reference: Zackriya-Solutions/meeting-minutes (Meetily) — local, privacy-first.

Manual minute-taking is slow, inconsistent and lossy; action items get dropped because nobody owns capturing them. P10 automates it — and adds the trainable NLP contribution the reference lacks: a fine-tuned dialogue→meeting summarizer and an explicit action-item / decision extractor, with per-item confidence and source provenance so a human can verify before circulating.

---

✅ How this repo meets every assignment requirement

---

🏗️ Pipeline

audio recording / transcript
  │  ingest: ASR (Whisper) if audio, else passthrough     ── D1 input routing
  ▼
segment + token-count
  │  chunk + map-reduce if the transcript is long          ── D2 long-vs-short routing
  │  ASR / segment quality check                           ── D3 quality gate
  ▼
abstractive SUMMARIZER  →  summary           (length / faithfulness gate ── D5)
  ▼
ACTION-ITEM extractor  →  {owner, task, due} (confidence / dedup / clarify ── D4)
  ▼
assemble minutes (Markdown + JSON): summary + decisions + action items

📦 Models & data (ids VERIFIED on the HF Hub)

The main dialogue/meeting corpora are non-commercial (CC-BY-NC) — fine for this academic project, flagged in docs/. No gold action-item dataset exists, so action labels are synthesized (weak imperative/commitment labeling). A bundled 3-meeting seed corpus with gold minutes powers the fully-offline demo, tests and eval.

🗂️ Repository layout

src/meetingai/
├── config.py  cli.py  logging_utils.py
├── data/         synthetic.py (seed meetings + gold) · transcript.py · datasets.py · download_dataset.py
├── asr/          transcriber.py (Whisper + passthrough fallback)
├── summarize/    summarizer.py (abstractive + extractive + map-reduce) · minutes.py
├── actions/      extractor.py (classifier + heuristic + slot-filling)
├── models/       model_registry.py
├── training/     train_summarizer.py · train_actions.py · evaluate.py · tune.py · metrics.py
├── agent/        state.py · policy.py (D1–D5) · tools.py · llm_orchestrator.py · minutes_agent.py
├── api/          schemas.py · dependencies.py · main.py · ui.py · app_combined.py
├── analysis/ autoreport/ monitoring/ automation/ grading/
configs/ · data/ · models/ · tests/ · docs/ · notebooks/ · app/ · deploy/ · sample_data/

---

🚀 Quickstart

pip install -e ".[ml,api,report]"      # add .[audio] (+ ffmpeg) for the Whisper ASR path

meetingai demo-agent --fast            # run the agent on the seed meetings (offline)
meetingai minutes --transcript sample_data/sample_transcript.txt --title "Standup" --fast

Train

meetingai --config configs/train.yaml train-summarizer   # fine-tune the summarizer (Seq2Seq, ROUGE)
meetingai --config configs/train.yaml train-actions      # fine-tune the action-item classifier
meetingai evaluate                                       # vs lead-3/TextRank + action-item F1

On Colab/GPU use the notebook (below) — it auto-profiles H100/A100/L4/T4.

Serve

meetingai serve --ui --port 7860       # FastAPI /minutes + /transcribe + Gradio UI at /ui

One-button train → report + slides + self-grade

meetingai autopilot --no-train         # eval → analysis → report.pdf + slides.pptx + bundle
meetingai grade

---

🤖 The agent (mandatory agentic component)

A deterministic FSM with five decision points acting on the model's own intermediate outputs, plus an optional LLM brain (anthropic, opt-in, validated, rule-fallback — kept off by default for privacy):

• D1 input routing (audio → Whisper vs transcript passthrough)
• D2 long-vs-short routing (chunk + map-reduce summarization for long meetings)
• D3 ASR / segment quality gate (flag low-confidence transcripts for review)
• D4 action-item confidence / dedup / clarify gate (drop weak/duplicate items; flag missing owner/due)
• D5 summary length / faithfulness gate (reject a degenerate summary, fall back)

Every step is timed + traced; same input + same models + brain disabled ⇒ identical output. See docs/agent_architecture.md.

☁️ Colab / H100 training

Open notebooks/Meeting_Minutes_Colab_Training_H100_AUTOPILOT.ipynb — mounts Drive, installs Colab-safe deps (+ ffmpeg, never touches torch), auto-profiles the GPU, fine-tunes the summarizer (+ action classifier) resume-safely, evaluates vs baselines, runs the agent, and generates the report/slides. Step-by-step: notebooks/COLAB_GUIDE.md.

🧪 Tests

pytest -q        # CPU-only, no model/audio downloads (seed meetings + extractive + heuristic)

📚 Docs index

docs/: problem_definition · data_description · data_card · model_selection · evaluation · agent_architecture · deployment · continual_learning_monitoring · privacy_robustness · project_plan · ethics_statement · architecture · model_card · slide_deck_outline · DESIGN_BRIEF.

📝 License

MIT — see LICENSE. Pretrained models keep their own licenses (table above). The dialogue/meeting training data is CC-BY-NC (research/academic only) — the trained model inherits a use restriction; flag before any commercial deployment (the permissive paths are pszemraj/qmsum-cleaned

• the MIT/Apache base checkpoints). Meeting recordings are sensitive PII — obtain all-party consent and prefer local processing (see docs/privacy_robustness.md).