╭────────────────────────────────────────────────────────────╮
│      ███████╗██████╗ ██╗  ██╗ █████╗ ██╗     ██╗           │
│      ██╔════╝██╔══██╗██║  ██║██╔══██╗██║     ██║           │
│      █████╗  ██║  ██║███████║███████║██║     ██║           │
│      ██╔══╝  ██║  ██║╚════██║██╔══██║██║     ██║           │
│      ███████╗██████╔╝     ██║██║  ██║███████╗███████╗      │
│      ╚══════╝╚═════╝      ╚═╝╚═╝  ╚═╝╚══════╝╚══════╝      │
╰────────────────────────────────────────────────────────────╯

What Ed4All does

• Converts source material into semantic, accessibility-oriented HTML with source provenance and automated validation.
• Builds digital courses with modules, learning objectives, activities, assessments, and machine-readable educational metadata.
• Packages courses for an LMS using IMS Common Cartridge (IMSCC).
• Indexes course content in a reusable local library with lexical, semantic, and hybrid-RRF retrieval.
• Generates grounded training data as SFT instruction pairs and DPO preference pairs derived from course content.
• Optionally trains a LoRA adapter and evaluates it alongside the course's retrieval system.

From source to course-grounded AI

flowchart LR
    materials["Books, PDFs, HTML,<br/>and learning materials"]
    semantik["SemantiK<br/>Accessible, structured HTML<br/>with source provenance"]
    courseforge["Courseforge<br/>Modular course content<br/>and an LMS-ready IMSCC"]
    trainforge["Trainforge<br/>Retrieval corpus plus<br/>SFT and DPO pairs"]
    libv2["LibV2<br/>Private course archive<br/>and optional LoRA training"]
    retrieval["Hybrid retrieval<br/>BM25 + dense search<br/>with rank-domain RRF"]
    answers["Course-grounded AI<br/>Evidence thresholds,<br/>refusal, and cited answers"]

    materials --> semantik --> courseforge --> trainforge --> libv2 --> retrieval --> answers

    classDef sourceNode fill:#eef6ff,stroke:#2563eb,color:#172554,stroke-width:2px;
    classDef buildNode fill:#f0fdf4,stroke:#16a34a,color:#14532d;
    classDef deliveryNode fill:#fff7ed,stroke:#ea580c,color:#7c2d12;
    classDef intelligenceNode fill:#faf5ff,stroke:#9333ea,color:#581c87;

    class materials sourceNode;
    class semantik,courseforge buildNode;
    class libv2 deliveryNode;
    class trainforge,retrieval,answers intelligenceNode;

One continuous workflow carries the source through accessible HTML, course design, LMS packaging, grounded-data synthesis, archival, retrieval, and citation validation. Optional LoRA training uses the generated training pairs as a separate operator opt-in; the course package, archive, and retrieval system remain complete deliverables without an adapter.

Retrieval that earns trust

Ed4All does more than make course content searchable. It builds a private, course-scoped evidence system designed to show its work:

• Lexical precision + semantic reach. BM25 catches exact terminology while dense retrieval finds conceptually related passages.
• Custom rank-domain RRF. Ed4All fuses rank positions instead of adding incompatible BM25 and cosine scores, rewarding evidence found by both arms without discarding strong single-arm results.
• Refusal before generation. Weak retrieval stops before the answer model can turn uncertainty into fluent guesswork.
• Citation-grounded answers. Returned citations must resolve to passages visible to the composer; unsupported answers are withheld.
• Useful with or without training. The retrieval system is a complete deliverable on its own and can also evaluate whether a LoRA adapter adds value beyond the indexed course.

The result is a local course intelligence layer that stays attached to the source: private indexes, reproducible ranking, inspectable evidence, and answers that can be traced back to the material. See the retrieval architecture.

Quick start

Ed4All requires Python 3.10 or newer. Tesseract OCR and Poppler improve extraction from scanned or image-heavy PDFs.

See the installation guide for platform dependencies, capability extras, Playwright, and the required third-party IMS Common Cartridge schemas.

git clone https://github.com/mdmurphy822/Ed4All.git
cd Ed4All
pip install -e ".[full]"

ed4all run textbook-to-course \
  --corpus <path-to-source> \
  --course-name <course-name>

Authoring and synthesis phases require a configured model provider. See the pipeline invocation guide for local and hosted OpenAI-compatible endpoint setup.

The default install leaves out the largest machine-learning dependencies. Add only the capabilities you need:

pip install -e '.[full,embedding]'
pip install -e '.[full,training]'

Before a production build, review the full-run playbook and licensing posture.

Choose your outcome

Convert documents to accessible HTML

Create remediated HTML without generating a course:

ed4all convert <source> --output <output-directory>

See the conversion guide.

Build and package a digital course

ed4all run textbook-to-course \
  --corpus <path-to-source> \
  --course-name <course-name>

This orchestrates conversion, course planning and generation, validation, IMSCC packaging, archival, and indexing according to the selected workflow and configuration.

Query existing course content

Query an archived and indexed course through the retrieval layer:

libv2 retrieve "<question>" --course <course-name> --engine hybrid-rrf

Hybrid RRF combines BM25 term matching with dense vector similarity. Results retain course and chunk provenance so downstream answer systems can cite the retrieved material. See retrieval and serving.

Generate training data or train an adapter

Training-pair synthesis produces SFT instructions and DPO preferences from the course's chunks and assessments. Adapter training is a separate, opt-in, GPU-bound stage:

# Build a course and explicitly include its training stages.
ed4all run textbook-to-course \
  --corpus <path-to-source> \
  --course-name <course-name> \
  --with-training

# Or train from an already archived course.
ed4all run trainforge_train \
  --course-name <course-name> \
  --base-model <supported-model>

Model licenses and provider terms determine whether generated pairs and trained derivatives are distributable. Read Licensing and ToS posture first. Full training runs and promotion decisions remain operator-driven.

Generated SFT and DPO pairs, retrieval indexes, and course artifacts always remain private local data; they are not published from this repository.

Why Ed4All

• One grounded content lineage. Every stage works from the same source material.
• Accessibility is part of the pipeline. Semantic structure and automated checks are designed to support WCAG 2.2 AA targets; final conformance still depends on the source, configuration, generated content, and human review.
• Standards-based packaging. Courseforge emits IMS Common Cartridge.
• Retrieval is a first-class deliverable. A course can be searched and queried without training an adapter.
• Training is explicit. LoRA stages do not attach to a default build unless the operator opts in.
• Providers are configurable. Authoring can use configured local or hosted OpenAI-compatible endpoints.

Components

• SemantiK converts documents into accessibility-oriented HTML with source provenance.
• Courseforge creates modular course content, learning activities, and IMS Common Cartridge packages.
• Trainforge creates tagged chunks, assessments, knowledge structures, SFT/DPO pairs, and optional LoRA training inputs.
• LibV2 archives and queries course content through lexical, semantic, and hybrid reciprocal-rank-fusion retrieval.

MCP orchestrates workflows; cli, gui, and lib provide the user and shared service surfaces.

Ways to run it

• Command line: ed4all run --help and the invocation guide.
• Browser: ed4all gui and the GUI guide.
• Containers: the provided Compose deployment and Docker guide.
• Long-running workflows: checkpoints, resume, and graceful stop are covered by the operations runbook.

Documentation

• Installation and local dependencies
• Architecture overview
• Pipeline flow
• Validation gates
• Licensing and ToS posture
• Component guides: SemantiK, Courseforge, Trainforge, and LibV2

License

Ed4All is available under the Apache License 2.0. See LICENSE.