Reproducible Google Colab GPU experiments for decision-model inference
See the code
Reproducible Google Colab GPU experiments for decision-model inference
English · 日本語
This repository records small, reproducible experiments that run Jev-like decision-model paths on Google Colab GPUs. Each experiment keeps its runner, notebook, pinned upstream revisions, sanitized inputs, and result JSON together under experiments/<slug>/.
The current snapshot contains five integrated GPU result sets: Laya, Kev-0.5B, Jevlike tiny option-attention, SemIf 4B, and OpenJev NLI 4B. They target different model paths and fixtures, so their probabilities, accuracy, latency, and VRAM values are not a common benchmark.
The repository is documentation-first: local checks do not download model weights.
git clone https://github.com/Sunwood-ai-labs/jev-colab-lab.git
Set-Location jev-colab-lab
# Run the model-free helper and fixture tests with uv.
uv run --no-project --with pytest pytest experiments/laya/tests experiments/openjev-nli/tests experiments/semif/tests -q
# Run the Jevlike regression suite with its declared torch dependency.
uv run --project experiments/jevlike --extra dev pytest experiments/jevlike/tests -q
# Check the Kev runner syntax without installing its model dependencies.
uv run --no-project python -m py_compile experiments/kev/t4_inference.py
For a model-backed local smoke test, follow the experiment README and use that experiment's uv project. A local CPU run is never reported as a Colab GPU result.
| Experiment | Target | Measurement focus | Evidence |
|---|---|---|---|
| Laya / ModernBERT | Colab T4 | choice, score, and noul decision outputs | runner · T4 JSON |
| Kev-0.5B | Colab T4 | packed/separate option probabilities and latency | runner · T4 JSON |
| Jevlike | Colab T4 | short synthetic training, held-out test, shuffled-context control | runner · T4 JSON |
| SemIf / Qwen3.5-4B | Colab L4 | direct next-token option-logit readout | runner · L4 JSON |
| OpenJev NLI 4B | Colab L4 | entailment, contradiction, and neutral scores | runner · L4 JSON |
The checked-in JSON is the canonical evidence. Jevlike timing fields retain their measurement boundary in the result schema; this landing page does not use them as a cross-experiment comparison point.
The five model paths have a JevDash Level 1 capture set outside Git. This is an independent single-episode observation set, not a completed comparative game-ability evaluation; the Jevlike path has a confirmed input-truncation defect. The public experiment pages link to the runners and sanitized evidence that can be reviewed here; the MP4 files themselves have no public URL in this repository and are intentionally kept out of Git.
These are independent single-episode demonstrations, not a model ranking. Video time is simulation time with synchronous inference waits omitted. The Kev and Jevlike presentation replays preserve the recorded model trajectory and state, changing only the HUD rendering. See the Japanese Colab summary for the measured selection results and limitations.
Start from the public capture evidence index for the five experiment-specific conduits.
status=success or status=ok is meaningful only together with the recorded GPU and runtime fields.The official Colab CLI currently runs on Linux and macOS, not Windows. On a Windows host, use WSL Ubuntu and keep the CLI state outside this repository:
uv tool install google-colab-cli
colab --help
Each experiment README gives the target accelerator, a unique jev-<slug> session name, an isolated session-state file, the upload/execute/download flow, and the required stop or ephemeral-run cleanup. GPU quota, entitlement, and authentication can still block a run; record that condition once instead of retrying indefinitely.
experiments/<slug>/
├── README.md # experiment-specific scope and reproduction
├── notebooks/ # Colab CLI notebooks where applicable
├── results/ # sanitized measurements and blocker records
├── scripts/ # runners and fixture validators
└── tests/ # model-free helper tests where applicable
docs/ # published bilingual guide
.github/workflows/ # docs deployment and public QA
NOTICE.md # upstream source and license boundaries
This is a research record, not the private TypeSafe Jev model, a production decision service, or a claim that the listed implementations are equivalent. Nimble 9B, OpenJev 35B, and DiffusionGemma remain deferred. See the full guide for measurement semantics, source revisions, and troubleshooting.
Original repository glue code and documentation are released under the MIT License. Upstream implementations, model weights, and their licenses remain separate; see NOTICE.md and each experiment's pinned source manifest. No model weights are stored in this repository.
Reproducible Google Colab GPU experiments for decision-model inference
See the code
Reproducible Google Colab GPU experiments for decision-model inference
English · 日本語
This repository records small, reproducible experiments that run Jev-like decision-model paths on Google Colab GPUs. Each experiment keeps its runner, notebook, pinned upstream revisions, sanitized inputs, and result JSON together under experiments/<slug>/.
The current snapshot contains five integrated GPU result sets: Laya, Kev-0.5B, Jevlike tiny option-attention, SemIf 4B, and OpenJev NLI 4B. They target different model paths and fixtures, so their probabilities, accuracy, latency, and VRAM values are not a common benchmark.
The repository is documentation-first: local checks do not download model weights.
git clone https://github.com/Sunwood-ai-labs/jev-colab-lab.git
Set-Location jev-colab-lab
# Run the model-free helper and fixture tests with uv.
uv run --no-project --with pytest pytest experiments/laya/tests experiments/openjev-nli/tests experiments/semif/tests -q
# Run the Jevlike regression suite with its declared torch dependency.
uv run --project experiments/jevlike --extra dev pytest experiments/jevlike/tests -q
# Check the Kev runner syntax without installing its model dependencies.
uv run --no-project python -m py_compile experiments/kev/t4_inference.py
For a model-backed local smoke test, follow the experiment README and use that experiment's uv project. A local CPU run is never reported as a Colab GPU result.
| Experiment | Target | Measurement focus | Evidence |
|---|---|---|---|
| Laya / ModernBERT | Colab T4 | choice, score, and noul decision outputs | runner · T4 JSON |
| Kev-0.5B | Colab T4 | packed/separate option probabilities and latency | runner · T4 JSON |
| Jevlike | Colab T4 | short synthetic training, held-out test, shuffled-context control | runner · T4 JSON |
| SemIf / Qwen3.5-4B | Colab L4 | direct next-token option-logit readout | runner · L4 JSON |
| OpenJev NLI 4B | Colab L4 | entailment, contradiction, and neutral scores | runner · L4 JSON |
The checked-in JSON is the canonical evidence. Jevlike timing fields retain their measurement boundary in the result schema; this landing page does not use them as a cross-experiment comparison point.
The five model paths have a JevDash Level 1 capture set outside Git. This is an independent single-episode observation set, not a completed comparative game-ability evaluation; the Jevlike path has a confirmed input-truncation defect. The public experiment pages link to the runners and sanitized evidence that can be reviewed here; the MP4 files themselves have no public URL in this repository and are intentionally kept out of Git.
These are independent single-episode demonstrations, not a model ranking. Video time is simulation time with synchronous inference waits omitted. The Kev and Jevlike presentation replays preserve the recorded model trajectory and state, changing only the HUD rendering. See the Japanese Colab summary for the measured selection results and limitations.
Start from the public capture evidence index for the five experiment-specific conduits.
status=success or status=ok is meaningful only together with the recorded GPU and runtime fields.The official Colab CLI currently runs on Linux and macOS, not Windows. On a Windows host, use WSL Ubuntu and keep the CLI state outside this repository:
uv tool install google-colab-cli
colab --help
Each experiment README gives the target accelerator, a unique jev-<slug> session name, an isolated session-state file, the upload/execute/download flow, and the required stop or ephemeral-run cleanup. GPU quota, entitlement, and authentication can still block a run; record that condition once instead of retrying indefinitely.
experiments/<slug>/
├── README.md # experiment-specific scope and reproduction
├── notebooks/ # Colab CLI notebooks where applicable
├── results/ # sanitized measurements and blocker records
├── scripts/ # runners and fixture validators
└── tests/ # model-free helper tests where applicable
docs/ # published bilingual guide
.github/workflows/ # docs deployment and public QA
NOTICE.md # upstream source and license boundaries
This is a research record, not the private TypeSafe Jev model, a production decision service, or a claim that the listed implementations are equivalent. Nimble 9B, OpenJev 35B, and DiffusionGemma remain deferred. See the full guide for measurement semantics, source revisions, and troubleshooting.
Original repository glue code and documentation are released under the MIT License. Upstream implementations, model weights, and their licenses remain separate; see NOTICE.md and each experiment's pinned source manifest. No model weights are stored in this repository.