For local usage:

• Only supported on Linux for now

• A CUDA Compute Capable Environment is required what does this mean?

• Highly recommended to use uv

• training and evals can be replicated via scripts in the scripts/ directory

• You will also need an OpenAI (or other provider) key in a .env somewhere if not finetuning and just evaluating.

• mamba models:

  • ensure you have mamba-ssm[causal-conv1d] installed. This adds significant overhead on first installation but these models cannot be run without these packages. Feel free to uv remove them if not using mamba.

• Other LLMs:

• Coming Soon

General Help:

• uv run -m module.file for any program you want to run (evals, training, etc...)
• bfloat16 is not functional at the moment, but pull requests for its funcitonality are appreciated and wanted.

For server usage:

• All scripts in scripts/slurm are written with the assumption that they are executed via sbatch from the root directory (i.e. sbatch /path/to/script). These were written for use on Worcester Polytechnic Institute's SLURM server, and may require refactoring depending on your environment.

• Other reqs are the same, highly recommended to run scripts/install_mamba.sh prior to anything else in order to install mamba-ssm[causal-conv1d] for your uv environment.