CS229 Project submission

For our project we used model implementations and reproduced previous papers from the following repositories:

• https://github.com/krasheninnikov/internalization
• https://github.com/EleutherAI/gpt-neox
• https://github.com/BlinkDL/RWKV-LM

For each, we have implemented new files and experiments for our 229 project:

• internalization
• • plot_neox_checkpoints.py
• • plot_rwkv_checkpoints.py
• • figure9_minimal.py
• • figure9_vision_minimal.py
• • *.pdf
• gpt-neox
• • generate_configs.py
• • generate_configs_410M.py
• • generate_configs_70M.py
• • train_sequentially.sh
• • train_sequentially_410M.sh
• • train_sequentially_70M.sh
• • *.yaml
• RWKV-LM
• • train_v5_sequential.sh