Reproduction Code for "On Predictability of Reinforcement Learning Dynamics for Large Language Models"

This repository corresponds to the paper:

On Predictability of Reinforcement Learning Dynamics for Large Language Models
https://arxiv.org/pdf/2510.00553

---

Installation

# clone codebase
git clone https://github.com/caiyuchen-ustc/Alpha-RL.git && cd Alpha-RL

# prepare environment
conda create -y -n AlphaRL python=3.11
conda activate AlphaRL
cd Alpha-RL

# install dependencies
pip install -r requirements.txt

Download_Model

You can access the checkpoint at the following link: Hugging Face - caiyuchen

# run
cd eval
sh download_hf.sh

Model Evaluation

sh reasoning_eval.sh

Singular Value Decomposition

sh svd.sh # Obtain the SVD decomposition of each matrix in a model

Obtain a Rank-k Model

sh upd_rank.sh 

t-SNE Visualization of Training Trajectories

cd analysis #eval/analysis
sh extract_rank1_u.sh #Extract U[:,0]
sh visualize_rank1_u_tsne.sh

PLS (Partial Least Squares) Trajectory Fitting

sh AlphaPLS.sh

AlphaRL Predict

sh AlphaPredVector.sh
sh AlphaRLBuildPredictModel.sh