Adaptive Difficulty-Driven Router

Final Project for CSE537 Artificial Intelligence. Router predicts the difficulty of math word problems from GSM8K dataset and sends the hard problems to WizardMath (7B parameter model) and the easy problems to Phi2 (2.7B parameter model). Two primary methods were used for the development of the router -

1. Random Forest + XGBoost (Stacked Classifier)
2. LinUCB (Multi Armed Bandits Algorithm)

Code Files -

1. phi2_solo_tester.py - contains the code to run phi2 on GSM8K problems. We used this to first generate the dataset for training our classifier (2000 problems of GSM8K) and then again to use as a baseline for testing our router (500 problems of GSM8K).
2. wizardmath_solo_tester.ipynb - contains essentially the same code as phi2_solo_tester.py, except it uses wizardmath instead. We used this for generating the wizardmath baseline.
3. difficulty_predictor.py - contains the code to train the router and run it on 500 problems of GSM8K to generate difficulty predictions.
4. ai_pipeline.ipynb - contains the code to take the predictions generated by difficulty_predictor.py and run the overall system.
5. linucb.ipynb - contains the code for our linucb implementation of the router.

Files 1-4 must be run in order, file 5 is independent.

Requirements -

Python 3.11.9
numpy 2.2.4
cuda 12.1