Machine Learning Final Project - Team 30 - Model Merging

This project demonstrates model merging techniques using SLERP (Spherical Linear Interpolation) to combine two language models: Breeze-7B (Traditional Chinese) and WizardMath-7B (Mathematical reasoning). The merged model is evaluated on multiple benchmarks to assess its performance compared to the baseline models.

---

Table of Contents

• Project Overview
• Requirements
• Installation
• Project Structure
• Usage
  • Step 1: Run Baseline Evaluations
  • Step 2: Merge Models
  • Step 3: Repair Merged Model
  • Step 4: Evaluate Merged Model
• Datasets
• Results
• Troubleshooting

---

Project Overview

This project aims to:

1. Establish baseline performance of Breeze-7B and WizardMath-7B on three benchmark datasets
2. Merge the two models using SLERP method with custom weight configurations
3. Evaluate the merged model's performance on the same benchmarks
4. Compare merged model performance against baseline results

Benchmark Datasets:

• GSM8K: Grade School Math problems (English)
• DRCD: Delta Reading Comprehension Dataset (Traditional Chinese)
• Math23k: Chinese elementary math word problems

---

Requirements

System Requirements

• GPU: CUDA-compatible GPU with at least 16GB VRAM (recommended)
• RAM: At least 32GB system memory
• Storage: At least 50GB free disk space
• Internet: Required for downloading models from Hugging Face Hub

Python Version

• Python 3.8 or higher

Required Libraries

All dependencies are listed below and can be installed via pip.

---

Installation

1. Clone or Navigate to Project Directory

cd /path/to/Machine_Learning_Final_Project

2. Create Virtual Environment (Recommended)

python3 -m venv venv
source venv/bin/activate  # On Linux/Mac
# OR
venv\Scripts\activate  # On Windows

3. Install Required Libraries

pip install torch transformers mergekit pyyaml tqdm bitsandbytes pillow accelerate huggingface_hub

Library Descriptions:

• torch: PyTorch deep learning framework
• transformers: Hugging Face transformers library
• mergekit: Model merging toolkit
• pyyaml: YAML configuration parser
• tqdm: Progress bar utility
• bitsandbytes: 4-bit quantization support
• pillow: Image processing library (compatibility)
• accelerate: Distributed training and inference
• huggingface_hub: Hugging Face Hub API for model downloads

4. Verify Installation

python3 -c "import torch; print('PyTorch version:', torch.__version__)"
python3 -c "import transformers; print('Transformers version:', transformers.__version__)"

---

Project Structure

Machine_Learning_Final_Project/
├── models/                     # (Optional) Local model storage
│   ├── Breeze-7B/              # Breeze-7B model files (if downloaded locally)
│   └── WizardMath-7B/          # WizardMath-7B model files (if downloaded locally)
├── baseline_Breeze_GSM8K.py    # Baseline: Breeze on GSM8K
├── baseline_Breeze_DRCD.py     # Baseline: Breeze on DRCD
├── baseline_Breeze_Math23k.py  # Baseline: Breeze on Math23k
├── baseline_Wizard_GSM8K.py    # Baseline: WizardMath on GSM8K
├── baseline_Wizard_DRCD.py     # Baseline: WizardMath on DRCD
├── baseline_Wizard_Math23k.py  # Baseline: WizardMath on Math23k
├── run_merge_SLERP.py          # Model merging script
├── repair_model.py             # Model alignment/repair script
├── check_answer.py             # Re-evaluate results with improved detection
├── benchmark_Merged_GSM8K.py   # Merged model evaluation on GSM8K
├── benchmark_Merged_DRCD.py    # Merged model evaluation on DRCD
├── benchmark_Merged_Math23k.py # Merged model evaluation on Math23k
├── GSM8K_dataset.csv           # GSM8K test dataset
├── DRCD_test.json              # DRCD test dataset (if available)
├── math_tw.json                # Math23k dataset
└── README.md                   # This file

---

Usage

Step 1: Run Baseline Evaluations

Before merging, establish baseline performance for both models on all three datasets.

Breeze-7B Baseline Evaluations

# GSM8K benchmark
python3 baseline_Breeze_GSM8K.py

# DRCD benchmark
python3 baseline_Breeze_DRCD.py

# Math23k benchmark
python3 baseline_Breeze_Math23k.py

WizardMath-7B Baseline Evaluations

# GSM8K benchmark
python3 baseline_Wizard_GSM8K.py

# DRCD benchmark
python3 baseline_Wizard_DRCD.py

# Math23k benchmark
python3 baseline_Wizard_Math23k.py

Expected Output:

• Each script will generate a JSON results file (e.g., GSM8K_results_Breeze.json)
• Progress bars will show evaluation progress
• Final accuracy metrics will be printed to console

Note: Each baseline evaluation may take 30-60 minutes depending on your hardware.

---

Step 2: Merge Models

Use SLERP (Spherical Linear Interpolation) to merge Breeze-7B and WizardMath-7B.

python3 run_merge_SLERP.py

What happens during merging:

1. Automatically downloads models from Hugging Face Hub (if not already cached):
   • MediaTek-Research/Breeze-7B-Instruct-v1_0
   • WizardLM/WizardMath-7B-V1.1
2. Applies SLERP with custom weight configurations:
   • self_attn layers: [0, 0.5, 0.5, 0.6, 0.7]
   • mlp layers: [1, 0.5, 0.7, 0.3, 0.2]
   • Other layers: 0.5 (default)
3. Saves merged model to ./merged_model/
4. Downloads and saves tokenizer files from Breeze-7B
5. Fixes configuration file for compatibility

Note: If you prefer to use local models, you can modify the paths in run_merge_SLERP.py (lines 20-21)

Expected Output:

📍 Current Workspace: /path/to/project
📦 Breeze Model: MediaTek-Research/Breeze-7B-Instruct-v1_0
📦 Wizard Model: WizardLM/WizardMath-7B-V1.1
🚀 Starting model merging... (The progress bar finishing means it's done)
✅ Confirmed X weight files exist.
🔧 Handling tokenizer files...
   Downloading tokenizer from Hugging Face...
   ✅ Tokenizer downloaded and saved
🔧 Rebuilding config.json...
✅ Fix completed: vocab_size 32001 -> 32000
✨ All rescue operations successful!

Output Directory: ./merged_model/

Notes:

• First-time merging will download models (~14GB each) and may take 30-60 minutes
• Subsequent runs will use cached models and complete in 10-30 minutes
• Models are cached in ~/.cache/huggingface/hub/ by default

---

Step 3: Repair Merged Model

After merging, run the repair script to align tokenizer and configuration files.

python3 repair_model.py

What happens during repair:

1. Downloads and replaces tokenizer files from WizardMath-7B (32k vocabulary)
2. Patches config.json to ensure vocab_size = 32000
3. Verifies model architecture type is set to "mistral"

Expected Output:

🔧 Starting repair for: ./merged_model
⬇️ Downloading standard Mistral/WizardMath tokenizer...
✅ Tokenizer files replaced.
📝 Patching config.json...
✅ Config fixed: vocab_size 32001 -> 32000
✨ Repair complete.

Why is this step necessary?

• Ensures tokenizer vocabulary matches the merged model weights
• Prevents token embedding size mismatches during inference
• Aligns configuration for compatibility with downstream tasks

Note: This step requires internet connection to download tokenizer files from Hugging Face Hub.

---

Step 4: Evaluate Merged Model

Evaluate the merged model on all three benchmarks.

# GSM8K benchmark
python3 benchmark_Merged_GSM8K.py

# DRCD benchmark
python3 benchmark_Merged_DRCD.py

# Math23k benchmark
python3 benchmark_Merged_Math23k.py

Expected Output:

• Each script generates a JSON results file (e.g., GSM8K_results_Merged.json)
• Progress bars show evaluation progress
• Final accuracy metrics are printed to console

Note: Make sure the MODEL_PATH in each benchmark script matches your merged model directory.

Important: Only run these benchmarks after completing Step 3 (model repair).

---

Datasets

1. GSM8K (Grade School Math 8K)

• Format: CSV file (GSM8K_dataset.csv)
• Columns: question, answer
• Language: English
• Task: Mathematical word problems
• Evaluation Metric: Exact numerical match

2. DRCD (Delta Reading Comprehension Dataset)

• Format: JSON file (DRCD_test.json)
• Language: Traditional Chinese
• Task: Reading comprehension
• Evaluation Metric: String matching (with normalization)

3. Math23k

• Format: JSON file (math_tw.json)
• Language: Traditional Chinese
• Task: Elementary math word problems
• Evaluation Metric: Numerical match with tolerance (0.01)

---

Results

Results are saved as JSON files in the project root directory:

Baseline Results

• GSM8K_results_Breeze.json
• GSM8K_results_WizardMath.json
• DRCD_results_Breeze.json
• DRCD_results_WizardMath.json
• Math23k_results_Breeze.json
• Math23k_results_WizardMath.json

Merged Model Results

• GSM8K_results_Merged.json
• DRCD_results_Merged.json
• Math23k_results_Merged.json

Each results file contains:

• Individual question-answer pairs
• Model responses
• Extracted answers
• Correctness flags (is_correct: true/false)
• Overall accuracy metrics

Re-evaluating Results with check_answer.py

The automated correctness detection in the benchmark scripts may have some inaccuracies, particularly with percentage handling and numerical format variations. Use check_answer.py to re-evaluate results with improved answer matching logic.

What check_answer.py fixes:

• Percentage format mismatches (e.g., gold answer "20%" but model outputs "20")
• Trailing periods in numbers (e.g., "12." treated as "12")
• Fraction format variations
• Better numerical precision handling

Usage:

1. Edit check_answer.py and modify line 206 to specify your results file:

   INPUT_FILE = "Math23k_results_Merged.json"  # Change to your target file
   

2. Optionally change the output file name on line 209:

   OUTPUT_FILE = "Math23k_results_Merged_percentage_fix.json"
   

3. Run the script:

   python3 check_answer.py
   

Output:

• Console will show updated accuracy and list all cases where the evaluation changed
• *_percentage_fix.json: Updated results file with corrected evaluations
• *_percentage_fix_changes.json: Detailed log of all evaluation changes

Recommendation: Run this on all Math23k results, as they frequently have percentage-related issues.

---

Troubleshooting

Issue: CUDA Out of Memory

Solution:

• Reduce TEST_LIMIT in the evaluation scripts (e.g., set to 50 instead of 100)
• Close other GPU-intensive applications
• The scripts already use 4-bit quantization to reduce memory usage

Issue: Model Download Failures

Solution:

• Check your internet connection
• Verify Hugging Face Hub is accessible
• If behind a firewall/proxy, configure environment variables:

  export HTTP_PROXY="http://your-proxy:port"
  export HTTPS_PROXY="https://your-proxy:port"
  

• Alternatively, manually download models and modify paths in run_merge_SLERP.py to use local directories

Issue: Merged Model Not Found

Solution:

• Verify that run_merge_SLERP.py completed successfully
• Check the output directory name (default: ./merged_model/)
• Update MODEL_PATH in benchmark scripts if needed

Issue: Missing Dataset Files

Solution:

• Ensure GSM8K_dataset.csv, DRCD_test.json, and math_tw.json are in the project root
• Check file names match those specified in the scripts
• Verify files are not corrupted (valid CSV/JSON format)

Issue: Import Errors

Solution:

# Reinstall dependencies
pip install --upgrade torch transformers mergekit pyyaml tqdm bitsandbytes pillow accelerate

# Verify CUDA availability
python3 -c "import torch; print('CUDA available:', torch.cuda.is_available())"

Issue: Permission Errors on Linux

Solution:

# Make scripts executable
chmod +x baseline_*.py benchmark_*.py run_merge_SLERP.py

---

Notes

• All scripts save intermediate results every 10 questions to prevent data loss
• Progress is shown via tqdm progress bars
• Models use 4-bit quantization to reduce memory requirements
• Temperature is set low (0.01-0.1) for deterministic mathematical reasoning
• The merged model inherits the tokenizer from Breeze-7B

---

Contact

For questions or issues related to this project, please contact the project maintainer.