1. Register the dataset

You first need to place your dataset file in ./prepare/data folder in .json format, and register it in ./prepare/data/dataset_info.json:

<dataset_name>:
{
  "file_name": "<dataset_name>.json",
  "columns": {"prompt":"prompt",
  "response":"response",
  "system":"system"},

Note that the format of your json file should be like:

[{ "prompt": "",
  "response": "",
  "system": ""},
  ...,
  ...
  ...,
  { "prompt": "",
  "response": "",
  "system": ""}]

please refer to LLaMA-Factory for more details.

2. Collect checkpoints for this dataset. You need to train LLMs on previous datasets to collect relevent checkpoints, forming data-checkpoint pairs for DnD's training.

We give an example of how to modify the training script:

### model
-model_name_or_path: ../models/Qwen2.5-0.5B-Instruct
+model_name_or_path: ../models/<your desired base model>


################# line9-21 of training_your_dataset.yaml #################

-lora_rank: 8
+lora_rank: <expected rank>
lora_target: all

### dataset
-dataset: ARC-c
+dataset: <dataset_name> # should be consistent with your json file name
template: qwen
cutoff_len: 2048
-max_samples: 5000
+max_samples: <expected sample>
overwrite_cache: true
preprocessing_num_workers: 16

### output
-output_dir: saves/common_sense_reasoning/ARC-c
+output_dir: saves/<task_name>/<dataset_name>

################# line9-21 of training_your_dataset.yaml #################



################# line28-33 of training_your_dataset.yaml #################
-per_device_train_batch_size: 1
-gradient_accumulation_steps: 8
-learning_rate: 1.0e-4
-num_train_epochs: 1.0
-lr_scheduler_type: cosine
-warmup_ratio: 0.01
#you can modify the training settings
+per_device_train_batch_size:
+gradient_accumulation_steps:
+learning_rate:
+num_train_epochs:
+lr_scheduler_type:
+warmup_ratio:

################# line28-33 of training_your_dataset.yaml #################

• After training, you need to do the following to get checkpoint collections.
  1. You need to observe the loss curve, and decide the starting point of fine-tuning for checkpoint collection.
  2. The trainer_state.json in the checkpoint folder (usually named checkpoint-xxx) needs to be modified, setting "save_steps"=1.
  3. You can follow the scripts in ./prepare/training_scripts folder that end with "finetune" to design your fine-tuning process.
  4. After running the scripts and obtaining multiple checkpoints, you can enter the saving directory, run ./workspace/datasets/process_datasets/post_process_ckpts.sh to clean your checkpoint folder, deleting config files and rename checkpoints to ease the process of data loading.

3. Calculate importance scores for the collected checkpoints.

DnD utilizes a weighted MSE for training, assigning different importance to different layers' weights. The specific importance is calculated by the channel-wise variance and we provide scripts in ./workspace/datasets, like : criterion_weight_for_<model_type>.py. You need to select a script and adjust it accordingly.

######################## on line 26-28 in ...<dataset_name>.py ########################
-DATASET_ROOT = "./data/common_sense_reasoning"
-CONFIG_ROOT = f"./workspace/datasets/common_sense_reasoning"
+DATASET_ROOT = "./data/<task_name>"
+CONFIG_ROOT = f"./workspace/datasets/<task_name>"
######################## on line 26-28 in ...<dataset_name>.py ########################



###################### on line 24 in ...<dataset_name>.py #######################

-dataset_tag = "ARC-c"
+dataset_tag = <your dataset_tag>

###################### on line 24 in ...<dataset_name>.py #######################




###################### on line 37 in ...<dataset_name>.py #######################

-datasets = ["ARC-e","OBQA","BoolQ","WinoGrande","PIQA","HellaSwag"]
+datasets = ["<dataset_name_1>","<dataset_name_2>",...,"<dataset_name_n>"]
# All datasets you collect for the target task

###################### on line 37 in ...<dataset_name>.py #######################

4. Create your training script. (<dataset_name> is decided by yourself. And we strongly recommend keeping this name in data registration, checkpoint collection, and DnD training consistent, since it can save much trouble.)

We use ./workspace/main/tasks/common_sense_reasoning/train_qwen0.5lora_ARC-c.py to give an example. You need to create your training script like ./workspace/main/tasks/<task_name>/train_<model_type>_<dataset_name>.py:


```diff

######################## on line 26-28 in ...<dataset_name>.py ########################
-DATASET_ROOT = "./data/common_sense_reasoning"
-CONFIG_ROOT = f"./workspace/datasets/common_sense_reasoning"
+DATASET_ROOT = "./data/<task_name>"
+CONFIG_ROOT = f"./workspace/datasets/<task_name>"
######################## on line 26-28 in ...<dataset_name>.py ########################



###################### on line 24 in ...<dataset_name>.py #######################

-dataset_tag = "ARC-c"
+dataset_tag = <your dataset_tag>

###################### on line 24 in ...<dataset_name>.py #######################




###################### on line 37 in ...<dataset_name>.py #######################

-datasets = ["ARC-e","OBQA","BoolQ","WinoGrande","PIQA","HellaSwag"]
+datasets = ["<dataset_name_1>","<dataset_name_2>",...,"<dataset_name_n>"]
# All datasets you collect for the target task

###################### on line 37 in ...<dataset_name>.py #######################


###################### on line 47 in ...<dataset_name>.py #######################

-max_text_length = xxx
+max_text_length = <The max prompt length in your dataset>

###################### on line 47 in ...<dataset_name>.py #######################



###################### on line 42-90 in ...<dataset_name>.py #######################
  config: dict[str, [float, int, str, dict]] = {
    # global setting
    "seed": SEED,
    "model_tag": os.path.basename(__file__)[:-3].split("_")[1],
    "need_test": False,
    "use_wandb": True,
    # data setting
-    "token_size": (10, 130),
+    "token_size": <suitable token size>
-    "real_length": 50,
+    "real_length": <number of checkpoints you like to use>
    "train_checkpoint_folders": [f"{DATASET_ROOT}/{dataset}" for dataset in datasets],
    "test_checkpoint_folder": "",
    "dataset_tag": dataset_tag,
    "generated_file": f"{CONFIG_ROOT}/{dataset_tag}/",
    # train setting
    "max_num_gpus": 8,
-    "batch_size": 64,
+    "batch_size": <suitable batch_size>
-    "num_workers": 8,
+    "num_workers": <suitable num_workers>
    "prefetch_factor": 1,
    "warmup_steps": 1,
-    "total_steps": 4000,
-    "learning_rate": 3e-5,
+    "total_steps": <your preferred training setting>
+    "learning_rate":
    "weight_decay": 0.1,
    "max_grad_norm": 1.0,
    "save_every": 100,
    "print_every": 20,
-    "num_texts": 128,
+    "num_texts": <suitable length of prompt batch>
    "save_folder": "./checkpoints",
    "noise_enhance": 0.0001,
    "criterion_weight": calculate_mean_criterion_weight([f"{CONFIG_ROOT}/{dataset}/criterion_weight.pt" for dataset in datasets]),
    "extractor_type":"BERT",
    "text_tokenizer":AutoTokenizer.from_pretrained(extractor),
    "extra_condition_module":
        AutoModel.from_pretrained(extractor,
        torch_dtype="auto").to(accelerator.device),
    "max_text_length":max_text_length,

-    "model_config": {
-        "features": [
-            (128, max_text_length, 384), (128, 200, 300),
-            (128, 100, 256), (256, 50, 200),
-            (512, 50, 200),
-            (1024, 25, 200), (1024, 10, 200), (2048, 10, 200),
-            (4296, 10, 130),
-        ],
-        "condition_dim": (128, max_text_length, 384),
-        "kernel_size": 9,
-    },
+     <your desired model size (the features actually denotes the shape transition of input embeddings, pretty convenient isn't it?)>
}
###################### on line 42-90 in ...<dataset_name>.py #######################

5. Train DnD model.

cd ./workspace/main
bash launch_multi.sh tasks/<task_name>/train_<model_type>_<dataset_name>.py <number_of_gpus>

Note that the adjustment of generation scripts is similar.