MergingTools is a Python-based automation tool designed to streamline the process of merging pre-trained language models. Leveraging the powerful capabilities of mergekit, MergingTools facilitates elaborate merges of models such as Llama, Mistral, GPT-NeoX, StableLM, and more, under resource-constrained situations. The tool supports both CPU and GPU execution, offering an out-of-core approach to perform merges efficiently with minimal resource requirements.
Ensure you have Python 3.6+ and pip installed on your system. Clone the repository and set up the environment by running:
git clone https://github.com/wutzcoding/MergingTools.git
cd AutoMerge
python -m pip install --upgrade pip
python setup_environment.py
To start a merge process, create a YAML configuration file specifying your merge parameters and run:
python automerge.py path/to/your/config.yml
For detailed information on creating a YAML configuration for your merge, see Merge Configuration.
AutoMerge uses YAML documents for specifying merge operations. Key elements include:
merge_method: Method used for merging (e.g., linear, slerp).models: Defines models for merging.base_model: Specifies the base model used in some merge methods.parameters: Holds various parameters such as weights and densities.For an example configuration and more details, refer to the examples/ directory.
After merging, you can share your model on the Hugging Face Hub. AutoMerge generates a basic README.md for your model card. Customize it as needed, and upload your model using the huggingface_hub library:
huggingface-cli login
huggingface-cli upload your_hf_username/my-cool-model ./output-model-directory
Contributions to AutoMerge are welcome! Please read our Contributing Guide for details on how to submit pull requests, report issues, and suggest improvements.
AutoMerge is open-source software licensed under the Apache2.0.
AutoMerge was built using mergekit, developed by Charles Goddard and supported by arcee.ai. Special thanks to the contributors of mergekit and the AI research community for their invaluable work.
MergingTools is a Python-based automation tool designed to streamline the process of merging pre-trained language models. Leveraging the powerful capabilities of mergekit, MergingTools facilitates elaborate merges of models such as Llama, Mistral, GPT-NeoX, StableLM, and more, under resource-constrained situations. The tool supports both CPU and GPU execution, offering an out-of-core approach to perform merges efficiently with minimal resource requirements.
Ensure you have Python 3.6+ and pip installed on your system. Clone the repository and set up the environment by running:
git clone https://github.com/wutzcoding/MergingTools.git
cd AutoMerge
python -m pip install --upgrade pip
python setup_environment.py
To start a merge process, create a YAML configuration file specifying your merge parameters and run:
python automerge.py path/to/your/config.yml
For detailed information on creating a YAML configuration for your merge, see Merge Configuration.
AutoMerge uses YAML documents for specifying merge operations. Key elements include:
merge_method: Method used for merging (e.g., linear, slerp).models: Defines models for merging.base_model: Specifies the base model used in some merge methods.parameters: Holds various parameters such as weights and densities.For an example configuration and more details, refer to the examples/ directory.
After merging, you can share your model on the Hugging Face Hub. AutoMerge generates a basic README.md for your model card. Customize it as needed, and upload your model using the huggingface_hub library:
huggingface-cli login
huggingface-cli upload your_hf_username/my-cool-model ./output-model-directory
Contributions to AutoMerge are welcome! Please read our Contributing Guide for details on how to submit pull requests, report issues, and suggest improvements.
AutoMerge is open-source software licensed under the Apache2.0.
AutoMerge was built using mergekit, developed by Charles Goddard and supported by arcee.ai. Special thanks to the contributors of mergekit and the AI research community for their invaluable work.