Model Introduction

We are excited to announce the release of a lightweight version from the Index series models: the Index-1.9B series. The open-source Index-1.9B series includes the following models:

• Index-1.9B base: The base model, with 1.9 billion non-embedding parameters, pre-trained on a 2.8T corpus mainly in Chinese and English. It leads in multiple evaluation benchmarks compared to models of the same level.
• Index-1.9B pure (this repository's model) : A control version of the base model with the same parameters and training strategy, but strictly filtered out all instruction-related data from the corpus to verify the impact of instructions on benchmarks.
• Index-1.9B chat: A dialogue model aligned with SFT and DPO based on the Index-1.9B base. We found that due to the introduction of a lot of internet community corpus in our pre-training, the model has significantly more interesting chatting capabilities.
• Index-1.9B character : Introduces RAG on top of SFT and DPO to achieve few-shots role-playing customization.

Note: This is the Base model, capable only of continuation and further training alignment, and cannot be directly interacted with.

• For the Chat model, see Index-1.9B-Chat
• For the Role-playing model, see Index-1.9B-Character

For more details, see our GitHub and Index-1.9B Technical Report

Evaluation Results

The Index-1.9B shows excellent performance in general understanding evaluations, leading compared to recently open-sourced small models and comparable to some 7B and models larger than 10B.

Evaluation code is based on OpenCompass with compatibility modifications. See the evaluate folder for details.