nlp-tutorial

News

• [2024/3] 🔥 nlp-bazel-tutorial provides support for T5 and large language models (LLMs) across both training and inference workflows
• [2023/12] 🔥 Building Model Training Code and Bazel-Based base/thirdparty Modules with ONNX for NLP Model Inference。

About

nlp-bazel-tutorial is a project encompassing NLP model training and C++-based engineering practices for inference, covering the following components:

• **Model Training:

  Named Entity Recognition (NER): SpanBERT、MRC-BERT、T5、LLM

  Text Error Correction: MacBERT、T5、LLM

  Text Classification: BERT Family of Models、LLM

• **C++-Based Engineering Practices:

  Build system managed with Bazel for cross-platform compilation CPU-optimized inference for traditional models and large language models (LLMs)

  Using ONNX、llama.cpp for model deployment

  Unified inference capabilities supporting: ONNX-compatible models llama.cpp-enabled LLMs

一、 Structures

.
├── base
├── chinese_error_correction
│   ├── macbert2csc
│   │   ├── macbert2onnx
│   │   └── onnx-cpp
│   │       └── model
│   ├── pycorrector
│   ├── qwen2csc
│   └── t52csc
│       ├── onnx-cpp
│       │   └── model
│       └── t52onnx
├── name_entity_recognition
│   ├── chatgpt2-ner
│   ├── mrc-ner
│   │   ├── mrc-for-flat-nested-ner
│   │   └── onnx-cpp
│   │       └── model
│   └── span-ner
│       ├── onnx-cpp
│       │   └── model
│       └── span-bert-ner-pytorch
├── text_classification
│   ├── bert-finetune
│   └── onnx-cpp
└── third_party

一、base

Base is pulled into many projects. For example, various ChromeOS daemons. So the bar for adding stuff is that it must have demonstrated wide applicability. Prefer to add things closer to where they're used (i.e. "not base"), and pull into base only when needed. In a project our size, sometimes even duplication is OK and inevitable.

二、 name entity recognition

Named entity recognition includes span ner and mrc ner.

1、span ner is reference paper of SpanNER: Named EntityRe-/Recognition as Span Prediction paper, the code is reference of [https://github.com/lonePatient/BERT-NER-Pytorch], On the basis of this codes, I add the codes for converting to onnxruntime and deployment in C++.

CLUENER

The overall performance of BERT on dev:

2、Mrc ner is advances in Shannon.AI. for more details, please see A Unified MRC Framework for Named Entity Recognition In ACL 2020. paper , the code is in [https://github.com/ShannonAI/mrc-for-flat-nested-ner] , On the basis of this codes, I add the codes for converting to onnxruntime and deployment in C++.

msra_zh

三、text error correction

The text correction module includes text error correction based on the MacBERT pre-trained model, T5 model-based approach, and large language models (LLMs). On a CPU, the T5 model achieves title-level inference speeds of 100ms. Additionally, it integrates llama.cpp to support LLM inference.

四、 Text Classification

Using the pre trained models for text classification。

THUCNews