• MindNLP
• • • News 📢
    • Installation
    • • Install from Pypi
      • Daily build
      • Install from source
      • Version Compatibility
    • Introduction
    • • Major Features
    • Quick Links
    • Supported models
    • License
    • Feedbacks and Contact
    • Acknowledgement
    • Citation

MindNLP

Installation |
Introduction |
Quick Links |

News 📢

• 🔥 Latest Features

  • 🤗 Hugging huggingface ecosystem, we use datasets lib as default dataset loader to support
    mounts of useful datasets.
  • 📝 MindNLP supports NLP tasks such as language model, machine translation, question answering, sentiment analysis, sequence labeling, summarization, etc. You can access them through examples.
  • 🚀 MindNLP currently supports industry-leading Large Language Models (LLMs), including Llama, GLM, RWKV, etc. For support related to large language models, including pre-training, fine-tuning, and inference demo examples, you can find them in the "llm" directory.
  • 🤗 Pretrained models support huggingface transformers-like apis, including 60+ models like BERT, Roberta, GPT2, T5, etc.
    You can use them easily by following code snippet:

    from mindnlp.transformers import AutoModel
    
    model = AutoModel.from_pretrained('bert-base-cased')
    

Installation

Install from Pypi

You can install the official version of MindNLP which uploaded to pypi.

pip install mindnlp

Daily build

You can download MindNLP daily wheel from here.

Install from source

To install MindNLP from source, please run:

pip install git+https://github.com/mindspore-lab/mindnlp.git
# or
git clone https://github.com/mindspore-lab/mindnlp.git
cd mindnlp
bash scripts/build_and_reinstall.sh

Version Compatibility

MindNLP version	MindSpore version	Supported Python version
master	daily build	>=3.7.5, <=3.9
0.1.1	>=1.8.1, <=2.0.0	>=3.7.5, <=3.9
0.2.x	>=2.1.0	>=3.8, <=3.9

Introduction

MindNLP is an open source NLP library based on MindSpore. It supports a platform for solving natural language processing tasks, containing many common approaches in NLP. It can help researchers and developers to construct and train models more conveniently and rapidly.

The master branch works with MindSpore master.

Major Features

• Comprehensive data processing: Several classical NLP datasets are packaged into friendly module for easy use, such as Multi30k, SQuAD, CoNLL, etc.
• Friendly NLP model toolset: MindNLP provides various configurable components. It is friendly to customize models using MindNLP.
• Easy-to-use engine: MindNLP simplified complicated training process in MindSpore. It supports Trainer and Evaluator interfaces to train and evaluate models easily.

Quick Links

• Documentation
• Tutorials
• Examples
• LLMs
• ...

Supported models

The table below represents the current support in the library for each of those models, whether they have support in Pynative mode or Graph mode.

Model	Pynative support	Graph Support
ALBERT	✅	✅
ALIGN	✅	❌
AltCLIP	✅	❌
Audio Spectrogram Transformer	✅	❌
Autoformer	✅ (Inference only)	❌
BaiChuan	✅	❌
Bark	✅	❌
BART	✅	❌
BERT	✅	✅
BLIP	TODO	✅
BLIP2	TODO	✅
BLOOM	✅	❌
ChatGLM	✅	❌
ChatGLM2	✅	❌
ChatGLM3	✅	❌
CLIP	✅	❌
CodeGen	✅	❌
ConvBERT	TODO	❌
CPM	✅	❌
CPM-Ant	✅	❌
CPM-Bee	✅	❌
MiniCPM	✅	❌
Deberta	✅	❌
Electra	TODO	❌
EnCodec	✅	❌
ERNIE	✅	✅
ERNIEM	✅	✅
ESM	✅	✅
Falcon	✅	❌
OpenAI GPT	✅	❌
OpenAI GPT-2	✅	✅
Gemma	✅	❌
GPT Neo	✅	❌
GPT NeoX	TODO	❌
GPT Pangu	✅	❌
GPTBigCode	✅	❌
Graphormer	✅	❌
Hubert	✅	❌
Llama	✅	❌
Llama2	✅	❌
CodeLlama	✅	❌
Longformer	✅	❌
LongT5	✅	❌
LUKE	✅	❌
MaskFormer	✅	❌
Mamba	✅	❌
mBART-50	✅	❌
Megatron-BERT	✅	❌
Megatron-GPT2	✅	❌
Mistral	✅	❌
Mixtral	✅	❌
MobileBERT	✅	❌
Moss	✅	❌
Nezha	✅	❌
OPT	✅	❌
Phi2	✅	❌
Pop2piano	✅	❌
Qwen2	✅	❌
Reformer	✅	❌
RegNet	Todo	❌
RoBERTa	✅	✅
RWKV	✅	❌
SeamlessM4T	✅	❌
SeamlessM4Tv2	✅	❌
StarCoder	✅	❌
T5	✅	❌
Timesformer	TODO	❌
Tinybert	✅	❌
wav2vec	✅	❌
Whisper	✅	❌
XLM	✅	❌
XLM-RoBERTa	✅	❌

License

This project is released under the Apache 2.0 license.

Feedbacks and Contact

The dynamic version is still under development, if you find any issue or have an idea on new features, please don't hesitate to contact us via Github Issues.

Acknowledgement

MindSpore is an open source project that welcome any contribution and feedback.
We wish that the toolbox and benchmark could serve the growing research
community by providing a flexible as well as standardized toolkit to reimplement existing methods
and develop their own new semantic segmentation methods.

Citation

If you find this project useful in your research, please consider citing:

@misc{mindnlp2022,
    title={{MindNLP}: a MindSpore NLP library},
    author={MindNLP Contributors},
    howpublished = {\url{https://github.com/mindlab-ai/mindnlp}},
    year={2022}
}