PROJECT NOT UNDER ACTIVE MANAGEMENT

This project will no longer be maintained by Intel. Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project. Intel no longer accepts patches to this project. If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.

Intel® Gaudi® Tutorials

These are the source files for the tutorials on the Developer Website

The tutorials provide step-by-step instructions for PyTorch and PyTorch Lightning on the Intel Gaudi AI Processor, from beginner level to advanced users. These tutorials should be run with a full Intel Gaudi Node of 8 cards.

IMPORTANT: To run these Jupyter Notebooks you will need to follow these steps:

1. Get access to an Intel Gaudi 2 Accelerator card or node. See the Get Access page on the Developer Website. Be sure to use port forwarding ssh -L 8888:localhost:8888 -L 7860:localhost:7860 -L 6006:localhost:6006 ... user@ipaddress to be able to access the notebook, run the Gradio interface, and use Tensorboard. Some of the tutorials use all of these features.
2. Run the Intel Gaudi PyTorch Docker image. Refer to the Docker section of the Installation Guide for more information. Running the docker image will allow you access to the entire software stack without having to worry about detailed Software installation Steps.

docker run -itd --name Gaudi_Docker --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --net=host --ipc=host vault.habana.ai/gaudi-docker/1.21.0/ubuntu22.04/habanalabs/pytorch-installer-2.6.0:1.21.0-555
docker exec -it Gaudi_Docker bash

3. Clone this tutorial in your $HOME directory: cd ~ && git clone https://www.github.com/habanaAI/Gaudi-tutorials
4. Install Jupyterlab: python3 -m pip install jupyterlab
5. Run the Jupyterlab Server, using the same port mapping as the ssh command: python3 -m jupyterlab_server --IdentityProvider.token='' --ServerApp.password='' --allow-root --port 8888 --ServerApp.root_dir=$HOME & and take the local URL and run that in your browser

The tutorials will cover the following domains and tasks:

Advanced

• Fine Tuning with LORA and Inference on Hugging Face Llama 2 70B model
• Full RAG application with TGI-gaudi
• Getting Started with vLLM
• Understanding vLLM on Gaudi
• Benchmarking on vLLM
• Getting Started with TGI-Gaudi
• RAG application with LocalGPT modified to run on Intel Gaudi
• How to setup and use DeepSpeed for Training Large Language Models

Intermediate

• GPU migration Tool
• Debug for Dynamic Shapes
• Running Simple Inference examples with HPU Graph
• Using Hugging Face Pipelines for Inference
• How to use the Gaudi Tensorboard Plug-in or Perfetto for Profiling
• Transformer Reinforcement Learning with Hugging Face
• Running DeepSpeed on PyTorch Lightning with GPT2
• BERT Fine Tuning using PyTorch Lightning

Getting Started

• Quick Start: Overall review of Model-References, Hugging Face and DeepSpeed
• Inference
• Fine Tuning
• Training a Classifier Basic PyTorch Tutorial
• Introduction to Lightning, Running a simple PyTorch Lightning model, using Gaudi plug-in

DISCLAIMER

Intel is committed to the respect of human rights and avoiding complicity in human rights abuses, a policy reflected in the Intel Global Human Rights Principles. Accordingly, by accessing the Intel material on this platform you agree that you will not use the material in a product or application that causes or contributes to a violation of an internationally recognized human right.

License

Intel® Gaudi® Tutorials is licensed under Apache License Version 2.0.

Datasets and Models

To the extent that any data, datasets, or models are referenced by Intel or accessed using tools or code on this site such data, datasets and models are provided by the third party indicated as the source of such content. Intel does not create the data, datasets, or models, provide a license to any third-party data, datasets, or models referenced, and does not warrant their accuracy or quality. By accessing such data, dataset(s) or model(s) you agree to the terms associated with that content and that your use complies with the applicable license.

Intel expressly disclaims the accuracy, adequacy, or completeness of any data, datasets or models, and is not liable for any errors, omissions, or defects in such content, or for any reliance thereon. Intel also expressly disclaims any warranty of non-infringement with respect to such data, dataset(s), or model(s). Intel is not liable for any liability or damages relating to your use of such data, datasets, or models.