Playing-with-ROCm

Here to show my experience about playing with ROCm with runable code, step-by-step tutorial to help you reproduce what I have did. If you have iGPU or dGPU of AMD, you may try Machine Learning with them.

NOTICE : For more easier tracking my update, I use 🆕 and 🔥 to flag the new hot topics.

Topics

Pysical AI & Robotics

• Create Virtual Reachy MINI Conversaion Robot with Mujoco at local 🆕
• ROCm Robotics RL Lab - 🆕🔥, I create this new repo with a sim & RL lab - lift cube with Panda Arm robot in Simulation.

Training

Finetuning

• LoRA with Radeon
• QLoRA with Radeon

Inference

• Deploy Deepseek-R1 in one GPU -AMD Instinct™ MI300X 🔥
• Deploy Llama 3.2 Vision quickly on AMD ROCm with Ollama
• Deploy vLLM service with Kubernetes over AMD ROCm GPU
• Deploy LLM with Radeon iGPU 780M
• Examples of using vLLM with ROCm 🆕 🔥
  • Help scripts to fast use vLLM with ROCm
  • Example: using docker compose to run multiple containers of vllm serve. e.g. gpu=0,1 for container-1 and gpu=6,7 for container-2.

MLOPS with ROCm

• vLLM
  • Neural Magic vLLM, nm-vllm
  • AIBrix
  • KubeAI : AI Inferencing Operator
  • vLLM Production Stack

Application/Demo

• RAG_LLM_QnA_Assistant, Step-by-step tutorial repo project to setup RAG Apps with ROCm
• Ask4ROCm_Chatbot, An chatbot app drive by RAG solution.
• LLM_Voice_Assistant , Use STT/TTS model from Picovoice.
• Easy-Wav2Lip-ROCm, Easy run Wav2Lip with ROCm over AMD GPU. Way2Lip is a project of Generalized Lip Sync Models
• Run EchoMimic with ROCm EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning
• Run LLama-3.2-vision with ROCm Ollama+Llama-3.2-vision+ROCm
• Deploy vLLM service with Kubernetes over AMD ROCm GPU , Turoial with sample codes.
• Play Qwen2.5–Omni with AMD GPU 🆕 🔥

---

Projects work over ROCm

These projects may not offical announce to support ROCm GPU. But they work fine base on my verification.

Wish List

Tracing

• Awesome-Multimodel-LLM
• Utiliy scripts of modal-labs/llm-serving

Misc

MCP

• write-my-first-mcp-server
• write-my-first-mcp-client

3rd-stuff

• Tutorial: vLLM deploy
• Summary: Awesome-Agent-Framework - https://github.com/AgentSpaceAI/Awesome-Agent-Framework - https://github.com/kyrolabs/awesome-agents

---

@misc{ Playing with ROCm,
  author = {He Ye (Alex)},
  title = {Playing with ROCm: share my experience and practice},
  howpublished = {\url{https://alexhegit.github.io/}},
  year = {2024--}
}