// Inference & local runtime

ROCm/FastFlowLM

Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.

Official · AMD ROCmNOASSERTIONActive

01Recent releases

ROCm/FastFlowLM v1.0.9 Patch

This is a maintenance release. It fixes **`flm validate` on Linux**, which wrongly reported "No NPU device found." in v1.0.8.
ROCm/FastFlowLM v1.0.8 Patch New model

This release adds **`minicpm-v-4.7:1b`**, a 1B vision-language model with toggleable thinking and tool calling. It also **opens up the source of the Gemma 4 engine**, so you can build it yourself and swap individual oper
ROCm/FastFlowLM v1.0.7 Patch New model

This release adds **`qwen3.8-mtp:27b`** — the largest dense model FastFlowLM has shipped to date, and the first to run **speculative decoding** on the NPU through its built-in MTP draft head. It also renames `hy-mt2:1.8b

ggml-org/llama.cpp LLM inference in C/C++ Inference & local runtime · official★ 132k

vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs Inference & local runtime · official★ 94.9k

mudler/LocalAI LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. Inference & local runtime★ 49.8k

exo-explore/exo Run frontier AI locally. Inference & local runtime★ 48.0k

ray-project/ray Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. Inference & local runtime★ 44.1k