// Inference & local runtime

google-ai-edge/LiteRT-LM

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.

Official · Google AI EdgeApache-2.0Active

01Recent releases

google-ai-edge/LiteRT-LM 0.19.0-nightly.20261020 Minorpre

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
google-ai-edge/LiteRT-LM v0.18.1 Patch

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
google-ai-edge/LiteRT-LM 0.19.0-nightly.20261019 Patchpre

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
google-ai-edge/LiteRT-LM 0.19.0-nightly.20261018 Patchpre

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
google-ai-edge/LiteRT-LM 0.19.0-nightly.20261017 Patchpre

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.

ggml-org/llama.cpp LLM inference in C/C++ Inference & local runtime · official★ 132k

vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs Inference & local runtime · official★ 94.9k

mudler/LocalAI LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. Inference & local runtime★ 49.8k

exo-explore/exo Run frontier AI locally. Inference & local runtime★ 48.0k

ray-project/ray Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. Inference & local runtime★ 44.1k