// Inference & local runtime

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Huawei Ascend

Official · vLLM 项目Apache-2.0Active

01Recent releases

vllm-project/vllm-ascend v0.30.0rc1 Patchpre

Community maintained hardware plugin for vLLM on Huawei Ascend
vllm-project/vllm-ascend v0.30.0rc2 Minorpre

Community maintained hardware plugin for vLLM on Huawei Ascend
vllm-project/vllm-ascend v0.28.0.rc1 Minorpre

Community maintained hardware plugin for vLLM on Huawei Ascend
vllm-project/vllm-ascend v0.27.1rc1 Patchpre

Community maintained hardware plugin for vLLM on Huawei Ascend
vllm-project/vllm-ascend v0.29.0.rc1 Minorpre

Community maintained hardware plugin for vLLM on Huawei Ascend

ggml-org/llama.cpp LLM inference in C/C++ Inference & local runtime · official★ 132k

vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs Inference & local runtime · official★ 94.9k

mudler/LocalAI LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. Inference & local runtime★ 49.8k

exo-explore/exo Run frontier AI locally. Inference & local runtime★ 48.0k

ray-project/ray Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. Inference & local runtime★ 44.1k