// Software stack
Run LLMs on your own computer
From one-click installs to command-line tuning: download open-weight models, run them locally and expose an OpenAI-compatible API to other tools.
5 pieces5 setup stepsscenario: Run models on your own machine
What goes in it
- 1 ★ 184k
Ollama Local inference & serving Open source
Easiest local model runner and API
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
- 2
LM Studio Local inference & serving Closed / commercial
GUI to download and try models
Desktop app to download and run local LLMs with an OpenAI-compatible server.
- 3 ★ 132k
llama.cpp Local inference & serving Open source
Underlying inference engine for fine-grained tuning
LLM inference in C/C++
- 4 ★ 45k
jan optional AI apps & chat clients Open source
Open-source offline chat client
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
- 5 ★ 156k
open-webui optional AI apps & chat clients Open source
Browser chat UI for many models
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
How to set it up
- 01Check what your VRAM / RAM can run with our VRAM calculator
- 02Beginners install LM Studio (GUI) or Ollama (one command:
ollama run <model>) - 03Add Open WebUI for a web interface or multiple users
- 04Connect other tools through the OpenAI-compatible API (Ollama listens on port 11434 by default)
- 05Switch to llama.cpp when you need fine control over quantisation, context and GPU layers
✓ Good fit
- Privacy, offline use, unlimited free use
✗ Look elsewhere if
- You need frontier capability: local 8–32B models are clearly weaker than top cloud models
Similar stacks: what’s different
- Local private LLMThat one lists trending, partly experimental GitHub projects; beginners should start with this mainstream set