Run AI Locally
Self-hosted LLM frontends, inference engines, and open-weight model families for on-device AI.
Run AI Locally.
No sign-ups, no data collection, no internet required. Frontends, engines, and open-weight models you can run entirely on your own hardware.
Cloud-based AI providers frequently use your prompts for data collection or model training. Running LLMs locally keeps everything — your prompts, documents, and outputs — entirely on your own machine, with no account, no logging, and no internet connection required once set up.
These are the apps you actually chat through — some are point-and-click desktop apps, others are browser-based UIs pointed at a local backend, and a few are terminal-only for people who want zero overhead.
LM Studio
The most beginner-friendly way to download and run local LLMs, with a built-in model browser and OpenAI-compatible local server.
Ollama
The de facto standard for running open-weight models locally — one command to pull and run a model, plus a REST API that most local-AI tools build on top of.
These are the backends that actually load and run the model weights — most frontends above are just a UI layered on top of one of these.
- ModelFit — checks your device against a live model database, ranking which LLMs and coding agents actually run well on your specific GPU or Mac, with dedicated cloud vs. local tool comparisons
- Can I Run This LLM? — enter your GPU VRAM and system RAM to see which open-weight models actually fit, plus estimated tokens-per-second
- VRAM Calculator (ApX Machine Learning) — detailed inference and fine-tuning VRAM estimator covering quantization, KV cache, batch size, and multi-GPU offloading
- As a rule of thumb, a 7B model in 4-bit quantization needs roughly 4 to 6GB of VRAM or unified memory, while a 70B model needs 40GB or more unless heavily quantized
Every family below is open-weight and runs through Ollama, llama.cpp, or LM Studio — pick based on your available RAM and what you're using it for, not just parameter count.
Meta Llama
The most widely adopted open-weight family with the largest community ecosystem, spanning 1B to 405B parameters with strong general reasoning.
Alibaba Qwen
27 models spanning 0.5B to 235B parameters — the widest size range of any family, with strong multilingual and coding performance.
DeepSeek
DeepSeek R1 introduced chain-of-thought reasoning that rivals proprietary frontier models, while V3 is a massive but efficient MoE model.
Mistral
Known for the best performance-per-parameter ratio in the open-weight space, using sliding window attention for efficiency at smaller sizes.
- SmolLM2 360M — the smallest model in most local databases, loads in about 0.5GB and runs on just 1GB of RAM, including old Macs, iPhones, and embedded boards
- LFM2 24B-A2B — a sparse mixture-of-experts model that only activates 2B of its 24B parameters per token, fitting in about 14GB and running well on any 16GB Mac
- Phi-4 Mini 3.8B — loads in 3.2GB and needs just 7GB minimum RAM, making it viable on most modern laptops
If you want an agentic coding assistant without sending code to the cloud, several tools below pair specifically with local models through Ollama rather than requiring a subscription API key.
Coding agents can execute terminal commands and modify files without additional confirmation if misconfigured. Run them in a container, VPS, or VM rather than on a machine with anything important on it, and avoid giving any agent full drive-root access.
For troubleshooting a specific setup or getting a second opinion on hardware before buying, these communities move faster than most written guides.