Local AI Infrastructure
A practical ranking for teams choosing open-weight models to run locally. The score balances model quality, hardware cost, quantization maturity, and serving-stack support.
| Rank | Model | Score | Best For | Local Hardware | Generation Support | Stack |
|---|---|---|---|---|---|---|
| #1 | Qwen3-235B-A22B Qwen · 235B MoE / 22B active · Open weights | 94 Q 98 / E 74 | General reasoning, multilingual work, agent planning Best quality tier for teams willing to run a serious local inference box. | Workstation 4 to 8 GPUs, each with 80GB VRAM, for high-quality serving; smaller labs should use distilled or heavily quantized variants AWQ / GPTQ / GGUF when available | Image: Text-only LLM. Best used to write and refine prompts for FLUX, SDXL, or SD3.5. Video: Use it to produce shot prompts with subject, action, camera move, duration, and negative prompts, then run those prompts in Wan or LTX-Video. Audio: Good for dialogue, narration, and voice direction; use Piper, XTTS, or Whisper separately. | vLLM, SGLang, llama.cpp variants |
| #2 | DeepSeek-R1 DeepSeek · Large MoE reasoning model · Open weights | 92 Q 97 / E 68 | Reasoning, math, coding, long problem solving Excellent reasoning model, but full-size local serving is infrastructure-heavy. | Workstation Multi-GPU server; use distilled variants for smaller machines FP8 / AWQ / GGUF distilled variants | Image: Strong at reasoning through visual requirements and generating structured image prompts. Video: Use it to break a scene into timed shots, check continuity, and create Wan/LTX/HunyuanVideo prompt batches. Audio: Strong at script logic, dialogue rewrites, and TTS-ready narration outlines. | vLLM, SGLang, llama.cpp for distills |
| #3 | Llama 3.3 70B Instruct Meta Llama · 70B dense · Llama community license | 88 Q 91 / E 72 | General chat, tool use, enterprise assistants Strong default choice when ecosystem support matters. | Workstation 2x 48GB GPU or 1x 80GB GPU with quantization Apple Mac Studio M4 Maxfrom $1,999; configure up to high unified memory on AppleApple Mac Studio M3 Ultrafrom $3,999; better for very large local models 4-bit / 5-bit GGUF, AWQ | Image: Good for creative direction, prompt variants, UI copy, and asset naming. Video: Use it to create reusable video prompt templates, scene outlines, and tool-call plans for ComfyUI or Diffusers. Audio: Good for assistant voice scripts and call-and-response dialogue drafts. | llama.cpp, Ollama, vLLM |
| #4 | Qwen2.5-Coder-32B-Instruct Qwen Coder · 32B dense · Open weights | 87 Q 89 / E 84 | Coding, repo assistance, code review One of the most practical local coding models for a single strong GPU. | Prosumer GPU 1x 24GB GPU with 4-bit quantization; 48GB preferred Apple Mac Studio M4 Maxfrom $1,999; configure memory/storage on AppleApple MacBook Pro M4/M5 Maxconfigure high-memory Max models on Apple Q4_K_M / Q5_K_M GGUF, AWQ | Image: Best for asset-pipeline code, metadata, prompt JSON, and batch generation scripts. Video: Use it to write batch scripts, ComfyUI workflow JSON helpers, frame naming rules, and render pipeline automation. Audio: Best for audio pipeline code, subtitle files, TTS batch scripts, and dialogue tooling. | Ollama, LM Studio, llama.cpp, vLLM |
| #5 | Mistral Small 3.1 Mistral · 24B dense · Open weights | 85 Q 86 / E 86 | Fast assistant, RAG, multilingual business tasks Good balance of quality, speed, and practical deployment cost. | Prosumer GPU 1x 24GB GPU quantized; 48GB for higher throughput Apple Mac Studio M4 Maxfrom $1,999; configure memory/storage on AppleApple MacBook Pro M4/M5 Maxconfigure high-memory Max models on Apple GGUF / AWQ | Image: Good for multilingual prompt writing, product image briefs, and RAG-assisted style guides. Video: Use it to convert multilingual briefs into short-form shot prompts, captions, and localized video scripts. Audio: Good for multilingual narration drafts and ASR/TTS workflow instructions. | Ollama, llama.cpp, vLLM |
| #6 | Gemma 3 27B Google Gemma · 27B dense · Gemma terms | 83 Q 84 / E 85 | Instruction following, lightweight multimodal-adjacent workflows A practical high-end local model when Google model family compatibility matters. | Prosumer GPU 1x 24GB GPU with quantization Apple Mac Studio M4 Maxfrom $1,999; practical for 24B-32B quantized modelsApple MacBook Pro M4/M5 Maxconfigure high-memory Max models on Apple 4-bit GGUF / quantized safetensors | Image: Good for instruction-following prompts and multimodal-adjacent planning; use an image model for pixels. Video: Use it to write concise scene instructions and QA checklists before generating clips in Wan/LTX/HunyuanVideo. Audio: Good for clean TTS scripts, labels, and accessibility descriptions. | Ollama, llama.cpp, Transformers |
| #7 | Phi-4 Microsoft Phi · 14B dense · Open weights | 82 Q 78 / E 93 | Small-machine reasoning, education, offline assistants Good for compact local deployments where latency and RAM matter. | Laptop / Mini PC Apple Silicon, 12-16GB VRAM, or modern CPU with enough RAM Apple Mac mini M4 Profrom $1,399; configure unified memory on AppleApple MacBook Proconfigure memory based on target model size Q4 / Q5 GGUF | Image: Lightweight prompt assistant for simple icons, screenshots, and UI asset briefs. Video: Use it for short prompt outlines, camera/motion wording, and simple image-to-video prompt cleanup. Audio: Lightweight narration and classroom-style explanation drafts. | Ollama, LM Studio, llama.cpp |
| #8 | Llama 3.1 8B Instruct Meta Llama · 8B dense · Llama community license | 80 Q 73 / E 96 | Fast local chat, simple agents, privacy-first assistants Still one of the easiest models to run everywhere. | Laptop / Mini PC 8-16GB unified memory or entry-level GPU Apple Mac mini M4from $599; choose higher memory for local LLMsApple MacBook Proconfigure memory based on target model size Q4_K_M / Q5_K_M GGUF | Image: Fast local prompt helper for SDXL/FLUX drafts and simple game-art briefs. Video: Use it as a fast local storyboard helper for short image-to-video or text-to-video prompt batches. Audio: Fast local helper for voiceover text, subtitles, and Piper/Coqui prompts. | Ollama, LM Studio, llama.cpp |
Most local LLMs do not directly generate MP4 files. Use them as the director and prompt engineer, then send the structured shot sheet to Wan, LTX-Video, HunyuanVideo, or another local video generation workflow.
Output scene goal, subject, action, camera move, duration, aspect ratio, style, seed notes, and negative prompts.
Use Wan or HunyuanVideo for heavier local text-to-video, LTX-Video for faster iteration, or image-to-video when you already have a key frame.
Paste the LLM shot prompt into the video workflow, attach reference images if needed, set frames/FPS/resolution, then render clips.
Compare outputs against the shot sheet, rewrite weak prompts, rename files, and prepare the next batch.
The ranking is not a pure benchmark table. It is a local-deployment score for builders deciding what can realistically run on their own machines.
Model family and intent signals such as Qwen, DeepSeek, Llama, Mistral, Gemma, Phi, reasoning, coder, or instruct variants.
Hugging Face downloads and likes, log-scaled so one viral repo does not completely dominate the list.
Laptop-scale models score higher for efficiency; workstation models can still rank high when quality justifies the hardware.
GGUF, AWQ, GPTQ, 4-bit, or quantized tags improve the score because they make local serving easier.
Recently updated models get a small boost so the daily sync can surface new releases.
Seed models are always retained, so temporary source gaps or noisy new repos cannot erase important baseline models.
Use the LLM ranking above for reasoning, chat, coding, prompt writing, and agents. Use these companion model families when the actual output needs to be an image, video, or audio file.
Text-to-image, image editing, style exploration, game art drafts
Hardware: 12-24GB VRAM for comfortable local use; more for higher resolution
Stack: ComfyUI, Automatic1111, Forge, Diffusers
Actual video rendering. Feed it LLM-written shot prompts, keyframe descriptions, camera moves, negative prompts, and optional reference images.
Hardware: 24GB+ VRAM recommended; multi-GPU for longer clips
Stack: ComfyUI video workflows, Diffusers, custom inference scripts
Speech recognition, narration, dialogue, simple sound voiceovers
Hardware: CPU works for many ASR/TTS cases; GPU improves throughput
Stack: Whisper.cpp, faster-whisper, Piper, Coqui TTS
Public leaderboards change formats often. This page exposes a stable internal JSON endpoint and keeps source links visible for manual audit./api/local-llm/rankings
Useful for open-weight model cards and downloadable checkpoints.
LMArenaHuman preference leaderboard; useful reference, but not a local-deployment benchmark.
Artificial AnalysisStrong model intelligence and speed comparisons; API access may require a plan.
Open LLM LeaderboardOpen benchmark ecosystem; dataset availability changes over time.