vLLM

github.com

An open-source LLM inference and serving engine optimized for high-throughput, low-latency local deployment, especially on Linux/NVIDIA systems.

LM Studio

lmstudio.ai

A user-friendly desktop application for running, managing, and chatting with local LLMs on Windows and macOS, featuring a graphical interface and easy model management.

LocalAI

localai.io

A self-hosted, OpenAI API-compatible server for running LLMs locally, supporting text, image, and audio modalities with Docker/Kubernetes scalability.

Nut Studio

nutstu.com

A GUI-based local LLM tool for Windows and Mac that supports 50+ open-source models and offers one-click setup, offline use, and ready-to-use AI agents.

GPT4All

gpt4all.io

A privacy-focused, offline desktop application for running and chatting with open-source LLMs on local hardware, emphasizing ease of use.

Llamafile

llamafile.ai

A portable, single-file deployment solution for running Llama-based models locally, enabling easy distribution and offline use.

Jan

jan.ai

A cross-platform desktop app for running local LLMs with a polished GUI, supporting multiple models and offline operation.