2. Services

Various services that are integrated with Harbor. The link in the service name will lead you to a dedicated page in Harbor's wiki with details on getting started with the service.

Frontends

This section covers services that can provide you with an interface for interacting with the language models.

• AnythingLLM Frontend, Partial Support, RAG
  The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.

• AstrBot Satellite, Frontend, Tools
  All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.

• BionicGPT Frontend
  on-premise LLM web UI with support for OpenAI-compatible backends

• Chat Nio Frontend
  Comprehensive LLM web interface with built-in marketplace

• ComfyUI Frontend, Workflows, Image
  The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.

• Hollama Frontend
  A minimal web-UI for talking to Ollama servers.

• HuggingFace ChatUI Frontend
  A chat interface using open source models, eg OpenAssistant or Llama. It is a SvelteKit app and it powers the HuggingChat app on hf.co/chat.

• KoboldCpp Satellite, Frontend, Backend
  KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.

• LibreChat Frontend
  Open-source ChatGPT UI alternative supporting multiple AI providers (Anthropic, AWS, OpenAI, Azure, Groq, Mistral, Google) with features like model switching, message search, and multi-user support. Includes integration with DALL-E-3 and various APIs.

• Lobe Chat Frontend, RAG
  An open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system.

• Mikupad Frontend
  LLM Frontend in a single HMTL file

• ol1 Frontend
  A simple Gradio app implementing an o1-like chain of reasoning with Ollama.

• Omnichain Frontend
  Visual programming for AI language models

• Onyx Frontend, Workflows, RAG
  Open Source AI Platform with Chat UI, RAG, MCP support, and 40+ document connectors.

• Open WebUI Frontend, RAG
  widely adopted and feature rich web interface for interacting with LLMs. Supports OpenAI-compatible and Ollama backends, multi-users, multi-model chats, custom prompts, TTS, Web RAG, RAG, and much much more.

• oterm Frontend, CLI
  The text-based terminal client for Ollama.

• Parllama Frontend
  TUI for Ollama

Backends

This section covers services that provide the LLM inference capabilities.

• AirLLM Backend
  70B inference with single 4GB GPU (very slow, though)

• Aphrodite Backend
  Large-scale LLM inference engine

• faster-whisper-server Backend, Audio, Partial Support
  Legacy version of Speaches, use that instead.

• KoboldCpp Satellite, Frontend, Backend
  KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.

• KTransformers Backend
  A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

• llama.cpp Backend
  LLM inference in C/C++

• lmdeploy Backend, Partial Support
  

• mistral.rs Backend
  Blazingly fast LLM inference.

• Modular MAX Backend
  MAX is a platform from Modular (creators of Mojo) for running LLMs.

• Nexa SDK Backend, Partial Support
  Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models.

• Ollama Backend
  Get up and running with Llama 3.2, Mistral, Gemma 3, and other large language models.

• openedai-speech Backend, Audio
  An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.

• Parler Backend, Audio
  Inference and training library for high-quality TTS models.

• SGLang Backend, Image
  SGLang is a fast serving framework for large language models and vision language models.

• Speaches Backend, Audio
  an OpenAI API-compatible speech server (formerly faster-whisper-server), both TTS and STT

• TabbyAPI Backend
  An OAI compatible exllamav2 API that's both lightweight and fast

• Text Generation Inference Backend
  Inference engine from HuggingFace.

• vLLM Backend
  A high-throughput and memory-efficient inference and serving engine for LLMs

Satellites

Additional services that can be integrated with various Frontends and Backends to enable more features.

• Activepieces Satellite, Workflows
  Open-source workflow automation platform with AI capabilities and 200+ app connectors.

• Agent Zero Satellite, RAG
  General-purpose personal assistant with Web RAG, persistent memory, tools, browser use and more.

• aichat Satellite, CLI, RAG
  All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents.

• Aider Satellite, CLI
  Aider is AI pair programming in your terminal.

• Airweave Satellite, RAG
  Airweave lets agents search any app by transforming its contents into agent-ready knowledge.

• AstrBot Satellite, Frontend, Tools
  All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.

• autogpt Satellite, Partial Support
  Create, deploy, and manage continuous AI agents that automate complex workflows.

• Bolt.new Satellite
  Prompt, run, edit, and deploy full-stack web applications.

• Browser Use Satellite, Partial Support
  AI-powered browser automation with web UI

• cloudflared Satellite, API, CLI, Infra
  A helper service allowing to expose Harbor services over the internet.

• cmdh Satellite, CLI
  Create Linux commands from natural language, in the shell.

• Cognee Satellite, API, RAG
  Knowledge engine for AI agent memory. Transforms documents into searchable graphs and vectors with built-in pipelines.

• DeerFlow Satellite, RAG, Workflows
  Community-driven deep research framework combining LLMs with web search, crawling, and multi-agent workflows for comprehensive research reports.

• Dify Satellite, Workflows
  An open-source LLM app development platform.

• Docling Satellite, RAG
  Transform documents into format ready for LLMs.

• Drawio Satellite
  AI-powered diagram creation tool - generate draw.io diagrams from natural language.

• Fabric Satellite, CLI
  LLM-driven processing of the text data in the terminal.

• Flowise Satellite, Workflows
  Drag & drop UI to build your customized LLM flow.

• gptme Satellite, CLI
  A simple CLI tool to interact with LLMs.

• Harbor Bench Satellite, CLI, Built-in, Eval
  Harbor's own tool to evaluate LLMs and inference backends against custom tasks.

• Harbor Boost Satellite, API, Built-in
  Connects to downstream LLM API and serves a wrapper with custom workflow. For example, it can be used to add a CoT (Chain of Thought) to an existing LLM API, and much more. Scriptable with Python.

• Home Assistant Satellite
  Open source home automation platform for managing and controlling smart home devices.

• JupyterLab Satellite
  Helper service to author/run Jupyter notebooks in Python with access to Harbor services.

• K6 Satellite, CLI
  A modern load testing tool, using Go and JavaScript - https://k6.io

• Karakeep Satellite
  Self-hosted bookmark manager with AI-powered automatic tagging via OpenAI or Ollama.

• Khoj Satellite, RAG
  AI second brain for chat, search, and agents with your docs. Supports local and cloud LLMs.

• KoboldCpp Satellite, Frontend, Backend
  KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.

• Kotaemon Satellite, RAG
  An open-source RAG-based tool for chatting with your documents.

• LangFlow Satellite, Workflows, RAG
  A low-code app builder for RAG and multi-agent AI applications.

• LangFuse Satellite, API, Infra
  Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets.

• Latent Scope Satellite
  A new kind of workflow + tool for visualizing and exploring datasets through the lens of latent spaces.

• LibreTranslate Satellite
  A free and open-source machine translation.

• LiteLLM Satellite, API, Infra
  LLM proxy that can aggregate multiple inference APIs together into a single endpoint.

• LitLytics Satellite, Partial Support, Workflows
  Simple analytics platform that leverages LLMs to automate data analysis.

• llama-swap Satellite, API
  Runs multiple llama.cpp servers on demand for seamless switching between them.

• lm-evaluation-harness Satellite, CLI, Eval
  A de-facto standard framework for the few-shot evaluation of language models.

• Local Deep Research Satellite
  Transforms complex questions into comprehensive, cited reports.

• LocalAI Satellite, Image
  Complete AI stack for running AI models locally. Allows downloading variety of LLMs, TTS/STT/Image models and running thme locally via Web UI.

• MCP Forge Satellite, Tools
  Gateway and admin UI for managing Model Context Protocol (MCP) servers, tools, and resources.

• mcpo Satellite, Tools
  Turn MCP servers into OpenAPI REST APIs - use them anywhere.

• MetaMCP Satellite, Tools
  Allows to manage MCPs via a WebUI, exposes multiple MCPs as a single server.

• MindsDB Satellite, API
  AI platform for integrating ML models with data sources via HTTP and MySQL APIs.

• Morphic Satellite, RAG
  An AI-powered search engine with a generative UI, similar to Perplexity and Perplexica.

• n8n Satellite, Workflows
  Fair-code workflow automation platform with native AI capabilities.

• nanobot Satellite, CLI, Tools
  Ultra-lightweight personal AI assistant with multi-channel messaging and scheduled tasks.

• Netdata Satellite, Infra
  Real-time infrastructure monitoring with per-second metrics for systems, containers, and applications.

• OmniParser Satellite, Image
  A simple screen parsing tool towards pure vision based GUI agent.

• Open Interpreter Satellite, CLI
  A natural language interface for computers.

• Open Notebook Satellite
  AI-powered research and note-taking platform with multi-provider LLM support, podcast generation, and content analysis.

• Open Terminal Satellite, API, Tools
  Remote terminal and file-management API for AI agents, pre-wired for Open WebUI and Harbor backends.

• Open WebUI Pipelines Satellite, API, Workflows
  UI-Agnostic OpenAI API Plugin Framework.

• OpenClaw Satellite, Tools
  Personal AI assistant with a self-hosted gateway, Control UI, and multi-channel messaging support.

• OpenCode Satellite, CLI
  AI coding assistant with server API, TUI, and IDE extensions. Supports multiple LLM providers.

• OpenFang Satellite, API, Tools
  Rust-based Agent Operating System for running autonomous AI agents with scheduling, 40 channel adapters, and 27 LLM providers.

• OpenHands Satellite, Partial Support
  A platform for software development agents powered by AI.

• OptiLLM Satellite, API, Infra
  Optimising LLM proxy that implements many advanced workflows to boost the performance of the LLMs.

• Perplexica Satellite, RAG
  An AI-powered search engine. It is an Open source alternative to Perplexity AI.

• PhotoPrism Satellite, Image
  AI-powered photo management app with face recognition, image classification, and automatic organization.

• Plandex Satellite, CLI
  AI driven development in your terminal.

• Postiz Satellite, Tools
  AI-powered social media scheduling tool for managing posts, analytics, and team collaboration.

• Presenton Satellite
  Open-source AI presentation generator with custom layouts, multi-model support, and PDF/PPTX export.

• Promptfoo Satellite, CLI
  Test your prompts, agents, and RAGs. A developer-friendly local tool for testing LLM applications.

• Qdrant Satellite, RAG, Infra
  Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine.

• RAGLite Satellite, RAG
  Python toolkit for Retrieval-Augmented Generation (RAG)

• Repopack Satellite, CLI
  A powerful tool that packs your entire repository into a single, AI-friendly file.

• Resume Matcher Satellite
  AI-powered tool for comparing resumes against job descriptions using local LLMs via Ollama.

• SearXNG Satellite
  A privacy-respecting, hackable metasearch engine. Highly configurable and can be used for Web RAG use-cases.

• Sim Studio Satellite, Workflows
  Open-source platform to build and deploy AI agent workflows with visual canvas editor.

• SQL Chat Satellite
  Chat-based SQL client, which uses natural language to communicate with the database.

• SuperGateway Satellite, CLI, Tools, Infra
  A simple and powerful API gateway for LLMs.

• TextGrad Satellite
  Automatic "Differentiation" via Text - using large language models to backpropagate textual gradients.

• Traefik Satellite, API, Infra
  A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.

• txtai RAG Satellite, RAG
  RAG WebUI built with txtai.

• Unsloth Satellite
  Jupyter Lab environment with Unsloth for fast LLM fine-tuning - 2x faster training with 70% less memory.

• Webtop Satellite
  Linux in a web browser supporting popular desktop environments.

• Windmill Satellite, Workflows
  Open-source developer platform for internal tools, workflows, and UIs with multi-language script support.