OpenRAG: An Open-Source RAG Platform for Enterprise AI

OpenRAG is an open-source RAG (Retrieval-Augmented Generation) platform designed to “turn data into context” for enterprise AI. The project is maintained and developed by the Langflow team, with support from IBM, and is licensed under Apache 2.0. OpenRAG’s goal is to unify three leading technologies—Langflow, OpenSearch, and Docling—into one complete, deployment-ready system. With a single installation, users can immediately have an intelligent document search system, LLM-based question answering, and semantic retrieval without having to configure each component separately.

Project Overview

OpenRAG was created under IBM’s leadership to “unlock” the power of RAG for the developer community. The project provides an open-source platform managed by the Langflow team under the Apache-2.0 license. The system’s primary language is Python (with a required version of ≥3.13), while the user interface is built with Next.js. According to the official GitHub page, OpenRAG has reached around 3,000 stars, around 268 forks, and about 39 contributors, showing that the community is growing quickly.

OpenRAG integrates three core components: Langflow (for building and deploying RAG workflows through a visual interface), OpenSearch (a semantic and vector search database), and Docling (for processing and extracting text from multiple document formats). Specifically, Langflow in OpenRAG handles agentic workflows such as document processing and retrieval, result re-ranking, and multi-agent orchestration, with a drag-and-drop interface that enables easy customization. OpenSearch provides large-scale vector search with enterprise-grade security and multi-user capabilities. Docling is responsible for handling input documents, such as complex PDFs and text images, to segment and normalize them before storing them in the data repository.

Architecture and Components

OpenRAG is deployed as a lightweight, container-based multi-service architecture. The overall model includes: (1) OpenRAG Backend Server – the central orchestration service that manages all other components; (2) Langflow Container – runs Langflow to edit and deploy flows; (3) Docling Serve – a local document processing service controlled by the backend; (4) OpenSearch – stores text indexes and embeddings; (5) OpenRAG Front-end – the web interface for users; and (6) External Connectors – integrations with cloud storage services such as Google Drive, OneDrive, SharePoint, AWS S3, and others to ingest documents into the system. The standard workflow is as follows: users upload documents through the interface or API; Docling extracts and normalizes the text; the OpenRAG backend coordinates with Langflow to embed vectors and index the content into OpenSearch; when a query is made, Langflow performs semantic search, ranks the results, and uses an LLM to generate a source-grounded answer.

graph LR subgraph Người dùng U[Người dùng (trình duyệt/API)] end subgraph OpenRAG_Platform A[Backend OpenRAG] B[Langflow (Xử lý Flow)] C[Docling (Xử lý Tài liệu)] D[OpenSearch (Lưu trữ)] F[Frontend (Giao diện Web)] E[Kho Đám mây/DS, v.v.] end U --> F F --> A A --> B A --> C A --> E B --> D C --> D

The diagram above illustrates the basic architecture of OpenRAG: the Backend service orchestrates the Langflow and Docling components, while Langflow and Docling connect to OpenSearch to store and retrieve content. External connections such as Drive and OneDrive are managed through the Backend, and users interact through the Frontend.

Installation and Deployment

OpenRAG provides multiple deployment options, suitable for both local environments and large-scale enterprise settings. The basic version requires Python ≥3.13 and Docker/Podman (8 GB RAM, ≥50 GB storage). There are two main approaches:

• Terminal-managed (uv/uvx): Use the uv tool (similar to pip) to install OpenRAG. The command uv run openrag or uvx openrag creates an isolated environment and launches the containers. The startup process guides users through the initial configuration (.env file) and automatically generates a Docker Compose file. The command-line interface (or TUI with the --tui option) provides an interactive menu to start/stop services, switch between CPU/GPU mode, reset to factory settings, and more.
• Self-managed (Docker/Podman/Kubernetes): Users prepare the .env file themselves (API keys, database settings, and so on) and then run docker-compose up (or Podman) to start the OpenRAG containers. A Helm chart is also provided for Kubernetes deployment (multi-cloud or on-prem). On Windows, OpenRAG must run inside WSL, as nested virtualization is not supported.

After installation, users need to complete the first onboarding process to set up the password and required environment variables. Once that is done, the system is ready: documents can be uploaded and queried through the web interface or API.

API, SDK, and CLI

OpenRAG fully supports REST APIs (FastAPI) along with client libraries (SDKs) for integration into applications.

• CLI/Terminal: As a Python application, OpenRAG provides the openrag command (run via uv run) to start or manage the system. In addition, the Terminal/TUI interface mentioned above makes operations easier.
• Python SDK: The openrag-sdk package allows developers to call the API in a more intuitive way. For example, after installing pip install openrag-sdk and setting OPENRAG_URL, the following simple Python code can run a query:

import asyncio from openrag_sdk import OpenRAGClient

async def main():
async with OpenRAGClient() as client:
response = await client.chat.create(message="What is RAG?")
print(response.response)
print(f"Chat ID: {response.chat_id}")
asyncio.run(main())

The code above initializes an OpenRAGClient and sends a question to the system, then prints the result. The SDK also supports streaming mode to receive responses progressively, along with source citation events. The OPENRAG_API_KEY and OPENRAG_URL values can be configured through environment variables or initialization arguments.

• TypeScript/JavaScript SDK: Similarly, the openrag-sdk library (npm) allows API calls from Node.js or browser applications. Example:

import { OpenRAGClient } from "openrag-sdk";

const client = new OpenRAGClient(); // automatically reads OPENRAG_URL/KEY from env
const response = await client.chat.create({ message: "What is RAG?" });
console.log(response.response);
console.log(Chat ID: ${response.chatId});

The example above shows how to create a client and send a query.

• Model Context Protocol (MCP): OpenRAG provides an MCP server (package openrag-mcp) that allows connection with AI assistants supporting MCP, such as Cursor or Claude Desktop. Simply run pip install openrag-mcp and set OPENRAG_URL/OPENRAG_API_KEY to allow language assistants to query knowledge from OpenRAG as a context source.

In short, beyond the web interface, OpenRAG also offers a CLI and SDKs for application integration, making it easy to perform query operations and basic management tasks such as listing or deleting conversations through code.

Integrations and Compatibility

OpenRAG is designed as an open platform and can connect with many systems and tools:

• Vector databases and language models: By default, OpenRAG uses OpenSearch as its semantic data store, keeping both the original text and embedded vectors. OpenSearch provides hybrid search (keyword + vector) and enterprise-grade security capabilities. Since Langflow supports many providers, users can integrate most major LLMs such as OpenAI, Anthropic, GPT4all, Bloom, Mistral, and others, as well as different embedding services. On IBM watsonx.data, OpenRAG can use frontier models from OpenAI/Anthropic or thousands of models provided by IBM.
• Cloud integrations: The system includes built-in OAuth connectors for importing documents from Google Drive, OneDrive, SharePoint, AWS S3, and more into OpenSearch. This allows OpenRAG to easily access enterprise data stored in the cloud.
• Document formats: Thanks to Docling, OpenRAG supports more than 20 file formats, including PDF, text images, PowerPoint, Excel, email, and more. Content is intelligently extracted and segmented while preserving structures such as tables and headings to maximize embedding quality.
• Scalability and orchestration: OpenRAG can be deployed on Kubernetes using a Helm chart, taking advantage of built-in auto-scaling and monitoring. Langflow provides agentic RAG flows with complex orchestration features such as multi-agent coordination and query planning to handle large queries or multi-step scenarios.

Overall, OpenRAG can fit into an existing AI ecosystem, from swapping models quickly to choosing alternative vector databases through customized flows. However, by default, OpenRAG is mainly “tied” to OpenSearch as its backend. Other tools and components can still be customized through the embedded Langflow interface.

Security and Privacy

OpenRAG inherits the security features of OpenSearch: it is open source and equipped with enterprise-grade security, including TLS, access control management, and multi-user controls. Deployments on IBM watsonx.data also add extra security and monitoring mechanisms such as governance, authentication, and auditing to ensure compliance with enterprise policies. All data remains under customer control, whether stored in OpenSearch or in their own infrastructure, helping avoid vendor lock-in. At the same time, privacy is mainly controlled by the enterprise: OpenRAG does not collect user data, and all analysis happens locally, whether on-premises or within a cloud VPC.

Performance and Scalability

OpenRAG is designed for enterprise scale. Heavy tasks such as embedding and vector search are handled by OpenSearch, a platform proven capable of processing billions of documents at Google-like scale. Through Docker and Kubernetes, OpenRAG services can run on large clusters, making use of multi-core processors and GPUs when needed, with support for switching between CPU and GPU modes through startup flags. Components such as Langflow and Docling can also scale across clusters, ensuring high load tolerance. IBM states that the watsonx.data version is intended for “high throughput and long-running tasks,” meaning OpenRAG applications can scale both horizontally and vertically depending on the infrastructure. Features such as caching, batching, and client-side buffering, for example with SDK streaming support, also help reduce latency. However, the actual limits in terms of throughput or response time still depend on hardware resources and document volume.

Community and Ecosystem

OpenRAG is a young but fast-growing project. On GitHub, it currently has around 3.0k stars, around 268 forks, and 39 main contributors. There have already been 52 releases, with the latest stable version being 0.3.1 in March 2026, and more than 100 open issues. The official documentation site links to GitHub and Discord/Discussions so contributors can ask questions and collaborate. IBM is also actively promoting OpenRAG, for example through watsonx.data. Overall, the community mainly operates in English, but there are also some introductory materials, such as blog posts from TecAdRise or IBM, that Vietnamese readers can refer to.

Comparison with Similar Projects

The table above summarizes the main differences: OpenRAG is a packaged solution with a web interface and flexible management, while LangChain and LlamaIndex are mainly libraries for custom programming. Haystack is also a complete RAG system, but it lacks a user-friendly UI and focuses mainly on production QA with Elasticsearch/Milvus. OpenRAG stands out for its “enterprise-ready” design through OpenSearch and its drag-and-drop Langflow interface.

Use Cases and Target Audience

OpenRAG is suitable for scenarios such as:

• Enterprise document search: building internal chatbots for companies, answering questions about policies, technical documents, emails, and more. For example, a support assistant can retrieve records, user guides, and call order-management APIs.
• Legal/compliance lookup: systems for searching regulations and contracts, then automatically matching knowledge against current legal requirements. The ability to handle complex formats such as PDF files with tables and formulas is especially useful.
• Business/CRM: sales assistants can extract information from CRM systems, reports, and call histories to summarize customer information and suggest strategies. Market research assistants can extract insights from business reports.
• Knowledge research: research or multidisciplinary teams can use OpenRAG to centralize knowledge from many sources, such as academic papers and reports, and combine it with LLMs to answer in-depth questions.

In general, OpenRAG is aimed at engineers, data scientists, and AI teams in large organizations that want to deploy RAG quickly without building everything from scratch. Organizations concerned about lock-in with boxed solutions often value OpenRAG because of its ability to be adjusted while keeping full control over data.

Limitations and Open Questions

• Tied to OpenSearch: The default version of OpenRAG uses OpenSearch as its semantic data store. This provides enterprise reliability, but it also means that if an organization already uses another vector database such as Pinecone, Milvus, or Qdrant, it may need to customize flows or accept an additional component. It is still unclear whether the standard version officially supports replacement options, as this is not clearly stated in the documentation.
• Advanced customization: Although Langflow allows custom flows, deeper modifications, such as changing embedding models or adding special processing steps, may require expert knowledge. The documentation mainly focuses on default scenarios, so more complex use cases, such as integrating internal LLMs or encrypting sensitive data, may require additional research.
• Multilingual support: The main interface and documentation are currently only available in English. If deployed in Vietnamese organizations, internal process guides may need to be translated or the interface localized. It is also not yet proven how well Docling handles Vietnamese text extraction.
• Real-world performance: Although the platform claims large-scale capability, clear performance figures such as average query latency or document ingestion speed have not been publicly detailed. It will require real testing with the enterprise’s own data and question sets.
• Infrastructure dependencies: Deployment requires Docker or Kubernetes, which may be challenging in environments with fixed legacy infrastructure, such as traditional Windows Server systems. There are also some warnings regarding WSL, especially about not using virtual machines inside WSL.

POC Evaluation Recommendations

To evaluate OpenRAG, we recommend running a pilot test with the following main steps:

1. Set up the environment: Install OpenRAG locally (dev/local) using uvx openrag or Docker Compose. Make sure onboarding is completed and verify that the UI and API are running stably.
2. Ingest test data: Prepare a representative dataset, such as PDFs, Excel files, and text images, and use the web interface to add documents. Check Docling’s segmentation results for text extraction accuracy and structure preservation, as well as the OpenSearch index to confirm complete content coverage. If cloud data sources such as Google Drive or SharePoint are used, test the connector functionality as well.
3. Querying and answer quality: Use both the chat interface and the SDK to ask questions against the uploaded documents. Evaluate the accuracy and completeness of the answers, whether citations are clearly provided, and the response time. Compare the results against your criteria, for example, at least 80% of simple questions answered correctly or closely enough.
4. Stability and performance: Create load test scenarios with multiple concurrent queries to measure system throughput. Record the average latency as the number of users or indexed documents increases. Evaluate whether the current hardware resources, such as CPU and RAM, meet the company’s requirements.
5. Integration testing: Check integration with specific LLMs, for example OpenAI, Anthropic, or internal LLMs, along with embedding models and enterprise security policies such as authentication and encryption. Test MCP with a tool such as Cursor if needed.
6. Visualization and UX: Evaluate the web interface, multilingual capabilities, user experience, and documentation. Consider whether internal training or document translation will be necessary.

Key metrics to track: answer accuracy, such as precision/recall on a sample question set; latency in milliseconds for each query type; document ingestion speed, such as files per minute; and resource cost in terms of CPU, GPU, and disk usage under a defined workload. At the same time, operational ease should also be considered, such as the complexity of upgrades and configuration. The evaluation results will help determine whether OpenRAG meets the organization’s business and technical requirements.

References: The main information was drawn from the official OpenRAG project page and related IBM and tech blog sources, ensuring the accuracy of this report.