Skip to main content

Install OpenRAG in a Python project with uv

Use uv to install OpenRAG as a managed or unmanaged dependency in a new or existing Python project.

When you install OpenRAG with uv, you will use the Terminal User Interface (TUI) to configure and manage your OpenRAG deployment.

For other installation methods, see Select an installation method.

Prerequisites​

  • Install Python version 3.13 or later.

  • Install uv.

  • Install Podman (recommended) or Docker.

    The OpenRAG team recommends, at minimum, 8 GB of RAM for container VMs. However, if you plan to upload large files regularly, more RAM is recommended. For more information, see Troubleshoot OpenRAG.

  • Install podman-compose or Docker Compose. To use Docker Compose with Podman, you must alias Docker Compose commands to Podman commands.

  • Gather the credentials and connection details for your preferred model providers. You must have access to at least one language model and one embedding model. If a provider offers both types, you can use the same provider for both models. If a provider offers only one type, you must select two providers.

    • OpenAI: Create an OpenAI API key.

    • Anthropic: Create an Anthropic API key. Anthropic provides language models only; you must select an additional provider for embeddings.

    • IBM watsonx.ai: Get your watsonx.ai API endpoint, IBM project ID, and IBM API key from your watsonx deployment.

    • Ollama: Deploy an Ollama instance and models locally, in the cloud, or on a remote server. Then, get your Ollama server's base URL and the names of the models that you want to use.

      info

      OpenRAG isn't guaranteed to be compatible with all models that are available through Ollama. For example, some models might produce unexpected results, such as JSON-formatted output instead of natural language responses, and some models aren't appropriate for the types of tasks that OpenRAG performs, such as those that generate media.

      The OpenRAG team recommends the following models when using Ollama as your model provider:

      • Language models: gpt-oss:20b or mistral-nemo:12b.

        If you choose gpt-oss:20b, consider using Ollama Cloud or running Ollama on a remote machine because this model requires at least 16GB of RAM.

      • Embedding models: nomic-embed-text:latest, mxbai-embed-large:latest, or embeddinggemma:latest.

      You can experiment with other models, but if you encounter issues that you are unable to resolve through other RAG best practices (like context filters and prompt engineering), try switching to one of the recommended models. You can submit an OpenRAG GitHub issue to request support for specific models.

  • Optional: Install GPU support with an NVIDIA GPU, CUDA support, and compatible NVIDIA drivers on the OpenRAG host machine. If you don't have GPU capabilities, OpenRAG provides an alternate CPU-only deployment.

Install and start OpenRAG with uv​

There are two ways to install OpenRAG with uv:

  • uv add (Recommended): Install OpenRAG as a managed dependency in a new or existing uv Python project. This is recommended because it adds OpenRAG to your pyproject.toml and lockfile for better management of dependencies and the virtual environment.

  • uv pip install: Use the uv pip interface to install OpenRAG into an existing Python project that uses pip, pip-tools, and virtualenv commands.

If you encounter errors during installation, see Troubleshoot OpenRAG.

Use uv add​

  1. Create a new uv-managed Python project:

    uv init PROJECT_NAME

  2. Change into your new project directory:

    cd PROJECT_NAME

    Because uv manages the virtual environment for you, you won't see a (venv) prompt. uv commands automatically use the project's virtual environment.

  3. Add OpenRAG to your project:

    • Add the latest version:

      uv add openrag

    • Add a specific version:

      uv add openrag==0.1.30

    • Add a local wheel:

      uv add path/to/openrag-VERSION-py3-none-any.whl

    For more options, see Managing dependencies with uv.

  4. Optional: If you want to use a pre-populated OpenRAG .env file, create one at ~/.openrag/tui before starting OpenRAG.

  5. Start the OpenRAG TUI:

    uv run openrag

Use uv pip install​

  1. Activate your virtual environment.

  2. Install the OpenRAG Python package:

    uv pip install openrag

  3. Optional: If you want to use a pre-populated OpenRAG .env file, create one at ~/.openrag/tui before starting OpenRAG.

  4. Start the OpenRAG TUI:

    uv run openrag

Set up OpenRAG with the TUI​

When you install OpenRAG with uv, you manage the OpenRAG services with the TUI. The TUI guides you through the initial configuration process before you start the OpenRAG services.

Your configuration values are stored in an OpenRAG .env file that is created automatically at ~/.openrag/tui. If OpenRAG detects an existing .env file in this directory, then the TUI can populate those values automatically during setup and onboarding.

Container definitions are stored in the docker-compose files in the same directory as the OpenRAG .env file.

You can use either Basic Setup or Advanced Setup to configure OpenRAG. This choice determines how OpenRAG authenticates with your deployment's OpenSearch instance, and it controls user access to documents stored in your OpenSearch knowledge base:

  • No-auth mode: If you select Basic Setup in the TUI, or your OpenRAG .env file doesn't include OAuth credentials, then the OpenRAG OpenSearch instance runs in no-auth mode.

    This mode uses one anonymous JWT token for OpenSearch authentication. There is no differentiation between users; all users that access your OpenRAG instance can access all documents uploaded to your knowledge base.

  • OAuth mode: If you select Advanced Setup in the TUI, or your OpenRAG .env file includes OAuth credentials, then the OpenRAG OpenSearch instance runs in OAuth mode.

    This mode uses a unique JWT token for each OpenRAG user, and each document is tagged with user ownership. Documents are filtered by user owner; users see only the documents that they uploaded or have access to through their cloud storage accounts.

    To enable OAuth mode after initial setup, see Ingest files with OAuth connectors.

info

You must use Advanced Setup if you want to use OAuth connectors to upload documents from cloud storage.

If OpenRAG detects OAuth credentials during setup, it recommends Advanced Setup in the TUI.

  1. In the TUI, select Basic Setup.

  2. Enter administrator passwords for the OpenRAG OpenSearch and Langflow services.

    The OpenSearch password is required, and a secure password is automatically generated if you don't provide one manually.

    The Langflow password is recommended but optional. If the Langflow password is empty, the Langflow server starts without authentication enabled. For more information, see Langflow settings.

    You can click Generate Password to create a Langflow password and username automatically.

  3. Optional: Under API Keys, enter your model provider credentials, or leave these fields empty if you want to configure model provider credentials during the application onboarding process.

    There is no material difference between providing these values now or during the application onboarding process. If you provide a credential now, it can be populated automatically during the application onboarding process if you enable the Use environment API key option.

    OpenRAG's core functionality requires access to language and embedding models. By default, OpenRAG uses OpenAI models. If you aren't sure which models or providers to use, you must provide an OpenAI API key to use OpenRAG's default model configuration.

  4. Optional: Under Others, edit the knowledge base paths if you don't want to use the default paths:

    • Documents Paths: One or more paths to directories are where OpenRAG looks for documents to ingest.
    • OpenSearch Data Path: Specify the path where you want OpenRAG to create your OpenSearch index.

  5. Click Save Configuration.

    Your passwords and API keys, if provided, are stored in the OpenRAG .env file at ~/.openrag/tui. If you modified any credentials that were pulled from an existing .env file, those values are updated in the .env file.

  6. Click Start OpenRAG to start the OpenRAG services.

    This process can take some time while OpenRAG pulls and runs the container images. If all services start successfully, the TUI prints a confirmation message:

    Services started successfully
    Command completed successfully

  7. Click Close, and then click Launch OpenRAG or navigate to localhost:3000 in your browser.

  8. Continue with the application onboarding process.

Complete the application onboarding process​

The first time you start the OpenRAG application, you must complete the application onboarding process to select language and embedding models that are essential for OpenRAG features like the Chat.

Some of these variables, such as the embedding models, can be changed seamlessly after onboarding. Others are immutable and require you to destroy and recreate the OpenRAG containers. For more information, see the OpenRAG environment variables reference.

You can use different providers for your language model and embedding model, such as Anthropic for the language model and OpenAI for the embedding model. Additionally, you can set multiple embedding models.

You only need to complete onboarding for your preferred providers.

info

Anthropic doesn't provide embedding models. If you select Anthropic for your language model, you must select a different provider for the embedding model.

  1. Enter your Anthropic API key, or enable Use environment API key to pull the key from your OpenRAG .env file.

  2. Under Advanced settings, select the language model that you want to use.

  3. Click Complete.

  4. Select a provider for embeddings, provide the required information, and then select the embedding model you want to use. For information about another provider's credentials and settings, see the instructions for that provider.

  5. Click Complete.

    After you configure the embedding model, OpenRAG uses your credentials and models to ingest some initial documents. This tests the connection, and it allows you to ask OpenRAG about itself in the Chat. If there is a problem with the model configuration, an error occurs and you are redirected back to the application onboarding screen. Verify that the credential is valid and has access to the selected model, and then click Complete to retry ingestion.

  6. Continue through the overview slides for a brief introduction to OpenRAG, or click Skip overview. The overview demonstrates some basic functionality that is covered in the quickstart and in other parts of the OpenRAG documentation.

Next steps​