Skip to main content

Install OpenRAG in a Python project with uv

Use uv to install OpenRAG as a managed or unmanaged dependency in a new or existing Python project.

When you install OpenRAG with uv, you will use an OpenRAG terminal session to configure and manage your OpenRAG deployment.

For other installation methods, see Select an installation method.

Prerequisites​

  • Install Python version 3.13.
  • Install uv.

  • Install Podman or Docker.

    A Docker or Podman VM must be running before you start OpenRAG unless you are running OpenRAG on a Linux-based VM, such as a Linux-based WSL image.

    The container VM must have sufficient resources to run the OpenRAG containers. The minimum recommendation is 8 GB of RAM and at least 50 GB of free disk space.

    For more information, see Troubleshoot OpenRAG.

  • Install podman-compose or Docker Compose. To use Docker Compose with Podman, you must alias Docker Compose commands to Podman commands.

  • Gather the credentials and connection details for one or more supported model providers:

    • OpenAI: Create an OpenAI API key.
    • Anthropic: Create an Anthropic API key. Anthropic provides language models only; you must select an additional provider for embeddings.
    • IBM watsonx.ai: Get your watsonx.ai API endpoint, IBM project ID, and IBM API key from your watsonx deployment.
    • Ollama: Deploy an Ollama instance and models locally, in the cloud, or on a remote server. Then, get your Ollama server's base URL and the names of the models that you want to use.

    OpenRAG requires at least one language model and one embedding model. If a provider offers both types of models, then you can use the same provider for both models. If a provider offers only one type, then you must configure two providers.

    Language models must support tool calling to be compatible with OpenRAG.

    For more information, see Complete the application onboarding process.

  • Optional: Install GPU support with an NVIDIA GPU, CUDA support, and compatible NVIDIA drivers on the OpenRAG host machine. If you don't have GPU capabilities, OpenRAG provides an alternate CPU-only deployment that is suitable for most use cases. The default CPU-only deployment doesn't prevent you from using GPU acceleration in external services, such as Ollama servers.

Install and start OpenRAG with uv​

Use uv add to install OpenRAG as a managed dependency in a new or existing uv Python project. This command adds OpenRAG to your pyproject.toml and lockfile. This provides better management of dependencies and the virtual environment as compared to commands like uv pip.

If you encounter errors during installation, see Troubleshoot OpenRAG.

  1. Create a new uv-managed Python project:

    uv init PROJECT_NAME --python 3.13

    The --python flag ensures that your project uses the minimum required Python version for OpenRAG. You can omit this flag if your system's default Python version is 3.13.

  2. Change into your new project directory:

    cd PROJECT_NAME

    Because uv manages the virtual environment for you, you won't see a (venv) prompt. uv commands automatically use the project's virtual environment.

  3. Add OpenRAG to your project:

    • Add the latest version:

      uv add openrag

    • Add a specific version:

      uv add openrag==0.1.30

    • Add a local wheel:

      uv add path/to/openrag-VERSION-py3-none-any.whl

    For more options, see Managing dependencies with uv.

  4. Optional: If you want to use a pre-populated OpenRAG .env file, create one at ~/.openrag/tui before starting OpenRAG.

  5. Start the OpenRAG terminal session:

    uv run openrag
    tip

    For a GUI terminal experience, use the --tui flag when starting OpenRAG.

Configure OpenRAG in the terminal​

When you install OpenRAG with uv, you manage the OpenRAG services in an OpenRAG terminal session. The terminal session guides you through the initial configuration process before you start the OpenRAG services.

Your configuration values are stored in an OpenRAG .env file that is created automatically at ~/.openrag/tui. If OpenRAG detects an existing .env file in this directory, then OpenRAG can populate those values automatically during setup and onboarding.

Container definitions are stored in the docker-compose files in the same directory as the OpenRAG .env file.

The first time you install OpenRAG, the terminal session issues a series of prompts to guide you through the initial configuration.

If OpenRAG detects an existing OpenRAG .env file, you might need to select the Reconfigure option in the terminal to access the configuration settings.

  1. Enter an administrator password for the OpenSearch service.

    This password is required, and a secure password is automatically generated if you don't provide one manually.

  2. Enter an administrator password for the Langflow service.

    This password is recommended but optional. If the Langflow password is empty, the Langflow server starts without authentication enabled. For more information, see Langflow settings.

  3. At the AI providers prompt, you can either enter your model provider credentials now, or press Enter to configure these options later during application onboarding.

    There is no material difference between providing these values now or during the application onboarding process. If you provide a credential now, it can be populated automatically during the application onboarding process if you enable the Use environment API key option.

    OpenRAG's core functionality requires access to language and embedding models. By default, OpenRAG uses OpenAI models. If you aren't sure which models or providers to use, you must provide an OpenAI API key to use OpenRAG's default model configuration.

  4. At the Langfuse tracing prompt, enter Y to enable the Langflow integration with Langfuse; otherwise, enter N. If you enter Y, provide the following Langfuse credentials:

    • Langfuse Secret Key: A secret key for your Langfuse project.
    • Langfuse Public Key: A public key for your Langfuse project.
    • Langfuse Host: Required for self-hosted Langfuse deployments. Leave empty for Langfuse Cloud.

  5. At the Documents path prompt, accept the default local documents path, or provide a path to a directory where you want OpenRAG to look for documents to ingest into your knowledge base.

  6. At the Cloud connectors and advanced settings prompt, enter Y if you want to configure any of the following additional settings; otherwise, enter N.

    info

    You must configure advanced settings if you want to enable either OAuth mode or cloud storage connectors during initial set up:

    • OAuth mode: Controls document ownership and access in your OpenRAG OpenSearch knowledge base. Without OAuth mode, there is no differentiation between users; all users that access your OpenRAG instance can access and manage all uploaded documents.
    • Cloud storage connectors: Enables ingestion of documents from external storage services.

    You can also configure these connectors later.

    • OAuth mode and cloud storage connectors: To configure one or more of these connectors, do the following:

      1. Register OpenRAG as an OAuth application in your cloud provider, and then obtain the app's OAuth credentials, such as a client ID and secret key. To enable multiple connectors, you must register an app and generate credentials for each provider.

      2. In your OpenRAG terminal session, enter the relevant OAuth credentials for the cloud connector prompts:

        The credentials can be populated automatically if OpenRAG detects these credentials in an OpenRAG .env file at ~/.openrag/tui.

      3. Register the redirect URIs shown in the terminal in your OAuth apps.

        The redirect URIs are used for the cloud storage connector webhooks. For Google, the redirect URIs are also used to redirect users back to OpenRAG after they sign in.

    • Webhook Base URL: If you entered OAuth credentials, you can set the base address for your OAuth connector endpoints. If set, the OAuth connector webhook URLs are constructed as WEBHOOK_BASE_URL/connectors/${provider}/webhook. This option is required to enable automatic ingestion from cloud storage.

    • OpenSearch Data Path: Use the default path, or specify the path where you want OpenRAG to create your OpenSearch index.

    • Langflow Public URL : Sets the base address to access the Langflow web interface where users interact with the Langflow editor in a browser. You must set this value if you want to run Langflow on a non-default port (7860)

  7. At the Start services now? prompt, press Y to start the OpenRAG services.

    This process can take some time while OpenRAG pulls and runs the container images. If all services start successfully, the terminal prints a confirmation message:

    Services are running.
    OpenRAG .env file

    Your configuration and credentials are stored in the OpenRAG .env file at ~/.openrag/tui. If you modified any credentials that were pulled from an existing .env file, those values are updated in the .env file.

  8. If the OpenRAG app doesn't launch automatically, select the Open OpenRAG in browser option to launch the OpenRAG app and start the application onboarding process. You can also manually navigate to localhost:3000 in a browser.

    If you provided Google OAuth credentials, you must sign in with Google before you are redirected to your OpenRAG instance.

Complete the application onboarding process​

The first time you start the OpenRAG application, you must complete the application onboarding process to select language and embedding models that are essential for OpenRAG features like the Chat.

To complete onboarding, you must configure at least one language model and one embedding model.

You can use different providers for your language model and embedding model, such as Anthropic for the language model and OpenAI for the embedding model. Additionally, you can select multiple embedding models.

info

Anthropic doesn't provide embedding models. If you select Anthropic for your language model, you must select a different provider for the embedding model.

  1. Enter your Anthropic API key, or enable Use environment API key to pull the key from your OpenRAG .env file.

  2. Under Advanced settings, select the language model that you want to use.

    Language models must support tool calling to be compatible with OpenRAG. Incompatible models aren't listed.

  3. Click Complete.

  4. Select a provider for embeddings, provide the required information, and then select the embedding model you want to use. For information about another provider's credentials and settings, see the instructions for that provider.

  5. Click Complete.

    After you configure the embedding model, OpenRAG uses your credentials and models to ingest some initial documents. This tests the connection, and it allows you to ask OpenRAG about itself in the Chat. If there is a problem with the model configuration, an error occurs and you are redirected back to the application onboarding screen. Verify that the credential is valid and has access to the selected model, and then click Complete to retry ingestion.

  6. Continue through the overview slides for a brief introduction to OpenRAG, or click Skip overview. The overview demonstrates some basic functionality that is covered in the quickstart and in other parts of the OpenRAG documentation.

Next steps​