> ## Documentation Index
> Fetch the complete documentation index at: https://docs.venice.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LlamaIndex Integration

> Build RAG pipelines, agents, and query engines with LlamaIndex using Venice's private, OpenAI-compatible chat models and embeddings via the OpenAILike client.

[LlamaIndex](https://www.llamaindex.ai/) is a data framework for building RAG pipelines, agents, and query engines over your own data. Venice works as an OpenAI-compatible backend — point the `OpenAILike` LLM and embedding clients at Venice's base URL and keep using the rest of the LlamaIndex API as usual.

## Prerequisites

* Python 3.9 or later
* A [Venice API key](/guides/getting-started/generating-api-key)

## Setup

Install LlamaIndex with the OpenAI-compatible LLM and embedding integrations:

<CodeGroup>
  ```bash pip theme={"system"}
  pip install llama-index llama-index-llms-openai-like llama-index-embeddings-openai-like
  ```

  ```bash uv theme={"system"}
  uv add llama-index llama-index-llms-openai-like llama-index-embeddings-openai-like
  ```
</CodeGroup>

Add your Venice API key to the environment:

```bash theme={"system"}
export VENICE_API_KEY=your-venice-api-key
```

<Warning>
  Keep API keys out of source control. Prefer environment variables or a secret manager in production.
</Warning>

## Configure Venice as the LLM

Venice speaks the OpenAI Chat Completions API. Use `OpenAILike` with Venice's base URL and set `is_chat_model=True`:

```python theme={"system"}
import os

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="venice-uncensored",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
    temperature=0.7,
)

response = llm.complete("Explain zero data retention in two sentences.")
print(str(response))
```

<Note>
  Set `context_window` to match the model you pick — `OpenAILike` can't infer it for non-OpenAI model IDs. Set `is_function_calling_model=True` only for models that support [function calling](/guides/features/function-calling).
</Note>

### Chat messages

```python theme={"system"}
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a concise, privacy-respecting assistant."),
    ChatMessage(role="user", content="Why does uncensored inference matter for research?"),
]

response = llm.chat(messages)
print(response.message.content)
```

## Streaming

```python theme={"system"}
for chunk in llm.stream_complete("Write a haiku about decentralization."):
    print(chunk.delta, end="", flush=True)
```

## Global defaults with Settings

Set Venice models once via `Settings` and every index, query engine, and agent will use them:

```python theme={"system"}
import os

from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding

Settings.llm = OpenAILike(
    model="zai-org-glm-5-1",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
)

Settings.embed_model = OpenAILikeEmbedding(
    model_name="text-embedding-bge-m3",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
)
```

## Embeddings

```python theme={"system"}
import os

from llama_index.embeddings.openai_like import OpenAILikeEmbedding

embed_model = OpenAILikeEmbedding(
    model_name="text-embedding-bge-m3",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
)

vector = embed_model.get_text_embedding("Venice AI provides private inference.")
print(f"Embedding dimension: {len(vector)}")
```

## RAG pipeline

Build a retrieval-augmented query engine over your documents. This uses the `Settings.llm` and `Settings.embed_model` configured above:

```python theme={"system"}
from llama_index.core import Document, VectorStoreIndex

documents = [
    Document(text="Venice AI provides private, uncensored AI inference with zero data retention."),
    Document(text="The Venice API is OpenAI-compatible, supporting chat, images, audio, video, and embeddings."),
    Document(text="Venice supports function calling, structured outputs, web search, and reasoning models."),
    Document(text="Privacy tiers include Private (zero retention) and Anonymized (third-party processed)."),
]

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=2)

response = query_engine.query("What privacy tiers does Venice offer?")
print(str(response))
```

To load your own files from a directory instead, use `SimpleDirectoryReader`:

```python theme={"system"}
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)
```

## Agents and tools

Use `FunctionAgent` to give Venice models tool access. Pick a model that supports [function calling](/guides/features/function-calling):

```python theme={"system"}
import asyncio
import os

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike


def get_venice_model_price(model_id: str) -> str:
    """Get the pricing for a Venice AI model."""
    prices = {
        "venice-uncensored": "Input: $0.20/1M, Output: $0.90/1M",
        "zai-org-glm-5-1": "Input: $1.75/1M, Output: $5.50/1M",
        "qwen3-5-9b": "Input: $0.10/1M, Output: $0.15/1M",
    }
    return prices.get(model_id, f"Model {model_id} not found in price list.")


llm = OpenAILike(
    model="zai-org-glm-5-1",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
)

agent = FunctionAgent(
    tools=[get_venice_model_price],
    llm=llm,
    system_prompt="You help users find the right Venice AI model. Use tools when needed.",
)


async def main() -> None:
    response = await agent.run("How much does zai-org-glm-5-1 cost?")
    print(str(response))


asyncio.run(main())
```

## Structured output

Wrap the LLM with `as_structured_llm` to validate the final answer against a Pydantic model:

```python theme={"system"}
import os

from pydantic import BaseModel, Field
from llama_index.core.llms import ChatMessage
from llama_index.llms.openai_like import OpenAILike


class PrivacySummary(BaseModel):
    summary: str = Field(description="One-sentence overview")
    benefits: list[str] = Field(description="Key privacy benefits")
    recommendation: str = Field(description="When to choose this approach")


llm = OpenAILike(
    model="zai-org-glm-5-1",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
)

structured_llm = llm.as_structured_llm(PrivacySummary)
result = structured_llm.chat([
    ChatMessage(role="user", content="Summarize why private inference beats providers that retain chat logs."),
])

summary = result.raw
print(summary.summary)
print(summary.benefits)
```

Browse models that support [structured responses](/guides/features/structured-responses) before relying on structured output in production.

## Venice-specific parameters

Pass Venice-only options through `additional_kwargs` using `extra_body`. For example, enable built-in web search with `venice_parameters`:

```python theme={"system"}
import os

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="venice-uncensored",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    context_window=32768,
    additional_kwargs={
        "extra_body": {
            "venice_parameters": {
                "enable_web_search": "auto",
            }
        }
    },
)

response = llm.complete("What are notable AI privacy developments this week?")
print(str(response))
```

You can also pass `extra_body` per call:

```python theme={"system"}
response = llm.complete(
    "Summarize today's headlines about decentralized AI.",
    extra_body={"venice_parameters": {"enable_web_search": "on"}},
)
```

See the [API specification](/api-reference/api-spec) for the full `venice_parameters` list (web scraping, citations, characters, thinking controls, and E2EE toggles).

## Recommended models

| Use case              | Model                            | Why                                |
| --------------------- | -------------------------------- | ---------------------------------- |
| General queries       | `venice-uncensored`              | Fast, cheap, uncensored            |
| Agents / tool calling | `zai-org-glm-5-1`                | Strong private flagship for agents |
| Complex reasoning     | `zai-org-glm-5-1`                | Better multi-step planning         |
| Embeddings (RAG)      | `text-embedding-bge-m3`          | Private embeddings                 |
| Budget / high volume  | `qwen3-5-9b`                     | Low cost per token                 |
| Code-focused agents   | `qwen3-coder-480b-a35b-instruct` | Optimized for code                 |

Model IDs rotate over time — confirm current IDs with [`GET /models`](/api-reference/endpoint/models/list) or the [models overview](/models/overview).

## Privacy advantage

LlamaIndex is typically used to build RAG systems over private documents, internal knowledge bases, and user data. Pairing it with Venice keeps that pipeline on private, uncensored inference:

* **Zero data retention** on private models — prompts, retrieved chunks, and tool payloads are not kept after the request
* **Uncensored analysis** when your data or questions would trip other providers' filters
* **OpenAI-compatible plumbing** so you can migrate existing LlamaIndex apps by swapping the LLM and embedding clients for `OpenAILike`

## Troubleshooting

<AccordionGroup>
  <Accordion title="401 Unauthorized">
    Confirm `VENICE_API_KEY` is set in the process that runs your app. Restart the shell or process after changing environment variables.
  </Accordion>

  <Accordion title="Model not found or unexpected endpoint errors">
    Use a current model ID from the [models page](/models/overview). Set `api_base` to `https://api.venice.ai/api/v1` with no trailing path — LlamaIndex appends `/chat/completions`.
  </Accordion>

  <Accordion title="Context window or token errors">
    `OpenAILike` can't infer the context window for non-OpenAI model IDs. Set `context_window` explicitly to match the model you're using.
  </Accordion>

  <Accordion title="Tools are ignored">
    Set `is_function_calling_model=True` and pick a model that supports [function calling](/guides/features/function-calling). Keep tool docstrings precise — LlamaIndex builds JSON schemas from signatures and docs.
  </Accordion>
</AccordionGroup>

<CardGroup cols={2}>
  <Card title="LlamaIndex Docs" icon="book" href="https://docs.llamaindex.ai/">
    Indexes, query engines, agents, and workflows
  </Card>

  <Card title="Venice Models" icon="database" href="/models/overview">
    Browse models and supported capabilities
  </Card>
</CardGroup>