LlamaIndex Integration | Venice API Docs

LlamaIndex is a data framework for building RAG pipelines, agents, and query engines over your own data. Venice works as an OpenAI-compatible backend — point the OpenAILike LLM and embedding clients at Venice’s base URL and keep using the rest of the LlamaIndex API as usual.

Prerequisites

Python 3.9 or later
A Venice API key

Setup

Install LlamaIndex with the OpenAI-compatible LLM and embedding integrations:

pip install llama-index llama-index-llms-openai-like llama-index-embeddings-openai-like

uv add llama-index llama-index-llms-openai-like llama-index-embeddings-openai-like

Add your Venice API key to the environment:

export VENICE_API_KEY=your-venice-api-key

Keep API keys out of source control. Prefer environment variables or a secret manager in production.

Configure Venice as the LLM

Venice speaks the OpenAI Chat Completions API. Use OpenAILike with Venice’s base URL and set is_chat_model=True:

import os

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="venice-uncensored",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
    temperature=0.7,
)

response = llm.complete("Explain zero data retention in two sentences.")
print(str(response))

Set context_window to match the model you pick — OpenAILike can’t infer it for non-OpenAI model IDs. Set is_function_calling_model=True only for models that support function calling.

Chat messages

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are a concise, privacy-respecting assistant."),
    ChatMessage(role="user", content="Why does uncensored inference matter for research?"),
]

response = llm.chat(messages)
print(response.message.content)

Streaming

for chunk in llm.stream_complete("Write a haiku about decentralization."):
    print(chunk.delta, end="", flush=True)

Global defaults with Settings

Set Venice models once via Settings and every index, query engine, and agent will use them:

import os

from llama_index.core import Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.openai_like import OpenAILikeEmbedding

Settings.llm = OpenAILike(
    model="zai-org-glm-5-1",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
)

Settings.embed_model = OpenAILikeEmbedding(
    model_name="text-embedding-bge-m3",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
)

Embeddings

import os

from llama_index.embeddings.openai_like import OpenAILikeEmbedding

embed_model = OpenAILikeEmbedding(
    model_name="text-embedding-bge-m3",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
)

vector = embed_model.get_text_embedding("Venice AI provides private inference.")
print(f"Embedding dimension: {len(vector)}")

RAG pipeline

Build a retrieval-augmented query engine over your documents. This uses the Settings.llm and Settings.embed_model configured above:

from llama_index.core import Document, VectorStoreIndex

documents = [
    Document(text="Venice AI provides private, uncensored AI inference with zero data retention."),
    Document(text="The Venice API is OpenAI-compatible, supporting chat, images, audio, video, and embeddings."),
    Document(text="Venice supports function calling, structured outputs, web search, and reasoning models."),
    Document(text="Privacy tiers include Private (zero retention) and Anonymized (third-party processed)."),
]

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine(similarity_top_k=2)

response = query_engine.query("What privacy tiers does Venice offer?")
print(str(response))

To load your own files from a directory instead, use SimpleDirectoryReader:

from llama_index.core import SimpleDirectoryReader, VectorStoreIndex

documents = SimpleDirectoryReader("./data").load_data()
index = VectorStoreIndex.from_documents(documents)

Agents and tools

Use FunctionAgent to give Venice models tool access. Pick a model that supports function calling:

import asyncio
import os

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike


def get_venice_model_price(model_id: str) -> str:
    """Get the pricing for a Venice AI model."""
    prices = {
        "venice-uncensored": "Input: $0.20/1M, Output: $0.90/1M",
        "zai-org-glm-5-1": "Input: $1.75/1M, Output: $5.50/1M",
        "qwen3-5-9b": "Input: $0.10/1M, Output: $0.15/1M",
    }
    return prices.get(model_id, f"Model {model_id} not found in price list.")


llm = OpenAILike(
    model="zai-org-glm-5-1",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
)

agent = FunctionAgent(
    tools=[get_venice_model_price],
    llm=llm,
    system_prompt="You help users find the right Venice AI model. Use tools when needed.",
)


async def main() -> None:
    response = await agent.run("How much does zai-org-glm-5-1 cost?")
    print(str(response))


asyncio.run(main())

Structured output

Wrap the LLM with as_structured_llm to validate the final answer against a Pydantic model:

import os

from pydantic import BaseModel, Field
from llama_index.core.llms import ChatMessage
from llama_index.llms.openai_like import OpenAILike


class PrivacySummary(BaseModel):
    summary: str = Field(description="One-sentence overview")
    benefits: list[str] = Field(description="Key privacy benefits")
    recommendation: str = Field(description="When to choose this approach")


llm = OpenAILike(
    model="zai-org-glm-5-1",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=32768,
)

structured_llm = llm.as_structured_llm(PrivacySummary)
result = structured_llm.chat([
    ChatMessage(role="user", content="Summarize why private inference beats providers that retain chat logs."),
])

summary = result.raw
print(summary.summary)
print(summary.benefits)

Browse models that support structured responses before relying on structured output in production.

Venice-specific parameters

Pass Venice-only options through additional_kwargs using extra_body. For example, enable built-in web search with venice_parameters:

import os

from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="venice-uncensored",
    api_base="https://api.venice.ai/api/v1",
    api_key=os.environ["VENICE_API_KEY"],
    is_chat_model=True,
    context_window=32768,
    additional_kwargs={
        "extra_body": {
            "venice_parameters": {
                "enable_web_search": "auto",
            }
        }
    },
)

response = llm.complete("What are notable AI privacy developments this week?")
print(str(response))

You can also pass extra_body per call:

response = llm.complete(
    "Summarize today's headlines about decentralized AI.",
    extra_body={"venice_parameters": {"enable_web_search": "on"}},
)

See the API specification for the full venice_parameters list (web scraping, citations, characters, thinking controls, and E2EE toggles).

Recommended models

Use case	Model	Why
General queries	`venice-uncensored`	Fast, cheap, uncensored
Agents / tool calling	`zai-org-glm-5-1`	Strong private flagship for agents
Complex reasoning	`zai-org-glm-5-1`	Better multi-step planning
Embeddings (RAG)	`text-embedding-bge-m3`	Private embeddings
Budget / high volume	`qwen3-5-9b`	Low cost per token
Code-focused agents	`qwen3-coder-480b-a35b-instruct`	Optimized for code

Model IDs rotate over time — confirm current IDs with GET /models or the models overview.

Privacy advantage

LlamaIndex is typically used to build RAG systems over private documents, internal knowledge bases, and user data. Pairing it with Venice keeps that pipeline on private, uncensored inference:

Zero data retention on private models — prompts, retrieved chunks, and tool payloads are not kept after the request
Uncensored analysis when your data or questions would trip other providers’ filters
OpenAI-compatible plumbing so you can migrate existing LlamaIndex apps by swapping the LLM and embedding clients for OpenAILike

Troubleshooting

401 Unauthorized

Confirm VENICE_API_KEY is set in the process that runs your app. Restart the shell or process after changing environment variables.

Model not found or unexpected endpoint errors

Use a current model ID from the models page. Set api_base to https://api.venice.ai/api/v1 with no trailing path — LlamaIndex appends /chat/completions.

Context window or token errors

OpenAILike can’t infer the context window for non-OpenAI model IDs. Set context_window explicitly to match the model you’re using.

Tools are ignored

Set is_function_calling_model=True and pick a model that supports function calling. Keep tool docstrings precise — LlamaIndex builds JSON schemas from signatures and docs.

LlamaIndex Docs

Indexes, query engines, agents, and workflows

Venice Models

Browse models and supported capabilities

​Prerequisites

​Setup

​Configure Venice as the LLM

​Chat messages

​Streaming

​Global defaults with Settings

​Embeddings

​RAG pipeline

​Agents and tools

​Structured output

​Venice-specific parameters

​Recommended models

​Privacy advantage

​Troubleshooting

LlamaIndex Docs

Venice Models

Prerequisites

Setup

Configure Venice as the LLM

Chat messages

Streaming

Global defaults with Settings

Embeddings

RAG pipeline

Agents and tools

Structured output

Venice-specific parameters

Recommended models

Privacy advantage

Troubleshooting