AWS Bedrock

Use Claude, Titan, Llama and other Bedrock models for LLM inference and embeddings.

AWS Bedrock

Overview

RAGLight supports AWS Bedrock for both LLM inference and embeddings. You can use it as a drop-in replacement for any other provider — just set provider=Settings.AWS_BEDROCK.

langchain-aws is included in RAGLight’s core dependencies. No extra install is needed.

---

Authentication

Authentication relies on the standard boto3 credential chain — the first available source is used:

1. Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION
2. Credentials file: ~/.aws/credentials
3. IAM role: automatic when running on EC2, ECS, or Lambda

export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_DEFAULT_REGION=us-east-1

---

Supported models

LLM

Model ID	Description
anthropic.claude-3-5-sonnet-20241022-v2:0	Default — Claude 3.5 Sonnet
anthropic.claude-3-haiku-20240307-v1:0	Fast and lightweight
amazon.titan-text-express-v1	Amazon Titan text
meta.llama3-8b-instruct-v1:0	Meta Llama 3 8B

Newer Claude models require cross-region inference profile IDs. On-demand invocation of recent Claude versions (e.g. Claude 3.7, Claude 4) is not supported with a bare anthropic.* model ID. You must use a cross-region inference profile prefixed with us., eu., or ap. — for example us.anthropic.claude-sonnet-4-5-20250929-v1:0.

To list available profiles in your account:

aws bedrock list-inference-profiles

Embeddings

Model ID	Description
amazon.titan-embed-text-v2:0	Default — Amazon Titan embeddings
cohere.embed-english-v3	Cohere English embeddings

Constants are available in Settings:

Settings.AWS_BEDROCK_LLM_MODEL       # anthropic.claude-3-5-sonnet-20241022-v2:0
Settings.AWS_BEDROCK_EMBEDDING_MODEL  # amazon.titan-embed-text-v2:0

---

Full example

from raglight.rag.simple_rag_api import RAGPipeline
from raglight.config.settings import Settings
from raglight.config.rag_config import RAGConfig
from raglight.config.vector_store_config import VectorStoreConfig
from raglight.models.data_source_model import GitHubSource

Settings.setup_logging()

# Embeddings via Bedrock Titan
vector_store_config = VectorStoreConfig(
    provider=Settings.AWS_BEDROCK,
    embedding_model=Settings.AWS_BEDROCK_EMBEDDING_MODEL,
    database=Settings.CHROMA,
    persist_directory="./bedrockDb",
    collection_name="bedrock_collection",
)

# LLM via Bedrock Claude
config = RAGConfig(
    provider=Settings.AWS_BEDROCK,
    llm=Settings.AWS_BEDROCK_LLM_MODEL,
    knowledge_base=[GitHubSource(url="https://github.com/Bessouat40/RAGLight")],
    k=5,
)

pipeline = RAGPipeline(config, vector_store_config)
pipeline.build()

response = pipeline.generate(
    "How can I create a RAGPipeline using raglight? Give me a Python example."
)
print(response)

---

Custom region

By default, AWS_DEFAULT_REGION (env var) or us-east-1 is used. You can override it per-model via the Builder API:

from raglight.rag.builder import Builder
from raglight.config.settings import Settings

rag = (
    Builder()
    .with_embeddings(
        Settings.AWS_BEDROCK,
        model_name=Settings.AWS_BEDROCK_EMBEDDING_MODEL,
        region_name="eu-west-1",
    )
    .with_vector_store(Settings.CHROMA, persist_directory="./db", collection_name="col")
    .with_llm(
        Settings.AWS_BEDROCK,
        model_name=Settings.AWS_BEDROCK_LLM_MODEL,
        region_name="eu-west-1",
    )
    .build_rag(k=5)
)

---

Mix and match

You can combine Bedrock with any other provider. For example, local HuggingFace embeddings with a Bedrock LLM:

vector_store_config = VectorStoreConfig(
    provider=Settings.HUGGINGFACE,
    embedding_model=Settings.DEFAULT_EMBEDDINGS_MODEL,
    database=Settings.CHROMA,
    persist_directory="./db",
    collection_name="my_collection",
)

config = RAGConfig(
    provider=Settings.AWS_BEDROCK,
    llm=Settings.AWS_BEDROCK_LLM_MODEL,
)

---

Use with raglight serve --ui

You can run the RAGLight REST API + chat UI with Bedrock by creating a .env file:

# .env
RAGLIGHT_DB=Chroma
RAGLIGHT_PERSIST_DIR=./bedrockDb
RAGLIGHT_COLLECTION=bedrock_collection
RAGLIGHT_LLM_PROVIDER=AWSBedrock
RAGLIGHT_LLM_MODEL=us.anthropic.claude-sonnet-4-5-20250929-v1:0

RAGLIGHT_EMBEDDINGS_PROVIDER=AWSBedrock
RAGLIGHT_EMBEDDINGS_MODEL=amazon.titan-embed-text-v2:0

AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=us-east-1

Then launch:

raglight serve --ui

If you use named profiles in ~/.aws/credentials, prefer setting explicit AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY in .env — profile names with special characters (parentheses, spaces) can cause parsing issues.

---

Summary

• Set provider=Settings.AWS_BEDROCK in both RAGConfig and VectorStoreConfig
• No extra install required — langchain-aws is bundled
• Credentials are resolved automatically via the boto3 chain
• Region defaults to AWS_DEFAULT_REGION env var or us-east-1
• Newer Claude models require cross-region inference profile IDs (e.g. us.anthropic.claude-...)
• Mix Bedrock with any other provider freely