Skip to main content

​
AWS Bedrock

​
Overview

RAGLight supports AWS Bedrock for both LLM inference and embeddings. You can use it as a drop-in replacement for any other provider — just set provider=Settings.AWS_BEDROCK.
langchain-aws is included in RAGLight’s core dependencies. No extra install is needed.

​
Authentication

Authentication relies on the standard boto3 credential chain — the first available source is used:
  1. Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION
  2. Credentials file: ~/.aws/credentials
  3. IAM role: automatic when running on EC2, ECS, or Lambda
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...
export AWS_DEFAULT_REGION=us-east-1

​
Supported models

​
LLM

Model IDDescription
anthropic.claude-3-5-sonnet-20241022-v2:0Default — Claude 3.5 Sonnet
anthropic.claude-3-haiku-20240307-v1:0Fast and lightweight
amazon.titan-text-express-v1Amazon Titan text
meta.llama3-8b-instruct-v1:0Meta Llama 3 8B
Newer Claude models require cross-region inference profile IDs. On-demand invocation of recent Claude versions (e.g. Claude 3.7, Claude 4) is not supported with a bare anthropic.* model ID. You must use a cross-region inference profile prefixed with us., eu., or ap. — for example us.anthropic.claude-sonnet-4-5-20250929-v1:0.To list available profiles in your account:
aws bedrock list-inference-profiles

​
Embeddings

Model IDDescription
amazon.titan-embed-text-v2:0Default — Amazon Titan embeddings
cohere.embed-english-v3Cohere English embeddings
Constants are available in Settings: 
Settings.AWS_BEDROCK_LLM_MODEL       # anthropic.claude-3-5-sonnet-20241022-v2:0
Settings.AWS_BEDROCK_EMBEDDING_MODEL  # amazon.titan-embed-text-v2:0

​
Full example

from raglight.rag.simple_rag_api import RAGPipeline
from raglight.config.settings import Settings
from raglight.config.rag_config import RAGConfig
from raglight.config.vector_store_config import VectorStoreConfig
from raglight.models.data_source_model import GitHubSource

Settings.setup_logging()

# Embeddings via Bedrock Titan
vector_store_config = VectorStoreConfig(
    provider=Settings.AWS_BEDROCK,
    embedding_model=Settings.AWS_BEDROCK_EMBEDDING_MODEL,
    database=Settings.CHROMA,
    persist_directory="./bedrockDb",
    collection_name="bedrock_collection",
)

# LLM via Bedrock Claude
config = RAGConfig(
    provider=Settings.AWS_BEDROCK,
    llm=Settings.AWS_BEDROCK_LLM_MODEL,
    knowledge_base=[GitHubSource(url="https://github.com/Bessouat40/RAGLight")],
    k=5,
)

pipeline = RAGPipeline(config, vector_store_config)
pipeline.build()

response = pipeline.generate(
    "How can I create a RAGPipeline using raglight? Give me a Python example."
)
print(response)

​
Custom region

By default, AWS_DEFAULT_REGION (env var) or us-east-1 is used. You can override it per-model via the Builder API: 
from raglight.rag.builder import Builder
from raglight.config.settings import Settings

rag = (
    Builder()
    .with_embeddings(
        Settings.AWS_BEDROCK,
        model_name=Settings.AWS_BEDROCK_EMBEDDING_MODEL,
        region_name="eu-west-1",
    )
    .with_vector_store(Settings.CHROMA, persist_directory="./db", collection_name="col")
    .with_llm(
        Settings.AWS_BEDROCK,
        model_name=Settings.AWS_BEDROCK_LLM_MODEL,
        region_name="eu-west-1",
    )
    .build_rag(k=5)
)

​
Mix and match

You can combine Bedrock with any other provider. For example, local HuggingFace embeddings with a Bedrock LLM: 
vector_store_config = VectorStoreConfig(
    provider=Settings.HUGGINGFACE,
    embedding_model=Settings.DEFAULT_EMBEDDINGS_MODEL,
    database=Settings.CHROMA,
    persist_directory="./db",
    collection_name="my_collection",
)

config = RAGConfig(
    provider=Settings.AWS_BEDROCK,
    llm=Settings.AWS_BEDROCK_LLM_MODEL,
)

​
Use with raglight serve --ui

You can run the RAGLight REST API + chat UI with Bedrock by creating a .env file: 
# .env
RAGLIGHT_DB=Chroma
RAGLIGHT_PERSIST_DIR=./bedrockDb
RAGLIGHT_COLLECTION=bedrock_collection
RAGLIGHT_LLM_PROVIDER=AWSBedrock
RAGLIGHT_LLM_MODEL=us.anthropic.claude-sonnet-4-5-20250929-v1:0

RAGLIGHT_EMBEDDINGS_PROVIDER=AWSBedrock
RAGLIGHT_EMBEDDINGS_MODEL=amazon.titan-embed-text-v2:0

AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
AWS_DEFAULT_REGION=us-east-1
Then launch: 
raglight serve --ui
If you use named profiles in ~/.aws/credentials, prefer setting explicit AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY in .env — profile names with special characters (parentheses, spaces) can cause parsing issues.

​
Summary

  • Set provider=Settings.AWS_BEDROCK in both RAGConfig and VectorStoreConfig
  • No extra install required — langchain-aws is bundled
  • Credentials are resolved automatically via the boto3 chain
  • Region defaults to AWS_DEFAULT_REGION env var or us-east-1
  • Newer Claude models require cross-region inference profile IDs (e.g. us.anthropic.claude-...)
  • Mix Bedrock with any other provider freely