Getting Started with Amazon S3 Vectors

Build a semantic search system using AWS's new serverless vector storage. Store millions of embeddings in S3 with sub-second query times and serverless pricing.

Open Seas 8 min read December 31, 2025
0
Getting Started with Amazon S3 Vectors
On this page (13 sections)

AWS launched S3 Vectors, native vector storage built directly into S3. No servers to provision, no clusters to manage, no indexes to tune. You store vectors like objects and query them with sub-second latency.

The pitch is compelling: serverless pricing with no cluster management, support for 2 billion vectors per index, and native Bedrock integration. But when should you actually use it?

Before Setting Sail

  • AWS account with Bedrock access
  • Python 3.9+
  • boto3 1.42+ (pip install boto3 --upgrade)
  • IAM permissions for S3 Vectors and Bedrock

When to Use S3 Vectors

S3 Vectors isn’t trying to replace your existing vector database. It’s optimized for a specific pattern:

Use CaseBest Choice
Real-time search (<50ms)OpenSearch, Pinecone
High query volume (thousands/min)OpenSearch, Pinecone
Large archive, occasional queriesS3 Vectors
RAG with Bedrock Knowledge BasesS3 Vectors
Cost-sensitive workloadsS3 Vectors

The sweet spot is large datasets with infrequent queries, such as document archives, historical embeddings, or RAG knowledge bases where you aren’t hammering the index constantly.

Captain's Note

S3 Vectors delivers ~100ms latency for warm queries and sub-second for cold queries. If you need single-digit milliseconds, stick with an in-memory vector database.

What We’re Building

We’ll build a movie recommendation system that:

  1. Creates a vector bucket and index in S3
  2. Generates embeddings with Amazon Titan
  3. Stores movie descriptions with metadata
  4. Queries for similar movies with metadata filtering

By the end, you’ll have a working semantic search system running entirely on serverless AWS infrastructure.

Step 1: Set Up the S3 Vectors Client

S3 Vectors uses a separate boto3 client from regular S3:

import boto3
import json

# S3 Vectors has its own client (not the regular s3 client)
s3vectors = boto3.client("s3vectors", region_name="us-east-1")
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

Rough Waters Ahead

S3 Vectors is available in us-east-1, us-west-2, and eu-west-1 at launch. Check the AWS documentation for current region availability.

Step 2: Create a Vector Bucket

Vector buckets are a new S3 bucket type specifically designed for vector storage:

# Create a vector bucket
bucket_name = "movie-embeddings-demo"

response = s3vectors.create_vector_bucket(
    vectorBucketName=bucket_name
)

print(f"Created bucket: {response['vectorBucketArn']}")
Output

Created bucket: arn:aws:s3vectors:us-east-1:123456789012:vector-bucket/movie-embeddings-demo

Captain's Note

Vector buckets automatically have Block Public Access enabled, preventing accidental exposure of your embeddings to the internet.

Step 3: Create a Vector Index

Within a bucket, you create indexes to organize your vectors. Each index specifies the embedding dimension and distance metric:

# Create an index for movie embeddings
# Titan Text Embeddings V2 produces 1024-dimensional vectors
index_name = "movies"

response = s3vectors.create_index(
    vectorBucketName=bucket_name,
    indexName=index_name,
    dimension=1024,
    distanceMetric="cosine"
)

print(f"Created index: {response['indexArn']}")
Output

Created index: arn:aws:s3vectors:us-east-1:123456789012:vector-bucket/movie-embeddings-demo/index/movies

Key parameters:

ParameterOptionsNotes
dimension1-16000Must match your embedding model
distanceMetriccosine, euclidean, dotProductCosine is most common for text

Step 4: Generate Embeddings with Bedrock

Now let’s create embeddings for some movie descriptions using Amazon Titan:

def generate_embedding(text: str) -> list[float]:
    """Generate an embedding using Amazon Titan Text Embeddings V2."""
    response = bedrock.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": text})
    )
    response_body = json.loads(response["body"].read())
    return response_body["embedding"]


# Sample movie data
movies = [
    {
        "key": "star-wars",
        "title": "Star Wars: A New Hope",
        "description": "A farm boy joins rebels to fight an evil empire in space.",
        "genre": "scifi",
        "year": 1977
    },
    {
        "key": "jurassic-park",
        "title": "Jurassic Park",
        "description": "Scientists create dinosaurs in a theme park that goes terribly wrong.",
        "genre": "scifi",
        "year": 1993
    },
    {
        "key": "finding-nemo",
        "title": "Finding Nemo",
        "description": "A clownfish father searches the ocean to rescue his lost son.",
        "genre": "family",
        "year": 2003
    },
    {
        "key": "the-matrix",
        "title": "The Matrix",
        "description": "A hacker discovers reality is a simulation controlled by machines.",
        "genre": "scifi",
        "year": 1999
    },
    {
        "key": "inception",
        "title": "Inception",
        "description": "A thief enters dreams to plant ideas in people's minds.",
        "genre": "scifi",
        "year": 2010
    }
]

# Generate embeddings for each movie
for movie in movies:
    movie["embedding"] = generate_embedding(movie["description"])
    print(f"Generated embedding for: {movie['title']}")
Output

Generated embedding for: Star Wars: A New Hope Generated embedding for: Jurassic Park Generated embedding for: Finding Nemo Generated embedding for: The Matrix Generated embedding for: Inception

Step 5: Store Vectors with Metadata

Upload the vectors to S3 with their metadata. Metadata enables filtering during queries:

# Prepare vectors for upload
vectors = [
    {
        "key": movie["key"],
        "data": {"float32": movie["embedding"]},
        "metadata": {
            "title": movie["title"],
            "description": movie["description"],
            "genre": movie["genre"],
            "year": movie["year"]
        }
    }
    for movie in movies
]

# Upload to S3 Vectors
response = s3vectors.put_vectors(
    vectorBucketName=bucket_name,
    indexName=index_name,
    vectors=vectors
)

print(f"Uploaded {len(vectors)} vectors")
Output

Uploaded 5 vectors

Metadata Types

S3 Vectors supports string, number, boolean, and list metadata types. All metadata is filterable by default; you can mark fields as non-filterable during index creation to exclude them from filter queries.

Step 6: Query for Similar Vectors

Now for semantic search: let’s find movies similar to a query:

def search_movies(query: str, top_k: int = 3) -> list[dict]:
    """Search for movies similar to the query."""
    # Generate query embedding
    query_embedding = generate_embedding(query)

    # Query the index
    response = s3vectors.query_vectors(
        vectorBucketName=bucket_name,
        indexName=index_name,
        queryVector={"float32": query_embedding},
        topK=top_k,
        returnDistance=True,
        returnMetadata=True
    )

    return response["vectors"]


# Search for space adventure movies
results = search_movies("epic space adventure with heroes fighting villains")

print("Search: 'epic space adventure with heroes fighting villains'\n")
for result in results:
    meta = result["metadata"]
    print(f"  {meta['title']} ({meta['year']})")
    print(f"  Distance: {result['distance']:.4f}")
    print(f"  Genre: {meta['genre']}\n")
Output

Search: ‘epic space adventure with heroes fighting villains’

Star Wars: A New Hope (1977) Distance: 0.3142 Genre: scifi

The Matrix (1999) Distance: 0.4521 Genre: scifi

Inception (2010) Distance: 0.5234 Genre: scifi

Star Wars is the top match for a space adventure query, right where we’d expect it.

Step 7: Filter by Metadata

Combine semantic search with metadata filters for more precise results:

# Search for scifi movies only
results = s3vectors.query_vectors(
    vectorBucketName=bucket_name,
    indexName=index_name,
    queryVector={"float32": generate_embedding("family friendly adventure")},
    topK=3,
    filter={"genre": "scifi"},  # Only return scifi movies
    returnDistance=True,
    returnMetadata=True
)

print("Search: 'family friendly adventure' (filtered to scifi)\n")
for result in results["vectors"]:
    meta = result["metadata"]
    print(f"  {meta['title']} - Distance: {result['distance']:.4f}")
Output

Search: ‘family friendly adventure’ (filtered to scifi)

Jurassic Park - Distance: 0.4892 Star Wars: A New Hope - Distance: 0.5124 The Matrix - Distance: 0.6341

Notice that Finding Nemo (the actual best match for “family friendly”) is excluded because we filtered to genre: "scifi".

Rough Waters Ahead

Filtering on non-filterable metadata fields returns a 400 Bad Request error. Make sure your filter fields weren’t marked as non-filterable during index creation.

Pricing Considerations

S3 Vectors pricing has three components:

ComponentCostNotes
Storage$0.06/GB/month1024-dim vector = 4KB
PUT operations$0.20/GBOne-time ingestion cost
Queries$2.50/million + data processingProcessing scales with index size

Example costs for 1 million 1024-dimensional vectors:

  • Storage: ~4GB × $0.06 = $0.24/month
  • Ingestion: 4GB × $0.20 = $0.80 one-time
  • 10K queries/month: ~$0.03/month

Compare this to a dedicated vector database instance running 24/7, and the savings are significant for low-query workloads.

Cost Optimization

S3 Vectors shines when you have millions of vectors but only query occasionally. For high-throughput workloads (thousands of queries per minute), the per-query costs can exceed a dedicated vector database.

Cleanup

Don’t forget to clean up resources when you’re done experimenting:

# Delete the index first
s3vectors.delete_index(
    vectorBucketName=bucket_name,
    indexName=index_name
)
print(f"Deleted index: {index_name}")

# Then delete the bucket
s3vectors.delete_vector_bucket(
    vectorBucketName=bucket_name
)
print(f"Deleted bucket: {bucket_name}")

Full Code

Here’s the complete implementation in one file:

"""
S3 Vectors Movie Recommendation Demo
Requires: boto3 >= 1.42, AWS credentials with S3 Vectors and Bedrock access
"""
import boto3
import json

# Initialize clients
s3vectors = boto3.client("s3vectors", region_name="us-east-1")
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

BUCKET_NAME = "movie-embeddings-demo"
INDEX_NAME = "movies"


def generate_embedding(text: str) -> list[float]:
    """Generate embedding using Amazon Titan Text Embeddings V2."""
    response = bedrock.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": text})
    )
    return json.loads(response["body"].read())["embedding"]


def setup_index():
    """Create vector bucket and index."""
    # Create bucket
    s3vectors.create_vector_bucket(vectorBucketName=BUCKET_NAME)
    print(f"Created bucket: {BUCKET_NAME}")

    # Create index (1024 dimensions for Titan V2)
    s3vectors.create_index(
        vectorBucketName=BUCKET_NAME,
        indexName=INDEX_NAME,
        dimension=1024,
        distanceMetric="cosine"
    )
    print(f"Created index: {INDEX_NAME}")


def load_movies():
    """Load sample movie data with embeddings."""
    movies = [
        {"key": "star-wars", "title": "Star Wars: A New Hope",
         "description": "A farm boy joins rebels to fight an evil empire in space.",
         "genre": "scifi", "year": 1977},
        {"key": "jurassic-park", "title": "Jurassic Park",
         "description": "Scientists create dinosaurs in a theme park that goes terribly wrong.",
         "genre": "scifi", "year": 1993},
        {"key": "finding-nemo", "title": "Finding Nemo",
         "description": "A clownfish father searches the ocean to rescue his lost son.",
         "genre": "family", "year": 2003},
        {"key": "the-matrix", "title": "The Matrix",
         "description": "A hacker discovers reality is a simulation controlled by machines.",
         "genre": "scifi", "year": 1999},
        {"key": "inception", "title": "Inception",
         "description": "A thief enters dreams to plant ideas in people's minds.",
         "genre": "scifi", "year": 2010},
    ]

    vectors = []
    for movie in movies:
        embedding = generate_embedding(movie["description"])
        vectors.append({
            "key": movie["key"],
            "data": {"float32": embedding},
            "metadata": {
                "title": movie["title"],
                "description": movie["description"],
                "genre": movie["genre"],
                "year": movie["year"]
            }
        })
        print(f"Generated embedding: {movie['title']}")

    s3vectors.put_vectors(
        vectorBucketName=BUCKET_NAME,
        indexName=INDEX_NAME,
        vectors=vectors
    )
    print(f"Uploaded {len(vectors)} vectors")


def search(query: str, top_k: int = 3, genre_filter: str = None):
    """Semantic search with optional genre filter."""
    query_embedding = generate_embedding(query)

    params = {
        "vectorBucketName": BUCKET_NAME,
        "indexName": INDEX_NAME,
        "queryVector": {"float32": query_embedding},
        "topK": top_k,
        "returnDistance": True,
        "returnMetadata": True
    }

    if genre_filter:
        params["filter"] = {"genre": genre_filter}

    response = s3vectors.query_vectors(**params)

    print(f"\nSearch: '{query}'" + (f" (genre: {genre_filter})" if genre_filter else ""))
    for result in response["vectors"]:
        meta = result["metadata"]
        print(f"  {meta['title']} ({meta['year']}) - Distance: {result['distance']:.4f}")


def cleanup():
    """Delete index and bucket."""
    s3vectors.delete_index(vectorBucketName=BUCKET_NAME, indexName=INDEX_NAME)
    s3vectors.delete_vector_bucket(vectorBucketName=BUCKET_NAME)
    print("Cleaned up resources")


if __name__ == "__main__":
    setup_index()
    load_movies()

    search("epic space adventure with heroes")
    search("family friendly adventure")
    search("mind-bending thriller", genre_filter="scifi")

    # Uncomment to clean up:
    # cleanup()

What’s Next

You’ve learned how to use S3 Vectors for serverless semantic search. From here, you could:

  • Scale up: Load a larger dataset (S3 Vectors supports up to 2 billion vectors per index)
  • Ship a full search app: Build an end-to-end semantic search pipeline on S3 Vectors and Bedrock in Building CFP Oracle
  • Add two-stage reranking: Pair vector recall with cross-encoder scoring in Cross-Encoders and Reranking
  • Add hybrid search: Combine sparse BM25 lexical recall, dense vectors, RRF fusion, and reranking in Two-Stage Hybrid Retrieval
  • Build a RAG pipeline: Connect to Claude for question-answering over your documents

Further Reading:

Found this walkthrough useful?
0
Mile Marker 106 emblem

Steven W. White · Software engineer and AI systems builder writing from the Florida coast

Independent engineering notes, measured hardware benchmarks, and runnable code from Mile Marker 106.

Keep Exploring

Related Guides

Concept Map →

Discussion

Loading comments...