RAG with Supabase and pgvector

Mayur Gohil

mayurgohil.com

Software Engineering @ Atyantik Technologies

Topics:

• Introduction to Vector Databases
• Supabase & PGVector Overview
• Setting Up a Supabase Project
• Demo with PGVector

What Are Vectors and Vector Databases?

Word Embedding Models:

• Transform words into numerical vectors, capturing semantic relationships in a continuous vector space.

Popular Embedding Models:

• Word2Vec
• GloVe
• BERT

d = √(x2−x1)2+(y2−y1)2

Vector database indexes and stored vector embeddings for fast retrival and similarity search

Challenges in Vector Databases

• Storing high-dimensional vectors and performing similarity searches across thousands of them can be computationally intensive and slow due to the complexity of distance calculations.

How Can We Address This Challenge?

• Indexing is a fundamental component of a vector database, serving as a data structure that streamlines the search process.
• It is a dedicated field of research with various methods available for calculating and implementing indexing.
• Learn more about indexing link.

Example:

E-commerce recommendation systems

• Represent products and user preferences as vectors.
• Perform rapid similarity searches for personalized recommendations

Introduction to SUPABASE

• Open-source Backend as a Service (BaaS) platform built on PostgreSQL.
• Provides RESTful APIs, GraphQL APIs, real-time subscriptions, authentication, storage, and edge functions.

Integration with PG Vector:

• Facilitates storage and querying of vector embeddings.
• Simplifies data architecture and improves performance.

Supabase PG Vector Features

• Unified Data Storage
• Efficient Similarity Search
• Advanced Querying Capabilities
• Scalability
• Seamless AI Integration
• Real-Time Updates

Setting Up a Supabase Project

1. Sign Up/In: Visit the Supabase website and create an account

2. Create a Project

Demo project

https://github.com/mayur161019/EmbedInbox.git

SQL code:

• Ensure the pgvector extension is enabled.

CREATE EXTENSION IF NOT EXISTS vector;

• Create the emails table.

CREATE TABLE emails ( id SERIAL PRIMARY KEY, -- Unique ID for each email 
subject TEXT NOT NULL, -- Subject of the email 
sender TEXT NOT NULL, -- Email address of the sender 
recipient TEXT[] NOT NULL, -- Array of recipients 
cc TEXT[], -- Optional CC recipients 
bcc TEXT[], -- Optional BCC recipients 
body TEXT NOT NULL, -- Full body of the email (raw content) 
created_at TIMESTAMPTZ DEFAULT NOW() -- Timestamp when the email was sent or received 
);

• Create the email_sections table.

CREATE TABLE email_sections ( id SERIAL PRIMARY KEY, -- Unique ID for each section 
email_id INT NOT NULL REFERENCES emails(id) ON DELETE CASCADE, -- Reference to parent email 
section_content TEXT NOT NULL, -- Content of the section (chunk) 
embedding VECTOR(1536), -- Embedding of the section 
section_order INT, -- Order of the section in the original email 
created_at TIMESTAMPTZ DEFAULT NOW() -- Timestamp for the section 
);

• Create an HNSW index on the section embeddings using the correct operator class.

CREATE INDEX section_embedding_hnsw_idx 
ON email_sections USING hnsw (embedding vector_cosine_ops);

Why Supabase is Ideal for AI applications

• Supabase Vector integrates the pgvector extension into PostgreSQL, enabling the storage, indexing, and querying of vector embeddings alongside traditional relational data.
• Pincode vs Supabase pgvector

Thank You!

Questions?