Vector Database
Vector Database
Let's understand these concepts using a real application, architecture diagrams, SQL examples, C#, Azure AI Search, and Retrieval-Augmented Generation (RAG).
The most important thing is that Vector Database, Embedding Model, LLM, and Semantic Kernel are four different things.
1. First, understand the four components
Component | What it does | Example |
|---|---|---|
Embedding model | Converts text into numerical representations (vectors) | Azure OpenAI embedding model |
Vector database | Stores and searches vectors based on similarity | Azure AI Search, Pinecone, Qdrant |
LLM | Understands prompts and generates answers | GPT models |
Semantic Kernel | Connects AI models, application functions, and other services | Microsoft Semantic Kernel for .NET |
Think of an insurance application where a user asks:
"My car windshield was damaged by hail. Will my insurance cover it?"
A normal database searches for matching keywords or applies explicit query rules.
A vector search system can retrieve information about glass damage, weather-related claims, and comprehensive coverage, even if the documents don't use the exact words from the question.
The LLM can then use those retrieved documents to explain the relevant policy terms.
Semantic Kernel can coordinate this workflow and, when authorized, call application functions.
2. What exactly is a vector?
A vector is an array of numbers.
For example:
Illustrative text embeddings
Sentence A: "My car is damaged"
[0.82, 0.15, 0.63]
Sentence B: "My vehicle has damage"
[0.80, 0.18, 0.61]
Sentence C: "I want to cook pasta"
[-0.31, 0.91, 0.12]
Notice that sentences A and B have similar numerical representations, while sentence C is different.
In a real AI application, vectors can have hundreds or thousands of dimensions.
For instance, you might use an embedding model producing 1,536 numbers per text chunk.
How does AI compare vectors?
It uses mathematical similarity or distance measures, such as:
Cosine similarity
Euclidean distance
Dot product
Conceptual view of vector similarity
The vector database identifies the closest matching vectors and returns their corresponding text and metadata.
Important: The vector database does not necessarily understand or generate an answer. Its main responsibility is retrieving relevant information.
3. How is a vector database different from SQL Server?
Suppose you have a SQL Server table:
CREATE TABLE InsurancePolicies
(
PolicyId INT PRIMARY KEY,
PolicyName NVARCHAR(200),
CoverageDetails NVARCHAR(MAX)
);And records describing auto insurance coverage.
Traditional SQL search
SELECT *
FROM InsurancePolicies
WHERE CoverageDetails LIKE '%hail%';This query finds text containing the word "hail."
But suppose a document says:
"Comprehensive coverage may include damage caused by severe weather conditions."
The user asks about hail damage, yet that text might not contain the exact word.
A keyword-only query can miss it.
Vector search
The query is converted into an embedding using the same compatible embedding model as the stored documents.
The vector search finds policy documents that are mathematically close in semantic meaning.
Feature | Traditional relational database | Vector search engine |
|---|---|---|
Main purpose | Business transactions and structured data | Semantic similarity and retrieval |
Search | SQL predicates, joins, full-text search | Nearest-neighbor similarity |
Data | Rows, columns, relationships | Vectors plus text and metadata |
Exact policy number | Excellent | Not ideal as primary lookup |
Similar meaning | Requires special features | Core strength |
Transactions and constraints | Strong relational support | Varies by product |
AI chatbot knowledge retrieval | Possible | Particularly well suited |
Important update: SQL Server can now store vectors!
You do not necessarily need a separate vector database.
SQL Server 2025 and supported Azure SQL offerings include a native VECTOR type and vector-distance capabilities. This allows SQL Server to handle relational data and semantic search together.
For example:
CREATE TABLE KnowledgeChunks
(
Id INT PRIMARY KEY,
Content NVARCHAR(MAX),
Embedding VECTOR(1536)
);The VECTOR(1536) column stores a vector containing 1,536 dimensions.
You can also calculate cosine distance:
DECLARE @query VECTOR(3) = '[0.8, 0.2, 0.6]';
SELECT TOP 5
Id,
Content,
VECTOR_DISTANCE(
'cosine',
Embedding,
@query
) AS Distance
FROM KnowledgeChunks
ORDER BY Distance;This example assumes the Embedding column is VECTOR(3) to match the demonstration query. A production VECTOR(1536) column needs a 1,536-dimensional query vector.
A smaller cosine distance indicates a closer match.
You would generally use SQL Server 2025 or Azure SQL when you want relational and vector capabilities in one database, and a dedicated search engine when your application requires more sophisticated retrieval capabilities.
4. How do I create a vector database?
Let's build a practical example using Azure AI Search + Azure OpenAI + .NET 8.
The architecture
Step 1 — Create the Azure resources
In the Azure portal, create:
Azure AI Search service.
Azure OpenAI resource with an embedding deployment, such as
text-embedding-3-small.Azure OpenAI chat model deployment for answering questions.
Optional Azure Blob Storage for source documents.
Azure AI Search is the storage and retrieval engine. Azure OpenAI generates embeddings; AI Search does not host the embedding model itself.
Step 2 — Create the vector index
You can use the Azure portal's import/vectorization workflow to create an index from supported document sources.
For a .NET application, install the Azure Search SDK:
dotnet new console -n VectorSearchDemo
cd VectorSearchDemo
dotnet add package Azure.Search.Documents
dotnet add package Azure.IdentityHere is an illustrative index creation example:
using Azure;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Indexes.Models;
var endpoint = new Uri(
"https://YOUR-SEARCH.search.windows.net");
var credential = new AzureKeyCredential(
Environment.GetEnvironmentVariable("SEARCH_ADMIN_KEY")!);
var client = new SearchIndexClient(endpoint, credential);
var fields = new List<SearchField>
{
new SimpleField("id", SearchFieldDataType.String)
{
IsKey = true
},
new SearchableField("content")
{
IsFilterable = false
},
new SimpleField("policyId", SearchFieldDataType.String)
{
IsFilterable = true
},
new VectorSearchField(
"contentVector",
1536,
"vector-profile")
};
var index = new SearchIndex("insurance-knowledge")
{
Fields = fields,
VectorSearch = new VectorSearch
{
Algorithms =
{
new HnswAlgorithmConfiguration("hnsw-config")
},
Profiles =
{
new VectorSearchProfile(
"vector-profile",
"hnsw-config")
}
}
};
await client.CreateOrUpdateIndexAsync(index);
This creates the schema and vector search configuration. It does not yet generate embeddings or upload documents.
The index has three important data elements:
Field | Purpose |
|---|---|
| Unique chunk ID |
| Original text |
| Numerical embedding |
| Metadata used for filtering |
The 1,536 dimensions must match your embedding model's actual configured output size.
Microsoft documents HNSW as an approximate nearest-neighbor search algorithm supported by Azure AI Search.
Step 3 — Generate embeddings
The embedding model converts each text chunk into a numerical array.
Conceptually:
string text = "Comprehensive insurance covers ...";
// Call your Azure OpenAI embedding deployment.
float[] vector = await GenerateEmbeddingAsync(text);
// vector.Length must match the index dimensions.
// Example: 1536GenerateEmbeddingAsync is a placeholder representing your embedding-service integration.
At this stage, you have something like:
{
"id": "chunk-101",
"policyId": "AUTO-001",
"content": "Hail damage may be covered under comprehensive insurance.",
"contentVector": [0.021, -0.034, 0.092]
}The vector is abbreviated for readability.
Step 4 — Upload documents and embeddings
Once the full embeddings have been generated, upload each record:
using Azure.Search.Documents;
var searchClient = new SearchClient(
endpoint,
"insurance-knowledge",
credential);
var records = new[]
{
new
{
id = "chunk-101",
policyId = "AUTO-001",
content = "Hail damage may be covered...",
contentVector = embedding // float[1536]
}
};
await searchClient.UploadDocumentsAsync(records);Here embedding must contain the actual generated embedding values.
Step 5 — Search by meaning
When someone asks:
"Does my insurance cover storm damage?"
You generate a query embedding, then search:
using Azure.Search.Documents.Models;
float[] queryVector =
await GenerateEmbeddingAsync(userQuestion);
var options = new SearchOptions
{
Size = 5,
VectorSearch = new VectorSearchOptions
{
Queries =
{
new VectorizedQuery(queryVector)
{
KNearestNeighborsCount = 5,
Fields = { "contentVector" }
}
}
}
};
var results =
await searchClient.SearchAsync<SearchDocument>(
null, options);This retrieves the nearest matching chunks.
For production, combine vector retrieval with keyword search and metadata filters where appropriate, especially for policy identifiers or customer-specific documents.
Comments
Post a Comment