Vector Database

 

Vector Database 

Let's understand these concepts using a real application, architecture diagrams, SQL examples, C#, Azure AI Search, and Retrieval-Augmented Generation (RAG).

The most important thing is that Vector Database, Embedding Model, LLM, and Semantic Kernel are four different things.

1. First, understand the four components

Component

What it does

Example

Embedding model

Converts text into numerical representations (vectors)

Azure OpenAI embedding model

Vector database

Stores and searches vectors based on similarity

Azure AI Search, Pinecone, Qdrant

LLM

Understands prompts and generates answers

GPT models

Semantic Kernel

Connects AI models, application functions, and other services

Microsoft Semantic Kernel for .NET

Think of an insurance application where a user asks:

"My car windshield was damaged by hail. Will my insurance cover it?"

A normal database searches for matching keywords or applies explicit query rules.

A vector search system can retrieve information about glass damage, weather-related claims, and comprehensive coverage, even if the documents don't use the exact words from the question.

The LLM can then use those retrieved documents to explain the relevant policy terms.

Semantic Kernel can coordinate this workflow and, when authorized, call application functions.

2. What exactly is a vector?

A vector is an array of numbers.

For example:

Illustrative text embeddings

Sentence A: "My car is damaged"

[0.82, 0.15, 0.63]

Sentence B: "My vehicle has damage"

[0.80, 0.18, 0.61]

Sentence C: "I want to cook pasta"

[-0.31, 0.91, 0.12]

These are invented three-dimensional vectors for illustration, not actual embedding-model outputs.

Notice that sentences A and B have similar numerical representations, while sentence C is different.

In a real AI application, vectors can have hundreds or thousands of dimensions.

For instance, you might use an embedding model producing 1,536 numbers per text chunk.

How does AI compare vectors?

It uses mathematical similarity or distance measures, such as:

  • Cosine similarity

  • Euclidean distance

  • Dot product

Conceptual view of vector similarity

Semantically related text tends to form neighborhoods in the embedding space. Real searches operate in much higher dimensions.

The vector database identifies the closest matching vectors and returns their corresponding text and metadata.

Important: The vector database does not necessarily understand or generate an answer. Its main responsibility is retrieving relevant information.

3. How is a vector database different from SQL Server?

Suppose you have a SQL Server table:

CREATE TABLE InsurancePolicies
(
    PolicyId INT PRIMARY KEY,
    PolicyName NVARCHAR(200),
    CoverageDetails NVARCHAR(MAX)
);

And records describing auto insurance coverage.

Traditional SQL search

SELECT *
FROM InsurancePolicies
WHERE CoverageDetails LIKE '%hail%';

This query finds text containing the word "hail."

But suppose a document says:

"Comprehensive coverage may include damage caused by severe weather conditions."

The user asks about hail damage, yet that text might not contain the exact word.

A keyword-only query can miss it.

Vector search

The query is converted into an embedding using the same compatible embedding model as the stored documents.

The vector search finds policy documents that are mathematically close in semantic meaning.

Feature

Traditional relational database

Vector search engine

Main purpose

Business transactions and structured data

Semantic similarity and retrieval

Search

SQL predicates, joins, full-text search

Nearest-neighbor similarity

Data

Rows, columns, relationships

Vectors plus text and metadata

Exact policy number

Excellent

Not ideal as primary lookup

Similar meaning

Requires special features

Core strength

Transactions and constraints

Strong relational support

Varies by product

AI chatbot knowledge retrieval

Possible

Particularly well suited

Important update: SQL Server can now store vectors!

You do not necessarily need a separate vector database.

SQL Server 2025 and supported Azure SQL offerings include a native VECTOR type and vector-distance capabilities. This allows SQL Server to handle relational data and semantic search together.

Microsoft Learn+1

For example:

CREATE TABLE KnowledgeChunks
(
    Id INT PRIMARY KEY,
    Content NVARCHAR(MAX),
    Embedding VECTOR(1536)
);

The VECTOR(1536) column stores a vector containing 1,536 dimensions.

You can also calculate cosine distance:

DECLARE @query VECTOR(3) = '[0.8, 0.2, 0.6]';

SELECT TOP 5
    Id,
    Content,
    VECTOR_DISTANCE(
        'cosine',
        Embedding,
        @query
    ) AS Distance
FROM KnowledgeChunks
ORDER BY Distance;

This example assumes the Embedding column is VECTOR(3) to match the demonstration query. A production VECTOR(1536) column needs a 1,536-dimensional query vector.

A smaller cosine distance indicates a closer match.

You would generally use SQL Server 2025 or Azure SQL when you want relational and vector capabilities in one database, and a dedicated search engine when your application requires more sophisticated retrieval capabilities.

4. How do I create a vector database?

Let's build a practical example using Azure AI Search + Azure OpenAI + .NET 8.

The architecture

Step 1 — Create the Azure resources

In the Azure portal, create:

  1. Azure AI Search service.

  2. Azure OpenAI resource with an embedding deployment, such as text-embedding-3-small.

  3. Azure OpenAI chat model deployment for answering questions.

  4. Optional Azure Blob Storage for source documents.

Azure AI Search is the storage and retrieval engine. Azure OpenAI generates embeddings; AI Search does not host the embedding model itself.

Microsoft Learn+1

Step 2 — Create the vector index

You can use the Azure portal's import/vectorization workflow to create an index from supported document sources.

For a .NET application, install the Azure Search SDK:

dotnet new console -n VectorSearchDemo
cd VectorSearchDemo

dotnet add package Azure.Search.Documents
dotnet add package Azure.Identity

Here is an illustrative index creation example:


using Azure;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Indexes.Models;

var endpoint = new Uri(
    "https://YOUR-SEARCH.search.windows.net");

var credential = new AzureKeyCredential(
    Environment.GetEnvironmentVariable("SEARCH_ADMIN_KEY")!);

var client = new SearchIndexClient(endpoint, credential);

var fields = new List<SearchField>
{
    new SimpleField("id", SearchFieldDataType.String)
    {
        IsKey = true
    },
    new SearchableField("content")
    {
        IsFilterable = false
    },
    new SimpleField("policyId", SearchFieldDataType.String)
    {
        IsFilterable = true
    },
    new VectorSearchField(
        "contentVector",
        1536,
        "vector-profile")
};

var index = new SearchIndex("insurance-knowledge")
{
    Fields = fields,
    VectorSearch = new VectorSearch
    {
        Algorithms =
        {
            new HnswAlgorithmConfiguration("hnsw-config")
        },
        Profiles =
        {
            new VectorSearchProfile(
                "vector-profile",
                "hnsw-config")
        }
    }
};

await client.CreateOrUpdateIndexAsync(index);

This creates the schema and vector search configuration. It does not yet generate embeddings or upload documents.

The index has three important data elements:

Field

Purpose

id

Unique chunk ID

content

Original text

contentVector

Numerical embedding

policyId

Metadata used for filtering

The 1,536 dimensions must match your embedding model's actual configured output size.

Microsoft documents HNSW as an approximate nearest-neighbor search algorithm supported by Azure AI Search.

Microsoft Learn+1

Step 3 — Generate embeddings

The embedding model converts each text chunk into a numerical array.

Conceptually:

string text = "Comprehensive insurance covers ...";

// Call your Azure OpenAI embedding deployment.
float[] vector = await GenerateEmbeddingAsync(text);

// vector.Length must match the index dimensions.
// Example: 1536

GenerateEmbeddingAsync is a placeholder representing your embedding-service integration.

At this stage, you have something like:

{
  "id": "chunk-101",
  "policyId": "AUTO-001",
  "content": "Hail damage may be covered under comprehensive insurance.",
  "contentVector": [0.021, -0.034, 0.092]
}

The vector is abbreviated for readability.

Step 4 — Upload documents and embeddings

Once the full embeddings have been generated, upload each record:

using Azure.Search.Documents;

var searchClient = new SearchClient(
    endpoint,
    "insurance-knowledge",
    credential);

var records = new[]
{
    new
    {
        id = "chunk-101",
        policyId = "AUTO-001",
        content = "Hail damage may be covered...",
        contentVector = embedding // float[1536]
    }
};

await searchClient.UploadDocumentsAsync(records);

Here embedding must contain the actual generated embedding values.

Step 5 — Search by meaning

When someone asks:

"Does my insurance cover storm damage?"

You generate a query embedding, then search:

using Azure.Search.Documents.Models;

float[] queryVector =
    await GenerateEmbeddingAsync(userQuestion);

var options = new SearchOptions
{
    Size = 5,
    VectorSearch = new VectorSearchOptions
    {
        Queries =
        {
            new VectorizedQuery(queryVector)
            {
                KNearestNeighborsCount = 5,
                Fields = { "contentVector" }
            }
        }
    }
};

var results =
    await searchClient.SearchAsync<SearchDocument>(
        null, options);

This retrieves the nearest matching chunks.

For production, combine vector retrieval with keyword search and metadata filters where appropriate, especially for policy identifiers or customer-specific documents.

Comments

Popular posts from this blog

𝗙𝗹𝘂𝗲𝗻𝘁𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 𝗶𝗻 𝗔𝗦𝗣.𝗡𝗘𝗧 𝗖𝗼𝗿𝗲 - 𝗖𝗹𝗲𝗮𝗻, 𝗙𝗹𝗲𝘅𝗶𝗯𝗹𝗲 𝗠𝗼𝗱𝗲𝗹 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 𝗳𝗼𝗿 𝗠𝗼𝗱𝗲𝗿𝗻 .𝗡𝗘𝗧 𝗔𝗽𝗽𝘀

12 essential distributed system patterns

API Security — How Attackers Break A