SAP HANA Cloud Vector Engine: How RAG Works on SAP, Explained With a Library

In 30 seconds

What it is Relevant passages, with sources, for a grounded answer.

You use it when You are about to design or review an SAP extension or AI use case that touches SAP HANA Cloud.

Next in this section Cloud Foundry vs Kyma on SAP BTP: Which Runtime Should Your App Use? →

Select allowed content → Create embeddings → Store text and vectors → Filter, then rank → Relevant passages, with sources, for a grounded answer. A learning overview, not a screenshot of a live system.

Foundation · 9 min read · Part 4 of 17 in SAP AI Components Explained

Your clerk types ‘how do I undo a payment’ and the search returns nothing, because the manual says ‘reversal’. Here is how meaning-based search fixes that, inside the database you already have.

SAP HANA Cloud vector engine is the focus of this post. SAP HANA Cloud is the database under most BTP apps. What makes it interesting for AI is the vector engine: the ability to store a passage of text next to a list of numbers that captures its meaning, and then ask the database for the passages whose meaning is closest to a question. That is the mechanism behind most grounded AI answers.

By the end you will be able to say in one sentence what SAP HANA Cloud does, draw how data moves through it, and use the design-review line that stops most mistakes: Similarity is not proof.

SAP HANA Cloud in one sentence

A database that can find things that mean the same, not only things spelled the same.


Its job	Keep your app’s data and your reference documents in one place, and search them by meaning for AI answers.
You need	A HANA Cloud instance with the vector engine, an embedding model (for example via AI Core, or in-database where available), and the documents you are allowed to search.
Layer	Data foundations
Design check	Similarity is not proof.

The everyday picture

A library with two catalogues. One lists books by exact title. The other lets you say ‘something like this’ and brings back the closest matches, ranked.

Why SAP HANA Cloud exists

Keyword search finds ‘payment reversal’ only if the document says ‘payment reversal’. A clerk who types ‘how do I undo a payment’ gets nothing. Vector search finds the right procedure because the meaning matches, even when the words do not. And because the vectors live in the same database as the business data, you can apply the user’s authorization filter in the same query.

How SAP HANA Cloud works, in four steps

Select allowed content. Only documents the user may see go into the index. Filter first, search second.
Create embeddings. A model turns each passage into a list of numbers that captures its meaning.
Store text and vectors. The vector sits in a column next to the text and its source reference.
Filter, then rank. SQL filters by authorization, then COSINE_SIMILARITY ranks the rest.

An embedding model turns each chunk of text into a vector of typically hundreds to a few thousand numbers. You store that vector in a REAL_VECTOR column alongside the text and its source reference. At question time the question is embedded the same way, and a SQL statement does two things in order: filter rows the user may see, then rank the remainder with COSINE_SIMILARITY. The top few passages go to the model, with their sources, and the model writes an answer using only those.

Where it sits in the SAP landscape

Documents are embedded once and stored with their source. At question time the same model embeds the question, and SQL filters by authorization before ranking.

A worked example, with numbers

The collections team has 640 procedure passages, each about 300 words, embedded with a 1,536-dimension model and stored with fields region and audience. Reem asks: ‘customer paid twice, what now’. Her question is embedded with the same model, and the query filters audience = ‘collections’, region = ‘UAE’ and status = ‘current’ first, leaving 210 of the 640 rows, then ranks them by cosine similarity. Top three: ‘Duplicate payment handling’ (0.89), ‘Refund request process’ (0.81), ‘Payment reversal FP08’ (0.77). Those three passages, with their titles and last-updated dates, go to the model, which drafts a four-line answer citing the first two. The 2023 version of the duplicate-payment procedure never reached the ranking step because its row is flagged superseded. Reem still opens the cited source before she acts.

How the data flows

The check that actually matters

Closest is not correct. The nearest passage might be last year’s procedure, or a draft. The database ranks; it does not judge. Always carry the source and date with each passage so a person can check, and never let an unauthorized passage reach the model in the hope of hiding its citation later; by then the disclosure has happened.

Similarity is not proof. The closest-matching procedure text may be last year’s version. The answer must show its source so a person can check.

HANA Cloud vector engine vs a separate vector database

Question	HANA Cloud vector engine	Separate vector database
Authorization	Same SQL filter as the business data	Must be re-implemented in the other system
Data movement	None; vectors sit next to the rows	Copy text and metadata out
Operations	One database to run	Two, plus sync
When to prefer	SAP-centric apps, strict access rules	Non-SAP stack already standardised on another store

What SAP HANA Cloud does, and what it does not

It does

Stores vectors as a native column type (REAL_VECTOR) and compares them with functions such as COSINE_SIMILARITY or L2DISTANCE in SQL.
Lets one query combine business filters (contract account, region) with similarity ranking.
Serves as the normal application database for your BTP apps at the same time.

It does not

Decide what is true. It returns what is closest, not what is correct.
Generate text. That is the language model, served through AI Core and the generative AI hub, post 12.
Enforce authorization by itself. Your query must apply the filter.

Three real situations

Payment procedure retrieval. A clerk asks how to handle a payment with a wrong reference. The app retrieves the three closest internal procedure passages and shows their titles and dates.

Product support search. A support team searches ten years of tickets by meaning, not keywords, and finds earlier cases with different wording.

Grounded assistant. An assistant answers only from retrieved passages the user is authorized to see, and cites each one.

In practice: what teams get wrong

The engine adds a REAL_VECTOR column type, functions such as COSINE_SIMILARITY and L2DISTANCE, and, in current releases, a vector index for approximate nearest-neighbour search on large tables, plus a separate knowledge graph engine for relationships. Exact index types, functions and limits depend on your HANA Cloud version. The practical design decisions are about chunking (how big a passage is), metadata (which fields you filter on), and refresh (how a changed procedure replaces its old vectors). The generative AI hub in AI Core also offers document grounding in its orchestration service, which can use a managed SAP HANA database as the vector store; if you build your own retrieval instead, keep the authorization filter in SQL where it belongs and do not assume a managed pipeline applies your business-level rules. Embedding models are commonly called through AI Core; the vector dimension is fixed by the model you choose, so changing models means re-embedding everything.

From the field. I have seen a team measure ‘search accuracy’ on clean PDFs and go live, then find the real procedure library was full of scanned tables that embedded as noise.

Other gotchas worth a line:

Chunk size changes results more than the model choice; test with real documents.
Re-embed when you change the embedding model; old and new vectors do not mix.
Vector search cost grows with table size; apply business filters and test performance with realistic volumes.
Vector column limits and functions vary by release; check the current SAP documentation.

Terms you will hear

Term	Plain meaning
Embedding	A list of numbers a model produces to represent the meaning of text.
Vector	That list, stored in the database.
Cosine similarity	A score from -1 to 1 for how close two vectors are.
Chunk	One passage of a document, embedded on its own.
RAG	Retrieval-augmented generation: find passages first, then let the model write from them.
Grounding	Giving the model evidence so it does not invent.
Vector index (ANN)	An index that makes approximate similarity search fast on big tables.

Quick self-check

In one sentence, what is SAP HANA Cloud’s job?
Which decision does it not make, and who makes it instead?
What is the one thing you would verify before calling it production-ready?

Show the answers

Keep your app’s data and your reference documents in one place, and search them by meaning for AI answers.
Decide what is true. It returns what is closest, not what is correct.
Chunk size changes results more than the model choice; test with real documents.

Key takeaways

A database that can find things that mean the same, not only things spelled the same.
Similarity is not proof.
Stores vectors as a native column type and compares them with COSINE_SIMILARITY or L2DISTANCE in SQL.
Decide what is true. It returns what is closest, not what is correct.
Vector column limits and functions vary by release; check the current SAP documentation.

FAQs

Do I need a separate vector database?

Not for most SAP scenarios. HANA Cloud holds vectors next to business data, which keeps authorization in one place.

What is RAG?

Retrieval-augmented generation: find the relevant passages first, then let the model write an answer using only those.

Keep going in the series

Previous: SAP Cloud Connector Next: Cloud Foundry and Kyma

Full map: SAP AI Components Explained — 17 posts from platform to agents.

Sources and research notes

Research checked on 21 October 2026. Product names, plans, models and regional availability change fast; the starting reference for this component is HANA Cloud vector engine. Examples use a fictional utility company and fictional cases (amounts, IDs, names); no customer landscape is described.

Recap

SAP HANA Cloud: keep your app’s data and your reference documents in one place, and search them by meaning for AI answers. The result is relevant passages, with sources, for a grounded answer. The one thing to remember: similarity is not proof.

Found this useful? The next post, Cloud Foundry and Kyma, builds on it. Share it with the SAP colleague who keeps asking what all these components actually do.

Frequently asked questions

Do I need a separate vector database?

Often not for SAP-centric scenarios. HANA Cloud can hold vectors next to business data, which keeps authorization in one place. A separate store can still make sense if your stack is already standardised on one.

What is RAG?

Retrieval-augmented generation: find the relevant passages first, then let the model write an answer using only those.