This page shows you how to use AlloyDB as a large language model(LLM) tool and generate vector embeddings based on an LLM.
For more information about using ML models with AlloyDB Omni, seeBuild generative AI applications.
AlloyDB lets you use an LLM hosted by Vertex AIto translate a text string into anembedding, which is the model's representation of the given text's semanticmeaning as a numeric vector. For more information about Vertex AIsupport for text embeddings, seeText embeddings.
Before you begin
To let AlloyDB generate embeddings, make sure you meet thefollowing requirements:
Regional restrictions
You can generate embeddings in regions where Generative AI on Vertex AI isavailable. For a list of regions, see Generative AI on Vertex AIlocations .
For AlloyDB, ensure that both the AlloyDB cluster and theVertex AI model you are querying are in the same region.
Required database extension
Ensure that the
google_ml_integration
extension, version1.2
or later, installed on yourAlloyDB database.CREATE EXTENSION google_ml_integration VERSION '1.2';
This extension is included with AlloyDB. You caninstall it on any database in your cluster.
Set the
google_ml_integration.enable_model_support
database flag tooff
.
Set up model access
Before you can generate embeddings from an AlloyDB database, youmust configure AlloyDB to work with a text embedding model.
To work with the cloud-based textembeddings-gecko
model, you need tointegrate your database withwith Vertex AI.
Grant database users access to generate embeddings
Grant permission for database users to execute the embedding
function to run predictions:
Connect a
psql
client to the cluster's primary instance, as described inConnect a psql client to an instance.At the psql command prompt, connect to the database and grant permissions:
\c DB_NAMEGRANT EXECUTE ON FUNCTION embedding TO USER_NAME;
Replace the following:
DB_NAME: the name of the database on which thepermissions should be granted
USER_NAME: the name of the user for whom thepermissions should be granted
Generate an embedding
AlloyDB provides a function that lets you translate text into avector embedding. You can then store that embedding in your database as vectordata, and optionally use pgvector
functions to base queries on it.
To generate an embedding using AlloyDB, use the embedding()
function provided by the google_ml_integration
extension:
SELECT embedding( 'MODEL_IDVERSION_TAG', 'TEXT');
Replace the following:
MODEL_ID
: the ID of the model to query.If you are using the Vertex AIModel Garden, then specify
textembedding-gecko@004
as the model ID. These are the cloud-based models thatAlloyDB can use for text embeddings. For more information, seeText embeddings.Optional:
VERSION_TAG
: the version tag of the model toquery. Prepend the tag with@
.If you are using one of the
textembedding-gecko
English models withVertex AI, then specify one of the version tags—for example,textembedding-gecko@004
,listed in Model versions.Google strongly recommends that you always specify the version tag.If you don't specify the version tag, then AlloyDB always uses the latest model version, which might lead to unexpected results.
TEXT
: the text to translate into a vector embedding.
The following example uses version 004
of the textembedding-gecko
English models to generate an embeddingbased on a provided literal string:
SELECT embedding('textembedding-gecko@004', 'AlloyDB is a managed, cloud-hosted SQL database service.');
Store embeddings
The embeddings generated using the google_ml_integrationextension are implemented as arrays of real
values.These generated embeddings are passed as inputs for pgvector
extensionfunctions.
To store this value in a table, add a real[]
column:
ALTER TABLE TABLE ADD COLUMN EMBEDDING_COLUMN real(DIMENSIONS);
After you create a column to store embeddings, you can populate it based onthe values already stored in another column in the same table:
UPDATE TABLE SET EMBEDDING_COLUMN = embedding('MODEL_IDVERSION_TAG', SOURCE_TEXT_COLUMN);
Replace the following:
TABLE
: the table nameEMBEDDING_COLUMN
: the name of the embedding column
MODEL_ID
: the ID of the model to query.If you are using the Vertex AIModel Garden, then specify
textembedding-gecko@004
as the model ID. These are the cloud-based models thatAlloyDB can use for text embeddings. For more information, seeText embeddings.Optional:
VERSION_TAG
: the version tag of the model toquery. Prepend the tag with@
.If you are using one of the
textembedding-gecko
English models withVertex AI, then specify one of the version tags—for example,textembedding-gecko@004
,listed in Model versions.Google strongly recommends that you always specify the version tag.If you don't specify the version tag, then AlloyDB always uses the latest model version, which might lead to unexpected results.
SOURCE_TEXT_COLUMN
: the name of the column storing the text to translate intoembeddings
Perform similarity search
You can use also use the embedding() function to translate thetext into a vector. You apply the vector to thepgvector
nearest-neighbor operator, <->
, to find the database rows with themost semantically similar embeddings.
Because embedding()
returns a real
array, you must explicitly cast theembedding()
call to vector
in order to use these values with pgvector
operators.
CREATE EXTENSION google_ml_integration VERSION '1.2'; CREATE EXTENSION IF NOT EXISTS vector; SELECT * FROM TABLE ORDER BY EMBEDDING_COLUMN::vector <-> embedding('MODEL_IDVERSION_TAG', 'TEXT') LIMIT ROW_COUNT
Google strongly recommends that you always use a stable version of yourchosen embeddings model. For most models, this means explicitly settinga version tag.
Calling the embedding()
function without specifying the version tag ofthe model is syntactically valid, but it is also error-prone.
If you omit the version tag when using a model in the Vertex AIModel Garden, then Vertex AI uses the latest versionof the model. This might not be the latest stable version. For moreinformation about available Vertex AI model versions, see Model versions.
A given Vertex AI model version always return the sameembedding()
response to given text input. If you don't specify modelversions in your calls to embedding()
, then a new published modelversion can abruptly change the returned vector for a given input,causing errors or other unexpected behavior in your applications.
To avoid these problems, always specify the model version.
What's next
- Create indexes and query vectors
- An example embedding workflow