Skip to main content

Image Embedding Models

ApertureDB stores images and their embeddings together, linked by a graph edge. A KNN query can traverse from matching descriptors directly to image blobs — no separate fetch step.

Runnable Notebooks

For setup and client configuration, see Client Configuration. For server setup options, see Server Setup.


CLIP​

CLIP embeds images and text into the same vector space, enabling text-to-image and image-to-image search. The clip-ViT-B-32 model from sentence-transformers is the simplest way to use it without PyTorch boilerplate:

pip install -U 'aperturedb>=1.0.1,<2' sentence-transformers Pillow requests
import requests
import numpy as np
from PIL import Image
from io import BytesIO
from sentence_transformers import SentenceTransformer
from aperturedb.CommonLibrary import create_connector

client = create_connector()
model = SentenceTransformer("clip-ViT-B-32") # 512-dimensional

# Create the vector index
client.query([{"CreateIndex": {
"target": "descriptor",
"class": "food_image_search",
"property": "_blob",
"kind": "hnsw",
"params": {"dimensions": 512, "metric": "CS"}
}}])

# Add image + embedding in one transaction
image_url = "https://example.com/butter_chicken.jpg"
resp = requests.get(image_url, timeout=10)
img = Image.open(BytesIO(resp.content)).convert("RGB")
emb = model.encode(img, normalize_embeddings=True).astype("float32")

client.query(
[
{"AddImage": {"class": "DishPhotos", "url": image_url, "_ref": 1, "properties": {"dish": "Butter Chicken", "cuisine": "Indian"}}},
{"AddDescriptor": {"class": "food_image_search", "connect": {"ref": 1, "class": "has_embedding"}, "properties": {"dish": "Butter Chicken"}}},
],
[emb.tobytes()]
)

Text-to-image search — CLIP text and image embeddings are comparable, so a text query returns visually matching images:

query_emb = model.encode("creamy curry", normalize_embeddings=True).astype("float32")

q = [
{"FindDescriptor": {"with_class": "food_image_search","engine": "HNSW", "k_neighbors": 5, "distances": True, "_ref": 1}},
{"FindImage": {"is_connected_to": {"ref": 1, "connection_class": "has_embedding"}, "blobs": True, "results": {"all_properties": True}}},
]
response, blobs = client.query(q, [query_emb.tobytes()])

The FindDescriptor → FindImage traversal returns matched images and metadata in one round trip.


Ingesting the Cookbook Dataset​

The Cookbook dataset (20+ dish photos) can be ingested with CLIP embeddings in one command using the ApertureDB CLI:

wget https://github.com/aperture-data/Cookbook/raw/refs/heads/main/scripts/load_cookbook_data.sh
bash load_cookbook_data.sh

This ingests all dish images with CLIP ViT-B/16 embeddings stored in the ViT-B/16 descriptor class. After ingestion, the Quick Start notebook's section 5c runs text-to-image search over all dish photos.


FaceNet​

For a large-scale example with a different model, the CelebA Face Similarity Search walkthrough uses FaceNet embeddings on 200k+ celebrity images with metadata-filtered KNN search (hair color, glasses, age).


Structured Ingestion with DataModels​

For bulk ingestion using typed Pydantic schemas, see Structured Ingestion with DataModels. This approach is used in the Cookbook dataset loader and the CelebA similarity search example.


What's Next​