Data Input & Output Formats
ApertureDB exchanges media and descriptors as binary buffers alongside JSON commands and results. Applications encode input data as bytes and decode the returned buffers into the types their libraries use.
CSV loaders handle input encoding for
Images and
Descriptors.
Applications that call db.query() directly supply and consume those buffers
alongside their commands. The examples assume an existing client connection
named db.
Image Encoding
An image buffer contains encoded image data. It can be written to a file or decoded in memory by an image library such as Pillow.
Retrieving Images
FindImage selects images using their metadata. With blobs: True, it returns
their binary data separately from the JSON results. Images may have been loaded
through the data loader tools or
added by an application.
Basic Image Retrieval
This query retrieves an existing image whose uuid property matches the given
value. It assumes that value identifies one image.
query = [{
"FindImage": {
"blobs": True, # Return image data
# Search by a unique property to retrieve one matching image.
"constraints": ["$uuid", "==", "127e46bf-48c0-42c1-8c70-6c0e24252c15"]
}
}]
results,blobs = db.query(query)
db.query() returns a JSON result array and a list of binary buffers. For
this single-image query, blobs[0] contains the image bytes. The returned
field indicates whether an image matched before the buffer is written to disk.
if db.last_query_ok() and results[0]["FindImage"]["returned"] != 0:
with open( 'output.jpg','wb' ) as file:
file.write(blobs[0])
blobs[0] is a Python bytes object. The
Pillow library decodes the saved file
and exposes its dimensions:
from PIL import Image
im = Image.open('output.jpg')
print(f"width: {im.width} height: {im.height}")
BytesIO provides the same image data to Pillow without saving a file:
from io import BytesIO
as_stream = BytesIO(blobs[0])
im = Image.open(as_stream)
print(f"width: {im.width} height: {im.height}")
Multiple Image Retrieval
A filter can match several images. This query requests up to ten images whose
animal property is dog:
query = [{
"FindImage": {
"blobs": True,
"limit":10,
"constraints": ["$animal", "==", "dog"]
}
}]
results,blobs = db.query(query)
The returned field gives the number of images in the response. The
limit bounds the
response size even when more images match the filter.
if db.last_query_ok() and results[0]["FindImage"]["returned"] != 0:
for i in range(results[0]["FindImage"]["returned"]):
with open( f'dog_{i}.jpg','wb' ) as file:
file.write(blobs[i])
For larger retrieval workloads, ParallelQuery and a QueryGenerator support concurrent queries and batch processing.
Complex Queries
group_by_source groups connected images by the objects that selected them.
This example requires the animal dataset linked in
Embedding Encoding. After downloading and extracting
it, load the image, entity, and connection CSV files:
from aperturedb.ParallelLoader import ParallelLoader
from aperturedb.ImageDataCSV import ImageDataCSV
from aperturedb.EntityDataCSV import EntityDataCSV
from aperturedb.ConnectionDataCSV import ConnectionDataCSV
data = ImageDataCSV("animals/image.adb.csv", blobs_relative_to_csv=True)
categories = EntityDataCSV("animals/classes.adb.csv")
connections = ConnectionDataCSV("animals/connection.adb.csv")
loader = ParallelLoader(db)
loader.ingest(data)
loader.ingest(categories)
loader.ingest(connections)
The loaded animal Entities contain scientific names and are connected to
images. The query selects animals by common name, returns their scientific
names, and retrieves the connected images. Each source Entity's
_uniqueid identifies its
group of images; each image's _blob_index identifies its binary buffer.
query=[{
"FindEntity": {
"with_class": "animal",
"_ref":1,
"results": {
"list": ["scientific_name", "_uniqueid"]
},
"constraints": ["$name", "in", {"array": ["Hedgehog", "Harbor seal"]}]
}
},{
"FindImage": {
"is_connected_to": {
"ref": 1
},
"results": {
"list": ["cute_id"]
},
"blobs": True,
"group_by_source": True
}
}]
r,b = db.query(query)
# create map from id to name.
id_to_name = { e["_uniqueid"]: e["scientific_name"] for e in r[0]["FindEntity"]["entities"] }
image_results = r[1]["FindImage"]["entities"]
for group_id in image_results.keys():
i = 0
for img in image_results[group_id]:
i = i + 1
filename= f"{id_to_name[group_id]}_{img['cute_id']}.jpg"
with open(filename ,'wb') as fp:
fp.write(b[img['_blob_index']])
The saved filenames combine each animal's scientific name with the image's
cute_id.
Uploading Images
ImageDataCSV loads images
in bulk. Applications can also add an individual image by sending its bytes
with an AddImage command.
Uploading a single Image
The following request reads house.jpg, attaches address properties, and uses
if_not_found to avoid adding another image with the same retail_uuid:
query=[{
"AddImage": {
"properties": {
"type": "house",
"addr_street": "123 Maple Street",
"addr_city": "Treetown",
"addr_state": "NY",
"addr_country": "US",
"retail_uuid": "e08c0f92-b08f-439d-ab94-345a6e137dc7"
},
"if_not_found": ["$retail_uuid", "==", "e08c0f92-b08f-439d-ab94-345a6e137dc7"]
}
}]
with open('house.jpg','rb') as fp:
image = fp.read()
result,_ = db.query( query, [image] )
if db.last_query_ok():
if result[0]["AddImage"]["status"] == 0:
print("Added Image!")
elif result[0]["AddImage"]["status"] == 2:
print("Image exists!")
Status 0 means AddImage added the image. Status 2 means an existing image
matched the if_not_found
condition, so the command did not add another one.
Embedding Encoding
Each embedding, or Descriptor, belongs to a DescriptorSet. Its vector must
contain the number of values specified by the set's dimensions. The input
is a dense vector; zero-valued elements cannot be omitted.
Setting up a simple model
The examples use a prepared animal image dataset. The public source images have CC-0 or CC-BY licenses.1 Download the archive and extract it:
unzip input-output-dataset.zip
The archive contains an animals directory with images and CSV files.
The embedding examples use imgbeddings with PyTorch. Install the packages
in the Python environment used by the application:
pip install imgbeddings torch # need pytorch or tensorflow.
imgbeddings uses CLIP to produce 768-dimensional vectors. The following
example loads the supplied cat image with Pillow:
from PIL import Image
from io import BytesIO
image = Image.open('animals/tabby-cat.jpg')
The model converts the image into an embedding. Its first initialization downloads the model weights; later runs use the cached copy.
from imgbeddings import imgbeddings
model = imgbeddings()
embeddings = model.to_embeddings(image)
The returned NumPy array groups embeddings by input image. embeddings[0]
is the vector for the first image, and this expression displays its first
ten values:
print(embeddings[0][0:10])
Adding an embedding
The cute animals DescriptorSet uses 768-dimensional vectors, the Flat
engine, and the L2 distance metric. Create it before adding descriptors:
query=[{
"AddDescriptorSet": {
"name": "cute animals",
"dimensions": 768,
"engine": "Flat",
"metric": "L2"
}
}]
r,_ = db.query(query)
This request adds the cat image and its descriptor together. The image's
_ref lets AddDescriptor connect the vector to its source image in the
same query. The binary inputs follow the command order: image bytes first,
then descriptor bytes.
query=[{
"AddImage": {
"_ref": 1,
"properties": {
"category" : "animals",
"animal" : "cat",
"cute_id": 42
}
}
}, {
"AddDescriptor": {
"set": "cute animals",
"connect": {
"ref":1
}
}
}]
img_bytes = BytesIO()
image.save(img_bytes,format=image.format)
r,_ = db.query( query, [img_bytes.getvalue(),embeddings[0].tobytes()])
A NumPy ndarray
provides its binary representation through tobytes(). A Pillow image can be
encoded into a BytesIO buffer using save(). An already encoded image file
can instead be read directly in binary mode.
When the image already exists, FindImage can declare the reference consumed
by AddDescriptor in place of AddImage.
Retrieving an Embedding
Descriptors can be selected by their properties, by graph connections, or by
similarity to another vector. This query finds the image with cute_id 42
and retrieves its connected descriptor's bytes:
query=[{
"FindImage": {
"_ref":1,
"constraints": ["$cute_id", "==", 42]
}
},{
"FindDescriptor": {
"set" : "cute animals",
"is_connected_to": { # this helps traverse the relationship
"ref":1
},
"blobs":True
}
}]
r, b = db.query(query )
assert b[0] == embeddings[0].tobytes(), "Consistency check failed!"
The assertion compares the returned bytes with the original vector's binary representation.
Adding Multiple Embeddings
The following loop creates an image and descriptor for each matching file in the animal dataset. Each pair has its own reference, and its image and vector buffers are appended in command order:
from pathlib import Path
query = []
blobs = []
i = 1
for p in Path("animals").iterdir():
parts = p.stem.split('_')
if len(parts) < 2:
continue
(animal,cute_id) = parts
# create the query for each item
query.extend( [{
"AddImage": {
"_ref": i,
"properties": {
"cute_id": int(cute_id),
"animal": animal.replace('-',' ')
}
}
},{
"AddDescriptor": {
"set" :"cute animals",
"connect": {
"ref": i
}
}
}] )
# prepare the data
image = Image.open(p)
embeddings = model.to_embeddings(image)
img_bytes = BytesIO()
image.save(img_bytes,format=image.format)
blobs.extend([img_bytes.getvalue(),embeddings[0].tobytes()])
i = i + 1
results, _ = db.query( query, blobs)
After ingestion, the database contains the images and their connected descriptors. For larger datasets, a QueryGenerator with ParallelQuery can divide the work into batches.
Retrieving Embeddings by Similarity Search
A query embedding can select nearby descriptors from the existing set.
This request generates the vector with the same model used for ingestion,
passes it to FindDescriptor, and retrieves the images connected to the
selected descriptors.
# First create our new embedding
image = Image.open("black-cat.jpg")
embeddings = model.to_embeddings(image)
# now use it for a neighbor search.
query=[{
"FindDescriptor": {
"_ref": 1,
"set": "cute animals",
"k_neighbors": 5,
"distances": True
}
}, {
"FindImage": {
"is_connected_to": {
"ref": 1
},
"blobs": True,
"results": {
"list": ["cute_id","animal"]
}
}
}]
results,blobs = db.query( query, [embeddings[0].tobytes()] )
# Finally, we take the results of our search and save them.
descriptors = results[0]["FindDescriptor"]["entities"]
images = results[1]["FindImage"]["entities"]
for (n,image) in enumerate(images):
filename="neighbor_{}_{}.jpg".format( image['animal'], image['cute_id'])
print("distance = {}".format( descriptors[n]["_distance"] ))
with open( filename, 'wb' ) as fp:
fp.write( blobs[image['_blob_index']] )
The example prints distances and saves images for up to five nearest neighbors, using each image's animal type and ID in its filename.
Read an object from an authorized HTTPS origin
A remote URL keeps an object's bytes at their existing location while ApertureDB stores searchable metadata. Reading those bytes requires a registered credential for the HTTPS origin, including for anonymous access.
The example requires a small, stable object at a public HTTPS origin under
your control, a user with object create/read/delete permissions, and a db
connection configured through client setup.
Set HTTPS_FIXTURE_ORIGIN to a canonical origin such as
https://media.example.com, HTTPS_FIXTURE_URL to a direct object URL under
that origin, and HTTPS_FIXTURE_SHA256 to the object's expected SHA-256 digest.
The origin must not already be registered by another credential.
Only public HTTPS destinations are supported. Redirects, query strings, fragments, and presigned URLs with query parameters are unsupported. Origin matching includes the port. The HTTPS credential restrictions define canonical origin syntax, Basic/Bearer keys, TLS checks, and transfer limits.
import hashlib
import os
import uuid
credential_name = "documentation_https_" + uuid.uuid4().hex
fixture_id = uuid.uuid4().hex
origin = os.environ["HTTPS_FIXTURE_ORIGIN"]
url = os.environ["HTTPS_FIXTURE_URL"]
expected_sha256 = os.environ["HTTPS_FIXTURE_SHA256"].lower()
def remote_query(commands):
response, blobs = db.query(commands)
if not isinstance(response, list):
raise RuntimeError(response)
for command in response:
for result in command.values():
if not isinstance(result, dict) or result.get("status") != 0:
raise RuntimeError(response)
return response, blobs
remote_query([{"CreateCredential": {
"name": credential_name, "service": "https", "key": {}, "resources": [origin]
}}])
remote_query([{"AddBlob": {
"url": url, "properties": {"documentation_fixture": fixture_id}
}}])
response, blobs = remote_query([{"FindBlob": {
"constraints": ["$documentation_fixture", "==", fixture_id], "blobs": True
}}])
assert response[0]["FindBlob"]["returned"] == 1
assert hashlib.sha256(blobs[0]).hexdigest() == expected_sha256
Credential creation registers access details; it does not fetch the object. The retrieval and digest comparison confirm that the returned bytes match the expected content. A metadata-only query can succeed even when fetching remote bytes fails.
For a generic fetch error, check the stored origin, credentials, public DNS, certificate, object size, timeout, and per-query limits. Default per-object limits are 16 MiB and 10 seconds; per-query limits also apply.
The cleanup commands remove only the metadata and credential created by this
example. They do not remove the remote object. Keep credential_name and
fixture_id until cleanup completes, including when an earlier step fails.
remote_query([{"DeleteBlob": {
"constraints": ["$documentation_fixture", "==", fixture_id]
}}])
remote_query([{"RemoveCredential": {"name": credential_name}}])
For private AWS S3 or Google Cloud Storage objects, register the corresponding
aws or gcp credential and bucket resources with
CreateCredential,
then use s3:// or gs:// object URLs. User URL credentials must carry their
own identity; they cannot borrow the server's default storage credentials.
Binary buffer formats
Binary inputs and outputs are Python bytes objects. Image libraries decode
encoded media buffers, while numerical libraries interpret descriptor buffers
using the vector's expected type and dimensions.