Connect Milvus data with any data source.

Milvus is an open-source vector database designed for efficiently storing and searching high-dimensional data, particularly useful in machine learning and similarity search applications. It offers powerful indexing and querying capabilities, making it a valuable tool for tasks like recommendation systems, image and text similarity matching, and more.

Load data into Milvus Read the docs

Sources: 590+
Standard support: 0.0.63

Overview

What is the Airbyte Milvus connector?

The Airbyte Milvus connector loads data from 590+ Airbyte sources into Milvus, with full refresh and incremental sync on a schedule you control.

01
Load from any source
Move data into Milvus from 590+ Airbyte sources on a schedule you control.
02
Incremental syncs
Pull only the records that changed since the last run instead of reloading everything.
03
Cloud or self-hosted
Run the Milvus connector on Cloud, OSS, Hybrid.

Setup

Live in twelve steps

First, navigate to the Milvus destination connector on Airbyte.
Click on the "Create a new connection" button.
Enter a name for your connector.
Enter your Milvus information and secret.
Enter the name of the Milvus database you want to connect to.
Enter the name of the schema you want to use.
Choose the tables you want to replicate.
Configure any additional settings, such as the replication frequency and the maximum number of rows to replicate.
Test the connection to ensure that it is working properly.
Save the connection and start the replication process. Note: It is important to have a basic understanding of Milvus before attempting to connect it to Airbyte. Additionally, it is recommended to consult the Airbyte documentation for more detailed instructions and troubleshooting tips.
‍
‍

Configuration fields

Milvus configuration fields
Field	Type	Description
embedding*	object	Embedding configuration
indexing*	object	Indexing configuration
processing*	object	Not documented.
omit_raw_text	boolean	Do not store the text that gets embedded along with the vector and the metadata in the destination. If set to true, only the vector and the metadata will be stored - in this case raw text for LLM use cases needs to be retrieved from another source.

Hybrid deployment

Sovereign by architecture, not by policy.

The data plane runs in your environment. Records, credentials and keys stay with you; the control plane only receives configuration and job metadata.

Cloud
OSS
Hybrid

Your environment

DataConnectorsCredentialsKeys

↕ metadata only

Airbyte control plane

OrchestrationMonitoringUI

FAQ

Milvus connector questions

What data can I load into Milvus with Airbyte?

Airbyte moves data from any supported source into Milvus on a schedule you control, with full refresh and incremental sync where the destination supports it.

Does the Milvus connector support incremental sync?

Yes. The Milvus destination supports full refresh and incremental sync, so Airbyte can load only the records that changed since the last sync.

Where can I run the Milvus connector?

Run the Milvus connector on Cloud, OSS, Hybrid. Hybrid deployments keep the data plane in your environment while the control plane orchestrates syncs.

Didn't find your answer? Talk to sales