Blog

Engineering

Create a Data App with the new MotherDuck Destination Connector

Build a data app using Airbyte and the MotherDuck destination connector to move, transform, and analyze data with a modern DuckDB workflow.

ENGINEERING

November 12, 2024

8 min read

Quinton Wall

Summarize with AI:

Create a Data App with the new MotherDuck Destination Connector

We just launched a new destination connector for MotherDuck to the Airbyte Marketplace. This connector is available for Cloud, OSS, and via PyAirbyte. MotherDuck is frequently used to replicate data from transactional databases such as Postgres, MongoDB, and other business systems like HubSpot, via data apps.

Data app architecture via https://motherduck.com/docs/key-tasks/data-apps/

‍

Let’s say we wanted to build an app in Streamlit to build a dashboard atop MotherDuck. To make things easy, we will use the Faker data source in Airbyte. Faker provides sample e-commerce data for a vehicle dealer. The data includes product, purchases, and users. MotherDuck is often used for for enabling OLAP use cases from OLTP data sources, so Faker is well suited for our data app example.

‍

Step 1: Create data pipeline

Start in the Airbyte UX by adding a new source to Faker. Once completed, tap create a connection to add the new MotherDuck connector. Search for “motherduck”, select the connector, and add your Access token. If you don't already have an Airbyte account, you can install on your local machine, or sign up for a free Cloud trial.

If you are unsure where to obtain your access token in MotherDuck, you can grab this via Settings > General, in the MotherDuck UI,then tap Create token if you don't already have one.

Test and save your connector, leaving the default values checked for which data to sync.

Tap next, and change the schedule type to manual. There is no need to use additional system resources for our sample app. We can save that for production. When ready, tap Set up connection, then Sync now to see data. In a moment, your data will be loaded into MotherDuck.

‍

Step 2: Verify data

At this stage your data should be synced with MotherDuck. Log in quickly to check it out. You should see the three streams: products, purchases, and users synced correctly. Great! The new MotherDuck connector makes it incredibly easy to unlock data from sources and load into a modern data warehouse for any use case.

Step : Build the data app

Now your data is in MotherDuck, it’s time to build a quick data app for accessing the data as a dashboard. I’m going to use Streamlit but you could just have easily used Evidence, React, Hex, or whatever frontend you choose.

Start by creating a .env file to store your MotherDuck access token:

Then, add the following code into a file called main.py:

‍

Looking at the code, we start by creating connection to MotherDuck. You use the duckdb library and simply pass in an md prefix along with your access token for authentication. Then, all we need is a query to access the data we want. I've prefilled a query to get a count of all product sales grouped by vehicle manufacturers, which I'll display in a data table. Of course, you can create dozens of charts using Streamlit's inbuilt options, or other integrated libraries like Plotly.

‍

All that is left is to start your app with:

Everything going to plan, in less than 50 lines of code you now have a data app connected to MotherDuck, and a fully functional data pipeline powered by Airbyte to keep data in sync. Let us know what you think of the new MotherDuck Destination Connector by sharing your experience in our Slack Community. If you don't already have an Airbyte account, you can install on your local machine, or sign up for a free Cloud trial now.

‍

Integrate with 700+ apps using Airbyte

Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.

Try it free Talk to sales

About the Author

Quinton Wall

Quinton is the Head of Developer Relations at Airbyte

Integrate with 700+ apps using Airbyte

Try Airbyte for free

Create a Data App with the new MotherDuck Destination Connector

Integrate with 700+ apps using Airbyte

Integrate with 700+ apps using Airbyte

Related posts