Blog

Engineering

Hands-on with the new AI Assistant

AI Assistant is now available in beta. Here are some tips to get started.

ENGINEERING

September 24, 2024

Quinton Wall

Summarize with AI:

If you caught the Airbyte 1.0 launch event, you saw that we just launched the alpha release of AI Assistant. We are incredibly excited for the potential of AI Assistant to dramatically improve the long tail of connectors. We want to do it right. Your feedback is critical to this. The following post will help you navigate the AI Assistant wizard, provide a few pointers where the alpha release may not function as expected, and show you how to leave feedback for the Product teams as we keep iterating.

List users in Notion

Let’s start with an example source you are likely already familiar with, Notion. We already have a connector in the Marketplace for this, but it’s a good representative example of an API that has some particular requirements that are quite common.

To get started, navigate to Builder > Create from Scratch and enter the endpoint URL and Users as the initial stream into the new Use AI Assist prompt. To keep things simple, we will retrieve a list of users, and build up from there. Notion’s API docs specify the URL endpoint for Users is https://api.notion.com/v1/users. In the example below, I’m going to paste in the specific stream that I am interested in, but AI Assistant is smart enough to discover all available streams simply by providing a Provider's API documentation homepage!

‍

Once you tap create, AI Assist will automatically build a configuration for you, and a Stream to return User data. Once complete, you can see that it has determined that the connection requires an API key for Authentication. AI Assist has already added a parameter where you can enter your Notion token, which you can retrieve from the Integration page in your Notion profile, as a testing value.

Keep an eye on the output, for now.

If you test your connection now however, it will fail. Notion, and many other endpoints require additional header information. In Notion’s case, they require a Notion-Version value. As mentioned, AI Assistant is in alpha. It will continuously get smarter at interrogating endpoints and should pick up additional required headers.Currently it doesn’t always get it perfect, but will already save you a significant amount of time vs. having to read through documentation, figuring out authentication headers, and so on. You can think of alpha as similar to the Centaur players in chess. AI will get the majority of the moves right and be much faster than you, but keep an eye on output to ensure it is fully correct. Over time, as AI Assistant learns more moves, the success coverage will only continue to improve.

For now, to resolve the missing Notion-Version header, you can add it into the Users stream. Then, re-running the test correctly returns 3 users in my Notion organization. Nice.

Start with a GET

As a general best practice, I like to start with a simple GET stream like Users which does not require an id or additional parameters. Once I know AI Assistant has generated everything correctly, and I’ve resolved any issues, I can create a new stream by having AI Assistant automatically discover available streams and let the platform do the rest. Just remember, if you have to add additional headers, you will need to do this again in the new stream, for now.

For example, I can add a new stream for Pages with AI Assistant automatically generating everything needed to query Notion, including pagination and JSON path to the results, all in about 30 seconds.

‍

Rick & Morty all the things!

We’ve seen how AI Assistant and be used against an API such as Notion that requires additional headers and some more complex API structures. Let’s try another API to demonstrate the power of AI Assistant to solve the challenges surrounding the long tail of connectors. We are going to use the Rick & Morty API. And yes, it really does it exist.

Now, I’m pretty confident that a Rick & Morty connector is not high on many integration providers' roadmap, but with AI Assistant, it doesn’t have to be. Let’s connect to it now by jumping back to Builder > Start from Scratch and enter the following information to create a data stream to retrieve a list of characters from the series. This time, we are providing AI Assistant the API documentation homepage and letting it automatically discover the stream just one of the many features found in leading API Management Software.

‍

Once AI Assistant has done its magic, you have a fully functional Stream. Tap Test and you see a list of Characters.

With the connection setup, we can use AI Assistant to suggest additional streams supported by the API. No more digging through documentation to discover what other data is available. Everything is handled for you, with just a tap.

You may have also noticed is that the Rick & Morty APIs conform to alpha success strategies above: No additional request headers, and GET requests. We are continuously iterating on AI Assistant and wanted to put it in the hands of our community as soon as possible. Over time, the AI models will continue to learn, but as you see in this post, AI Assistant will already save you significant amounts of time. That’s more time for coffee. Win-win.

Contribute to the Airbyte Community 💗

Now that you’ve created a new connector using AI Assistant, you can contribute this back to the community by publishing it to GitHub. All you need to do is tap the Publish button, select Contribute to Airbyte, enter a few details and you are done. Our team will review the PR, and if it is accepted, your connector will be discoverable in the new Airbyte Marketplace for everyone to benefit.

We want to hear from you!

That’s it for a quick tour of the new AI Assistant alpha release. As we continue to build out the capabilities of AI Assistant to address the challenges of the long tail of connectors, your feedback is especially important. The more endpoints we test, the more comprehensive the product will become. If you find an endpoint which doesn’t work as expected, or requires unique configuration, let us know by creating a GitHub Issue for the product team to investigate. We would also love to gather any additional feedback in the AI Connector Builder Assist (alpha) discussion channel.

We are really excited about AI Assist. Welcome to Airbyte 1.0!

Integrate with 700+ apps using Airbyte

Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.

Try it free Talk to sales

About the Author

Quinton Wall

Quinton is the Head of Developer Relations at Airbyte

Integrate with 700+ apps using Airbyte

Try Airbyte for free

Hands-on with the new AI Assistant

Integrate with 700+ apps using Airbyte

Integrate with 700+ apps using Airbyte

Related posts