This week we hosted a webinar, Building AI Apps with (Customer) Context where we shared learning and product updates. A key takeaway in our learnings that we kept hearing from customers and developers building AI apps was a consistent theme -- not only does every AI app need data, but as more and more developers are building apps leveraging LLMs, they need a way to easily allows customers to add their data for context.
If you missed the webinar, the recording is below.
Blueprint of Contextual AI Apps This trend of apps that leverage LLMs and customer contextual data requires a different architectural approach to development. On the backend, you need some form of data hub model. This hub should make it easy to enable customer onboarding without you having to collect credentials or data directly. Developers need to offer a secure self-service approach through some form of API or onboarding app to streamline the process.
As an administrator or app developer, you should be able to manage these connections and more importantly govern what data is shared with the LLM to ensure the right privacy controls. And, finally, when ready, move the customer data into some form of staging store before being processed. App developers want control on the final context store that they use, and want a standard staging store to give them flexibly to process the data into any system they choose. We’ve found Amazon S3 buckets are ideal for a staging store. S3 is very flexible. It can easily handle structured and unstructured data, and S3 can write to data lakes or warehouses once data syncs are complete.
A typical implementation pattern for contextual AI apps
Contextual AI Data Pipelines On the front end, apps are being built in modern frameworks such as React, Node, Electron, etc. They utilize LLMs and natural language prompts, enriched with contextual data from vector stores to perform logic, which used to be hard coded by developers. Eliminating this custom logic and allowing AI/LLMs to derive insight from customer context results in lightweight apps with significantly less code required to build and deploy. This allows app developers and product teams to ship more frequently.
The ability to ship fast, with confidence that the LLM is going to return accurate results, is only made possible through the processing of the right customer data from the staging store. Once it is moved into an iceberg or vector store, app developers can build with confidence without requiring coding to prevent data leaks and maintain strong data governance practices. We see this as an evolution of the traditional ELT data pipelines.
The evolution of data pipelines for AI apps
Airbyte Embedded This trend towards contextual apps has also seen a shift in the persona of people building data pipelines. Historically, a data engineering team would be responsible for moving data between systems and preparing it for consumption. Now, with LLMs and AI, app development teams can load this data into their context store and rely on providers such as OpenAI , Google , and Anthrophic to provide models that interpret prompts. Further, we are also seeing an increasing number of AI agents and bots consuming data pipelines to feed downstream workflows or apps, therefor only increasing the need to connect customer data to feed relevant context to AI apps.
This is why we built Airbyte Embedded : app developers and agents can quickly onboard customers to feed contextual stores with relevant data for consumption by LLMs, and deliver accurate insights. Airbyte Embedded delivers a data hub where app developers can expose APIs and widgets to onboard their customers, and centrally manage where they want to push the cleansed data into a staging store for AI processing. Airbyte Embedded leverages all of the benefits of the Airbyte Platform such as scalability, security, and access to hundreds of connectors including all major business apps, and full support for structured and unstructured data.
What’s Next? AI is evolving rapidly. We are kicking off a monthly AI webinar series with new episodes dropping the first Tuesday of every month. Make sure you check out the events page , join the AI community slack channel , and subscribe to YouTube to be notify of topics and agendas.
