Agentic Data Engineering Resources

Resource

What Is an Agentic Framework?

Learn why agentic frameworks have 3 distinct layers and how production teams fail when they treat governed data access as an afterthought.

Pedro Lopez

June 5, 2026

Last updated September 8, 2026

Summarize with AI:

An agentic framework is a software platform that provides prebuilt components, LLM integration, tool use, memory management, planning, and orchestration for developing, deploying, and managing AI agents.

The term spans three distinct layers: reasoning patterns, orchestration frameworks, and data infrastructure. Treating those layers as one category hides the source of many production failures.

Teams often blame prompts or orchestration when the real constraint is governed access to fresh, permissioned data. Framework selection shapes how agents reason and coordinate work, but in production, authentication, permissions, schema mismatch, and data freshness often determine whether the system works at all.

TL;DR

Agentic frameworks span three distinct layers: reasoning patterns, orchestration frameworks, and data infrastructure.
Most definitions focus on the first two layers, but production bottlenecks often come from governed data access.
Four core design patterns appear across frameworks: Reflection, Tool Use, Planning, and multi-agent collaboration.
Start with the simplest approach that works, then map data infrastructure requirements before choosing an orchestration framework.

What Are the Three Layers of an Agentic Framework?

An agentic framework spans three layers: reasoning patterns, orchestration frameworks, and data infrastructure. Treating framework as a single category is a common architectural mistake in agentic AI.

Layer	What It Is	What It Handles	Examples	Who Owns It
Reasoning Patterns	Architectural patterns for how an LLM reasons and acts during inference	Deciding what to think, when to act, and how to self-correct	ReAct, Reflexion, Planning, Tool Use	AI/ML researchers designing agent cognitive loops
Orchestration Frameworks	Software libraries and software development kits (SDKs) that implement reasoning patterns with state management, tool routing, and multi-agent coordination	Coordinating agent workflows in code, state persistence, conditional routing, human-in-the-loop, checkpointing	Graph-based, role-based, event-driven, or managed runtimes	AI engineers building and deploying agent applications
Data Infrastructure	The data stores, authentication systems, normalization layers, access control systems, and context delivery pipelines beneath frameworks	Getting governed, fresh, correctly permissioned enterprise data to agents across hundreds of software-as-a-service (SaaS) tools	Agent connectors, context stores, entity resolution, managed auth, MCP servers	Data and platform engineers (or purpose-built platforms like Airbyte's agentic platform)

These layers are related, but they solve different engineering problems.

Layer 1: Reasoning Patterns

Reasoning patterns specify how an agent structures its inference loop. They carry no software dependency and cannot be installed or imported.

ReAct interleaves intermediate reasoning with environment interaction. Reflexion adds a reflection phase. Plan-and-Execute uses a planning phase followed by an execution phase. Engineers can implement these patterns directly or through a chosen framework.

Layer 2: Orchestration Frameworks

Orchestration frameworks are software systems that implement one or more reasoning patterns with state machines, graphs, message-passing protocols, and developer APIs. ReAct-style reasoning, for example, can run as a state machine that tracks reasoning steps, validation, failure handling, and timeouts.

These implementations turn abstract patterns into software shaped by practical engineering decisions.

Layer 3: Data Infrastructure

Data infrastructure governs what data an agent can access and under what conditions. Tool execution is mediated by interfaces, sandboxes, and policies, and that separation supports governance and auditability in enterprise agent platforms.

Data infrastructure includes auth systems that manage credentials across SaaS tools, normalization layers that reconcile different schemas, access control systems that enforce row-level permissions, and context delivery pipelines that ensure agents work with fresh data. Orchestration frameworks decide how agents use data in a workflow, while data infrastructure determines how that data is sourced, secured, and structured.

When an enterprise agent needs context from tickets, conversations, documentation, and accounts before generating a response, the orchestration framework invokes tool functions for each source. The underlying connections remain part of the data infrastructure layer.

What Are the Four Core Design Patterns?

Four design patterns appear repeatedly across agent systems: reflection, tool use, planning, and multi-agent collaboration. They are building blocks, not mutually exclusive categories.

Reflection

Reflection adds a feedback step in which the model critiques its own output and revises it. A single-agent implementation generates output, prompts the LLM to critique it, then uses the critique to produce an improved version. A multi-agent implementation creates separate generator and critic agents.

Tool-enhanced reflection goes further by giving the LLM tools that evaluate its output, like running code through unit tests to check whether it generates correct results.

Tool Use

Tool use lets the LLM call external functions and work with live systems and information. The model decides which functions to call, such as web search, calendar access, email sending, code execution, or database queries, to carry out external actions.

With tool use, agents can access current information, interact with external systems, and take external actions while preserving the LLM's reasoning capability.

Planning

Planning is explicit task decomposition before execution. The LLM breaks a complex goal into sub-tasks rather than attempting everything in a single pass. Plans can be static, generated once, or dynamic, revised as new information emerges during execution.

Planning often combines with tool use, where individual sub-tasks require external tool calls.

Multi-Agent Collaboration

Multi-agent collaboration distributes work across agent roles with different responsibilities. A coder agent, code reviewer agent, and test writer agent can each receive specialized system prompts. A parallel research system can decompose complex queries into sub-queries handled by separate agent instances and aggregated by an orchestrator.

The framework determines how these patterns are combined, constrained, and observed in production workflows.

What Types of Agentic Frameworks Exist?

In the orchestration layer, frameworks take several architectural approaches to coordination. They differ mainly in how they manage state, route work, and structure collaboration.

Architecture Style	Primary Differentiator	Best Fit
Graph-based state machine	Explicit state management with checkpoints and conditional routing	Complex, stateful workflows requiring fine-grained control and human-in-the-loop
Role-based multi-agent	Manager-worker pattern mapping to real team structures	Workflows that mirror organizational roles
Event-driven multi-agent	Distributed environments with conversational collaboration patterns	Research-style multi-agent systems where agents negotiate, critique, and collaborate dynamically
Managed runtime	Built-in tool use and handoffs for rapid prototyping	Lightweight multi-agent routing with triage patterns
Minimalist code-based	Simple tool execution with minimal abstraction	Lightweight agent experiments requiring code execution
Modular hierarchical	Modular coordination inside a broader cloud ecosystem	Teams building within an existing platform stack

Framework, Runtime, And Harness

Framework, runtime, and harness describe different categories within the orchestration layer. Frameworks provide composable primitives and abstractions for tools, vector stores, and memory. Runtimes address production infrastructure concerns such as durable execution, streaming, human-in-the-loop support, and thread-level persistence.

Harnesses are batteries-included. They provide default prompts, built-in planning tools, and opinionated tool handling. That distinction explains why some products feel lightweight while others feel more operationally heavy.

Comparing Orchestration Approaches

Orchestration approaches differ mainly in how much control they expose and how much they abstract away. Some runtimes use a directed graph with typed state, nodes, edges, and conditional routing. This gives engineers explicit control over every state transition.

Harnesses get a team to a working agent quickly but constrain what can change without fighting defaults. Runtimes require more upfront investment but give greater control over production infrastructure. Match the floor and ceiling to the timeline: prototype in a harness, produce in a runtime.

How Does MCP Fit Into the Agentic Framework Stack?

MCP fits into the stack as a protocol layer between agents and external tools or data sources. MCP is an open protocol built on JSON-RPC 2.0 that standardizes how AI applications communicate with external data sources and tools.

Before MCP, each of M AI applications needed a bespoke integration for every one of N data sources: M × N custom implementations. MCP reduces this to M + N. Each AI application implements MCP clients, each data source implements an MCP server, and any compliant client can communicate with any compliant server through standardized capability negotiation during initialization.

MCP standardizes discovery, capability negotiation, and invocation. Authentication credential management, schema normalization, row-level access controls, entity resolution, and data freshness monitoring remain in the data infrastructure beneath it.

The orchestration framework sits above MCP. The data infrastructure behind MCP servers operates below it.

What Do Frameworks Miss in Production?

Frameworks often miss the implementation work that production systems depend on, especially data integration, governance, and monitoring. Context engineering, the discipline of preparing and delivering the right information so agents can reason accurately, depends on infrastructure beneath the framework.

Authentication And Credential Management Across Sources

Authentication and credential management become difficult as agents connect to more sources. OAuth makes the scaling problem clear. Connecting an AI agent to one SaaS API via Open Authorization (OAuth) 2.0 takes an afternoon. Connecting to many takes much longer. Every provider implements OAuth differently, and integration code written for one source does not transfer cleanly to another.

That identity layer exists outside the orchestration framework.

Schema Normalization And Entity Resolution

Schema normalization and entity resolution keep cross-system context consistent. Shared context breaks when the same business concept exists differently across enterprise systems. A Date field in one system may include a timestamp while the corresponding field in another does not. A customer_id in one system may be a string while the same field in a data warehouse is an integer.

Identity mismatch adds another problem: the same customer appears under different names and identifiers across systems with no shared key.

Access Controls And Data Freshness

Access controls and data freshness determine whether an agent can use enterprise data safely and accurately. Enterprise data carries access controls at the source. When an agent retrieves data into a vector store or context store, those source-system controls are not automatically carried forward.

Permissions must be enforced before retrieval. Data freshness creates a parallel challenge. When a customer queries a support agent about a recent order, that agent needs data from the last few minutes, not the last scheduled batch run. Orchestration frameworks do not track pre-fetching with freshness time-to-live values, Change Data Capture (CDC)-based incremental sync, or freshness metadata. When retrieval, permissions, freshness, or context assembly fail, the model responds confidently with incomplete information.

What Should You Build First To Get AI Agents Working In Production?

Choose an agentic framework by starting with the simplest approach that works, then evaluating predictability, control, and data requirements. The predictability-agency tradeoff determines whether a team needs predefined workflows or dynamic agent behavior. Workflows rely on predefined code paths for well-defined tasks, while agents allow models to direct their own processes and tool usage when greater flexibility is needed. The most common framework selection error is choosing a fully autonomous architecture for a deterministic workflow use case. That introduces unnecessary non-determinism, debugging complexity, and latency with no corresponding benefit.

Evaluating floor and ceiling means balancing speed to prototype against production control. High-floor frameworks, harnesses with built-in prompts, default tools, and opinionated patterns, get a team to a working agent fastest. The ceiling arrives when those built-in assumptions conflict with production requirements, such as custom routing logic, specific state management patterns, or integration with existing infrastructure. Low-floor frameworks require understanding graph-based state machines and explicit control flow before writing the first agent. The payoff is full control over checkpointing, conditional routing, human-in-the-loop interrupts, and fault-tolerant resumption.

Build the orchestration layer and the data infrastructure layer from the start, and treat them as different engineering problems. Before evaluating any orchestration framework, map which sources agents need, what authentication models those sources use, what permission structures must be enforced, and what freshness requirements apply. Data infrastructure handles governed access to enterprise data: authentication, schema normalization, entity resolution, access controls, and freshness. Teams that treat these as one problem debug data infrastructure failures with orchestration tools, then waste time changing prompts, routing logic, or frameworks while the actual failure sits in the retrieval pipeline.

Airbyte's agentic platform operates in the data infrastructure layer beneath orchestration frameworks. It provides 50+ agent connectors built on Airbyte's broader 700+ replication connector ecosystem, along with governed SaaS access, Managed Auth across sources with different auth models, and row-level and user-level access controls. It works with orchestration frameworks through Agent MCP and the Agent SDK.

Talk to sales to see how Airbyte's agentic platform supports production AI agents with permission-aware data access, or try Airbyte's agentic platform today.

Frequently Asked Questions

What is the difference between an agentic framework and an agent framework?

Most sources use these terms interchangeably. The meaningful distinction here is architectural: reasoning patterns, orchestration, and data infrastructure are separate layers, even when people use framework as a catch-all term.

Is LangChain an agentic framework?

In the orchestration sense, yes. Framework can also describe adjacent layers, but in the orchestration layer, frameworks provide abstractions and integrations for building agent applications.

Do agentic frameworks handle enterprise data access?

No. Orchestration frameworks handle reasoning, state management, and tool routing. Governed access to enterprise SaaS tools, including authentication management, schema normalization, row-level security, and entity resolution across sources, sits in the data infrastructure layer beneath frameworks.

What role does MCP play in agentic frameworks?

Model Context Protocol (MCP) standardizes how agents communicate with external tools and data sources, replacing M×N custom integrations with a common standard. It solves the protocol problem, but not the data infrastructure behind it.