OpenAI on AWS: When to Use the API, Bedrock, or Managed Agents

Chuck Hernandez

May 5, 2026

•

9 min read

OpenAI coming to AWS is easy to misread.

It does not mean every enterprise AI workload should move to AWS. It is not that the API is suddenly the wrong path. It means architecture teams now have a sharper set of choices: direct API, OpenAI models on Bedrock, or a managed agent runtime inside AWS.

That is where a lot of architecture conversations will get sharper.

The decision is no longer just "which model should we use?" It is: should this workflow call the OpenAI API directly, use OpenAI models through Amazon Bedrock, or run as a managed agent inside AWS?

Those are different choices. They imply different ownership, governance, runtime, and engineering trade-offs.

Start with the workflow

The wrong move is to start with the platform.

Start with the shape of the work:

Is this a single request/response interaction, or a multi-step workflow?
Does the application already own state, retries, permissions, and logging?
Does the workflow need to use tools or take action across systems?
Does it need to pause for approval and resume later?
Does the data already live inside AWS?
Does the enterprise need IAM, PrivateLink, CloudTrail, encryption, guardrails, or AWS procurement paths?

Those answers matter more than the announcement itself.

OpenAI’s April 28 announcement introduced three limited-preview paths on AWS: OpenAI models on Amazon Bedrock, Codex on Amazon Bedrock, and Amazon Bedrock Managed Agents powered by OpenAI. AWS described OpenAI models on Bedrock as inheriting AWS controls such as IAM, PrivateLink, guardrails, encryption, and CloudTrail logging.

That does not make the choice automatic. It makes the decision more concrete.

Path 1: Use the OpenAI API when your app owns the workflow

The direct API path is still the cleanest answer for many use cases.

Use it when you are embedding intelligence into an application, product feature, internal tool, or backend service and your system already owns the workflow around the model.

Examples:

summarizing support tickets inside an existing customer service application
extracting structured fields from contracts in a document workflow
generating product recommendations inside a commerce app
drafting first-pass emails inside a CRM workflow
classifying inbound requests before routing them to a human

In this model, the application owns the important production behavior: database writes, permissions, queues, retries, user experience, audit logs, exception handling, and business rules.

The OpenAI API provides model intelligence. Your application provides the operating system around it.

That is the right trade-off when you want maximum control and the workflow is already well-structured. The API is not "less enterprise." It is just lower-level. You own more of the system.

OpenAI’s Responses API gives teams agentic primitives, not a full operating model. It supports tools, multimodal inputs, conversation state, function calling, remote MCP, multi-turn interactions, and stateful context when needed.

But your application still owns the surrounding architecture: durable workflow state, permission boundaries, approval gates, retries, recovery, evaluation, observability, deployment, and cost

That means you need to build or integrate:

state persistence
tool execution
approval gates
retries and recovery
observability
evaluation
permissions
deployment
cost controls

For many teams, that is exactly what they want. For others, that is the burden they are trying to avoid.

Path 2: Use OpenAI models on Bedrock when AWS controls matter

Amazon Bedrock is the broader AWS platform for working with foundation models. OpenAI models on Bedrock means AWS customers can access OpenAI models through the AWS environment and controls they already use.

This path makes sense when the workflow is still mostly model-driven, but enterprise constraints make direct API access harder.

Use this path when:

the data already lives in AWS
the security team wants AWS-native identity and access controls
procurement wants usage aligned to AWS cloud commitments
network architecture requires PrivateLink or AWS-controlled paths
platform teams already standardize on Bedrock for model access
centralized logging through CloudTrail matters

This is not necessarily about building a full agent. It is about placing OpenAI model access inside the enterprise's existing cloud operating model.

A good example is an internal knowledge workflow for an AWS-heavy company. The workflow may pull from S3, Redshift, DynamoDB, or an internal document pipeline. The application still owns orchestration, but the model access, logging, identity, and procurement story sit more naturally inside AWS.

That can remove a lot of friction.

It can also create a false sense of completeness. Bedrock gives you the AWS-native control plane around model access. It does not automatically define your business workflow, approval logic, evaluation criteria, fallback behavior, or operating owner.

Those still have to be designed.

Path 3: Use Managed Agents when the workflow needs state and action

Amazon Bedrock Managed Agents, powered by OpenAI, is different from model access.

This is the path for workflows that do not look like one prompt and one answer. OpenAI describes Bedrock Managed Agents as a way to build agents that maintain context, execute multi-step workflows, use tools, and take action across complex business processes.

AWS adds more technical detail: managed agents have their own identity, log each action, run in the customer's environment, and use Amazon Bedrock AgentCore as the default compute environment.

That is the line where the architecture changes.

You are no longer just asking a model to produce an answer. You are asking an agent to operate across steps.

Examples:

a finance workflow that reviews an invoice, checks purchase order history, asks for approval, updates an ERP queue, and logs the exception
a customer support workflow that reads ticket history, checks entitlement, drafts a response, escalates edge cases, and updates the case record
an IT workflow that diagnoses an issue, runs approved checks, opens a remediation ticket, and waits for human confirmation before action
a sales operations workflow that reads account context, prepares follow-up, checks CRM state, and routes next steps

These workflows need state. They need tool access. They need identity. They need auditability. They often need to pause and resume.

That is why the stateful agent work between OpenAI and AWS matters. In February, OpenAI described the Stateful Runtime Environment for Agents in Amazon Bedrock as a way to handle multi-step work that depends on prior actions, multiple tool outputs, approvals, system state, and secure guardrails. AWS has also added AgentCore capabilities around managed harnesses, microVM sessions, filesystem persistence, stateful MCP, progress notifications, and resumable work.

That is the technical value of a managed agent runtime. It reduces the amount of scaffolding a team has to build before an agent can survive real work.

It does not remove implementation discipline. It moves the focus from basic runtime plumbing to workflow design, permission boundaries, tool contracts, quality checks, and operating ownership.

Do not confuse managed runtime with managed correctness

A managed agent runtime can help with state, identity, tool use, and execution. It does not define what “good” means for the business. For customer-facing agents, teams still need scenario-based evals, golden datasets, brand rules, failure-mode tests, and circuit breakers for high-impact actions. The runtime can make the work observable and recoverable. It cannot define the customer experience standard for you.

Path	Use When	Watch Out For
Direct OpenAI API	Your application owns orchestration, state, permissions, UX, logging, retries, and business rules.	You must build or integrate the production scaffolding yourself.
OpenAI Models on Bedrock	You want OpenAI model access inside AWS controls, procurement, identity, logging, and network patterns.	AWS model access does not automatically solve workflow design or agent runtime needs.
Bedrock Managed Agents	The workflow needs state, tools, actions, identity, logs, approvals, resumability, and governance.	Do not use a managed agent runtime for simple request/response work.

The simplest rule is this:

Use the API when the application is the system of control.

Use Bedrock when AWS needs to be the model access and governance layer.

Use Managed Agents when state, tools, identity, approvals, logs, resumability, and governance are part of the problem

Do not default to the heaviest path

The managed-agent path is powerful, but it should not become the default answer for every AI use case.

If all you need is classification, extraction, summarization, drafting, or a model call inside an existing workflow, the API may be cleaner. If your main issue is security approval, procurement, AWS-native networking, or centralized logging, OpenAI models on Bedrock may be the right path.

Managed Agents become more compelling when the workflow has multiple steps, multiple tools, intermediate state, approvals, and actions that need to be explained later.

That is the practical distinction.

A lot of AI architecture mistakes come from skipping this step. Teams either overbuild a full agent platform for a simple model call, or they underbuild a fragile API chain for a workflow that clearly needs state, permissions, audit, and recovery.

Both are expensive.

The better move is to classify the workflow before choosing the implementation path.The wrong architecture usually starts when teams pick a platform before they understand who needs to own state, tools, approvals, evaluation, recovery, and control.