Skip to content
FrameworkHunt

OpenAI Agents SDK

OpenAI · Python

Small primitive set — agents, handoffs, guardrails, sessions — with tracing built in rather than bolted on.

Assessed at openai/openai-agents-python@ebb746dc00 · reviewed 18 Aug 2026 · rubric v1.0.0 · reviewer Fizz

86/100

architecture score

Assessment

Six axes, each 0–4, weighted to 100. Every basis line below is backed by citations pinned to the reviewed commit.

Control flow & multi-agent composition

3/4 · weight 20%

Composition is handoffs plus agents-as-tools, and the documentation is explicit that a handoff is exposed to the model as a tool call — so routing is model-decided by default rather than authored. Deterministic chaining is possible and shipped as an official example. Held at 3: there is a documented, exercised path, but no authored graph and no way to replay a routing decision from a persisted point.

State, memory & durability

3/4 · weight 20%

Session is a runtime-checkable protocol with get_items, add_items, pop_item and clear_session, and SQLite and hosted OpenAI-conversation implementations ship in-tree. Held at 3 rather than 4: sessions persist conversation history, not in-flight run state — there is no checkpoint of a partially executed run, so a crash mid-run replays from conversation history rather than resuming where it stopped.

Tools, integrations & interoperability

4/4 · weight 15%

Function tools, hosted tools and MCP servers are all first-class, with schemas derived from Python signatures. The tool surface is not just present but defended: dedicated tests cover tool guardrails, name-collision policy, tool-choice reset and approval call-id reuse — enforcement at the tool boundary rather than a happy-path demo.

Reliability, safety & human controls

4/4 · weight 15%

Guardrails are a typed mechanism where a tripwire halts execution, not advisory scoring: GuardrailFunctionOutput carries tripwire_triggered and the runner raises InputGuardrailTripwireTriggered. Tests exercise sync and async guardrails on both input and output paths, and human approval is shipped as official examples including a custom-rejection variant.

Observability & evaluation

4/4 · weight 15%

Tracing is in the library rather than in a companion product. There is a tracing package with a documented processor interface and provider, so spans can be routed to a third-party backend, and an official test asserts trace and span structure for real agent runs. This is the axis where the SDK is strongest relative to peers.

Developer experience, portability & maintainability

3/4 · weight 15%

There is a canonical quickstart, a documented testing story, and a large official example tree covering basics through agent patterns. Held at 3: the primitive set is small and quick to learn, but the SDK is oriented around OpenAI's own Responses API surface, so portability across providers is a configuration exercise rather than a first-class abstraction.

Strengths

  • Tracing is part of the library with a public processor interface, so runs are observable without adopting a separate commercial platform.

    oa-ob-srcoa-ob-test

  • Guardrails halt execution by contract rather than returning an advisory score, and both the input and output paths are covered by tests.

    oa-rs-srcoa-rs-test

  • The primitive set is deliberately small — agents, handoffs, guardrails, sessions — so the whole model fits in your head on day one.

    oa-dx-docoa-cf-doc

Limitations

  • Sessions persist conversation history, not in-flight run state, so a crash mid-run resumes by replaying conversation rather than continuing from where execution stopped.

    oa-sm-srcoa-sm-sqlite

  • Routing is model-decided by default: a handoff is a tool call the model chooses, so control flow is not authored, diffable or replayable the way an explicit graph is.

    oa-cf-doc

  • The SDK is oriented around OpenAI's own API surface, so running it against other providers is configuration rather than a first-class portability abstraction.

    oa-dx-doc

Best for

  • Teams already on OpenAI models who want traced, guard-railed agents quickly without standing up separate observability infrastructure.

    oa-ob-srcoa-rs-src

  • Delegation-shaped problems — a triage agent routing to specialists — where letting the model pick the next agent is genuinely the right design.

    oa-cf-doc

Avoid if

  • You need a long-running process to resume mid-run after a crash; conversation-history sessions do not checkpoint partial execution.

    oa-sm-src

  • You need routing to be reviewable and deterministic — the default composition puts that decision inside the model.

    oa-cf-doc

Patterns worth stealing

  • Model a safety check as a tripwire that halts execution, not as a score the caller may ignore. An enforcement point that returns advice is not an enforcement point.

    oa-rs-srcoa-rs-test

  • Derive the model-facing tool schema from the typed function signature so the contract cannot drift from the implementation.

    oa-ti-src

  • Ship the rejection path, not just the approval path. Most human-in-the-loop designs are only tested on approve, and rejection is where state gets corrupted.

    oa-rs-hitl

Evidence ledger

All 20 citations behind this assessment. Each was fetched at the reviewed commit and checked for the exact text it claims.

  • official-docnativedocumented

    The docs state a handoff is surfaced to the model as a tool named transfer_to_<agent>, so which agent runs next is a model decision at inference time.

    docs/handoffs.md — Handoffs

    pinned ebb746dc00b0 · blob 093c9a1cdd9c6 · observed 18 Aug 2026

  • sourcenativeobserved

    Handoffs ship as their own package including explicit conversation-history handling across the transfer boundary.

    src/agents/handoffs/history.py

    pinned ebb746dc00b0 · blob 9b14dbc7a5739 · observed 18 Aug 2026

  • examplenativedemonstrated

    An official example chains agents in code rather than letting the model route, showing deterministic composition is available when the author wants it.

    examples/agent_patterns/deterministic.py

    pinned ebb746dc00b0 · blob 30bef35e25ddc · observed 18 Aug 2026

  • official-docnativedocumented

    The runner documentation describes how a run consumes and produces conversation items.

    docs/running_agents.md

    pinned ebb746dc00b0 · blob 3f3e64beb6159 · observed 18 Aug 2026

  • sourcenativeobserved

    Session is a runtime-checkable Protocol over conversation items. The unit of persistence is the conversation item, not a run checkpoint.

    src/agents/memory/session.py — class Session

    pinned ebb746dc00b0 · blob 26690c2c71602 · observed 18 Aug 2026

  • sourcenativeobserved

    SQLiteSession provides a working local persistence implementation of the Session protocol without extra dependencies.

    src/agents/memory/sqlite_session.py — class SQLiteSession

    pinned ebb746dc00b0 · blob 286b063f03f00 · observed 18 Aug 2026

  • official-docnativedocumented

    The tools documentation covers hosted tools and Python functions turned into tools via the function_tool decorator.

    docs/tools.md

    pinned ebb746dc00b0 · blob 9f16e93846564 · observed 18 Aug 2026

  • official-docnativedocumented

    MCP has dedicated first-party documentation, so external tool servers are an intended integration path rather than a community adapter.

    docs/mcp.md

    pinned ebb746dc00b0 · blob 3104f023eb3d8 · observed 18 Aug 2026

  • sourcenativeobserved

    Tool JSON schemas are generated from signatures and docstrings, so the typed Python contract is the source of truth for the model-facing schema.

    src/agents/function_schema.py — function_schema

    pinned ebb746dc00b0 · blob 378715dcb4e59 · observed 18 Aug 2026

  • testnativedemonstrated

    An official test pins behaviour when two tools claim the same name — a real failure mode at the tool boundary, not a happy path.

    tests/test_tool_name_collision_policy.py

    pinned ebb746dc00b0 · blob d67e7b1dbaf37 · observed 18 Aug 2026

  • official-docnativedocumented

    Guardrails are documented as running alongside agents to validate input and output.

    docs/guardrails.md

    pinned ebb746dc00b0 · blob 9b258e824d301 · observed 18 Aug 2026

  • sourcenativeobserved

    The tripwire_triggered field is documented in source as halting execution, making the guardrail an enforcement point rather than an advisory score.

    src/agents/guardrail.py — GuardrailFunctionOutput.tripwire_triggered

    pinned ebb746dc00b0 · blob 07475c8183835 · observed 18 Aug 2026

  • testnativedemonstrated

    Tests cover sync and async guardrails on both input and output paths and assert typed tripwire exceptions, evidencing enforced failure behaviour.

    tests/test_guardrails.py

    pinned ebb746dc00b0 · blob 275623ba8b348 · observed 18 Aug 2026

  • examplenativedemonstrated

    An official example covers not just approval but the rejection path, which is the branch most human-in-the-loop demos omit.

    examples/agent_patterns/human_in_the_loop_custom_rejection.py

    pinned ebb746dc00b0 · blob 597f6954275cf · observed 18 Aug 2026

  • official-docnativedocumented

    Tracing has first-party documentation describing built-in collection of agent runs.

    docs/tracing.md

    pinned ebb746dc00b0 · blob d73a644209f42 · observed 18 Aug 2026

  • sourcenativeobserved

    A documented processor interface lets spans be forwarded to any backend, so tracing is not locked to a single vendor sink.

    src/agents/tracing/processor_interface.py — class TracingProcessor

    pinned ebb746dc00b0 · blob d0f18bde38a26 · observed 18 Aug 2026

  • testnativedemonstrated

    Official tests assert the emitted trace and span shape for real runs, so instrumentation is a tested contract rather than best effort.

    tests/test_agent_tracing.py

    pinned ebb746dc00b0 · blob c094439947284 · observed 18 Aug 2026

  • official-docnativedocumented

    The quickstart documents install and a first agent in-repo, so the getting-started path is pinned to the reviewed revision.

    docs/quickstart.md

    pinned ebb746dc00b0 · blob ed3534d3dbdc8 · observed 18 Aug 2026

  • examplenativedemonstrated

    The smallest official example constructs an Agent and runs it through Runner, confirming the documented surface is the real one.

    examples/basic/hello_world.py

    pinned ebb746dc00b0 · blob 169290d6f59af · observed 18 Aug 2026

  • examplenativedemonstrated

    An official example persists a session to a file and replays it, confirming the Session protocol is usable end to end.

    examples/memory/file_session.py

    pinned ebb746dc00b0 · blob e62dbd167fa88 · observed 18 Aug 2026