Bedrock AgentCore Runtime Latency: What Actually Matters (Multi-Language Benchmarking)

Bedrock AgentCore Runtime Latency: What Actually Matters (Multi-Language Benchmarking)Sibonelo N.

This article presents comprehensive latency benchmarking results for AWS Bedrock AgentCore Runtime...

This article presents comprehensive latency benchmarking results for AWS Bedrock AgentCore Runtime across four programming languages (Go, Node.js, Java, Python), three framework options (raw boto3, Strands SDK, LangChain), multiple network configurations, and caller locations. The goal is to provide data-driven guidance on what design choices affect latency and what does not.

Background

AgentCore Runtime is the compute layer for hosting agents and tools in Bedrock AgentCore. Customers frequently ask:

  • Which language should I write my runtime container in?
  • Does PUBLIC vs VPC network mode affect performance?
  • Should I use Strands SDK, LangChain, or raw SDK calls?
  • Does the client SDK language matter?
  • Where should my caller run for best latency?

This article answers each question with measured data from a controlled test environment.

Test Environment

  • Region: us-east-1
  • Protocol: HTTP (port 8080)
  • Model (for real agent tests): Claude Haiku 4.5 via cross-region inference profile
  • VPC: Private subnets with com.amazonaws.us-east-1.bedrock-agentcore interface endpoint
  • Caller locations tested: In-VPC Lambda (private subnet) and external (boto3 with connection reuse)
  • All container images built for linux/arm64 (AgentCore Firecracker VMs are ARM64)

Finding 1: Container language does not affect warm latency

Four echo runtimes were deployed, each implementing the same trivial HTTP server (receive JSON, return it with metadata). All use the HTTP protocol on port 8080.

4-Language Latency Comparison

Language Image Size In-VPC Median External Mean
Go 22 MB 85ms 968ms
Node.js 229 MB 80ms 950ms
Java (Corretto 21) 494 MB 77ms 945ms
Python (3.12 + gunicorn) 52 MB 84ms 937ms

All four are within 10ms of each other on warm in-VPC calls. The platform routing floor dominates. Language choice should be driven by team expertise and ecosystem, not latency.

Finding 2: Caller location is the single biggest factor

The same runtime invoked from three different locations:

VPC vs External

Caller Location Mean Latency What Adds Cost
In-VPC Lambda (VPC endpoint) 80-90ms Platform routing only
External (boto3, connection reuse) 937-968ms Network RTT to region
External (AWS CLI, no reuse) 1,750-1,830ms Network RTT + TLS + SDK init

Placing callers in the same VPC with a bedrock-agentcore interface endpoint provides 5-10x latency improvement. This is the single most impactful optimization.

Finding 3: PUBLIC vs VPC-MODE runtime has no latency impact

VPC-MODE places the container's ENI in your private subnet. PUBLIC mode runs on platform-managed infrastructure.

PUBLIC vs VPC-MODE

Language PUBLIC Median VPC-MODE Median
Go 87ms 85ms
Node.js 80ms 84ms
Java 77ms 85ms
Python 84ms 87ms

Identical once warm. VPC-MODE has higher cold-start spikes (ENI attachment takes 500-2500ms on first invocation) and takes longer to provision (~2.5 minutes vs 5 seconds for PUBLIC). Use VPC-MODE for security and compliance (network isolation, access to private resources), not for performance.

The 3-way comparison (External vs In-VPC PUBLIC vs In-VPC VPC-MODE) shows the full picture:

3-Way Comparison

Finding 4: Client SDK language does not affect warm latency

Three Lambda functions were deployed, each written in its native language, calling its corresponding VPC-mode runtime:

Lambda Language SDK Warm Median Cold Start
Go aws-sdk-go-v2 98ms 904ms
Node.js @aws-sdk/client-bedrock-agentcore 115ms 873ms
Python boto3 96ms 2,631ms

Warm latency is identical. Cold start differs: Go and Node.js initialize in under 1 second, Python (boto3) takes 2.6 seconds due to interpreter and credential chain resolution overhead.

Finding 5: Framework choice adds zero measurable overhead

Three variants of the same SageMaker SME agent were deployed, all using Claude Haiku 4.5 with the same system prompt and question:

Framework Comparison

Framework Framework Overhead Image Size
Raw boto3 (invoke_model) 0.2ms 180 MB
Strands Agents SDK 0.2ms 347 MB
LangChain (ChatBedrock) 0.1ms 438 MB

Framework processing overhead is under 0.3ms for all three. The model inference call (5-26 seconds depending on output length) accounts for over 95% of end-to-end latency. Choose frameworks based on developer productivity:

  • Raw boto3: Lightest image, full control, manual orchestration
  • Strands SDK: Built-in tool registration, memory management, conversation handling
  • LangChain: Rich ecosystem of chains, RAG integrations, prompt templates

Finding 6: Connection pool size matters only at high concurrency

Connection pool stress tests with boto3 (urllib3 backend) at varying pool sizes:

Connection Pool Stress

Concurrency Pool=2 Mean Pool=10 Mean (default) Pool=50 Mean
1 thread 84ms 81ms 81ms
10 threads 153ms 139ms 132ms
50 threads 204ms 155ms 113ms
100 threads 254ms 138ms 142ms

No errors at any concurrency level. The default pool size (10) is adequate for most workloads.

Finding 7: Model inference dominates real agent latency

For a real SageMaker SME agent answering technical questions:

Echo vs Real Agent

Component Latency Percentage of E2E
Container processing (framework + parsing) less than 1ms less than 0.01%
Platform routing (SigV4, DDB, Firecracker) 95-375ms 3-5%
Model inference (Claude Haiku 4.5) 5,700-26,000ms 95%+

The model call scales with output token count (approximately 5ms per output token for Haiku 4.5).

What actually matters (ranked)

Rank Factor Impact
1 Caller location (in-VPC vs external) 5-10x
2 Model choice (for real agents) Determines total time
3 SDK cold start (Go/Node vs Python) 3x on first call only
4 Connection pool size Mild at high concurrency
5 Container language Zero impact
6 Runtime network mode (PUBLIC vs VPC) Zero impact
7 Client SDK language (warm) Zero impact

Optimization Recommendations

  1. Place callers in-VPC with a bedrock-agentcore interface endpoint for 5-10x latency reduction.

  2. Reuse SDK clients across invocations. Never create a new client per call. In Lambda, initialize at module scope outside the handler.

  3. Do not optimize container language for latency. Pick what your team knows. The platform routing floor makes it invisible.

  4. Use VPC-MODE for security, not performance. Identical once warm.

  5. Consolidate multi-tool agents into fewer runtimes. Each sequential InvokeAgentRuntime call adds platform routing overhead.

  6. Use streaming for better perceived performance. Model inference is the bottleneck.

Code Configuration Support

Language Code Config (zip to S3) Container Config (Docker)
Python Yes (3.10 through 3.14) Yes
Node.js Yes (NODE_22) Yes
Go No Yes
Java No Yes

Architecture 

Caller (Lambda / ECS / App)
  -> [VPC Endpoint or Public Endpoint]
    -> AgentCore Platform (SigV4 validation, session lookup, routing)
      -> Container (PUBLIC: platform VM | VPC: ENI in your subnet)
        -> [Optional] Bedrock InvokeModel (model inference)
Enter fullscreen mode Exit fullscreen mode

Source Code

All source code, raw data, Lambda functions, and charts are available here:

GitHub logo Neloh / agentcore-latency-benchmarks

AgentCore Runtime latency benchmarking: Go, Node.js, Java, Python across PUBLIC/VPC modes

AgentCore Runtime Latency Benchmarks

Comprehensive latency benchmarking of AWS Bedrock AgentCore Runtime across 4 programming languages (Go, Node.js, Java, Python), multiple network configurations, and caller locations.

Key Findings






































Factor Impact on Warm Latency

Caller location (in-VPC vs external)
5-10x - the #1 factor
Model choice (for real agents) Determines total time (95%+)
SDK cold start (Go/Node vs Python) 3x on first call only
Connection pool size Mild at high concurrency
Container language Zero impact
Runtime network mode (PUBLIC vs VPC) Zero impact
Client SDK language (warm) Zero impact

Test Results

1. Four-Language Echo Comparison (In-VPC)

All languages perform identically once warm. The platform routing floor (~80ms in-VPC) dominates.

4-Language Latency Comparison

2. In-VPC vs External Caller

Calling from within the VPC (via bedrock-agentcore interface endpoint) is 5-10x faster than calling over the public internet.

VPC vs External

3. Full Comparison: Echo vs Real Agent (SME + Claude Haiku 4.5)

Model inference (Bedrock) accounts for 95%+ of…


View on GitHub


References