AgentOps is the operational discipline for deploying and managing AI agents in production, analogous to how MLOps extends DevOps. The post outlines a four-pillar framework implemented with Amazon Bedrock AgentCore: Governance & Security (multi-account architecture, IAM, Cedar policies, tool governance), Build & Operations (versioned agent/tool/memory artifacts with CI/CD pipelines), Evaluation (four levels: tool, conversation turn, session outcome, and system metrics, in both on-demand and online modes), and Observability (four telemetry layers using OpenTelemetry, CloudWatch, and AgentCore dashboards). A reference architecture maps these pillars across the full DevOps lifecycle from planning through production monitoring, with guidance on multi-account AWS setup, AgentCore Runtime versioning, agent registry, memory governance, and integration with third-party observability tools.
