Observability engineer for AI agent platforms

Intellias; United States; Remote; USD 130,000 - 190,000; Full time

Job summary

Intellias is seeking an Observability Engineer in the United States to build behavioral monitoring and auditability for an enterprise AI Agent Platform. You will transform agent telemetry, anomalies, safety signals, and governance events into actionable dashboards and alerts to monitor production activity for operational and audit purposes.

Responsibilities include designing dashboards with New Relic/NRQL, defining behavioral signals, and building auditability views while keeping technical

Qualifications

• Experience with New Relic and NRQL for dashboards and alert conditions.
• Ability to design behavioral monitoring signals such as anomalies, drift, safety signals, and policy denials.
• Experience building auditability views surfacing governance evidence for consumption.
• Observability domain segregation between technical monitoring and audit/security evidence.
• Proficiency in configuring, tuning, and validating production telemetry alerts.
• Python scripting for metrics emission and observability validation.
• Experience with environments requiring formal audit/compliance evidence.

Responsibilities

• Design and implement behavioral monitoring dashboards and alerts.
• Define and monitor behavioral signals including anomalies and drift.
• Build auditability views surfacing governance and compliance evidence.
• Maintain clear segregation between observability and audit data stores.
• Configure and validate alerts based on production telemetry.
• Develop Python-based utilities for metrics emission and validation.
• Incorporate security and governance signals into monitoring.

Skills

New Relic

NRQL

Anomaly monitoring

Alert tuning

Python

Tools

CloudWatch

SIEM

Job description

Описание

We are looking for an Observability Engineer to build behavioral monitoring and auditability capabilities for an enterprise AI Agent Platform being developed from the ground up. The role will focus on transforming agent telemetry, behavioral anomalies, safety signals, and governance events into actionable dashboards and alerts. You will work at the intersection of observability, AI agent behavior, and governance , helping ensure that production agent activity can be effectively monitored, validated, and surfaced for both operational and audit purposes.

Задачи

• Design and implement behavioral monitoring dashboards and alerts using New Relic and NRQL.
• Define and monitor behavioral signals including anomalies, drift, safety events, and policy denials.
• Build auditability views that surface relevant governance and compliance evidence.
• Maintain clear segregation between technical observability and audit/security evidence across monitoring systems.
• Configure, tune, and validate alerts based on production agent telemetry.
• Verify dashboard accuracy and telemetry consistency to ensure monitoring data is reliable.
• Develop Python-based utilities for metrics emission, validation, and observability testing.
• Implement monitoring patterns for AI agent anomaly and behavioral drift detection.
• Validate that sensitive information, including PII, is appropriately handled in telemetry and traces.
• Incorporate security and governance signals, such as policy decisions and access denials, into behavioral monitoring.
• Collaborate with security and governance stakeholders to support enterprise compliance and audit requirements.

Требования

• New Relic and NRQL for behavioral dashboards and alert conditions
• Behavioral monitoring signals (anomaly detection, drift, safety signals, policy denials)
• Auditability views surfacing governance evidence for consumption
• Observability domain segregation (Technical in CloudWatch/New Relic vs Audit in dedicated evidence/SIEM stores)
• Alert validation and dashboard data-accuracy verification
• Python (metrics emission and validation)
• 4+ years of observability engineering
• Behavioral / anomaly monitoring and alert tuning on production telemetry
• Working in environments with formal audit/compliance evidence requirements
• Будет плюсом: Anomaly/drift detection techniques for agent telemetry; PII redaction validation in traces surfaced through dashboards; Cedar policy decisions / Contract-Policy Engine denials as monitoring inputs; Enterprise security/compliance standards mapping

Условия

That’s why we offer a range of benefits that support your well-being and charge your professional growth.

We are committed to fostering equity, diversity, and inclusion as an equal opportunity employer.