Principal AI Operations Engineer (Remote)

Inspira Financial; Oak Brook (IL); Remote; USD 180,000 - 240,000; Full time

Job summary

Inspira Financial seeks a Principal AI Operations Engineer to define instrumenting and measuring AI systems across the enterprise. You will drive adoption of telemetry, observability, and governance standards, working with vendor and in-house AI capabilities to ensure audit-ready evidence and controlled operation.

The role focuses on building enterprise-wide telemetry, evaluation frameworks, drift detection, guardrails, and incident management, partnering with Risk, Compliance, Legal, and

Responsibilities

• Define the enterprise telemetry standard for AI systems, including traces, prompts, tool calls, retrieval, tokens, latency, and cost.
• Build the logging and observability layer to provide a single view of AI activity across custom apps, agent platforms, and vendor-embedded AI.
• Integrate AI telemetry with enterprise logging, APM, security monitoring, alerting, and incident response platforms for visibility and audit retention.
• Establish retention, redaction, masking, and access patterns for AI conversation and trace data to meet privacy and legal obligations.
• Publish reference implementations and starter kits for teams adopting the telemetry standard.
• Design the evaluation framework for AI systems with golden datasets, offline evaluation, and CI regression gates.
• Define drift criteria and build detection for output quality, retrieval decay, input distribution shift, and silent model updates.
• Build runtime guardrails for agents, including tool access, spend limits, escalation paths, and human-in-the-loop triggers.
• Establish the AI incident model with severity taxonomy, detection, alerting, rollback, and postmortem processes.
• Translate NIST AI RMF, ISO/IEC 42001, and related obligations into operational controls.
• Build the evidence layer for internal audit, external examination, and client due diligence.
• Integrate AI operational data with governance, risk, and compliance platforms for continuous control monitoring.
• Maintain the operational AI inventory with owners, risk classifications, and control coverage.
• Define the assurance pattern for vendor AI, including telemetry attestations and monitoring.
• Build metrics linking AI usage, quality, cost, and risk to business outcomes for leadership reporting.
• Own token and platform cost visibility by team and use case to drive optimization.
• Run working sessions, document patterns, and coach teams through their first fully instrumented AI deployment.
• Collaborate with Risk, Compliance, Legal, Privacy, and Security to incorporate risk-triad feedback.
• Perform other duties as assigned.

Job description

The Principal AI Operations Engineer reports to the Sr. Director, AI Center of Excellence, in the Technology Department. This role defines how AI systems at Inspira Financial are instrumented, measured, monitored, and evidenced, and drives adoption of those standards across the enterprise, spanning both vendor-provided and internally built AI capabilities. Primary duties include building the enterprise telemetry and observability standard for AI systems; designing evaluation, drift detection, and agent guardrail frameworks; translating AI risk and compliance frameworks into operational controls and audit evidence; maintaining the enterprise AI system inventory; and teaching other engineering teams to adopt these standards independently.

Duties & Responsibilities:

1. Define the enterprise telemetry standard for AI systems, covering traces, prompts, tool calls, retrieval, tokens, latency, and cost, built on a vendor-neutral schema such as OpenTelemetry GenAI semantic conventions.
2. Build the logging and observability layer that provides the AI Center of Excellence, business units, and the AI Governance Committee with a single view of AI activity across custom applications, agent platforms, and vendor-embedded AI.
3. Integrate AI telemetry with enterprise log aggregation, application performance monitoring, security monitoring, alerting, and incident response platforms to support operational visibility, investigation, and audit retention.
4. Establish retention, redaction, masking, and access patterns for AI conversation and trace data that satisfy privacy, records retention, and legal hold obligations.
5. Publish reference implementations and starter kits so teams adopting the telemetry standard inherit a working path.
6. Design the evaluation framework for AI systems, including golden datasets, offline evaluation harnesses, automated and LLM-as-judge scoring, and regression gates in CI.
7. Define drift criteria for each class of AI system and build detection for output quality degradation, retrieval decay, input distribution shift, and silent model updates from providers or vendors.
8. Build runtime guardrails and boundary enforcement for agents, including tool and action authorization, scope and spend limits, escalation paths, human-in-the-loop triggers, and hard-stop conditions.
9. Establish the AI incident model, including severity taxonomy, detection, alerting, rollback, postmortem process, and conversion of incidents into new test cases.
10. Translate NIST AI RMF, ISO/IEC 42001, model risk management practice, and applicable privacy and recordkeeping obligations into operational controls engineers can implement.
11. Build the evidence layer that automates capture of artifacts needed for internal audit, external examination, and client due diligence.
12. Integrate AI operational data with governance, risk, and compliance platforms to support continuous control monitoring, automated evidence collection, risk reporting, and audit readiness.
13. Maintain the operational AI inventory across custom builds, agents, integrations, and vendor AI features, including an owner, risk classification, and control coverage for each entry.
14. Define the assurance pattern for vendor AI, including required telemetry, attestations, detection of vendor model changes, and compensating monitoring.
15. Build metric frameworks connecting AI usage, quality, cost, and risk posture to business outcomes, and report these to CoE leadership, business unit leaders, and the AI Governance Committee.
16. Own token and platform cost visibility and attribution by team and use case and lead resulting optimization work.
17. Run working sessions, document patterns, review designs, and coach teams through their first fully instrumented AI deployment.
18. Partner with Risk, Compliance, Legal, Privacy, and Security to incorporate feedback from the risk triad and ensure adequate detection, response, and mitigation measures are in place.
19. Perform other duties as assigned.