AI Observability Engineer

Galent; Plano (TX); In loco; USD 90.000 - 130.000; Tempo pieno

Descrizione del lavoro

Galent in Plano, TX seeks an experienced AI Observability Engineer to design, implement, and manage enterprise-scale observability for ML, GenAI, and LLM platforms. You will establish end-to-end visibility into AI workloads, monitor model performance, latency, cost, data quality, drift, and user experience.

The role requires building dashboards with Grafana/Tableau, creating metrics with Prometheus/PromQL, and collaborating with AI/ML, Platform, and SRE teams to improve production reliability.

Competenze

• Proven years of experience in observability, monitoring, analytics, or telemetry engineering.
• Proven years of experience creating operational dashboards and visualizations.
• Experience supporting production-scale applications in cloud or distributed environments.

Mansioni

• Build and maintain observability solutions for AI/ML applications and platform services.
• Develop dashboards and alerts using Grafana and Tableau.
• Monitor AI service health, latency, throughput, error rates, and reliability metrics.
• Create and maintain metrics using Prometheus and PromQL.
• Analyze operational and time-series data to identify trends and performance issues.
• Define and monitor SLIs, SLOs, and service health indicators.
• Partner with AI/ML, Platform, and SRE teams to improve production reliability.
• Support incident troubleshooting and root cause analysis

Conoscenze

Observability engineering

Dashboard development

Telemetry analysis

Strumenti

Prometheus

Grafana

Thanos

OpenTelemetry

Jira

Descrizione del lavoro

We are seeking an experienced AI Observability Engineer to design, implement, and manage enterprise-scale observability solutions for Machine Learning (ML), Generative AI, and Large Language Model (LLM) platforms. This role will be responsible for establishing end-to-end visibility into AI workloads by monitoring model performance, inference reliability, system health, latency, cost, data quality, drift, and user experience.

The ideal candidate will have strong expertise in modern observability ecosystems including Prometheus, Grafana, Thanos, and data visualization technologies, along with experience analyzing telemetry and operational data to improve the reliability, performance, and scalability of AI systems in production

Responsibilities

• Build and maintain observability solutions for AI/ML applications and platform services.
• Develop dashboards and alerts using Grafana and Tableau.
• Monitor AI service health, latency, throughput, error rates, and reliability metrics.
• Create and maintain metrics using Prometheus and PromQL.
• Analyze operational and time-series data to identify trends and performance issues.
• Define and monitor SLIs, SLOs, and service health indicators.
• Partner with AI/ML, Platform, and SRE teams to improve production reliability.
• Support incident troubleshooting and root cause analysis

Required Qualifications

Experience

• Proven years of experience in observability, monitoring, analytics, or telemetry engineering.
• Proven years of experience creating operational dashboards and visualizations.
• Experience supporting production-scale applications in cloud or distributed environments.
• Experience working with large-scale metrics and monitoring systems.

Preferred Skills

• Experience monitoring AI/ML or GenAI workloads.
• Familiarity with OpenTelemetry.
• Knowledge of model performance and data drift concepts.
• Experience working in Agile environments using Jira.

We are an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, citizenship status, age, disability, genetic information, protected veteran status, or any other characteristic protected by applicable law.

https://www.e-verify.gov/sites/default/files/everify/posters/IER_RighttoWorkPoster.pdf