Sr Observability Engineer

IT Associates; Irvine (CA); Hybrid; USD 150.000 - 210.000; Teilzeit

Zusammenfassung

IT Associates in Irvine, CA is seeking a Senior Observability Engineer to design and optimize end-to-end monitoring across distributed systems and hybrid environments. This role emphasizes real-time monitoring, intelligent alerting, anomaly detection, and automated remediation to reduce operational risk and improve service reliability.

The candidate will lead observability initiatives, drive best practices, and integrate CI/CD/IaC to automate telemetry collection and fault diagnosis, enabling

Qualifikationen

• 10+ years of experience in observability engineering and monitoring.
• 7+ years with APM, monitoring, observability and event management tools.
• Bachelor's degree in computer science or engineering.

Aufgaben

• Design end-to-end observability solutions across applications and infrastructure.
• Improve monitoring maturity and reduce alert fatigue across platforms.
• Define automated alerting aligned with SLOs/SLAs and governance.
• Automate configurations using CI/CD pipelines and IaC (Helm, Ansible, Terraform).
• Lead cross-functional teams and provide strategic guidance on incidents.

Kenntnisse

APM tools

Monitoring

Observability

Dynatrace

AppDynamics

Splunk

Prometheus

Grafana

Cortex

CI/CD

IaC

Java

Python

Terraform

Helm

Ansible

Team leadership

Ausbildung

Bachelor's degree in computer science or engineering

Tools

Helm

Ansible

Terraform

Jobbeschreibung

• Contract length: 6-9 months, potential for CTH
• Location: Irvine, CA local only, no remote candidates - hybrid, onsite at the client site 1-3 days a week

As a Senior Observability Engineer, you will be responsible for designing, implementing, migrating, and optimizing end‑to‑end monitoring and observability solutions to ensure the reliability, performance, and resilience of distributed systems and application services for our clients. You will play a critical role in advancing observability maturity across complex legacy, distributed and hybrid environments by enabling proactive detection, rapid diagnosis, efficient resolution of incidents and using AI capabilities.

This role requires deep expertise in real‑time monitoring, intelligent alerting, anomaly detection, and automated remediation, with a strong focus on reducing operational risk, minimizing alert fatigue, and improving overall service reliability.

Required Qualifications: (Must have)

• Overall 10+ years of experience out of which, 7+ years of solid experience with APM, monitoring, observability and event management tools including Dynatrace/AppDynamics, Splunk, Cortex, Prometheus, Grafana, and Netcool.
• Experience with ITSM, ticketing tools and their integration with monitoring tools.
• Proficiency in Application Workloads (Binary, Java, Python, .NET, Batch Jobs).
• Exposure to CI/CD pipelines and IaC (Infrastructure as Code).
• Strong in analytical and problem-solving skills for diagnosing complex issues
• Effective in communication, individual leadership, and cross-functional team collaboration.
• Ability to think outside the box, sensitivity towards business impacts, and self-awareness to refine processes.
• Bachelor7s degree in computer science or engineering field.

Preferred Qualifications: (Good to have)

• Proficiency in broader aspects of monitoring and observability (APM, System Monitoring, Logs, Tracing, Visualization, Reporting and Integration)
• Experience in automation/programming/coding to an extent that can instrument monitoring solutions for a given platform/tooling/practice.
• Certified professional in Dynatrace/AppDynamics, Splunk, ITIL or AI.

What You Will Do:

• Assess the current state of monitoring and observability across applications and systems, including identifying alert fatigue, monitoring gaps, and coverage deficiencies.
• Define and execute strategies to incrementally improve the monitoring and observability maturity of platforms, applications, and infrastructure.
• Design and implement end‑to‑end observability solutions that provide comprehensive visibility into business transactions, service dependencies, and underlying technical components.
• Establish and promote monitoring best practices focused on noise reduction, controlled metric cardinality, and the prevention of duplicate or redundant telemetry.
• Define and implement automated alerting strategies aligned with Service Level Objectives (SLOs) and Service Level Agreements (SLAs) to ensure actionable and meaningful alerts.
• Develop and enforce monitoring audit standards to support governance, compliance, and regulatory requirements.
• Act as an escalation point for complex or critical monitoring‑related incidents and provide strategic guidance and recommendations to engineering and operations teams.
• Automate monitoring configurations, policy management, and telemetry collection using CI/CD pipelines and Infrastructure as Code (IaC) practices with tools such as Helm, Ansible, and Terraform.
• Build reusable automation frameworks and standardized reporting solutions to support consistent monitoring rollouts, configuration management, and operational insights.
• Leverage AI and machine learning techniques to enhance observability outcomes, including intelligent anomaly detection, alert noise reduction, predictive incident identification, automated root‑cause analysis, and data‑driven insights to improve service reliability and operational efficiency.