Senior Observability & AI Ops Engineer

BCforward; Phoenix (AZ); Hybrid; USD 120,000 - 140,000; Full time

Job summary

BCforward seeks a Senior Observability Operations Engineer for its Phoenix, AZ hybrid team. The role focuses on managing Dynatrace, Splunk, and OpenSearch/Elasticsearch platforms, with Linux and Kubernetes support and a drive toward AI-assisted operations.

The candidate will automate tasks, optimize monitoring, and design robust logging and alerting. 3 days on-site weekly with flexible shifts and a weekend rotation may apply.

Qualifications

• Bachelor's degree or higher in CS/IT/Engineering or equivalent.
• 6–10+ years IT infrastructure or observability operations experience.
• 4+ years administering Dynatrace, Splunk, OpenSearch, or Elasticsearch.
• Strong troubleshooting and analytical skills.
• Excellent communication and stakeholder management.

Responsibilities

• Administer and optimize enterprise observability platforms including Dynatrace, Splunk, and OpenSearch/Elasticsearch.
• Design, deploy, configure, and maintain monitoring, logging, tracing, and alerting solutions.
• Manage large-scale OpenSearch/Elasticsearch clusters, including indexing strategies, performance tuning, shard optimization, backups, and capacity planning.
• Configure Dynatrace OneAgent, ActiveGate, Synthetic Monitoring, Real User Monitoring (RUM), Digital Experience Monitoring (DEM), Davis AI, and Application Performance Monitoring (APM).
• Administer Splunk Enterprise, Universal Forwarders, Indexers, Search Heads, Cluster Manager, Deployment Server, and Splunk ITSI.
• Develop dashboards, alerts, reports, and executive operational metrics.
• Support Linux-based infrastructure and Kubernetes environments (Docker/OpenShift/Rancher preferred).
• Implement observability best practices using OpenTelemetry, distributed tracing, metrics, logs, and events.
• Perform root cause analysis for production incidents using observability platforms.
• Collaborate with Platform Engineering, SRE, DevOps, Infrastructure, and Application teams.
• Automate operational tasks using Python, Shell scripting, REST APIs, Terraform, or Ansible.
• Participate in incident, problem, change, and release management processes.
• Drive platform upgrades, patching, security compliance, and operational governance.
• Improve platform reliability through automation, self-healing, and AI-assisted operations.

Skills

Observability Platforms

Dynatrace Administration

OpenSearch Administration

Elasticsearch Administration

Grafana

Kibana

OpenTelemetry

Kafka

Education

Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience

Tools

Docker

OpenShift

Rancher

Terraform

Ansible

Git

REST APIs

Job description

BCforward seeks a Senior Observability Operations Engineer for its Phoenix, AZ hybrid team. The role focuses on managing Dynatrace, Splunk, and OpenSearch/Elasticsearch platforms, with Linux and Kubernetes support and a drive toward AI-assisted operations.

The candidate will automate tasks, optimize monitoring, and design robust logging and alerting. 3 days on-site weekly with flexible shifts and a weekend rotation may apply.