Senior Observability Engineer (Splunk / SignalFx) - US Citizen or Green Card only

Nextiya LLC; Fairfax (VA); On-site; USD 140,000 - 190,000; Full time

Job summary

Zohorecruit in Fairfax, VA is seeking a Senior Observability Engineer to architect and optimize Splunk Enterprise and Splunk Observability Cloud for large-scale environments.

You will build telemetry pipelines with OpenTelemetry, SignalFx agents, and Splunk forwarders, and instrument cloud-native apps to capture traces, metrics, and logs.

Collaborate with SRE and development teams to define SLIs/SLOs, share best practices, and keep the monitoring stack secure and well documented.

Qualifications

• 4+ years in platform observability, SRE, or DevOps.
• Proven architecture experience for large-scale enterprise systems with Splunk/Splunk Observability.
• Hands-on with Splunk Cloud/Enterprise and SignalFx.
• OpenTelemetry familiarity and configuring OTel Collectors.
• Cloud & container experience: AWS/Azure/GCP, Kubernetes, Docker.
• IaC with Terraform, Ansible, or CloudFormation.
• Strong IT infra background incl. Linux/Windows, networks.
• Scripting: Python, Bash; Git for version control.
• Understanding of microservices, REST, and load balancing.
• Knowledge of SQL/NoSQL databases.
• CI/CD, Kubernetes, configuration management, security hardening.
• Strong analytical and cross-functional communication.

Responsibilities

• Architect, deploy, and optimize Splunk Enterprise and Observability Cloud environments.
• Build telemetry pipelines with OpenTelemetry Collector, SignalFx agents, and Splunk forwarders.
• Design dashboards, real-time charts, and alerting to reduce alert fatigue.
• Instrument microservices to capture traces, metrics, and logs for end-to-end visibility.
• Collaborate with SRE/Dev teams to define SLIs/SLOs and conduct root-cause analysis.
• Document architecture, configurations, and troubleshooting per SDLC and security standards.

Skills

Observability

SRE/DevOps

Cross-functional communication

Requirements analysis

Tools

Splunk Enterprise/Splunk Observability Cloud

SignalFx

OpenTelemetry

Kubernetes (EKS/GKE)

Docker

Terraform

Ansible

CloudFormation

Git

Jenkins

SQL/NoSQL databases

Job description

Senior Observability Engineer (Splunk / SignalFx) - US Citizen or Green Card only

Fairfax, United States | Posted on 10/09/2026

• Platform Architecture: Architect, deploy, and optimize Splunk Enterprise log management and Splunk Observability Cloud (SignalFx) environments.
  
• Data Ingestion & Pipelines: Build and maintain telemetry data pipelines using the OpenTelemetry (OTel) Collector, SignalFx Smart Agents, and Splunk Universal/Heavy Forwarders.
  
• Monitoring & Alerting: Design high-performance dashboards, real-time streaming charts, and dynamic alerting mechanisms using SignalFlow or advanced SPL (Search Processing Language) to eliminate alert fatigue.
  
• Distributed Tracing & APM: Instrument cloud-native microservices applications (Java, Python, Go, Node.js) to capture distributed traces, metrics, and logs for end-to-end transaction visibility.
  
• Collaboration & Coaching: Partner with SRE and Development teams to establish Service Level Indicators (SLIs) and Service Level Objectives (SLOs), providing training on root-cause analysis using the modern Splunk ecosystem.
  
• Best Practice: Maintain comprehensive documentation of system architecture, configurations, and troubleshooting procedures aligned with SDLC (Software Development Life Cycle) best practices.
  
• Up-to-date Security Posture: Stay current with emerging trends in cloud security, vulnerability management, threat intelligence, and system hardening to enhance monitoring strategies.
  

Requirements

Qualifications

• Experience: 4+ years of dedicated engineering experience in platformObservability, Site Reliability Engineering (SRE), or DevOps.
  
• Proven experience in solution architecture for large-scale enterprise systems with a focus on observability tools such asSplunk Enterprise SecurityorSignalFx (Splunk Observability Cloud).
  
• Tooling Expertise: Proven, hands-on experience with Splunk Cloud/Enterprise and SignalFx (Splunk Infrastructure Monitoring, Splunk APM, and Splunk Log Observer).
  
• OpenTelemetry: Deep familiarity with OpenTelemetry standards, including configuring OTel Collectors for distributed tracing and streaming metrics.
  
• Cloud & Containerization: Strong expertise monitoring AWS, Azure, or GCP infrastructure alongside container orchestration platforms like Kubernetes (EKS/ GKE) and Docker.
  
• Infrastructure as Code (IaC): Experience automating monitoring deployments via Terraform, Ansible, or CloudFormation.
  
• Strong background in IT infrastructure including cloud platforms (AWS, Azure), virtualization (VMware), operating systems (Linux/Windows), and network fundamentals (DNS, VPNs, firewalls).
  
• Expertise in scripting languages like Python, Bash shell scripting; familiarity with version control systems such as Git for managing codebases.
  
• Deep understanding of system design principles including microservices architecture, load balancing, RESTful APIs, and system integration techniques.
  
• Knowledge of database systems including SQL (MySQL, PostgreSQL), NoSQL databases (MongoDB), and data management best practices.
  
• Experience with DevOps practices such as CI/CD pipelines (Jenkins), container orchestration (Kubernetes), configuration management (GPO), and system hardening for security compliance.
  
• Strong analytical skills in requirements analysis, business analysis, and technical troubleshooting support across
  
• Excellent cross-functional communication skills to translate technical concepts into clear documentation for stakeholders at all levels.
  

Preferred Qualifications

• Splunk Enterprise Certified Admin, Splunk Cloud Certified Admin, or official Splunk Observability Cloud certifications.
  
• Proficiency in scripting or coding languages (Python, Go, or Bash) to write custom automation scripts or OpenTelemetry plug-ins.