Lead Software Engineer - Observability Infrastructure Monitoring

Splunk Inc.; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

A leading technology company seeks a Lead Software Engineer for their Observability team, responsible for developing world-class monitoring tools. Candidates should have extensive experience in backend systems, particularly with Golang, and possess strong skills in cloud technologies and distributed systems. This position offers opportunities to influence product direction and collaborate across teams in a dynamic environment.

Benefits

Medical benefits

Dental benefits

Vision benefits

401(k) plan

Paid time off

Qualifications

• 8-10 years of relevant experience in software engineering.
• Strong understanding of observability tools and practices.
• Ability to mentor and communicate effectively.

Responsibilities

• Lead and execute tasks as the main engineering contributor.
• Organize sprint planning and other engineering processes.
• Mentor engineers and lead code reviews.

Skills

Golang

Python

Java

Microservices architecture

Distributed systems

OpenTelemetry

Prometheus

Grafana

Kubernetes

Cloud-native applications

Education

Bachelor’s degree in Computer Science, Engineering, or related fields

Tools

Kubernetes

OpenShift

AWS

GCP

NoSQL databases

Job description

Lead Software Engineer - Observability Infrastructure Monitoring

Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most importantly, each other’s success. Learn more about Splunk careers and how you can become a part of our journey!

As applications and systems become more sophisticated and user experience is at high stake, observability—the ability to monitor and understand systems and their impact on users—becomes one of the biggest challenges for engineering teams. We are building world-class tools to help engineers deliver better, faster, and more reliable applications.

About the Role

Splunk is looking for hardworking and creative full stack Software Engineers to join our Splunk Observability team, working on systems that empower our customers to quickly detect and resolve infrastructure and application issues. You will be a key contributor and lead for the user-facing web application for Splunk Observability Infrastructure Monitoring. Splunk’s Observability Suite is a leader in the Gartner Magic Quadrant, and Infrastructure Monitoring is one of the most used and popular products in this suite.

Responsibilities

1. Operate as a lead engineer, independently execute tasks, and own outcomes.
2. Organize processes such as sprint planning, retrospectives, backlog refinement, and program increment (PI) planning.
3. Understand business direction, promote innovative ideas, and align with multi-functional teams; influence product management on features, product direction, and priorities.
4. Lead design discussions, ensuring both functional and non-functional requirements (like scale, security, operability, usability) are considered.
5. Lead development of new product features, improve designs, and participate in code reviews.
6. Build relationships with cross-area teams to understand dependencies and collaborate effectively.
7. Mentor senior engineers through design and code reviews, seminars, and pair programming.
8. Implement actions to improve the team’s engineering and operational practices.

Requirements

1. Proven experience in architecting and delivering complex, large-scale backend systems.
2. Expertise in Golang for backend development; experience with Python or Java is a plus.
3. Hands-on experience with OpenTelemetry, Prometheus, and Grafana for observability and monitoring in distributed environments.
4. Deep understanding of microservices architecture and distributed systems, ensuring scalability, performance, and reliability.
5. Experience with cloud-native applications in Kubernetes or OpenShift, and server clusters in AWS, GCP, or on-premises.
6. Experience with relational and NoSQL databases.
7. Knowledge of AI technologies is a plus.
8. Experience with observability tools is advantageous.
9. Ability to make architectural decisions, set technical direction, and mentor engineers.
10. Strong communication skills, verbal and written.
11. Bachelor’s degree in Computer Science, Engineering, or related fields; 8-10 years of relevant experience.

We value diversity and are an equal opportunity employer. All qualified applicants will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or other protected characteristics. Reasonable accommodations are available for individuals with disabilities.

Note: The base pay ranges vary by location, and additional incentives, equity, or cash awards may apply. Benefits include medical, dental, vision, 401(k), paid time off, and more. Learn more about our benefits.