Lead Software Engineer - Observability Infrastructure Monitoring

Splunk; Washington (District of Columbia); Hybrid; USD 139,000 - 193,000; Full time

Job summary

An innovative company is seeking a Lead Software Engineer to drive the development of cutting-edge observability tools. This role involves architecting and delivering large-scale backend systems, mentoring engineers, and collaborating with cross-functional teams. You'll lead the user-facing web application for a top-tier observability suite, ensuring performance, scalability, and reliability. With a commitment to diversity and inclusion, this company offers flexible working arrangements and a comprehensive benefits package. Join a passionate team dedicated to making machine data accessible and valuable for everyone.

Benefits

Medical Insurance

Dental Insurance

Vision Insurance

401(k) with Match

Paid Time Off

Flexible Working Arrangements

Equity Options

Long-term Cash Awards

Qualifications

• 8-10 years of experience in backend systems architecture and design.
• Expertise in Golang with experience in Python or Java is a plus.

Responsibilities

• Lead the development of new product features and own the outcomes.
• Mentor senior engineers and promote innovative ideas within the team.

Skills

Golang

Python

Java

OpenTelemetry

Prometheus

Grafana

Microservices Architecture

Cloud-native Applications

AI Technologies

Strong Communication Skills

Education

Bachelor's Degree in Computer Science

Tools

Kubernetes

OpenShift

AWS

GCP

Relational Databases

NoSQL Databases

Job description

Lead Software Engineer - Observability Infrastructure Monitoring

Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most importantly to each other’s success. Learn more about Splunk careers and how you can become a part of our journey!

As applications and systems become more sophisticated and user experience is at high stake, observability—the ability to monitor and understand the systems and how they impact users—becomes one of the biggest challenges for engineering teams. We are building world-class tools to help engineers deliver better, faster, and more reliable applications.

About the Role

Splunk is looking for hardworking and creative full-stack Software Engineers to join our Splunk Observability team. You will work on systems that empower our customers to quickly detect and resolve their infrastructure and application issues.

You will be a key contributor and lead for the user-facing web application for Splunk Observability Infrastructure Monitoring. Splunk’s Observability Suite is a leader in the Gartner Magic Quadrant, and Infrastructure Monitoring is one of the most used and popular products in this suite.

Responsibilities

1. Operate as a lead engineer, independently execute tasks, and own the outcomes.
2. Own and organize processes for the team such as sprint planning, retrospectives, backlog refinements, and program increment (PI) planning.
3. Understand the business direction, promote innovative ideas, and align with multi-functional teams; influence PM on features, product direction, and priorities.
4. Lead design discussions, ensuring functional and non-functional requirements (like scale, security, operability, usability) are considered.
5. Lead the development of new product features, clarify and improve designs, and participate in code reviews.
6. Build relationships with cross-area teams to understand dependencies and set up collaborations.
7. Mentor senior engineers through design and code reviews, tech seminars, pair programming, etc.
8. Take actions to improve the team’s engineering and operational practices.

Requirements

• Proven track record of architecting, designing, and delivering complex, large-scale backend systems.
• Expertise in Golang for backend development; experience with Python or Java is a plus.
• Strong hands-on experience with OpenTelemetry, Prometheus, and Grafana for observability and monitoring in distributed environments.
• Deep understanding of microservices architecture and distributed systems, ensuring scalability, performance, and reliability.
• Experience developing cloud-native applications in Kubernetes or OpenShift, server clusters in AWS, GCP, or on-premises.
• Rich experience with relational and NoSQL databases.
• Knowledge and experience leveraging AI technologies is a BIG plus.
• Knowledge and experience with observability tools is an advantage.
• Ability to drive architectural decisions, set technical direction, and mentor engineers.
• Strong communication skills, both verbal and written.
• Bachelor’s degree in Computer Science, Computer Engineering, or related fields; approximately 8-10 years of relevant experience.
We value diversity at our company. All qualified applicants will receive consideration without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or other protected characteristics. Reasonable accommodations are provided for individuals with disabilities. Please contact us to request accommodations.

Splunk, a Cisco company, is an Equal Opportunity Employer and all qualified applicants will receive consideration regardless of legally protected characteristics.

Note:

Base Pay Range

• SF Bay Area, Seattle Metro, and New York City Metro Area: $174,800 - $240,350/year
• California (excluding SF Bay Area), Washington (excluding Seattle Metro), Washington DC Metro, and Massachusetts: $157,320 - $216,315/year
• Other cities and states: $139,840 - $192,280/year

Splunk offers flexible working arrangements, including remote and in-office roles. The pay varies by location and experience. In addition to base pay, the role may include incentive compensation, equity, or long-term cash awards.

Benefits include medical, dental, vision, 401(k) with match, paid time off, and more. Learn more at https://splunkbenefits.com.