Senior Go Engineer – Observability & Alerting at Grafana

Grafana Labs; United States; Remote; USD 139,000 - 167,000; Full time

Job summary

Grafana Labs is seeking a Senior Go Engineer for Observability & Alerting, with a remote-first setup. The role emphasizes distributed systems, cloud infrastructure, and open-source collaboration across Grafana’s alerting and SLOs products.

Compensation benchmarks note a UK range with RSUs providing upside, and on-call ownership is part of the role. The position welcomes candidates from the UK and beyond, reflecting Grafana’s global remote culture.

Responsibilities

• Take an active role in influencing our roadmap and your own career objectives
• Work with your team to deliver new features, then use the results to iterate and improve
• Drive projects from initial ideation all the way to operations once in the hands of customers
• Embrace our open-source culture and contribute to other projects that may not directly fall within your team's scope
• Design, build, operate, and maintain critical systems, owning the reliability, performance, and availability
• Be a part of your team's on-call rotations and take ownership of the services you’re running
• Mentor and support other team members, participate in design discussions and collaborate with the team
• Learn new skills by gaining a deeper understanding of our cloud product and customers, and getting to know the codebase of a large distributed system

Skills

Go

Prometheus

Alerting infrastructure

Distributed systems

Job description

Senior Go Engineer – Observability & Alerting at Grafana — Remote

This Senior Software Engineer role at Grafana Labs sits squarely in the observability and distributed systems space — one of the hottest engineering domains in 2024–2025. Skills in Go, Prometheus, alerting infrastructure, and large-scale cloud systems are in exceptionally high demand across companies like Datadog, Cloudflare, HashiCorp, and AWS.

Compensation benchmarking: The GBP 103,958–124,750 range is competitive for a senior backend engineer in the UK, broadly aligned with top-quartile UK tech salaries. For context, Levels.fyi and similar platforms show senior backend roles at UK unicorns averaging £90k–£120k. Grafana's range skews slightly above median, plus RSUs add meaningful upside given the company's high-profile VC backing (Sequoia, Lightspeed).

Green flags for career growth:

• Ownership from ideation to production — excellent for building a well-rounded senior-to-staff engineer portfolio
• Open-source contribution embedded in the role — builds public profile and community reputation
• SLOs product scope in addition to Alerting — breadth of systems experience across two products
• Mentoring responsibilities signal a path toward staff/principal engineering or engineering management
• Transparent, defined career growth pathways explicitly called out

Watch out for: On-call rotation is part of the role — ensure you're comfortable with production ownership before applying. The codebase is large and complex, which is great for learning but can mean a steep ramp-up period.

Overall, this is a strong opportunity for engineers wanting to deepen expertise in distributed systems, cloud infrastructure, and open-source engineering at a globally recognized company.

Grafana Labs is the company behind Grafana Cloud, the fully managed observability platform trusted by more than 10,000 organizations to ensure reliability, resolve incidents faster, and optimize telemetry at scale. Built on open source and open standards and designed for interoperability across any stack, Grafana Cloud brings AI to observability and observability to AI, giving teams unified visibility so they can see, understand, and act on all their disparate data, wherever it lives. Customers including Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce rely on Grafana Labs. We are a 100% remote company with team members across 40+ countries, backed by leading investors including Lightspeed Venture Partners, Sequoia Capital, GIC, Coatue, J.P. Morgan, CapitalG, and Lead Edge Capital. Learn more at grafana.com and follow us on LinkedIn and X .

We're scaling fast and staying true to what makes us different: an open-source legacy, a global collaborative culture, and a passion for meaningful work. Our team thrives in an innovation-driven environment where transparency, autonomy, and trust fuel everything we do.

You may not meet every requirement, and that's okay. If this role excites you, we'd love you to raise your hand for what could be a truly career-defining opportunity.

This is a remote opportunity and we would be interested in applications from the UK.

The Opportunity:

The Grafana Alerting squad operates at the core of the Grafana open-source project (link), and our mission is to provide the most in-depth way to let users manage their alerts. The backend-focused team works closely with customers and the Grafana Cloud teams to make Alerting work on-prem and at infinite scale in Cloud. The team's scope spans both Grafana Alerting and the SLOs product, so engineers get to work across two products that are deeply connected in how they help users detect and respond to problems.

Grafana Alerting is trusted by major organizations worldwide, monitoring essential medical devices and critical infrastructure. We thrive on collective creativity and diverse perspectives — every team member is encouraged to contribute ideas that shape our product into a dependable tool.

Our domain is quite large. Here are some recent projects to give you an idea of the kind of work we do:

• Build distributed systems to scale alert ingestion and delivery of Prometheus-based alerts
• Unify Alerting setup and delivery to work the same for Grafana and Prometheus-based alerts
• Build distributed systems to scale alert evaluation of Grafana-managed alerts, reaching over 500 evaluations per second in production

What You'll Be Doing:

• Take an active role in influencing our roadmap and your own career objectives
• Work with your team to deliver new features, then use the results to iterate and improve
• Drive projects from initial ideation all the way to operations once in the hands of customers
• Embrace our open-source culture and contribute to other projects that may not directly fall within your team's scope
• Design, build, operate, and maintain critical systems, owning the reliability, performance, and availability
• Be a part of your team's on-call rotations and take ownership of the services you’re running
• Mentor and support other team members, participate in design discussions and collaborate with the team
• Learn new skills by gaining a deeper understanding of our cloud product and customers, and getting to know the codebase of a large distributed system

As we are remote-first and our engineering organization is largely remote, we provide guidance and meet regularly using video calls, so an independent attitude and good communication skills are a must.

What Makes You a Great Fit:

• You are a motivated self-starter with a bias towards action
• You are customer focused — we build everything with our users in mind
• You have a passion for creating intuitive products that fit customers' needs
• Pragmatism: you are able to take on complex challenges and break them down to achieve short feedback loops — analyze, design, and build modular solutions, deliver MVPs, gather data and feedback, then progress iteratively
• Collaboration and communication: the smallest unit we have is a team; good communication skills are a must in a fully remote setup
• Solid experience with at least one programming language — we use Go, but familiarity with Python, Java, C++, Rust, or similar translates well
• Some experience delivering projects from gathering requirements and brainstorming all the way to shipping to customers in a self-driven way
• Some experience with developing software that runs in the Cloud or some experience with systems engineering
• Experience writing clean, robust, and performant software that is easily maintained by others

Compensation & Rewards:

In the UK, the compensation range for this role is GBP 103,958 – GBP 124,750. Actual compensation may vary based on level, experience, and skillset as assessed throughout the interview process. All of our roles include Restricted Stock Units (RSUs), giving every team member ownership in Grafana Labs’ success. We believe in shared outcomes — RSUs help us stay aligned and invested as we scale globally.

*Compensation ranges are country specific. If you are applying for this role from a different location than listed above, your recruiter will discuss your specific market’s defined pay range & benefits at the beginning of the process.

Why You'll Thrive at Grafana Labs:

• 100% Remote, Global Culture – As a remote-only company, we bring together talent from around the world, united by a culture of collaboration and shared purpose.
• Scaling Organization – Tackle meaningful work in a high-growth, ever-evolving environment.
• Transparent Communication – Expect open decision-making and regular company-wide updates.
• Innovation-Driven – Autonomy and support to ship great work and try new things.
• Open Source Roots – Built on community-driven values that shape how we work.
• Empowered Teams – High trust, low ego culture that values outcomes over optics.
• Career Growth Pathways – Defined opportunities to grow and develop your career.
• Approachable Leadership – Transparent execs who are involved, visible, and human.
• Passionate People – Join a team of smart, supportive folks who care deeply about what they do.
• In-Person Onboarding – We want you to thrive from day 1 with your fellow new ‘Grafanistas’ to learn all about what we do and how we do it.
• Balance is Key – We operate a global annual leave policy of 30 days per annum. 3 days of your annual leave entitlement are reserved for Grafana Shutdown Days to allow the team to really disconnect. *We will comply with local legislation where applicable.

Equal Opportunity Employer: Grafana Labs is an equal opportunities employer. We welcome applications from everyone regardless of race, colour, nationality, origin, caste, sex, gender reassignment identity or expression, sexual orientation, age, religion or belief, disability, veteran status, genetic information, pregnancy, maternity, marital, family or carer status, or any other characteristic which is protected by local law.

Grafana Labs may utilize AI tools in its recruitment process to assist in matching information provided in CVs to job postings. The recruitment team will continue to review inbound CVs manually to identify alignment with current openings.

For information about how your personal data is used once you've applied to a job, check out our privacy policy.

Is this role truly fully remote or is there an office requirement? This is a fully remote position with no office requirement. Grafana Labs is a 100% remote company with engineers across 40+ countries, though this specific role requires applicants based in the UK. What is the salary range for this Senior Software Engineer role in the UK? The UK compensation range is GBP 103,958 to GBP 124,750, depending on level, experience, and skillset. All roles also include Restricted Stock Units (RSUs). Do I need to know Go to apply for this role? Go is the primary language used by the Alerting team, but Grafana Labs accepts candidates with strong experience in Python, Java, C++, Rust, or similar languages, as those skills translate well. Is there an on-call requirement for this engineering role? Yes, being part of the team's on-call rotation is explicitly listed as part of the role, as engineers are expected to take ownership of the services they build and run. How much annual leave do Grafana Labs employees receive? Grafana Labs operates a global annual leave policy of 30 days per year, including 3 reserved Grafana Shutdown Days to help the team fully disconnect.

Remote work & career advice that actually works.