Sr. Staff Production Engineer - Data Platform

Cacheflow; San Francisco (CA); On-site; USD 228,600 - 314,250; Full time

Job summary

Databricks is seeking a Sr. Staff Production Engineer to lead the strategic vision for the operational stability of our internal Databricks-on-Databricks environment.

You will guide the shift from traditional SRE models to an agent-driven, self-healing architecture supporting mission-critical workloads. You will drive 99.99% availability for compute, storage, and control plane services, while defining scalable strategies across 100+ regions and evangelizing Chaos Engineering practices across

Qualifications

• BS/MS/PhD in Computer Science, or a related field.
• 10+ years of production-level experience as a Software Engineer or SRE in distributed, multi-cloud environments.
• You write code to solve operational problems and build automation to remove toil.
• Familiarity with LLM infrastructure, training/inference pipelines, or agentic frameworks is a plus.
• Operational grit and calm under pressure in large-scale systems.
• Experience building long-range technical roadmaps and cross-functional alignment.

Responsibilities

• Architect future self-healing infrastructure at scale with AI agents.
• Own the Data Platform's operational integrity enabling high availability for compute, storage and control planes.
• Establish the next generation of change safety protocols with automation and guardrails.
• Lead technical standards and evangelize modern SRE practices including Chaos Engineering.

Skills

10+ years experience

Code-focused

AI/LLM mindset

Operational grit

Strategic influence

Education

BS/MS/PhD in Computer Science

Tools

Scala

Java

Go

Python

Job description

RDQ126R106

At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers - and customer obsessed - we leap at every opportunity to tackle technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines. And we're only getting started.

Our Data Platform organization is the backbone of the world’s leading Data and AI platform. We operate a massive, multi-cloud (AWS, Azure, GCP), multi-AI, multi-region stack that powers thousands of the world’s most demanding workloads. As we move into the era of Agentic AI, we are not just scaling infrastructure; we are building a new generation of Agentic Observability and Reliability systems.

As a Sr. Staff Production Engineer, you will lead the strategic vision for the operational stability of our internal "Databricks-on-Databricks" environment. You will transition our infrastructure from traditional SRE models toward an agent-driven, self-healing architecture, ensuring that our platform—and the agents operating within it—remain rock-solid for mission-critical customer workloads.

The Impact You Will Have

• Architecting Agentic Reliability: Define and drive the design of future "self-healing" infrastructure at scale where AI agents proactively detect, diagnose, and remediate production incidents before they impact customers.
• Data Platform Optimization: Own the operational integrity of the Data Platform that powers our internal AI models, ensuring 99.99% availability for the compute, storage, and control plane services used by thousands of Databricks engineers.
• High-Scale Operational Excellence: Establish the next generation of "Change Safety" protocols, utilizing automation and agentic guardrails to manage complex deployments across 100+ global regions.
• Leadership in Chaos & Scale: Serve as a technical bar-raiser for the team, evangelizing modern SRE practices (including Chaos Engineering) to navigate the structural transformation of the industry toward agentic, autonomous systems.

What We Look For

• BS/MS/PhD in Computer Science, or a related field
• Technical Depth: 10+ years of production-level experience as a Software Engineer or SRE in highly distributed, multi-cloud environments.
• Engineering Persona: You write code to solve operational problems. You are not a traditional sys-admin; you build frameworks, automation, and tooling (Scala, Java, Go, or Python) to eliminate toil.
• Platform & AI Mindset: Deep understanding of distributed data platforms and a passion for leveraging AI/ML to revolutionize infrastructure management. Familiarity with LLM infrastructure, training/inference pipelines, or agentic frameworks is a significant plus.
• Operational Grit: Proven ability to remain calm and decisive under pressure. You have navigated large-scale distributed systems through hyper-growth and have a track record of driving incident-to-roadmap loops.
• Strategic Influence: Experience building long-range technical roadmaps and driving cross-functional alignment. You are comfortable challenging senior leadership with data-driven insights.

Pay Range Transparency

Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles. Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Based on the factors above, Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above. For more information regarding which range your location is in visit our page here.

Zone 1 Pay Range

$228,600 - $314,250 USD

About Databricks

Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook.

At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here.

Our Commitment to Diversity and Inclusion

At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.

Compliance

If access to export-controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone.