Staff Software Engineer - AI Gateway

databricks

New York (NY)

On-site

USD 190,000 - 261,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Databricks is seeking an experienced backend/infrastructure engineer to power the Unity AI Gateway. You will design and implement the routing layer that directs requests to the right model, balancing quality, cost, latency, and budgets.

You will collaborate with research, platform, infra, and AI/ML teams to improve reliability and efficiency of distributed workloads, enabling developers and data scientists to build and run AI on Databricks.

Qualifications

  • 8+ years of experience in backend or infrastructure engineering.
  • Strong programming skills in Scala or Go.
  • Experience with distributed systems, high-throughput APIs, or cloud-native infrastructure.
  • Familiarity with service-oriented architecture, deployment pipelines, and system observability.
  • Strong product and ownership mindset; building the right solution.
  • Experience with real-time serving, AI infrastructure, or GPU orchestration.
  • Exposure to platforms like Bedrock, Vertex AI, or Azure ML.
  • Experience building developer platforms or internal tools supporting AI workflows.

Responsibilities

  • Build the routing layer that picks the right model for each request across quality, cost, latency, availability, and remaining budget.
  • Improve reliability, latency, and efficiency of distributed AI workloads.
  • Collaborate with research, platform, infra, and AI/ML teams to deliver seamless end-to-end experiences.
  • Shape how developers and data scientists build and interact with AI on Databricks.

Skills

Scala
Go

Job description

At Databricks, we are passionate about enabling data and AI teams to solve the world's toughest problems, from making the next mode of transportation a reality to accelerating medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers, and customer obsessed, we leap at every opportunity to solve technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines. And we're only getting started.

As part of the Unity AI Gateway team, you'll build the control plane that every AI request at Databricks passes through: partner and self-hosted models, agents, coding assistants, and MCP servers. It enforces budgets, routes each request to the right model, and applies guardrails inline. Unity AI Gateway is one of our fastest-growing businesses, moving trillions of tokens per week for thousands of customers.

The impact you will have:
  • Build the routing layer that picks the right model for each request across quality, cost, latency, availability, and remaining budget
  • Improve reliability, latency, and efficiency of distributed AI workloads
  • Collaborate with research, platform, infra, and AI/ML teams to deliver seamless end-to-end experiences
  • Shape how developers and data scientists build and interact with AI on Databricks
What we look for:
  • 8+ years of experience in backend or infrastructure engineering
  • Strong programming skills in Scala, or Go
  • Experience with distributed systems, high-throughput APIs, or cloud-native infrastructure
  • Familiarity with service-oriented architecture, deployment pipelines, and system observability
  • Strong product and ownership mindset. You care about building the right solution, not just any solution
  • Experience with real-time serving, AI infrastructure, or GPU orchestration
  • Exposure to platforms like Bedrock, Vertex AI, or Azure ML
  • Experience building developer platforms or internal tools supporting AI workflows

Pay Range Transparency

Databricks is committed to fair and equitable compensation practices. The pay range(s) for this role is listed below and represents the expected salary range for non-commissionable roles or on-target earnings for commissionable roles. Actual compensation packages are based on several factors that are unique to each candidate, including but not limited to job-related skills, depth of experience, relevant certifications and training, and specific work location. Based on the factors above, Databricks anticipates utilizing the full width of the range. The total compensation package for this position may also include eligibility for annual performance bonus, equity, and the benefits listed above. For more information regarding which range your location is in visit our page here .

Local Pay Range

$190,000 — $261,250 USD

About Databricks

Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and 70% of the Fortune 500 — rely on the Databricks Data + AI Platform to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. To learn more, follow Databricks on LinkedIn , X , YouTube , Instagram .

Benefits

At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here .

Our Commitment to Diversity and Inclusion

At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.

Compliance

If access to export-controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone.

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey.Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiringprocess or thereafter. Any information that you do provide will be recorded and maintained in aconfidential file.

As set forth in Databricks’s Equal Employment Opportunity policy,we do not discriminate on the basis of any protected group status under any applicable law.

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection.As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measurethe effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categoriesis as follows:

A \"disabled veteran\" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A \"recently separated veteran\" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An \"active duty wartime or campaign badge veteran\" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An \"Armed forces service medal veteran\" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Manager Compensation R&D
Senior Manager Compensation R&D

databricks • United States

On-site
USD 174,000 - 300,000
Sr. Automation Engineer - IT Asset Management
Sr. Automation Engineer - IT Asset Management

Databricks Inc. • Mountain View (CA)

On-site
USD 130,000 - 190,000
Manager, Field Engineering - Strategic Digital Native Business
Manager, Field Engineering - Strategic Digital Native Business

Databricks Inc. • Washington

Hybrid
USD 173,000 - 237,000
Sr. Partner Solutions Architect - Customer Lake & Lakewatch
Sr. Partner Solutions Architect - Customer Lake & Lakewatch

databricks • United States

On-site
USD 219,000 - 301,000
Software Engineering Intern (2027) Bellevue, Washington; Mountain View, California; San Francisco, California; Vancouver, Canada
Software Engineering Intern (2027) Bellevue, Washington; Mountain View, California; San Francisco, California; Vancouver, Canada

Databricks Inc. • Mountain View (CA)

On-site
USD 74,000 - 83,000
Enterprise Account Executive - Retail
Enterprise Account Executive - Retail

databricks • Texas

Hybrid
USD 312,000 - 428,000
Associate Forward Deployed Engineer
Associate Forward Deployed Engineer

Databricks Inc. • Mountain View (CA)

Hybrid
USD 120,000 - 160,000
Hybrid office
Travel opportunities
Manager, Field Engineering (Specialist) - HCLS
Manager, Field Engineering (Specialist) - HCLS

Databricks Inc. • United States

On-site
USD 197,000 - 271,000
Sr. Staff Software Engineer, AI Native Web Platform
Sr. Staff Software Engineer, AI Native Web Platform

Databricks Inc. • Mountain View (CA)

On-site
USD 198,000 - 273,000
Senior Financial Analyst - Strategic GTM Finance
Senior Financial Analyst - Strategic GTM Finance

Databricks Inc. • Mountain View (CA), San Francisco (CA)

On-site
USD 137,000 - 189,000