Staff Data Engineer: Databricks ML Platform Lead

redventures

United States

Remote

USD 150,000 - 220,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Medical, dental and vision coverage
Life and disability insurance
Flexible spending accounts
Matched 401(k)
Employee assistance
Paid parental bonding leave
Flexible paid time off

Job summary

redventures seeks a senior data platform leader to design and modernize large-scale data platforms, focusing on Databricks and ML infrastructure. You will collaborate across engineering teams and external partners to deliver governed, cost-conscious data inputs for production models.

Responsibilities include defining platform architecture, operating Databricks environments, building feature stores and vector databases, and delivering secure data contracts with strong observability and cost

Qualifications

  • 10+ years of data engineering experience designing and operating large-scale platforms.
  • Production experience with Databricks, SQL, Spark and Python.
  • Experience using Terraform to manage data platform infrastructure.
  • Familiarity with workflow orchestration tools such as Airflow or Dagster.
  • Experience with streaming or messaging platforms, such as Kafka, Kinesis or similar.
  • Knowledge of data governance, quality and observability practices, including access controls and PII safeguards.
  • Experience with DevOps and FinOps practices, such as CI/CD, data contracts and platform cost optimization.
  • Ability to explain technical trade-offs to non-engineering stakeholders and guide discussions with external partners.

Responsibilities

  • Turn stakeholder needs into platform architecture decisions and guide technical design across teams.
  • Modernize and operate Databricks environments used by multiple engineering groups.
  • Build AI and machine learning data infrastructure, including feature stores and vector databases.
  • Develop pipelines for embeddings, retrieval-augmented generation, and versioned training data.
  • Set standards for governance, data quality, observability, access control and protection of personal information.
  • Improve platform cost efficiency and delivery practices through CI/CD, infrastructure automation and data contracts.
  • Mentor engineers and lead design reviews to strengthen technical quality.

Tools

Databricks
SQL
Spark
Python
Terraform
Airflow
Dagster
Kafka
Kinesis

Job description

redventures seeks a senior data platform leader to design and modernize large-scale data platforms, focusing on Databricks and ML infrastructure. You will collaborate across engineering teams and external partners to deliver governed, cost-conscious data inputs for production models.

Responsibilities include defining platform architecture, operating Databricks environments, building feature stores and vector databases, and delivering secure data contracts with strong observability and cost

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Platform Engineer: Databricks, Governance & Real-time ML Ops
Data Platform Engineer: Databricks, Governance & Real-time ML Ops

AgenticBricks.com • Seattle (WA)

On-site
USD 160,000 - 210,000
Lead Databricks Data Engineer | Cloud Platform Leader
Lead Databricks Data Engineer | Cloud Platform Leader

Fa Ewjt Saasfaprod1 • New York (NY)

On-site
USD 140,000 - 170,000
Databricks Platform Lead & Data Engineering Manager
Databricks Platform Lead & Data Engineering Manager

iMPact Business Group • Grand Rapids (MI)

Hybrid
USD 140,000 - 190,000
Databricks Solutions Architect: Lead Data & ML Platform
Databricks Solutions Architect: Lead Data & ML Platform

Centraprise • San Francisco (CA)

On-site
USD 180,000 - 260,000
Data Platform Engineer (Databricks)
Data Platform Engineer (Databricks)

Luxoft • Irvine (CA)

Hybrid
USD 120,000 - 180,000
Senior Databricks Platform Engineer (AWS)
Senior Databricks Platform Engineer (AWS)

Modernizing Medicine, Inc. • Boca Raton (FL)

On-site
USD 140,000 - 170,000
Medical/Dental/Vision
401(k) matching
Paid time off
+2
Sr. Staff Platform/Data Reliability Engineer, Databricks (R5537)
Sr. Staff Platform/Data Reliability Engineer, Databricks (R5537)

Remote • San Francisco (CA)

On-site
USD 200,000 - 270,000
Senior Data Platform Engineer — AI & Analytics
Senior Data Platform Engineer — AI & Analytics

team.blue Global • United States

Remote
USD 140,000 - 200,000
Senior Platform Engineer - Databricks & AI Platform
Senior Platform Engineer - Databricks & AI Platform

Vizient • Irving (TX)

On-site
USD 102,400 - 179,000
Incentive eligible
Senior Data Platform Engineer: Databricks Lakehouse Lead
Senior Data Platform Engineer: Databricks Lakehouse Lead

Buildertrend • United States

Hybrid
USD 130,000 - 160,000
Health packages
401(k) with matching
Paid time off
+3