Staff Data Engineer: AI/ML Data Infra & Databricks Leader

Red Ventures

United States

Hybrid

USD 130,000 - 220,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health Insurance Coverage (medical, in
Life Insurance
Disability Insurance
Flexible Spending Accounts
Holiday Pay
401(k) with match
Employee Assistance Program
Parental Bonding Benefit
Generous PTO

Job summary

Red Ventures is hiring a Staff Data Engineer to own the design and modernization of Databricks environments and to build AI/ML data infrastructure at scale. You will drive architecture decisions, lead multi-team initiatives, and ensure governed, cost-efficient data inputs for ML workloads.

You will design LLM-ready pipelines, feature stores, and vector databases while establishing data governance standards and observability.

Qualifications

  • 10+ years of data engineering experience designing and operating large-scale data platforms.
  • Production experience with Databricks, SQL, Spark and Python.
  • Infrastructure-as-code experience with Terraform for data platforms.
  • Experience with an orchestration tool such as Airflow or Dagster.
  • Experience with streaming platforms such as Kafka, Kinesis or Rudderstack.
  • Experience with data governance frameworks and PII controls.
  • Experience with data quality and observability tools.
  • DevOps and FinOps experience with CI/CD and data contracts.

Responsibilities

  • Own design and modernization of Databricks environments across multiple teams.
  • Build AI/ML data infrastructure including feature stores and vector databases.
  • Develop LLM-ready pipelines with embedding, retrieval and data-versioning.
  • Establish data governance, quality, observability and access controls.
  • Drive platform cost optimization and DevOps practices with data contracts.
  • Mentor engineers and lead design reviews across teams.

Skills

Databricks
SQL
Spark
Python
Terraform
Airflow
Dagster
Kafka
Kinesis
Rudderstack
RBAC
PII controls
Unity Catalog
Data governance
Great Expectations
DevOps

Tools

Databricks
SQL
Spark
Python
Terraform
Airflow
Dagster
Kafka
Kinesis
Rudderstack
Unity Catalog
DataHub
Collibra
Atlan
Great Expectations

Job description

Red Ventures is hiring a Staff Data Engineer to own the design and modernization of Databricks environments and to build AI/ML data infrastructure at scale. You will drive architecture decisions, lead multi-team initiatives, and ensure governed, cost-efficient data inputs for ML workloads.

You will design LLM-ready pipelines, feature stores, and vector databases while establishing data governance standards and observability.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Data Engineer: Databricks ML Platform Lead
Staff Data Engineer: Databricks ML Platform Lead

redventures • United States

Remote
USD 150,000 - 220,000
Medical, dental and vision coverage
Life and disability insurance
Flexible spending accounts
+4
Staff AI & Data Engineer, Databricks — Remote
Staff AI & Data Engineer, Databricks — Remote

Eliassen Group • United States

Remote
USD 110,000 - 124,000
Senior AI/ML Environments Engineer
Senior AI/ML Environments Engineer

FDE Times • Mountain View (CA)

On-site
USD 166,000 - 220,000
Remote Data & AI Engineer: Databricks & ML Ops
Remote Data & AI Engineer: Databricks & ML Ops

RevStar Consulting Inc • Florida

Remote
USD 120,000 - 180,000
Paid Time Off
Remote-First Working Environment
Comprehensive Health Coverage
+5
Senior Staff Data Pipelines Architect
Senior Staff Data Pipelines Architect

Databricks Inc. • San Francisco (CA)

Hybrid
USD 228,600 - 314,250
Comprehensive benefits
Diversity and inclusion commitment
Senior Data Platform Engineer — Databricks & Cloud Infra
Senior Data Platform Engineer — Databricks & Cloud Infra

Rivian • Atlanta (GA), Palo Alto (CA)

On-site
USD 111,000 - 139,000
Data Engineer Mexico
Data Engineer Mexico

V 4c Ai • United States

Remote
USD 85,000 - 120,000
Senior Data & AI Field Engineer - Customer Solutions
Senior Data & AI Field Engineer - Customer Solutions

Databricks Inc. • Denver (CO)

On-site
USD 182,000 - 250,000
Benefits
Data & AI Engineer — Databricks Lakehouse, Remote
Data & AI Engineer — Databricks Lakehouse, Remote

RevStar Consulting Inc • Idaho

Hybrid
USD 120,000 - 180,000
Paid Time Off
Remote-First Working Environment
Comprehensive Health Coverage
+6
Senior Data & AI Forward-Deployed Engineer
Senior Data & AI Forward-Deployed Engineer

Neon • San Francisco (CA)

On-site
USD 182,000 - 250,000