Senior ML Data Pipeline Engineer (Spark/Databricks, LLMs)

SD Solutions

United States

Remote

USD 120,000 - 160,000

Full time

6 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

SD Solutions is seeking a data engineer to own ML data pipelines in a fast-moving analytics environment. You will build and run batch pipelines on Spark and Databricks, onboard new data sources, and embed LLM steps to enhance batch processing.

With 5+ years in data engineering or MLOps, you bring hands-on Spark and Databricks expertise, focusing on reliability, scalability, and collaboration with product teams.

Qualifications

  • 5+ years in data engineering, MLOps or ML platform roles.
  • Production Spark: complex transformations, window functions, large joins.

Responsibilities

  • Own ML data pipelines: build and run batch pipelines on Spark and Databricks.
  • Onboard new data sources: integrate internal databases, external APIs, partner feeds.
  • Use LLMs inside pipelines: add LLM steps to batch jobs.

Skills

Data engineering
MLOps
LLMs in pipelines
Batch data processing

Tools

Spark
Databricks

Job description

SD Solutions is seeking a data engineer to own ML data pipelines in a fast-moving analytics environment. You will build and run batch pipelines on Spark and Databricks, onboard new data sources, and embed LLM steps to enhance batch processing.

With 5+ years in data engineering or MLOps, you bring hands-on Spark and Databricks expertise, focusing on reliability, scalability, and collaboration with product teams.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior MLOps Engineer (Data & ML Platform) (Fully Remote)
Senior MLOps Engineer (Data & ML Platform) (Fully Remote)

SD Solutions • United States

Remote
USD 120,000 - 160,000
Senior ML Data Engineer: Real-Time ML Pipelines & DataOps
Senior ML Data Engineer: Real-Time ML Pipelines & DataOps

Cedent • United States

Hybrid
USD 55,104 - 82,656
Health
Dental
Vision Insurance
Senior AI Data Engineer: Databricks, Spark & ML Pipelines
Senior AI Data Engineer: Databricks, Spark & ML Pipelines

HCL Global Systems Inc • Princeton (NJ)

On-site
USD 150,000 - 190,000
Senior Data Engineer — LLM & Spark Platform (Delta Lake)
Senior Data Engineer — LLM & Spark Platform (Delta Lake)

Altimetrik • Cary (NC)

On-site
USD 120,000 - 170,000
Senior Data Engineer - Scalable Pipelines & ML-Ready Data
Senior Data Engineer - Scalable Pipelines & ML-Ready Data

Compunnel Inc. • Jersey City (NJ)

On-site
USD 100,000 - 130,000
Senior Data Engineer for ML & Analytics Pipelines
Senior Data Engineer for ML & Analytics Pipelines

TechDigital Group • Minneapolis (MN)

On-site
USD 120,000 - 165,000
Senior Databricks Engineer - Scala & Spark
Senior Databricks Engineer - Scala & Spark

Diverse Lynx, LLC • Quincy (MA)

On-site
USD 130,000 - 170,000
LLM Ops Engineer — Databricks Lakehouse & MLOps
LLM Ops Engineer — Databricks Lakehouse & MLOps

Health Business Solutions LLC • Florida

On-site
USD 120,000 - 180,000
ML Data Engineer: Scale Pipelines & MLOps
ML Data Engineer: Scale Pipelines & MLOps

PODS Enterprises, LLC • Clearwater (FL)

On-site
USD 120,000 - 180,000
Senior MLOps Engineer
Senior MLOps Engineer

Seminole Hard Rock Support Services • Florida

On-site
USD 140,000 - 190,000