Remote Senior Data Engineer - AI Data Pipelines

Eightelevengroup; Clayton (MO); On-site; USD 110,000 - 124,000; Full time

Benefits offered by this job

401(k) matching plan

Medical, dental, and vision plans

Paid company holidays

Regular engagement check‑ins

Job summary

Eightelevengroup is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines in a remote, contract‑to‑hire role for a Fortune 50 healthcare insurer. You will work on production data solutions powering analytics, ML, and AI initiatives within a modern distributed data environment.

You will collaborate with cross‑functional teams, implement data quality and governance, and contribute to live production systems while leveraging CI/CD and Git workflows.

Qualifications

• Senior‑level, hands‑on experience building production data pipelines in distributed data environments.
• Experience with Databricks Data Engineering, Spark, or Hadoop in production.
• Strong SQL and Python/PySpark for data‑engineering tasks.
• DevOps, CI/CD pipelines, and Git‑based workflows.

Responsibilities

• Design, build, and maintain scalable batch and streaming data pipelines.
• Develop production data solutions using SQL, Python, and PySpark.
• Troubleshoot, optimize, and enhance existing pipelines for performance.
• Create reusable ingestion, transformation, and data‑product patterns for analytics.
• Implement data‑quality checks, monitoring, and lineage for production pipelines.
• Apply CI/CD practices to test, version, and deploy data assets across environments.
• Collaborate with engineers, data scientists, and platform teams on healthcare solutions.
• Support AI‑enabling data assets like feature tables and embeddings.
• Ensure governance, cataloging, and discoverability in regulated environments.

Skills

Databricks

Spark

SQL

Python

PySpark

Git

CI/CD

ETL/ELT

Parquet/Delta

Linux scripting

Tools

Databricks

Spark

Hadoop

Job description

Eightelevengroup is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines in a remote, contract‑to‑hire role for a Fortune 50 healthcare insurer. You will work on production data solutions powering analytics, ML, and AI initiatives within a modern distributed data environment.

You will collaborate with cross‑functional teams, implement data quality and governance, and contribute to live production systems while leveraging CI/CD and Git workflows.