Databricks Data Engineer - Onsite, Azure & MLOps

Berkley Alternative Markets IO; Chesterfield (MO); On-site; USD 110,000 - 140,000; Full time

Job summary

Berkley Alternative Markets IO in Missouri is seeking a Databricks Data Engineer to design, build, and maintain scalable data pipelines in cloud environments. You will work with Databricks, Delta Lake, PySpark, MLflow, and Azure Databricks to enable analytics, AI, and ML at scale.

The role requires on-site work Monday–Thursday at Manassas, VA or Chesterfield, MO, collaboration with data scientists and product managers, and a strong focus on governance, observability, and CI/CD automation.

Qualifications

• Minimum 3 years of experience in Databricks, PySpark, Python, DevOps, data engineering.
• Bachelor's degree in CS/IS/Engineering or related field required.
• Experience with Azure Databricks and MLOps is essential.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines.
• Migrate legacy SSIS logic to cloud-native Databricks pipelines.
• Partner with data scientists and product manager to deploy models.
• Ensure observability, governance, and CI/CD for pipelines.

Skills

Databricks

PySpark

Python

SQL

ML/AI

DevOps

Data governance

Education

Bachelor’s degree in CS/IS/Eng

Master’s degree preferred

Tools

Databricks

Delta Lake

Azure

MLflow

GitHub Actions

Job description

Berkley Alternative Markets IO in Missouri is seeking a Databricks Data Engineer to design, build, and maintain scalable data pipelines in cloud environments. You will work with Databricks, Delta Lake, PySpark, MLflow, and Azure Databricks to enable analytics, AI, and ML at scale.

The role requires on-site work Monday–Thursday at Manassas, VA or Chesterfield, MO, collaboration with data scientists and product managers, and a strong focus on governance, observability, and CI/CD automation.