Databricks Data Engineer — Azure & ML Pipelines

Berkley Alternative Markets Tech; Manassas (VA); On-site; USD 90,000 - 130,000; Full time

Benefits offered by this job

Health insurance

401(k) plan

Paid time off

Job summary

A leading insurance technology firm in Manassas, VA, seeks a Databricks Data Engineer to design and maintain scalable data pipelines. This role involves working with advanced tools like Databricks and PySpark to drive analytics and machine learning initiatives. The ideal candidate will have significant experience in Databricks, Python, and data engineering practices. The company offers a competitive salary and benefits package, emphasizing innovation and teamwork in transforming the insurance industry.

Qualifications

• Minimum of 3 years of experience in Databricks, PySpark notebooks, and Python.
• Experience with MLOps and CI/CD for ML.
• Proficiency in SQL, Python, and cloud data platforms.

Responsibilities

• Design, build, and maintain data pipelines using Azure Databricks.
• Convert existing SSIS logic into cloud-native pipelines.
• Collaborate with data scientists to deploy ML models.

Skills

Databricks

PySpark

Python

DevOps

Data Engineering

SQL

Machine Learning

Education

Bachelor’s degree in Computer Science or related field

Tools

Azure Databricks

Delta Lake

MLflow

Power BI

Job description

A leading insurance technology firm in Manassas, VA, seeks a Databricks Data Engineer to design and maintain scalable data pipelines. This role involves working with advanced tools like Databricks and PySpark to drive analytics and machine learning initiatives. The ideal candidate will have significant experience in Databricks, Python, and data engineering practices. The company offers a competitive salary and benefits package, emphasizing innovation and teamwork in transforming the insurance industry.