Databricks Data Engineer: Build Scalable Pipelines

Zurich North America; New Mexico; On-site; USD 27,681 - 41,522; Full time

Benefits offered by this job

Azure cloud experience

Job summary

Zurich Capability Center is hiring a Data Engineer specialized in Databricks to design, build, and optimize scalable data pipelines and data platforms across lakes, warehouses, and analytics environments. You will partner with data scientists and analysts to deliver reliable datasets and production-ready features.

Required skills include Python, SQL, Databricks, Spark, and ETL/ELT design, with strong data modeling and Git proficiency. Preference for Azure Data services and MLOps exposure.

Qualifications

• Bachelor's degree in CS/Engineering/IS/Math or related field.
• 3+ years in Data Engineering, Data Warehousing or Big Data.
• Strong Python and SQL programming skills.
• Hands-on with Databricks, Apache Spark, and distributed processing.
• Experience designing ETL/ELT data pipelines.
• Experience with Data Lakes and modern data architectures.
• Knowledge of Star Schema and Dimensional Modeling.
• Git and version control experience.
• English level B2 or higher.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks and cloud-based data platforms.
• Develop, optimize, and automate ETL/ELT processes to support analytics, reporting, and ML initiatives.
• Collaborate with Data Scientists, Analysts, ML Engineers, and stakeholders to deliver reliable datasets.
• Implement data ingestion frameworks from multiple structured and unstructured sources.
• Build and maintain data models, data lakes, and data warehouses for analytics.
• Ensure data quality, governance, and security across platforms.
• Optimize Spark workloads and Databricks for performance and cost efficiency.
• Support deployment and operationalization of ML solutions with production-ready datasets.
• Collaborate with Azure, IT, and DevOps to implement CI/CD and DataOps.
• Monitor and improve data platform performance and reliability.

Skills

Python

SQL

Databricks

Apache Spark

ETL/ELT pipelines

Data Lakes

Data modeling

Git

English B2

Education

Bachelor's Degree in CS/Engineering/IS/Math or related field

Tools

Azure Databricks

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Analytics

Azure DevOps

Delta Lake

Airflow

Databricks Workflows

MLflow

Kafka

Job description

Zurich Capability Center is hiring a Data Engineer specialized in Databricks to design, build, and optimize scalable data pipelines and data platforms across lakes, warehouses, and analytics environments. You will partner with data scientists and analysts to deliver reliable datasets and production-ready features.

Required skills include Python, SQL, Databricks, Spark, and ETL/ELT design, with strong data modeling and Git proficiency. Preference for Azure Data services and MLOps exposure.