Senior Data Engineer with Databricks

xebiacee; Stati Uniti; Remoto; USD 100.000 - 150.000; Tempo pieno

Descrizione del lavoro

xebiacee is seeking a data engineer to design and build scalable data pipelines on Databricks, enabling production workloads and multi-region onboarding for analytics, data science, and MLOps.

You will develop pipelines with Python and dbt, migrate legacy flows, and collaborate with data architects to optimize performance. Remote work from within the European Union is supported; a valid work permit is required.

Competenze

• Strong professional experience in data engineering, including production-grade Databricks pipelines.
• Good command of Python for data processing and experience with dbt.
• Solid understanding of pipeline design, orchestration, and scalability.
• Experience with legacy pipeline migration and cloud data platforms such as Azure, AWS, or GCP.
• Knowledge of data modeling, including medallion architecture or dimensional models; financial or market data experience is relevant.
• Familiarity with CI/CD for data pipelines, independent ownership, and clear team communication.

Mansioni

• Design and implement scalable pipelines and transformations using Databricks, Python, and dbt.
• Standardize existing pipelines and migrate or refactor legacy data flows.
• Support onboarding datasets from the United States and Asia-Pacific regions.
• Work with data architects and analytics teams to shape data solutions.
• Optimize pipeline performance and resource use under capacity constraints.
• Contribute to foundations for future data science and MLOps work.

Conoscenze

Databricks

Python

dbt

CI/CD

Data modeling

Pipeline orchestration

Strumenti

Databricks

dbt

Descrizione del lavoro

Role overview

Build and improve scalable data pipelines, with a focus on production workloads running on Databricks. The work includes modernizing legacy flows, onboarding data from multiple regions, and laying foundations that can support future analytics, data science, and machine learning operations.

Responsibilities

• Design and implement scalable pipelines and transformations using Databricks, Python, and dbt.
• Standardize existing pipelines and migrate or refactor legacy data flows.
• Support onboarding datasets from the United States and Asia-Pacific regions.
• Work with data architects and analytics teams to shape data solutions.
• Optimize pipeline performance and resource use under capacity constraints.
• Contribute to foundations for future data science and MLOps work.

Requirements

• Strong professional experience in data engineering, including production-grade Databricks pipelines.
• Good command of Python for data processing and experience with dbt.
• Solid understanding of pipeline design, orchestration, and scalability.
• Experience with legacy pipeline migration and cloud data platforms such as Azure, AWS, or GCP.
• Knowledge of data modeling, including medallion architecture or dimensional models; financial or market data experience is relevant.
• Familiarity with CI/CD for data pipelines, independent ownership, and clear team communication.

Nice to have

• Experience with data science environments, MLOps, or machine learning lifecycle support in Databricks.
• Experience collaborating across distributed teams or contributing to large, multi-region data onboarding.

Benefits and work setup

• Work must be performed from within the European Union; a valid work permit is required.