Senior Data Engineer with Databricks

Xebia Group; United States; Remote; USD 110,000 - 170,000; Full time

Job summary

Xebia is seeking a Data Engineer to design and implement scalable data pipelines on Databricks, standardize and migrate legacy pipelines, and develop data transformations using dbt and Python. You will onboard new datasets across regions (US, APAC) and collaborate with data architects and analytics teams to enable data-driven solutions.

The role requires strong experience with cloud platforms (Azure/AWS/GCP), data modeling, and CI/CD practices for data pipelines, with ownership mindset and

Qualifications

• Strong experience as a Data Engineer.
• Hands-on experience with Databricks (production-grade pipelines).
• Very good knowledge of Python for data processing.
• Experience with dbt and data transformation layers.
• Solid understanding of data pipeline design, orchestration, and scalability.
• Experience working with legacy systems and pipeline migration.
• Experience with cloud-based data platforms (Azure / AWS / GCP).
• Knowledge of data modeling (medallion architecture, dimensional models).
• Experience working with financial or market data.
• Familiarity with CI/CD practices for data pipelines
• Ability to work independently and take ownership of data solutions.
• Strong communication skills and team collaboration mindset.

Responsibilities

• Design scalable data pipelines on Databricks.
• Standardize existing pipelines (gold standard approach).
• Migrate and refactor legacy data pipelines.
• Develop data transformations using dbt and Python.
• Support onboarding of new datasets (US, APAC regions).
• Collaborate with data architects and analytics teams.
• Contribute to future Data Science and MLOps foundations.
• Work under capacity constraints and optimize performance and resource usage.

Skills

Databricks

Python

dbt

Data pipelines

Cloud platforms

CI/CD for data

Data modeling

Legacy migration

Communication skills

Tools

Azure

AWS

GCP

dbt

Job description

Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable products, platforms, and data-driven solutions.

We specialize in Artificial Intelligence, Data and Cloud, Intelligent Automation, and Digital Products, combining deep technical expertise with a strong focus on engineering excellence and a people-first culture.

In the CEE region, we’re a team of nearly 1,000 experts delivering modern applications, data platforms, and AI solutions for clients such as Millennium, ING, Play, Edenred, Arabian Drilling, FedEx, Leroy Merlin, Truecaller, Volotea, Schmitz Cargobull, and many, many more. We work with leading technologies including AWS, Azure, GCP, Databricks, and Snowflake, and combine strong engineering culture with a consulting mindset and a continuous focus on growth and knowledge sharing.

You will be:

• Designing and implementing scalable data pipelines on Databricks.
• Standardizing existing pipelines (gold standard approach).
• Migrating and refactoring legacy data pipelines.
• Developing data transformations using dbt and Python.
• Supporting onboarding of new datasets (US, APAC regions).
• Collaborating with data architects and analytics teams.
• Contributing to future Data Science and MLOps foundations.
• Working under capacity constraints and optimizing performance and resource usage.

Your profile:

• Strong experience as a Data Engineer.
• Hands-on experience with Databricks (production-grade pipelines).
• Very good knowledge of Python for data processing.
• Experience with dbt and data transformation layers.
• Solid understanding of data pipeline design, orchestration, and scalability.
• Experience working with legacy systems and pipeline migration.
• Experience with cloud-based data platforms (Azure / AWS / GCP).
• Knowledge of data modeling (medallion architecture, dimensional models).
• Experience working with financial or market data.
• Familiarity with CI/CD practices for data pipelines
• Ability to work independently and take ownership of data solutions.
• Strong communication skills and team collaboration mindset.

Work from the European Union region and a work permit are required.

Nice to have:

• Experience with Data Science environments or MLOps.
• Knowledge of ML lifecycle support in Databricks.
• Experience working in global, distributed teams.
• Exposure to large-scale, multi-region data onboarding projects.

Recruitment Process:

CV review – HR call – Interview – Client Interview – Decision