Senior Data Engineer / Databricks / End-to-End Ownerhp

Motion Recruitment; Chicago (IL); On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Medical, Dental, and Vision Insurance

Vacation Time

Stock Options

Job summary

Motion Recruitment in Chicago is seeking a Data Engineer to support a large-scale cloud data platform. You will build modern data pipelines using Azure Databricks, Python, SQL and Spark in a production environment, collaborating with engineers and business partners to turn raw data into analytics-ready datasets.

This hands-on role emphasizes end-to-end pipeline ownership, data governance, and evolving the platform within an enterprise.

Qualifications

• 6+ years of hands-on data engineering experience.
• Strong experience building pipelines using Databricks.
• Production data processing with Python and SQL.
• Working knowledge of Apache Spark and Delta Lake.
• Experience with cloud data storage (ADLS, object storage).
• Experience with data modeling and schema design.
• Familiarity with Git, CI/CD concepts, and basic testing/monitoring.

Responsibilities

• Own end-to-end data pipelines from ingestion to analytics-ready datasets.
• Collaborate with BI, AI, and product teams to support reporting and insights.
• Contribute to data governance and platform evolution in a cloud environment.

Skills

Databricks

Python

SQL

Spark

Delta Lake

Git

CI/CD

Tools

Azure Databricks

ADLS/Object Storage

Job description

A well-established, global professional services organization is expanding its Data & AI engineering team and hiring a Data Engineer to support a large-scale cloud data platform initiative. This role is based in Chicago and focuses on building modern data pipelines using Azure Databricks, Python, SQL, and Spark in a production environment.

This is a hands-on engineering opportunity for someone who enjoys owning data pipelines end-to-end and wants exposure to enterprise-scale data challenges. You’ll work alongside experienced engineers and business partners to turn raw data into trusted, analytics-ready datasets that power reporting, insights, and AI use cases.

This team is early in its Databricks journey, which means your work will matter immediately. You’ll help shape how pipelines are built, how data is governed, and how the platform evolves over time. It’s ideal for a strong mid-level engineer who wants to deepen their Databricks expertise while working on a highly visible, long-term transformation initiative.

Required Skills & Experience

• 6+ years of hands-on data engineering experience
• Strong experience building pipelines using Databricks
• Proficiency in Python and SQL for production data processing
• Working knowledge of Apache Spark and Delta Lake
• Experience with cloud data storage (ADLS, object storage)
• Experience with data modeling and schema design
• Familiarity with Git, CI/CD concepts, and basic testing/monitoring

Desired Skills & Experience

• Azure Databricks in an enterprise environment
• Experience with Unity Catalog and Databricks Jobs/Workflows
• Exposure to streaming or CDC pipelines
• Experience supporting BI or AI use cases
• Familiarity with Infrastructure as Code

What You Will Be Doing

Tech Breakdown

• 70% Azure Databricks / Spark / Delta Lake
• 20% Python & SQL development
• 10% CI/CD, testing, and optimization

Daily Responsibilities

• 80% Hands-On Engineering
• 10% Collaboration with BI, AI, and product teams
• 10% Documentation, reviews, and platform improvements

The Offer

• Bonus eligible

You Will Receive The Following Benefits

• Medical, Dental, and Vision Insurance
• Vacation Time
• Stock Options

Applicants must be currently authorized to work in the US on a full-time basis now and in the future.

Posted By: Rose Baldonado