Senior Data Platform Engieer

Motion Recruitment Partners LLC; Lake Forest (IL); On-site; USD 90,000 - 140,000; Full time

Job summary

An established industry player is seeking a Senior Data Engineer to join their innovative Data Engineering Center of Excellence. This role offers the opportunity to implement cutting-edge data processing systems using technologies like Databricks and Snowflake, while collaborating with various teams to enhance data reliability and accuracy. The position promises a dynamic work environment where your contributions will directly impact the efficiency of data engineering practices. With a focus on both batch and streaming ETL processes, you will be at the forefront of data engineering, shaping the future of data management in a company recognized as one of the best workplaces. If you're passionate about data and eager to make a difference, this role is for you.

Benefits

Performance bonus eligibility

Medical insurance

Dental insurance

Vision insurance

Life insurance

PTO and holidays

6% 401(k) match

Employee discounts

Parental leave

Tuition reimbursement

Qualifications

• 5+ years of experience in data engineering with a focus on ETL processes.
• Strong knowledge of data processing systems and data modeling.

Responsibilities

• Implement scalable data processing systems using Databricks and Snowflake.
• Design solutions to ensure data reliability and accuracy.

Skills

Spark

Python

Scala

Snowflake

Databricks

Data Management Lifecycle

Shell Scripting

Tools

Databricks Workflows

Apache Airflow

Docker

Kubernetes

Jenkins

Github Actions

Gitlab CI/CD

AWS Glue

AWS Athena

AWS S3

Job description

We are looking for a Senior Data Engineer to join one of the 100 Best Companies to Work For, with offices in the Chicago suburbs as well as in the city. The role will be within their Data Engineering 'Center of Excellence' and will impact and improve the way Data Engineering teams work.

Responsibilities

1. Implement highly efficient, reusable, and scalable data processing systems and pipelines in Databricks and Snowflake.
2. Design and implement technical solutions to ensure data reliability and accuracy.
3. Develop data models and mappings; build new data assets required by users; perform exploratory data analysis.
4. Educate data engineering teams on adopting new data patterns and tools.
5. Engage with AI, Platform, and Business Analytics teams as SME to build pipelines and data assets.

Desired Skills & Experience

• Experience with batch and streaming ETL using Spark, Python, Scala, Snowflake, or Databricks.
• Experience with workflow tools like Databricks Workflows, Apache Airflow, or Luigi.
• Experience preparing structured and unstructured data for data science models.
• Knowledge of containerization and orchestration technologies (Docker, Kubernetes) and shell scripting.
• Experience with CI/CD pipelines using Jenkins, Github Actions, or Gitlab CI/CD.
• Familiarity with AWS services such as Glue, Athena, Lambda, S3, and DynamoDB.
• Experience with data management lifecycle, including data quality functions like standardization, transformation, and matching.

The Offer

• Performance bonus eligibility.
• Benefits include medical, dental, vision, and life insurance; PTO and holidays; 6% 401(k) match; employee discounts; parental leave; donation matching; tuition reimbursement; wellbeing programs.

Applicants must be authorized to work in the US on a full-time basis now and in the future.