Lead Data Engineer, Databricks & Azure

84.51°; Chicago, Cincinnati (IL, OH); On-site; USD 125,000 - 207,000; Full time

Benefits offered by this job

Health insurance

401(k) with Roth option and matching

Health Savings Account

Paid time off + holidays

Job summary

84.51° is a retail data science and media company driving personalized shopper experiences for Kroger and partners. As Lead Data Engineer, you will design and implement data ingestion, storage, transformation, and distribution solutions on Azure Cloud, leveraging Databricks, Spark, PySpark, JSON, and SQL to power customer personalization at scale.

You will collaborate with Product, Data Science, and Architecture to set strategy, mentor engineers, and build scalable enterprise applications in a

Qualifications

• 5+ years of professional data development experience.
• Strong understanding of Agile Principles (Scrum).
• 5+ years proven experience with Spark (Databricks, PySpark).
• Full understanding of ETL concepts and data warehousing.

Responsibilities

• Take ownership of features and drive them through the SDLC, including external and internal applications and process improvements.
• Develop business domain understanding and bridge to technical stakeholders.
• Set technical strategy in partnership with Data Science, Product, Architecture, and Kroger partners.
• Oversee teams of Engineers including offshore/onshore resources.
• Mentor junior resources.
• Lead design and development of Databricks-based solutions.
• Develop REST APIs hosted in Kubernetes.
• Perform unit and integration testing.
• Collaborate with senior resources to ensure best practices.
• Bring new perspectives and continuous improvement mindset.

Skills

Spark

Python

Kubernetes

REST APIs

Agile

ETL

Data Warehousing

Education

Bachelor's degree in Computer Science / MIS / Math / Analytics

Tools

Databricks

PySpark

Job description

84.51° is a retail data science and media company driving personalized shopper experiences for Kroger and partners. As Lead Data Engineer, you will design and implement data ingestion, storage, transformation, and distribution solutions on Azure Cloud, leveraging Databricks, Spark, PySpark, JSON, and SQL to power customer personalization at scale.

You will collaborate with Product, Data Science, and Architecture to set strategy, mentor engineers, and build scalable enterprise applications in a