DC Data Platform Lead — Scale ETLs & Impact

Afresh; United States; On-site; USD 168,000 - 273,000; Full time

Job summary

Afresh, the AI platform for grocery, is hiring a Staff Data Engineer for the Distribution Center Solutions team to design and scale ETL pipelines handling large customer datasets, using PySpark and DBT.

You will lead technical efforts, mentor engineers, and collaborate with product, engineering, and applied science to deliver scalable data solutions that accelerate customer onboarding and power our ML grocery platform.

Qualifications

• Significant experience designing and maintaining ETLs that process large-scale datasets.
• Proficiency with Python, PySpark, SQL, and experience working on platforms/tools like Databricks, Snowflake, or DBT.
• 2+ years experience in a technical lead role (e.g. Tech Lead or Engineering Manager).

Responsibilities

• Engineering & Data Pipelines: Design, build, and optimize robust ETLs using PySpark and DBT to process large-scale customer datasets while developing tools and frameworks to streamline data integrations and improve scalability.
• Technical Leadership & Strategy: Define the technical vision for DC data architecture, mentor engineers, and manage external contractors to ensure the team delivers high-quality, practical solutions for current and future needs.
• Cross-functional Collaboration: Partner with product, engineering, and applied science teams to scope work and deliver data solutions that address real-world challenges in customer data quality and product feature requirements.

Skills

ETL design

Python

PySpark

SQL

Technical leadership

Tools

Databricks

Snowflake

DBT

Job description

Afresh, the AI platform for grocery, is hiring a Staff Data Engineer for the Distribution Center Solutions team to design and scale ETL pipelines handling large customer datasets, using PySpark and DBT.

You will lead technical efforts, mentor engineers, and collaborate with product, engineering, and applied science to deliver scalable data solutions that accelerate customer onboarding and power our ML grocery platform.