Data Engineer

Mill; San Bruno (CA); On-site; USD 185,000 - 210,000; Full time

Job summary

Mill is seeking a Data Engineer in California to own end-to-end data systems from ingestion to customer-facing recommendations. You will architect data warehouse models, tune recommendation logic, and collaborate with product, engineering, analytics, and marketing teams to drive insight and impact.

You will design and maintain scalable pipelines, operate the recommendation engine with multi-source data, and ensure data quality with monitoring and CI/CD practices.

Qualifications

• 5+ years of experience operating data engineering systems in production.
• Proficient in Python and data tooling (dbt, Airflow, Fivetran).
• Strong SQL skills and experience with a cloud data warehouse (Snowflake, BigQuery, Redshift).
• Experience with recommendation systems or multi-source data pipelines.
• Experience with CI/CD for data pipelines and model updates.

Responsibilities

• Design, build, and maintain scalable data pipelines across Mill's product and operational systems.
• Operate the customer-facing recommendation engine, including LLM-based logic when useful.
• Design transformation pipelines for food data from multiple sources.
• Partner with analytics and marketing for self-serve tools.
• Own data quality monitoring and observability tooling.

Skills

Python

SQL

dbt

Airflow

Fivetran

CI/CD

LLM-based logic

data pipelines

cloud data warehouse

Tools

dbt

Airflow

Fivetran

Terraform

Job description

The Role

As a Data Engineer at Mill, you\'ll touch systems end-to-end — from raw ingestion to the recommendation a customer sees in the app to managing the data warehouse. You\'ll architect a warehouse model one week and tune recommendation logic the next. You\'ll partner closely with product, engineering, data analytics, and marketing teams.

What You\'ll Do

• Design, build, and maintain scalable data pipelines across Mill\'s product and operational systems
• Build and operate the customer-facing recommendation engine — including LLM-based logic where useful — that turns characterized food waste data into actionable recommendations: purchasing suggestions, anomaly explanations, operational nudges
• Design transformation and integration pipelines for food data coming from multiple sources — including agent-based reconciliation where it helps — handling schema changes, validation, and consistency issues
• Partner with data analytics and marketing teams to support self-serve analytics tools
• Own data quality monitoring — build alerting, validation frameworks, and observability tooling
• Bring CI/CD discipline to pipeline — automated tests, staged rollouts, and rollback paths — and track recommendation accuracy over time so we know whether a change actually helped
• Define and maintain the metrics, table endorsements, and business logic that analysts and stakeholders rely on — so everyone across the company is working from the same numbers

What We\'re Looking For

• 5 years of experience operating data engineering systems in production
• Have built and operated data pipelines in production using Python and tools like dbt, Airflow, Fivetran, or similar — including handling failures, backfills, and schema changes after launch
• Strong SQL skills and experience with a cloud data warehouse (e.g., Snowflake, BigQuery, Redshift)
• Experience with recommendation systems or pipelines that combine multiple data sources into a single product-facing output, in production — including recommendation logic built with LLMs
• Have set up CI/CD for data pipelines or product logic (automated testing, staged rollout, rollback), and have measured whether a change to a recommendation or model actually improved outcomes, not just shipped it
• A bias toward clarity and action
• Comfort working in a collaborative environment where data consumers are partners, not just stakeholders

Nice to Have

• Exposure to distributed systems concepts (partitioning, consistency, fault tolerance)
• Hands-on experience with infrastructure as code (Terraform, Pulumi) in a cloud environment
• Experience with Hex, Mixpanel, Tableau, or similar BI/analytics tools
• Familiarity with data contract or data mesh patterns
• Experience with event tracking or product analytics

The estimated base salary range for this position is $185k to $210k, which does not include the value of benefits or a potential equity grant. A wide range of factors are considered in making compensation decisions, including but not limited to skill sets, market conditions, experience and training, licensure and certifications, and business and organizational needs.