Senior Data Engineer — Recommendations & Data Platform

Mill; San Bruno (CA); On-site; USD 185,000 - 210,000; Full time

Job summary

Mill is a waste prevention technology company tackling food waste at its source. We are seeking a Data Engineer to architect data pipelines, own the end-to-end data warehouse, and ship actionable insights via a customer-facing recommendation engine.

You'll partner with product, engineering, data analytics, and marketing to ensure data quality, governance, and scalable analytics across Mill's product and operations. Salary ranges between $185k–$210k.

Qualifications

• Designed, built, or operated a recommendation system in production, combining multiple data sources into a single customer-facing output.
• Experience building recommendation or personalization logic using LLMs in a live product.
• Built data pipelines in production using Python, SQL, and tools like dbt, Airflow, Fivetran against a cloud data warehouse, including handling failures, backfills, and schema changes.
• Brought CI/CD discipline to data pipelines or product logic with automated testing and staged rollout.
• 5 years of experience operating data engineering systems in production.
• A bias toward clarity and action.

Responsibilities

• Build and operate the customer-facing recommendation engine that turns food waste data into actionable recommendations — purchasing suggestions, anomaly explanations, operational nudges — including LLM-based logic where useful
• Aggregate and unify disparate data streams to guarantee the precision, reliability, and protection of our analytical assets.
• Co-create and drive Mill's self-serve analytics strategy with analysts and data scientists — defining golden datasets, setting SLAs for freshness and reliability, and establishing the governance model that keeps self-serve trustworthy at scale
• Own data quality monitoring against those SLAs — build alerting, validation frameworks, and observability tooling so golden dataset issues get caught before they become business problems
• Collaborate with software engineers to instrument new product features and ensure event data flows cleanly into the warehouse, meeting the governance and quality bar required for it to become part of a golden dataset
• Design, build, and maintain scalable data pipelines across Mill's product and operational systems
• Define and maintain the metrics, table endorsements, and business logic that analysts and stakeholders rely on — so everyone across the company is working from the same numbers

Skills

Data modeling

Python

SQL

LLMs

CI/CD

Observability

Tools

dbt

Airflow

Fivetran

Snowflake

BigQuery

Redshift

Job description

Mill is a waste prevention technology company reimagining what it means to eliminate waste, starting with food. We build smart systems and infrastructure for homes, businesses, and municipalities that transform food scraps from landfill-bound waste into valuable resources, including chicken feed. Tens of thousands of Mill’s residential food recyclers are already helping households divert millions of pounds of food scraps every year, paving the way for our upcoming launch of Mill Commercial—the industry’s first end-to-end solution for managing, understanding, and preventing food waste in commercial environments (e.g. grocery, restaurants, food services). At Mill, we are passionate about building easy-to-use, beautifully designed technologies that keep food in the food system and out of landfills.

As a Data Engineer at Mill, you\'ll touch systems end-to-end — from raw ingestion to the recommendation a customer sees in the app to managing the data warehouse. You\'ll architect a warehouse model one week and tune recommendation logic the next. You\'ll partner closely with product, engineering, data analytics, and marketing teams.

What You\'ll Do

• Build and operate the customer-facing recommendation engine that turns food waste data into actionable recommendations — purchasing suggestions, anomaly explanations, operational nudges — including LLM-based logic where useful
• Aggregate and unify disparate data streams to guarantee the precision, reliability, and protection of our analytical assets.
• Co-create and drive Mill\'s self-serve analytics strategy with analysts and data scientists — defining golden datasets, setting SLAs for freshness and reliability, and establishing the governance model that keeps self-serve trustworthy at scale
• Own data quality monitoring against those SLAs — build alerting, validation frameworks, and observability tooling so golden dataset issues get caught before they become business problems
• Collaborate with software engineers to instrument new product features and ensure event data flows cleanly into the warehouse, meeting the governance and quality bar required for it to become part of a golden dataset
• Design, build, and maintain scalable data pipelines across Mill\'s product and operational systems
• Define and maintain the metrics, table endorsements, and business logic that analysts and stakeholders rely on — so everyone across the company is working from the same numbers

What We\'re Looking For

• Have designed, built, or operated a recommendation system in production — one that combines multiple data sources into a single customer-facing output — not just contributed data to someone else\'s model
• Experience building recommendation or personalization logic using LLMs (prompt-based scoring, retrieval-augmented generation, agent-based reasoning) in a live product, not just a prototype
• Have built and operated data pipelines in production using Python, SQL, and tools like dbt, Airflow, Fivetran, or similar against a cloud data warehouse (e.g., Snowflake, BigQuery, Redshift) — including handling failures, backfills, and schema changes after launch
• Have brought CI/CD discipline to data pipelines or product logic (automated testing, staged rollout, rollback), with a track record of measuring whether a change actually improved outcomes, not just shipping it
• 5 years of experience operating data engineering systems in production
• A bias toward clarity and action

Nice to Have

• Exposure to distributed systems concepts (partitioning, consistency, fault tolerance)
• Hands-on experience with infrastructure as code (Terraform, Pulumi) in a cloud environment
• Experience with Hex, Mixpanel, Tableau, or similar BI/analytics tools
• Familiarity with data contract or data mesh patterns
• Experience with event tracking or product analytics

The estimated base salary range for this position is $185k to $210k, which does not include the value of benefits or a potential equity grant. A wide range of factors are considered in making compensation decisions, including but not limited to skill sets, market conditions, experience and training, licensure and certifications, and business and organizational needs.