Senior Principal Data Product Engineer — Lakehouse Quality

Publicis Groupe Holdings B.V; Chicago (IL); On-site; USD 133,000 - 247,000; Full time

Job summary

Epsilon is seeking a Senior Principal Engineer to own the quality and reliability of data products on our lakehouse platform. This hands-on contributor role sits in the pipelines and in the SQL, accountable for data that feeds client work - where correctness and timeliness are product requirements.

You'll build automated QC, diagnose large-scale data issues, and evolve Spark and SQL pipelines while mentoring engineers and analysts across the team.

Qualifications

• 10+ years hands-on with production data across data engineering, data operations, or data platform work.
• Expert SQL, with performance tuning and strong production Python (PySpark preferred).
• Track record designing data quality, validation, monitoring, or reconciliation systems.
• Accountability for production batch pipelines, not only building them.
• Experience with large-scale distributed platforms and data warehouses.
• Software engineering practices: version control, code review, automated testing, CI/CD.
• Comfort with high-cardinality data and deduplication; skeptical of overly-clean data.
• Ability to explain upstream changes to non-engineers.
• Bachelor's degree in a technical field or equivalent experience.

Responsibilities

• Own the quality and reliability of datasets on the lakehouse platform from ingestion through publication.
• Build automated QC layers for completeness, distribution, referential integrity, and anomaly detection.
• Diagnose failures and discrepancies across large data sets and ensure freshness and correctness.
• Automate recurring operations and reduce toil with self-service tooling.
• Evolve Spark and SQL pipelines, optimize refresh strategy and cost, and improve onboarding of sources.
• Set engineering standards and mentor engineers and analysts on the team.

Skills

Hands-on data production

Expert SQL

PySpark

Data quality systems

Production batch pipelines

Distributed platforms

CI/CD

Code review

Troubleshooting large datasets

Mentoring engineers

Education

Bachelor's degree in a technical field

Tools

Databricks

Spark

Greenplum

Postgres

Snowflake

Redshift

Job description

Epsilon is seeking a Senior Principal Engineer to own the quality and reliability of data products on our lakehouse platform. This hands-on contributor role sits in the pipelines and in the SQL, accountable for data that feeds client work - where correctness and timeliness are product requirements.

You'll build automated QC, diagnose large-scale data issues, and evolve Spark and SQL pipelines while mentoring engineers and analysts across the team.