Lead Data Engineer, Consumer

Versant Media

New York (NY)

On-site

USD 170,000 - 190,000

Full time

6 days ago
Be an early applicant
Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Versant Media is seeking a hands-on Lead Data Engineer to guide end-to-end data product engineering, delivering trusted Silver-layer datasets from source systems into the enterprise data platform. This role blends technical leadership with delivery accountability across ingestion, transformation, governance, and team execution.

Based in New York, NY, the position focuses on building scalable data pipelines, data quality, and governance practices to enable Gold-layer analytics and AI consumption.

Qualifications

  • 8+ years in data engineering or software/data-platform work, with 2+ years leading engineers or delivery teams.
  • Proven design and delivery of enterprise-scale ingestion and transformation pipelines.
  • Strong hands-on skills in SQL, Python, Spark/PySpark and modern ELT/ETL patterns.

Responsibilities

  • Lead delivery of source-to-Silver data products across the data stack.
  • Oversee a distributed/offshore team of Data Engineers and drive delivery commitments.
  • Translate roadmaps into engineering plans with milestones and dependencies.
  • Design, oversee ingestion, transformation, standardization, orchestration, and publication of data.

Skills

SQL
Python
Spark/PySpark
ELT/ETL
Databricks
Delta Lake
CI/CD
Bronze/Silver/Gold
change-data-capture

Tools

Databricks
Delta Lake
Unity Catalog

Job description

Versant Media is seeking a hands-on Lead Data Engineer to guide end-to-end data product engineering, delivering trusted Silver-layer datasets from source systems into the enterprise data platform. This role combines technical leadership with delivery accountability across ingestion, transformation, governance, and team execution.

Key Responsibilities
  • Lead delivery of source-to-Silver data products.
  • Lead and develop a distributed/offshore team of Data Engineers responsible for delivery from source ingestion through curated Silver-layer data products.
  • Translate product roadmaps and requirements into engineering plans, including milestones, estimates, dependencies, and delivery commitments.
  • Design and oversee ingestion, transformation, standardization, orchestration, and publication of data from operational, SaaS, file, streaming, API, and partner sources.
  • Ensure Silver-layer data products are validated, standardized, documented, reusable, performant, secure, and ready for Gold-layer analytics and AI consumption.
  • Establish repeatable patterns for batch, incremental, change-data-capture, and event-driven processing.
  • Build scalable engineering foundations through reusable frameworks, templates, libraries, and pipeline patterns that reduce repeated effort across products and domains.
  • Apply modern engineering practices, including source control, peer review, automated testing, CI/CD, observability, release management, and incident remediation.
  • Define and maintain standards for naming, partitioning, schema evolution, error handling, replay/recovery, performance, cost management, and documentation.
  • Partner with the Data Platform team to use approved workspace, compute, storage, security, and deployment patterns.
  • Identify technical debt and lead pragmatic improvements to increase delivery speed, reliability, and maintainability.
  • Collaborate with Data Modelers on canonical entities, conformed dimensions, data contracts, source-to-target mappings, and enterprise modeling standards.
  • Partner with Product Managers and domain leaders to clarify outcomes, source-system realities, priority use cases, and acceptance criteria.
  • Coordinate dependencies with source-system owners, platform teams, analytics teams, and external partners.
  • Establish data profiling, reconciliation, quality testing, freshness monitoring, lineage, and alerting for each delivered data product.
  • Ensure source-to-Silver traceability, including authoritative source identification, transformation logic, ownership, metadata, and data-quality expectations.
  • Implement access controls, sensitive-data classification, masking, retention, and regional/data-residency requirements.
  • Drive resolution of data defects, schema changes, pipeline failures, and quality issues through clear ownership and service-level expectations.
  • Set priorities, technical direction, delivery expectations, and quality standards for the engineering team.
  • Coach engineers in data engineering, cloud development, testing, observability, and product-oriented delivery practices.
  • Create an effective onshore/offshore delivery model with defined handoffs, overlap hours, documentation standards, ceremonies, and escalation paths.
  • Communicate delivery progress, risks, tradeoffs, and decisions clearly to technical and business stakeholders.
  • Build a culture of ownership, continuous improvement, and reliable execution.
Required Qualifications
  • 8+ years of data engineering, software engineering, or data-platform experience, including 2+ years leading engineers, technical workstreams, or large-scale delivery.
  • Demonstrated experience designing and delivering enterprise-scale ingestion and transformation pipelines.
  • Strong hands-on expertise in SQL, Python, Spark/PySpark, and modern ELT/ETL patterns.
  • Experience with Databricks, Delta Lake, or a comparable lakehouse platform; experience with cloud storage, orchestration, and CI/CD.
  • Strong understanding of Bronze/Silver/Gold (or equivalent layered data-platform patterns).
  • Experience implementing data quality, observability, lineage, metadata, and production support practices.
  • Proven ability to partner with architects, data modelers, product leaders, and domain stakeholders.
  • Experience leading distributed teams and creating effective delivery practices across time zones.
  • Strong communication skills, including the ability to explain technical choices, risks, and tradeoffs to non-technical stakeholders.
Technologies
  • SQL
  • Python
  • Spark
  • PySpark
  • ELT
  • ETL
  • Databricks
  • Delta Lake
  • CI/CD
  • Bronze/Silver/Gold
  • change-data-capture
Preferred Qualifications
  • Experience with Unity Catalog or comparable governance, catalog, and access-control capabilities.
  • Experience with data contracts, schema evolution, change data capture, APIs, event streaming, and large-volume data processing.
  • Experience supporting multiple data domains, global data products, or regional data-residency requirements.
  • Experience with infrastructure-as-code, DataOps, and automated environment provisioning.
  • Experience in a regulated, high-scale, or operationally sensitive environment.
Role Details
  • Location: New York, NY (onsite)
  • Compensation: USD 170,000 - 190,000 per year
  • Experience Level: 8+ years
Additional Information
  • As part of the selection process, external candidates may be required to attend an in-person interview with a VERSANT Media employee at one of its locations prior to a hiring decision.
  • Qualified individuals with a disability or disabled veterans who require support during the application and/or recruitment process may request a reasonable accommodation by emailing candidateaccessibility@versantmedia.com.
  • VERSANT Media is committed to fair and equitable compensation practices.
  • VERSANT Media is not accepting unsolicited assistance from search firms for this employment opportunity.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer, Ad Sales & Finance
Lead Data Engineer, Ad Sales & Finance

Versant Media • New York (NY)

On-site
USD 170,000 - 190,000
Lead Data Engineer, Consumer
Lead Data Engineer, Consumer

Versant • New York (NY)

On-site
USD 170,000 - 190,000
Lead Data Engineer, Ad Sales & Finance
Lead Data Engineer, Ad Sales & Finance

SmartRecruiters, Inc. • New York (NY), Northern (KY)

Hybrid
USD 170,000 - 190,000
Lead Data Engineer, Ad Sales & Finance
Lead Data Engineer, Ad Sales & Finance

Versant • New York (NY)

On-site
USD 170,000 - 190,000
Lead Data Engineer, Consumer
Lead Data Engineer, Consumer

Versant3 • New York (NY)

On-site
USD 150,000 - 230,000
Lead Data Engineer, Data Platform & Pipelines (Onsite NYC)
Lead Data Engineer, Data Platform & Pipelines (Onsite NYC)

Versant Media • New York (NY)

On-site
USD 170,000 - 190,000
Lead Data Engineer, Ad Sales & Finance
Lead Data Engineer, Ad Sales & Finance

Versant3 • New York (NY)

On-site
USD 150,000 - 190,000
Data Product Engineering Lead — Enterprise Data Platform
Data Product Engineering Lead — Enterprise Data Platform

SmartRecruiters, Inc. • New York (NY), Northern (KY)

Hybrid
USD 170,000 - 190,000
Lead Data Engineer, Data Platform & AI Readiness
Lead Data Engineer, Data Platform & AI Readiness

Versant Media • New York (NY)

On-site
USD 170,000 - 190,000
Lead Data Engineer
Lead Data Engineer

Long Ridge Partners • New York (NY)

On-site
USD 175,000 - 225,000