Data Product Engineering Lead — Enterprise Data Platform

SmartRecruiters, Inc.; New York, Northern (NY, KY); Hybrid; USD 170,000 - 190,000; Full time

Job summary

VERSANT Media in New York seeks a hands-on Data Product Engineering Lead to lead engineering resources to deliver trusted, scalable data products from source systems through the Silver layer of our enterprise data platform.

This role turns product and business needs into reliable, governed data capabilities: ingesting source data, applying standardized transformations, implementing data-quality controls, and publishing Silver-layer datasets for analytics, reporting, AI, and operations.

Qualifications

• 8+ years of data engineering, software engineering, or data-platform experience, including 2+ years leading engineers, technical workstreams, or large-scale delivery.
• Demonstrated experience designing and delivering enterprise-scale ingestion and transformation pipelines.
• Strong hands-on expertise in SQL, Python, Spark/PySpark, and modern ELT/ETL patterns.
• Experience with Databricks, Delta Lake, or a comparable lakehouse platform; experience with cloud storage, orchestration, and CI/CD.
• Strong understanding of Bronze/Silver/Gold or equivalent layered data-platform patterns.
• Experience implementing data quality, observability, lineage, metadata, and production support practices.
• Proven ability to partner with architects, data modelers, product leaders, and domain stakeholders.
• Experience leading distributed teams and creating effective delivery practices across time zones.
• Strong communication skills and the ability to explain technical choices, risks, and tradeoffs to non-technical stakeholders.

Responsibilities

• Lead delivery of source-to-Silver data products.
• Lead and develop a distributed/offshore team of Data Engineers responsible for delivery from source ingestion through curated Silver-layer data products.
• Translate product roadmaps and requirements into actionable engineering plans, milestones, estimates, dependencies, and delivery commitments.
• Design and oversee ingestion, transformation, standardization, orchestration, and publication of data from operational, SaaS, file, streaming, API, and partner sources.
• Ensure Silver-layer data products are validated, standardized, documented, reusable, performant, secure, and ready for Gold-layer analytics and AI consumption.
• Establish repeatable patterns for batch, incremental, change-data-capture, and event-driven processing.
• Build scalable engineering foundations with reusable data-engineering frameworks and patterns.
• Apply modern engineering practices including source control, peer review, automated testing, CI/CD, observability, release management, and incident remediation.
• Define and maintain standards for naming, partitioning, schema evolution, error handling, replay/recovery, performance, cost management, and documentation.
• Partner with the Data Platform team to use approved workspace, compute, storage, security, and deployment patterns.
• Identify technical debt and lead pragmatic improvements that increase delivery speed, reliability, and maintainability.
• Work with Data Modelers to implement canonical entities, conformed dimensions, data contracts, source-to-target mappings, and enterprise modeling standards.
• Work with Product Managers and domain leaders to clarify intended outcomes, source-system realities, priority use cases, and acceptance criteria.
• Coordinate dependencies with source-system owners, platform teams, analytics teams, and external partners.
• Establish data profiling, reconciliation, quality testing, freshness monitoring, lineage, and alerting for every delivered data product.
• Ensure source-to-Silver traceability, including authoritative source identification, transformation logic, ownership, metadata, and data-quality expectations.
• Implement appropriate access controls, sensitive-data classification, masking, retention, and regional/data-residency requirements.
• Drive resolution of data defects, schema changes, pipeline failures, and quality issues through clear ownership and service-level expectations.
• Set clear priorities, technical direction, delivery expectations, and quality standards for the engineering team.
• Coach engineers in data engineering, cloud development, testing, observability, and product-oriented delivery practices.
• Create a healthy onshore/offshore delivery model with defined handoffs, overlap hours, documentation standards, ceremonies, and escalation paths.
• Communicate delivery progress, risks, tradeoffs, and decisions clearly to technical and business stakeholders.
• Build a culture of ownership, continuous improvement, and reliable execution.

Skills

SQL

Python

Spark/PySpark

ELT/ETL patterns

Databricks

Delta Lake

CI/CD

Tools

Databricks

Delta Lake

Job description

VERSANT Media in New York seeks a hands-on Data Product Engineering Lead to lead engineering resources to deliver trusted, scalable data products from source systems through the Silver layer of our enterprise data platform.

This role turns product and business needs into reliable, governed data capabilities: ingesting source data, applying standardized transformations, implementing data-quality controls, and publishing Silver-layer datasets for analytics, reporting, AI, and operations.