Lead Data Engineer, Ad Sales & Finance

Versant Media; New York (NY); On-site; USD 170,000 - 190,000; Full time

Job summary

Versant Media is seeking a Lead Data Engineer to guide the delivery of reliable, scalable source-to-Silver data products used for downstream analytics, reporting, AI, and day-to-day operations. This role leads a distributed/offshore engineering team and owns ingestion, transformation, data-quality controls, governance, and the operating model for consistent release-ready outputs.

You will partner with product managers and domain leaders to align on outcomes, implement data profiling and lineage,

Qualifications

• 8+ years of data-engineering, software engineering, or data-platform experience, including 2+ years leading engineers or large-scale delivery.
• Proven track record delivering enterprise-scale ingestion and transformation pipelines.
• Hands-on expertise with SQL, Python, Spark/PySpark, and modern ELT/ETL patterns.

Responsibilities

• Lead and develop a distributed/offshore Data Engineering team delivering work from source ingestion through curated Silver-layer data products.
• Convert roadmaps and requirements into engineering plans, milestones, estimates, and delivery commitments.
• Design and manage ingestion, transformation, standardization, orchestration, and publication of data across sources.
• Ensure Silver-layer data products are validated, standardized, documented, reusable, performant, secure, and prepared for Gold-layer analytics and AI consumption.
• Create repeatable processing patterns for batch, CDC, and event-driven workflows.
• Partner with the Data Platform team to use approved workspace, compute, storage, security, and deployment patterns.

Skills

SQL

Python

Spark

PySpark

ELT/ETL patterns

Databricks

Delta Lake

CI/CD

Unity Catalog

APIs

Data governance

Data quality

Tools

Databricks

Delta Lake

CI/CD

Unity Catalog

APIs

Event streaming

Job description

Versant Media is seeking a Lead Data Engineer to guide the delivery of reliable, scalable source-to-Silver data products used for downstream analytics, reporting, AI, and day-to-day operations. This role leads a distributed/offshore engineering team and owns ingestion, transformation, data-quality controls, governance, and the operating model for consistent release-ready outputs.

Key Responsibilities

• Lead and develop a distributed/offshore Data Engineering team delivering work from source ingestion through curated Silver-layer data products.
• Convert product roadmaps and requirements into engineering plans, milestones, estimates, dependency mapping, and delivery commitments.
• Design and manage ingestion, transformation, standardization, orchestration, and publication of data across operational, SaaS, file, streaming, API, and partner sources.
• Ensure Silver-layer data products are validated, standardized, documented, reusable, performant, secure, and prepared for Gold-layer analytics and AI consumption.
• Create repeatable processing patterns for batch, incremental, change-data-capture, and event-driven workflows.
• Develop reusable engineering frameworks, templates, libraries, and pipeline patterns to reduce repeated effort across products and domains.
• Apply modern engineering practices including source control, peer review, automated testing, CI/CD, observability, release management, and incident remediation.
• Define and maintain standards for naming, partitioning, schema evolution, error handling, replay and recovery, performance, cost management, and documentation.
• Partner with the Data Platform team to use approved workspace, compute, storage, security, and deployment patterns.
• Identify technical debt and drive practical improvements that increase delivery speed, reliability, and maintainability.
• Collaborate with Data Modelers on canonical entities, conformed dimensions, data contracts, source-to-target mappings, and enterprise modeling standards.
• Work with Product Managers and domain leaders to align on intended outcomes, source-system realities, priority use cases, and acceptance criteria.
• Coordinate dependencies with source-system owners, platform teams, analytics teams, and external partners.
• Implement data profiling, reconciliation, quality testing, freshness monitoring, lineage, and alerting for each delivered data product.
• Maintain source-to-Silver traceability, including authoritative source identification, transformation logic, ownership, metadata, and data-quality expectations.
• Establish and enforce access controls, sensitive-data classification, masking, retention, and regional or data-residency requirements.
• Drive resolution of data defects, schema changes, pipeline failures, and quality issues through clear ownership and service-level expectations.
• Set team priorities, technical direction, delivery expectations, and quality standards.
• Coach engineers in data engineering, cloud development, testing, observability, and product-oriented delivery practices.
• Maintain a healthy onshore/offshore delivery model with defined handoffs, overlap hours, documentation standards, ceremonies, and escalation paths.
• Communicate delivery progress, risks, tradeoffs, and decisions clearly to technical and business stakeholders.
• Build a culture of ownership, continuous improvement, and reliable execution.

Required Qualifications

• 8+ years of data engineering, software engineering, or data-platform experience, including 2+ years leading engineers, technical workstreams, or large-scale delivery.
• Demonstrated experience designing and delivering enterprise-scale ingestion and transformation pipelines.
• Strong hands-on expertise in SQL, Python, Spark or PySpark, and modern ELT or ETL patterns.
• Experience with Databricks, Delta Lake, or a comparable lakehouse platform, plus cloud storage, orchestration, and CI/CD.
• Strong understanding of Bronze/Silver/Gold (or equivalent layered data-platform) patterns.
• Experience implementing data quality, observability, lineage, metadata, and production support practices.
• Proven ability to partner with architects, data modelers, product leaders, and domain stakeholders.
• Experience leading distributed teams and establishing effective delivery practices across time zones.
• Strong communication skills, including the ability to explain technical choices, risks, and tradeoffs to non-technical stakeholders.
• Experience with Unity Catalog or comparable governance, catalog, and access-control capabilities.
• Experience with data contracts, schema evolution, change data capture, APIs, event streaming, and large-volume data processing.
• Experience supporting multiple data domains, global data products, or regional data-residency requirements.
• Experience with infrastructure-as-code, DataOps, and automated environment provisioning.
• Experience in a regulated, high-scale, or operationally sensitive environment.

Technologies

• SQL, Python, Spark, PySpark
• ELT/ETL
• Databricks, Delta Lake
• CI/CD
• Bronze/Silver/Gold
• Unity Catalog
• APIs, event streaming
• Infrastructure-as-code, DataOps

Location and Compensation

• Location: New York, NY (onsite)
• Salary Range: USD 170,000 - 190,000 per year

Additional Information

• External candidates may be required to attend an in-person interview with a VERSANT Media employee at one of its locations prior to a hiring decision.
• Versant Media is an equal employment opportunity employer.
• If you require a reasonable accommodation during the application and/or recruitment process due to a disability, you may submit a request to [e-mail].
• Compensation practices include a good faith pay range; actual pay is based on factors such as skills, qualifications, experience, and location, and may include additional forms of compensation and benefits.
• Versant Media is not accepting unsolicited assistance from search firms; resumes submitted without a valid written Statement of Work will be deemed the sole property of VERSANT and no fee will be paid if the candidate is hired.