Data Engineering Team Lead

Jobtailor; New York (NY); On-site; USD 170,000 - 230,000; Full time

Job summary

Jobtailor is seeking a Senior Data Engineering Manager to lead and grow a team responsible for production data platforms. You will set standards for ELT/ETL pipelines, data modeling, and lakehouse patterns, partnering with Product, Data Science, and Engineering to drive data freshness and performance.

The role requires deep Python/SQL skills, Spark/Databricks expertise, and hands-on cloud experience. You will mentor engineers, define architecture, and ensure robust testing, metadata, and

Qualifications

• 6+ years building and operating production data systems at scale.
• Experience leading a team or mentoring senior engineers.
• Experience with hiring, performance management, and career development.
• Hands-on AWS or similar cloud experience including storage, compute, networking basics, and cost controls.
• CI/CD for data and infrastructure as code.

Responsibilities

• Manage and grow a team of Data Engineers, including goal-setting, 1:1s, feedback, hiring, and performance management.
• Set architecture and design standards for ELT/ETL pipelines using Python, SQL, Spark, and Databricks.
• Make build-vs-buy and platform trade-off decisions.
• Drive dimensional modeling, star schemas, and incremental data patterns across the lakehouse and warehouse.
• Own SLOs and on-call rotations for data platforms.
• Scale monitoring, alerting, capacity, and cost management.
• Build roadmaps, sequence delivery, and drive measurable improvements in data freshness, completeness, and query performance.

Skills

Data Engineering Leadership

Python

SQL

Team Leadership

Tools

Databricks

Airflow

Databricks Workflows

Dbt

Monitoring Tools

Job description

• Manage and grow a team of Data Engineers, including goal-setting, 1:1s, feedback, hiring, and performance management
• Set architecture and design standards for ELT/ETL pipelines using Python, SQL, Spark, and Databricks
• Make build-vs-buy and platform trade-off decisions
• Drive dimensional modeling, star schemas, and incremental data patterns across the lakehouse and warehouse
• Set technical standards for streaming and event-data jobs, idempotency, and exactly-once semantics
• Ensure automated testing, anomaly detection, validation, lineage, metadata, and documentation
• Own SLOs and on-call rotations for data platforms
• Scale monitoring, alerting, capacity, and cost management
• Build roadmaps, sequence delivery, and drive measurable improvements in data freshness, completeness, and query performance
• Partner with Product, Data Science, and Application Engineering on source selection, feature readiness, experiment design, APIs, and semantic layers
• Lead design and code reviews, run brown-bag sessions, and coach engineers

Requirements

• 6+ years building and operating production data systems at scale
• Prior experience leading a team or mentoring senior engineers
• Experience managing people or leading ambiguous, high-stakes initiatives to clear results
• Experience with hiring, performance management, and career development
• Deep fluency with Python and SQL
• Expert knowledge of Spark, Databricks, and lakehouse patterns
• Strong data modeling skills
• Experience running workloads in AWS or a similar cloud, including storage, compute, networking basics, and cost controls
• Hands-on experience with Airflow, Databricks Workflows, or dbt
• Experience with CI/CD for data and infrastructure as code
• Ability to operate as both an individual technical contributor and a people manager

Core Competencies

Demonstrates expertise in managing and mentoring Data Engineers while setting architectural standards for ELT/ETL pipelines using Python, SQL, and Spark. Proven ability to drive data modeling, performance management, and collaboration across teams to enhance data systems.

Highest-signal resume keywords

• Data Engineering Leadership
• Python Proficiency
• SQL Expertise
• Spark and Databricks Knowledge
• AWS Experience

ATS Optimization Keywords

Hard Skills

• Data Modeling
• ELT/ETL Pipeline Design
• Dimensional Modeling
• Incremental Data Patterns
• Automated Testing
• Anomaly Detection
• CI/CD for Data
• Infrastructure as Code
• Performance Management
• Career Development

Soft Skills

• Team Management
• Mentoring
• Feedback Delivery
• Goal-Setting
• Collaboration

Industry Keywords

• Lakehouse Patterns
• Streaming Data
• Event-Data Jobs
• SLOs
• Cost Management

Tools & Technologies

• Databricks
• Airflow
• Databricks Workflows
• Dbt
• Monitoring Tools