Data Engineer Leader- Job Code: JAKD-KGPG-OLDH-BDM

Alchemy Job; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

An established industry player is seeking a Technical Leader for their Data Engineering practice in New York. This role focuses on delivering cutting-edge data engineering solutions within the media industry. The ideal candidate will leverage over a decade of experience, leading a talented team while designing scalable data pipelines and optimizing workflows for analytics and machine learning. If you're passionate about transforming data into actionable insights and thrive in a collaborative environment, this opportunity is perfect for you. Join a forward-thinking team dedicated to innovation and excellence in data management.

Qualifications

• 12+ years in data engineering with a focus on big data and cloud technologies.
• Proficiency in Python, PySpark, and SQL with hands-on experience in Databricks.

Responsibilities

• Lead a team of 15+ data engineering professionals in a hybrid environment.
• Design, develop, and optimize scalable data pipelines using Python and Databricks.

Skills

Python

PySpark

Databricks

Snowflake

AWS

SQL

Data Governance

Problem-solving

Cross-functional Communication

Tools

AWS S3

AWS Glue

AWS Lambda

AWS Redshift

AWS Athena

Retool

Tableau

Power BI

Mode Analytics

Job description

Data Engineer Leader - Job Code: JAKD-KGPG-OLDH-BDM

Be among the first 25 applicants two weeks ago.

Get AI-powered advice on this job and more exclusive features.

About the Role

We are seeking a Technical Leader for our Data Engineering practice to join our onsite team in New York, focusing on data engineering solutions in the media industry. The ideal candidate will have over 10 years of experience in account management and delivery leadership, with deep expertise in data engineering using Python, PySpark, Databricks, Snowflake, and AWS. Experience working with Nielsen and social media data is essential, and familiarity with Retool for internal tools is a plus. The candidate should be skilled in designing scalable data pipelines, managing large-scale media datasets, and optimizing workflows for analytics, reporting, and machine learning.

Key Responsibilities

1. Manage a large account with delivery and revenue responsibilities.
2. Lead a team of 15+ data engineering professionals in a hybrid (on-shore + off-shore) environment.
3. Provide thought leadership in data engineering.
4. Design, develop, and optimize scalable data pipelines using Python, PySpark, and Databricks.
5. Work with Nielsen, social media, and streaming data to generate insights for media performance and audience analytics.
6. Implement ETL/ELT processes within Snowflake and AWS data ecosystem.
7. Manage AWS-based data architectures including S3, Glue, Lambda, Athena, and Redshift.
8. Collaborate with data scientists, analysts, and business teams to ensure data availability and usability.
9. Develop data models, warehouses, and marts optimized for analytics and BI.
10. Leverage Databricks for distributed data processing and performance tuning.
11. Ensure data governance, security, and compliance best practices in media data processing.
12. Utilize Retool (basic knowledge required) to build internal tools for operational efficiency.
13. Troubleshoot data quality issues, performance bottlenecks, and optimize accordingly.

Required Skills & Experience

1. 12+ years in data engineering, focusing on big data and cloud technologies.
2. Proficiency in Python, PySpark, and SQL.
3. Hands-on experience with Databricks and Apache Spark workload optimization.
4. Experience with media datasets, especially Nielsen ratings, social media engagement, and digital content analytics.
5. Expertise in Snowflake, including data modeling, Snowpipe, and performance tuning.
6. Solid experience with AWS cloud services such as S3, Glue, Lambda, Redshift, Athena.
7. Knowledge of streaming data processing (Kafka, Kinesis, etc.) is a plus.
8. Familiarity with Retool for internal tool development.
9. Strong problem-solving skills and proactive approach to data challenges.
10. Excellent cross-functional communication skills.

Preferred Qualifications

1. Experience in media audience measurement, ad performance, or content engagement analytics.
2. Knowledge of machine learning integration in data pipelines.
3. Exposure to data visualization tools like Tableau, Power BI, or Mode Analytics.
4. Experience in Agile/Scrum environments.

Seniority Level

• Director

Employment Type

• Full-time

Job Function

• IT Services and IT Consulting

Referrals can double your chances of interviewing at Alchemy Job.

Get notified about new Data Engineer roles in New York, United States.

Salary ranges and recent postings are available for various Data Engineer positions in New York.