Lead AWS Data Engineer

Jobtailor; Town of Florida (NY); On-site; USD 120,000 - 170,000; Full time

Job summary

Jobtailor is seeking a senior data engineering leader in the New York area to guide a high-performing team across data pipelines, architecture, and governance. You will partner with business leaders, data scientists, and IT to translate needs into scalable technical solutions.

The role requires strong leadership, advanced data modeling, and deep experience with cloud data platforms and real-time processing. On-site collaboration in NY is expected.

Qualifications

• 7–10 years of data engineering experience with leadership and PM exposure.
• Experience with data pipeline orchestration and real-time processing.
• Proficiency in Python/Java/Scala and SQL.

Responsibilities

• Lead and manage the data engineering team, incl. planning and performance.
• Develop data standards, processes, and best practices.
• Translate business needs into technical requirements with cross-functional teams.
• Ensure scalable, secure data solutions aligned with goals.
• Drive adoption of data tools and methodologies across the org.
• Oversee data documentation and architecture artifacts.
• Manage technical debt and modernize legacy systems.
• Provide leadership in resolving complex data issues.

Skills

Leadership

Project management

AWS cloud data platforms

Data governance

Data modeling

Education

Bachelor's or Master's in CS/IT

Tools

Apache Airflow

Apache NiFi

AWS Glue

Apache Kafka

Spark Streaming

AWS Lambda

Snowflake

Prefect

Dbt

Job description

• Lead and manage the data engineering team, including project planning, resource allocation, and performance management
• Develop and enforce data engineering standards, processes, and best practices
• Collaborate with business leaders, data scientists, and IT teams to prioritize data-related business needs and translate them into technical requirements
• Ensure data solutions are scalable, secure, and aligned with organizational goals
• Drive adoption of technologies, tools, and methodologies that improve data engineering efficiency, reliability, and scalability
• Oversee data documentation, including data architecture diagrams, process flows, and technical specifications
• Manage technical debt and modernize legacy systems
• Provide technical leadership in resolving complex data issues and guide troubleshooting and problem-solving efforts
• Lead transformative initiatives and coordinate efforts across multiple departments
• Develop future leaders and establish a culture of accountability, integrity, and high-quality delivery

Requirements

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field
• 7–10 years of experience in data engineering, with significant leadership and project management experience
• Extensive experience with data pipeline orchestration tools such as Apache Airflow and Luigi
• Experience with data integration platforms such as Apache NiFi and AWS Glue
• Strong leadership and project management skills
• Experience leading cross-functional teams and managing multiple projects simultaneously
• Expertise in AWS cloud data platforms and big data ecosystems
• Proficiency in advanced data modeling, including enterprise data warehouses, data lakes, and data marts
• In-depth knowledge of data governance, including data cataloging, data lineage, and data quality management
• Experience with Master Data Management (MDM) and metadata management
• Advanced programming and scripting skills in Python, Java, or Scala
• Experience with real-time data processing and event-driven architectures using Apache Kafka, Spark Streaming, and AWS Lambda
• Experience with AWS, Snowflake, and Azure
• Experience with AWS Glue, AWS AppFlow, Prefect, dbt, S3, and Qlik
• SQL, Python, and Java skills
• Ability to influence and engage senior leaders with data-driven insights
• Ability to resolve complex, high-stakes problems with innovative solutions

Core Competencies

Demonstrates expertise in data engineering leadership, project management, and advanced data modeling, with a strong focus on AWS cloud platforms and data governance. Capable of driving transformative initiatives and fostering a culture of accountability and high-quality delivery.

Highest-signal resume keywords

• Data Engineering Leadership
• Project Management
• AWS Cloud Data Platforms
• Data Pipeline Orchestration
• Advanced Data Modeling

ATS Optimization Keywords

Hard Skills

• Data Pipeline Orchestration
• Advanced Data Modeling
• Programming in Python
• Programming in Java
• Programming in Scala
• SQL
• Data Governance
• Master Data Management
• Real-Time Data Processing
• Event-Driven Architectures

Soft Skills

• Leadership
• Problem-Solving
• Influencing Senior Leaders
• Collaboration
• Accountability

Industry Keywords

• Data Engineering
• Data Integration
• Data Architecture
• Data Quality Management
• Data Cataloging
• Data Lineage
• Big Data Ecosystems
• Data Lakes
• Data Warehouses
• Data Marts

Tools & Technologies

• Apache Airflow
• Apache NiFi
• AWS Glue
• Apache Kafka
• Spark Streaming
• AWS Lambda
• Snowflake
• Azure
• Prefect
• Dbt