Lead Data Engineer: AI-Driven Pipelines on AWS

Talentola; Charlotte (NC); On-site; USD 110,000 - 150,000; Full time

Job summary

Talentola is seeking an experienced pipeline-centric data engineer to build out ETL and Data Operations frameworks for data preparation and normalization. The role focuses on creating scalable pipelines and secure data platforms within the AWS ecosystem, including Glue, S3, and Iceberg.

You will mentor a team of data engineers, collaborate with business units, and translate complex challenges into data-driven solutions.

Qualifications

• Experience designing scalable data pipelines and ETL processes.
• Strong SQL, data modeling, and data governance fundamentals.
• Proficiency with AWS Glue, S3, and related services.
• Ability to mentor and lead a small team of engineers.

Responsibilities

• Lead design, development, and maintenance of scalable data pipelines and ETL processes.
• Architect and manage data platforms on the AWS ecosystem for analytics.
• Provide mentorship and guidance to the data engineering team.
• Translate complex business needs into data-driven solutions and communicate with stakeholders.

Skills

AWS ecosystem

PySpark

Apache Iceberg

S3

SQL

Data Pipelines

Generative AI

Leadership / Mentorship

Stakeholder communication

ETL design

Tools

MySQL

Db2

PostgreSQL

Snowflake

Job description

Talentola is seeking an experienced pipeline-centric data engineer to build out ETL and Data Operations frameworks for data preparation and normalization. The role focuses on creating scalable pipelines and secure data platforms within the AWS ecosystem, including Glue, S3, and Iceberg.

You will mentor a team of data engineers, collaborate with business units, and translate complex challenges into data-driven solutions.