Data Engineer Lead- FullTime Only

Talentola; Charlotte (NC); On-site; USD 110,000 - 150,000; Full time

Job summary

Talentola is seeking an experienced pipeline-centric data engineer to build out ETL and Data Operations frameworks for data preparation and normalization. The role focuses on creating scalable pipelines and secure data platforms within the AWS ecosystem, including Glue, S3, and Iceberg.

You will mentor a team of data engineers, collaborate with business units, and translate complex challenges into data-driven solutions.

Qualifications

• Experience designing scalable data pipelines and ETL processes.
• Strong SQL, data modeling, and data governance fundamentals.
• Proficiency with AWS Glue, S3, and related services.
• Ability to mentor and lead a small team of engineers.

Responsibilities

• Lead design, development, and maintenance of scalable data pipelines and ETL processes.
• Architect and manage data platforms on the AWS ecosystem for analytics.
• Provide mentorship and guidance to the data engineering team.
• Translate complex business needs into data-driven solutions and communicate with stakeholders.

Skills

AWS ecosystem

PySpark

Apache Iceberg

S3

SQL

Data Pipelines

Generative AI

Leadership / Mentorship

Stakeholder communication

ETL design

Tools

MySQL

Db2

PostgreSQL

Snowflake

Job description

We’re seeking an experienced pipeline-centric data engineer to put it to good use in building out ETL and Data Operations framework (Data Preparation / Normalization and Ontological processes).

Technical Skills:

• Lead the design, development, and maintenance of scalable data pipelines and ETL processes, ensuring data integrity and accessibility for business intelligence and advanced analytics.
• Architect and manage robust data platforms on the "AWS ecosystem", leveraging services like Glue, PySpark, Apache Iceberg, IAM, S3, and Secrets Manager to build a secure and efficient data infrastructure.
• Provide technical guidance and mentorship to a team of data engineers, fostering a culture of high performance and continuous learning.
• Utilize deep expertise in various RDBMS (e.g., MySQL, Db2, PostgreSQL, Snowflake) and different data formats (e.g., JSON, Parquet) to drive strategic data initiatives.
• Champion the adoption of modern data technologies such as Apache Iceberg with AWS Glue to optimize data lake performance and analytics capabilities.
• Apply advanced proficiency in Generative AI to automate and streamline data analysis, development, and documentation processes.
• Business Acumen and Stakeholder Communication
• Translate complex business challenges into clear, data-driven solutions, effectively communicating technical concepts and project progress to both technical and non-technical stakeholders, including senior leadership.
• Act as a key liaison between the data engineering team and business units, providing data-backed insights and recommendations that directly influence business strategy.
• Manage concurrent projects in a dynamic, research-oriented environment, ensuring timely delivery and high-quality outcomes.
• Experience in insurance domain preferrable
• AWS Data Engineering certification good to have