Lead AWS Data Engineer at SFE Maryland

SFE; Maryland; On-site; USD 140,000 - 190,000; Full time

Job summary

Wishalicious is seeking a senior data engineer to design and operate scalable data lakes, marts and data mesh architectures. You will own data integration pipelines using Python, AWS data services, and DBT, ensuring data quality and timely delivery.

You will collaborate with business stakeholders and architects to shape the enterprise data strategy, models, and migration roadmap, while mentoring junior engineers and improving performance across ETL/ELT pipelines.

Qualifications

• 12+ years of hands-on experience in Python, AWS Data Services, DBT, Airflow, SQL and PySpark.
• Very good knowledge of SQL and Data Warehousing lifecycle is required.
• Experience in building data pipelines and orchestrating with DBT and Airflow.
• Experience with data migrations and development of ODS, EDW, Data Lake and Data Marts.
• Experience fixing performance issues and data modeling exposure is a must.
• Nice-to-have: cloud ETL/ELT tools like Glue/EMR or similar.
• Experience with Snowflake, DB2, PostgreSQL and other databases is a plus.
• Excellent communication to liaise with Business & IT stakeholders.
• Experience in planning projects and estimating effort.
• Exposure to Agile ways of working.

Responsibilities

• Lead design of reliable, secure data lakes, data marts and data meshes.
• Lead design, development, and maintenance of data integration solutions using Python, AWS Data Services, and DBT to ensure data quality.
• Collaborate with stakeholders to define enterprise data strategy, models, and migration roadmap.
• Translate complex constraints into business insights and manage cross-functional expectations.
• Develop, modify, configure and debug existing data pipelines per business requirements.
• Troubleshoot and optimize code for performance.
• Manage new requirements, review jobs, perform gap analysis and fix performance issues.
• Mentor junior/mid-level engineers on best practices and modern data patterns.
• Document all data flow, mappings, sessions and workflows.
• Handle tickets and perform issue analysis in Agile / POD contexts.

Skills

Python

AWS Data Services

DBT

Apache Airflow

SQL

PySpark

Data Warehousing

Data pipelines

Data migrations

Data modeling

Performance tuning

Data engineering best practices

Agile

ETL / ELT tooling

Snowflake

DB2

PostgreSQL

Communication

Project planning

Effort estimation

Tools

DBT

Apache Airflow

Snowflake

Glue

EMR

PostgreSQL

DB2

Job description

Must Have Technical/Functional Skills:

• 12+ years of solid hands-on experience in Python, AWS Data Services, DBT, Apache Airflow (on Astronomer platform), SQL and PySpark
• Very Good hands-on knowledge on SQL and Data Warehousing life cycle is an absolute requirement.
• Experience in creating data pipelines and orchestrating using DBT & Apache Airflow
• Significant experience with data migrations and development of Operational Data Stores, Enterprise Data Warehouses, Data Lake and Data Marts.
• Experiencing in fixing performance issues / parallelism, Data model exposure is a must.
• Good to have: Experience with cloud ETL and ELT in one of the tools like Glue/EMR or any other ELT tool
• Experience in using Snowflake, DB2, Postgres and other database technologies is plus.
• Excellent communication skills to liaise with Business & IT stakeholders.
• Expertise in planning execution of a project and efforts estimation.
• Exposure to working in Agile ways of working

Roles & Responsibilities:

• Lead the design of reliable, secure, and highly available data lakes, data marts and data meshes
• leads the design, development, and maintenance of data integration solutions using Python, AWS Data Services, DBT ensuring data pipeline & data quality
• Collaborate with business stakeholders and architects to define the enterprise data strategy, data models, and migration Roadmap
• Translate complex technical constraints into business insights and manage expectations across cross-functional teams
• Develop, modify, configure & debug existing data pipeline as per the business requirement.
• Troubleshoot and resolve technical issues. Debug, tune and optimize code for optimal performance
• Manage the new requirements, Review the existing jobs, Perform gap analysis & Fixing performance issues, etc.
• Guide, coach, and upskill junior and mid-level data engineers on best practices, coding standards, and modern data patterns
• Document all data flow & mappings, sessions and workflows
• Ticket handling and problem ticket analysis skills in Agile /POD approach'