Lead AWS Data Engineer

Tata Consultancy Services; Owings Mills (MD); On-site; USD 110,000 - 125,000; Full time

Benefits offered by this job

Discretionary incentive

Medical coverage

Family leaves

Auto & Home Insurance

Commuter benefits

Vacation & Holidays

Legal & financial assistance

401K & retirement

Job summary

Tata Consultancy Services is seeking a senior data engineer to lead the design and implementation of data lakes, data marts, and data meshes. You will own data integration pipelines, collaborating with business stakeholders to align data models with enterprise strategy.

Ideal candidates have 12+ years of hands-on experience with Python, AWS Data Services, DBT, Apache Airflow, SQL and PySpark, and are skilled at agile delivery and performance tuning. On-site opportunity in the US.

Qualifications

• 12+ years hands-on experience in Python, AWS Data Services, DBT, Apache Airflow (Astronomer), SQL and PySpark.
• Strong SQL and Data Warehousing lifecycle knowledge required.
• Experience in building data pipelines and orchestrating with DBT & Apache Airflow.
• Data migrations and development of ODS, EDW, Data Lake and Data Marts.
• Performance tuning and data modeling exposure essential.
• Cloud ETL/ELT experience with Glue/EMR or similar tools is a plus.
• Experience with Snowflake, DB2, Postgres and other databases is beneficial.
• Excellent communication to liaise with Business & IT stakeholders.
• Ability to plan execution and estimate effort; familiar with Agile.

Responsibilities

• Lead design of data lakes, data marts, and data meshes that are reliable and secure.
• Develop and maintain data integration solutions using Python and AWS Data Services with DBT and Airflow.
• Collaborate with stakeholders to define enterprise data strategy, data models, and migration roadmaps.
• Translate technical constraints into business insights and manage cross-functional expectations.
• Modify, configure, and debug existing data pipelines to meet business needs.
• Troubleshoot, tune, and optimize code for performance.
• Review new requirements, perform gap analysis, and fix performance issues.
• Mentor junior/mid-level data engineers on best practices and data patterns.
• Document data flows, mappings, sessions and workflows.
• Handle tickets in an Agile/POD framework.

Skills

Python

AWS Data Services

DBT

Apache Airflow

SQL

PySpark

Data Warehousing

Data Pipelines

Agile

Communication

Education

Bachelor's degree in Computer Science

Tools

Airflow

DBT

Snowflake

DB2

Postgres

Job description

Must Have Technical/Functional Skills

• 12+ years of solid hands-on experience in Python, AWS Data Services, DBT, Apache Airflow (on Astronomer platform), SQL and PySpark


• Very Good hands-on knowledge on SQL and Data Warehousing life cycle is an absolute requirement.


• Experience in creating data pipelines and orchestrating using DBT & Apache Airflow


• Significant experience with data migrations and development of Operational Data Stores, Enterprise Data Warehouses, Data Lake and Data Marts.


• Experiencing in fixing performance issues / parallelism, Data model exposure is a must.


• Good to have: Experience with cloud ETL and ELT in one of the tools like Glue/EMR or any other ELT tool


• Experience in using Snowflake, DB2, Postgres and other database technologies is plus.


• Excellent communication skills to liaise with Business & IT stakeholders.


• Expertise in planning execution of a project and efforts estimation.


• Exposure to working in Agile ways of working

Roles & Responsibilities

• Lead the design of reliable, secure, and highly available data lakes, data marts and data meshes


• leads the design, development, and maintenance of data integration solutions using Python, AWS Data Services, DBT ensuring data pipeline & data quality


• Collaborate with business stakeholders and architects to define the enterprise data strategy, data models, and migration Roadmap


• Translate complex technical constraints into business insights and manage expectations across cross-functional teams


• Develop, modify, configure & debug existing data pipeline as per the business requirement.


• Troubleshoot and resolve technical issues. Debug, tune and optimize code for optimal performance


• Manage the new requirements, Review the existing jobs, Perform gap analysis & Fixing performance issues, etc.


• Guide, coach, and upskill junior and mid-level data engineers on best practices, coding standards, and modern data patterns


• Document all data flow & mappings, sessions and workflows


• Ticket handling and problem ticket analysis skills in Agile /POD approach

TCS Employee Benefits Summary

• Discretionary Annual Incentive.


• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.


• Family Support: Maternal & Parental Leaves.


• Insurance Options: Auto & Home Insurance, Identity Theft Protection.


• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.


• Time Off: Vacation, Time Off, Sick Leave & Holidays.


• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.


• Salary Range: $110,000 - $125,000 a year

Qualifications

BACHELOR OF COMPUTER SCIENCE