Senior AWS Data Engineer - Data Pipelines & Lakes Lead

Tata Consultancy Services Limited; Owings Mills (MD); On-site; USD 110,000 - 125,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage:Medical

Family Support: Leaves

Commuter Benefits

Certification & Training Reimbursement

Vacation and Holidays

Job summary

Tata Consultancy Services Limited in Owings Mills, MD, is seeking an experienced Engineer in Technology to lead data engineering initiatives. The role focuses on building reliable data lakes, data lakes, data meshes, and end-to-end data pipelines using Python, AWS services, DBT, and Airflow.

The ideal candidate brings 12+ years of hands-on experience, strong SQL/data warehousing knowledge, and excellent stakeholder communication.

Qualifications

• 12+ years of hands-on experience in Python, AWS Data Services, DBT, Airflow (Astronomer), SQL and PySpark.
• Strong SQL and data warehousing lifecycle knowledge is an absolute requirement.
• Experience in creating data pipelines and orchestrating using DBT & Apache Airflow.
• Significant experience with data migrations and development of ODS, EDW, Data Lake and Data Marts.
• Experience fixing performance issues / parallelism; data modeling exposure is a must.
• Nice to have: cloud ETL/ELT tools like Glue/EMR.
• Experience with Snowflake, DB2, Postgres and other databases is a plus.
• Excellent communication with business and IT stakeholders.
• Expertise in planning execution and effort estimation.
• Exposure to Agile ways of working.

Responsibilities

• Lead the design of reliable, secure, and highly available data lakes, data marts and data meshes.
• Lead the design, development, and maintenance of data integration solutions using Python, AWS Data Services, DBT ensuring data pipeline and data quality.
• Collaborate with business stakeholders and architects to define the enterprise data strategy, data models, and migration roadmap.
• Translate complex technical constraints into business insights and manage expectations across cross-functional teams.
• Develop, modify, configure and debug existing data pipelines as per business requirements.
• Troubleshoot and resolve technical issues; debug, tune and optimize code for performance.
• Manage new requirements, review existing jobs, perform gap analysis and fix performance issues.
• Guide and upskill junior/mid-level data engineers on best practices and data patterns.
• Document all data flow and mappings, sessions and workflows.
• Ticket handling and problem ticket analysis in Agile/POD environments.

Skills

AWS

Python

Education

Bachelor of Computer Science

Job description

Tata Consultancy Services Limited in Owings Mills, MD, is seeking an experienced Engineer in Technology to lead data engineering initiatives. The role focuses on building reliable data lakes, data lakes, data meshes, and end-to-end data pipelines using Python, AWS services, DBT, and Airflow.

The ideal candidate brings 12+ years of hands-on experience, strong SQL/data warehousing knowledge, and excellent stakeholder communication.