Lead Aws Data Engineer

Tata Consultancy Services; Owings Mills (MD); On-site; USD 110,000 - 125,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage:Medical

Family Support: Leaves

Commuter Benefits

Certification & Training Reimbursement

Vacation and Holidays

Job summary

Tata Consultancy Services Limited in Owings Mills, MD, is seeking an experienced Engineer in Technology to lead data engineering initiatives. The role focuses on building reliable data lakes, data lakes, data meshes, and end-to-end data pipelines using Python, AWS services, DBT, and Airflow.

The ideal candidate brings 12+ years of hands-on experience, strong SQL/data warehousing knowledge, and excellent stakeholder communication.

Qualifications

• 12+ years of hands-on experience in Python, AWS Data Services, DBT, Airflow (Astronomer), SQL and PySpark.
• Strong SQL and data warehousing lifecycle knowledge is an absolute requirement.
• Experience in creating data pipelines and orchestrating using DBT & Apache Airflow.
• Significant experience with data migrations and development of ODS, EDW, Data Lake and Data Marts.
• Experience fixing performance issues / parallelism; data modeling exposure is a must.
• Nice to have: cloud ETL/ELT tools like Glue/EMR.
• Experience with Snowflake, DB2, Postgres and other databases is a plus.
• Excellent communication with business and IT stakeholders.
• Expertise in planning execution and effort estimation.
• Exposure to Agile ways of working.

Responsibilities

• Lead the design of reliable, secure, and highly available data lakes, data marts and data meshes.
• Lead the design, development, and maintenance of data integration solutions using Python, AWS Data Services, DBT ensuring data pipeline and data quality.
• Collaborate with business stakeholders and architects to define the enterprise data strategy, data models, and migration roadmap.
• Translate complex technical constraints into business insights and manage expectations across cross-functional teams.
• Develop, modify, configure and debug existing data pipelines as per business requirements.
• Troubleshoot and resolve technical issues; debug, tune and optimize code for performance.
• Manage new requirements, review existing jobs, perform gap analysis and fix performance issues.
• Guide and upskill junior/mid-level data engineers on best practices and data patterns.
• Document all data flow and mappings, sessions and workflows.
• Ticket handling and problem ticket analysis in Agile/POD environments.

Skills

AWS

Python

Education

Bachelor of Computer Science

Job description

Job Description

Must Have Technical/Functional Skills

• 12+ years of solid hands-on experience in Python, AWS Data Services, DBT, Apache Airflow (on Astronomer platform), SQL and PySpark
• Very Good hands-on knowledge on SQL and Data Warehousing life cycle is an absolute requirement.
• Experience in creating data pipelines and orchestrating using DBT & Apache Airflow
• Significant experience with data migrations and development of Operational Data Stores, Enterprise Data Warehouses, Data Lake and Data Marts.
• Experiencing in fixing performance issues / parallelism, Data model exposure is a must.
• Good to have: Experience with cloud ETL and ELT in one of the tools like Glue/EMR or any other ELT tool
• Experience in using Snowflake, DB2, Postgres and other database technologies is plus.
• Excellent communication skills to liaise with Business & IT stakeholders.
• Expertise in planning execution of a project and efforts estimation.
• Exposure to working in Agile ways of working

Roles & Responsibilities

• Lead the design of reliable, secure, and highly available data lakes, data marts and data meshes
• leads the design, development, and maintenance of data integration solutions using Python, AWS Data Services, DBT ensuring data pipeline & data quality
• Collaborate with business stakeholders and architects to define the enterprise data strategy, data models, and migration Roadmap
• Translate complex technical constraints into business insights and manage expectations across cross-functional teams
• Develop, modify, configure & debug existing data pipeline as per the business requirement.
• Troubleshoot and resolve technical issues. Debug, tune and optimize code for optimal performance
• Manage the new requirements, Review the existing jobs, Perform gap analysis & Fixing performance issues, etc.
• Guide, coach, and upskill junior and mid-level data engineers on best practices, coding standards, and modern data patterns
• Document all data flow & mappings, sessions and workflows
• Ticket handling and problem ticket analysis skills in Agile /POD approach

TCS Employee Benefits Summary:

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
• Salary Range: $110,000 - $125,000 a year

Location

Owings Mills, MD

Function

TECHNOLOGY

Role

Engineer

ID

432597

Skills

AWS | Python

Salary Range

$110,000 - $125,000

Desired Candidate Profile

Qualifications

BACHELOR OF COMPUTER SCIENCE