Aws Data Engineer(including Redshift) Hyderabad/Chennai/Bangalore

Tata Consultancy Services; Hyderabad, Chennai District, Bengaluru; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

Tata Consultancy Services is seeking a data engineer with 4-5+ years of AWS and data warehousing experience to design, build, and maintain real-time and batch data pipelines. The role focuses on API integration, ETL/ELT processes, and delivering scalable data solutions within an agile team.

Ideal candidates will have strong SQL, Python (pandas, numpy, Flask), and hands-on experience with Redshift/BigQuery, along with data onboarding and validation practices to ensure data quality across

Qualifications

• 4-5+ years of experience with AWS (including Redshift) and other cloud technologies.
• 4-5+ years of experience designing and maintaining data warehouses (e.g., Redshift, Big Query) as well as building complex ETL/ELT pipelines.
• Advanced SQL proficiency, including concepts such as query optimization, indexing, and partitioning.
• Hands-on experience with Python for data-related processes, using libraries like pandas, numpy, and frameworks like Flask.

Responsibilities

• Exceptional communication skills and can communicate effectively to both technical and non-technical stakeholders.
• Proven track record of delivering well-designed, well-tested and performant software solutions and pipelines within an agile environment.
• Lead the development of automated data management and reporting solutions, reducing manual reporting efforts and significantly improving reporting efficiency and turnaround time.

Skills

AWS

Data warehousing

SQL

Python

ETL/ELT

API integrations

Real-time data

Batch data pipelines

Agile

Tools

Redshift

BigQuery

Pandas

Numpy

Flask

Job description

Role & responsibilities

• Exceptional communication skills and can communicate effectively to both technical and non-technical stakeholders.
• 4-5+ years of experience with AWS (including Redshift) and other cloud technologies
• 4-5+ years of experience designing and maintaining data warehouses (e.g., Redshift, Big Query) as well as building complex ETL/ELT pipelines.
• Expertise in working with real-time and batch data pipelines, including API-based integration workflows.
• Experience in maintaining and executing data onboarding blueprints which map source and target data strategies and aggregations.
• Advanced SQL proficiency, including concepts such as query optimization, indexing, and partitioning.
• Hands-on experience with Python for data-related processes, using libraries like pandas, numpy, and frameworks like Flask.
• Experience with data monitoring, detection, troubleshooting as well as validation frameworks.
• Proven track record of delivering well-designed, well-tested and performant software solutions and pipelines within an agile environment.

Preferred candidate profile

• Lead the development of automated data management and reporting solutions, reducing manual reporting efforts and significantly improving reporting efficiency and turnaround time.
• Develop and maintain Python scripts to interact with various advertising platforms through API integrations (e.g., Facebook Ads, Google Ads, Programmatic Ads Platforms), enabling seamless data extraction and loading to desired platforms.
• Provide mentorship and technical guidance to team members, promoting the adoption of best practices in Python scripting, data engineering, and API integrations.
• Development and maintain data quality checks and procedures, helping stakeholders overcome data issues before they impact the quality of the reporting solutions.