Aws Data Engineer-Virtual Drive -15-Oct

Tata Consultancy Services; Pune District, Bengaluru; Remote; INR 2,800,000 - 5,200,000; Full time

Job summary

Tata Consultancy Services is seeking an AWS Data Engineer to design and maintain large-scale data pipelines using AWS Data Factory to load data into Snowflake. The role requires collaboration with cross-functional teams to deliver robust data processing solutions.

The candidate will develop complex SQL, optimize performance in Snowflake/Redshift, implement Pandas-based quality checks, and ensure scalability, security, and reliability across data systems.

Qualifications

• 6-9 years of experience as an AWS Data Engineer with Python
• Proficiency in AWS services such as S3, Lambda, Step Functions, etc.
• Big-data tech exposure with Spark (PySpark), Hive or Impala would be a plus

Responsibilities

• Design, develop, test, deploy, and maintain large-scale data pipelines using AWS Data Factory (ADF) to load data into Snowflake
• Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Snowflake/Redshift
• Implement data quality checks using Pandas and perform statistical analysis on large datasets
• Ensure scalability, security, and reliability of the system by implementing best practices around data engineering

Skills

Python

SQL

Spark (PySpark)

Snowflake

Education

B.Tech / B.E. degree

Tools

AWS (S3, Lambda, Step Functions)

AWS Data Factory (ADF)

Snowflake

Redshift

Pandas

Job description

TCS-- AWS Data Engineer --Virtual Drive--15 Oct for Bengaluru & Pune

Roles and Responsibilities

• Design, develop, test, deploy, and maintain large-scale data pipelines using AWS Data Factory (ADF) to extract, transform, load data from various sources into Snowflake.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business needs.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in NoSQL databases like Redshift.
• Implement data quality checks using Pandas and perform statistical analysis on large datasets.
• Ensure scalability, security, and reliability of the system by implementing best practices around data engineering.

Desired Candidate Profile

• 6-9 years of experience as an AWS Data Engineer with expertise in Python programming language.
• Strong understanding of AWS services such as S3, Lambda functions, Step Functions etc. .
• Proficiency in working with big-data technologies like Spark (PySpark), Hive or Impala would be a plus.
• Educational background in B.Tech / B.E. degree from any specialization.