AWS Data Engineer

Tata Consultancy Services; Uttar Pradesh; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Tata Consultancy Services (TCS) in Noida is seeking an AWS Data Engineer to design, build, and optimize scalable data pipelines and cloud-based data platforms using Python, PySpark, and AWS data services.

The ideal candidate has 4 to 7 years of experience with big data processing, data warehousing, and data governance, and can collaborate with stakeholders, analysts, and architects to deliver reliable data solutions.

Qualifications

• Experience designing scalable data pipelines and cloud-based data platforms.
• Hands-on experience with big data processing, data integration, cloud migration, and data warehousing on AWS.
• Proficient in Python, PySpark, SQL, and Spark SQL.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and optimize ETL/ELT workflows for large-scale data processing.
• Develop data solutions using AWS services such as Glue, EMR, S3, Lambda, Redshift, Athena, and RDS.
• Implement data ingestion and transformation processes from multiple sources.
• Work with structured and unstructured datasets in cloud environments.
• Optimize Spark jobs and improve data processing performance.
• Develop and maintain data lakes and data warehouse solutions.
• Collaborate with business stakeholders, data analysts, and architects to understand data requirements.
• Ensure data quality, governance, and security standards are followed.
• Participate in code reviews, testing, deployment, and support activities.

Skills

AWS Cloud

Python

PySpark

Apache Spark

SQL

Data Engineering

ETL Development

Data Warehousing

Tools

AWS Glue

Amazon S3

Amazon EMR

AWS Lambda

Amazon Redshift

Athena

RDS

IAM

CloudWatch

Job description

TCS Hiring | Azure Data Engineer | Noida

Greetings from Tata Consultancy Services (TCS)!

We are looking for As an AWS Data Engineer with strong expertise in Python, PySpark, and AWS Data Services to design, build, and optimize scalable data pipelines and cloud-based data platforms. The ideal candidate should have hands-on experience in big data processing, data integration, cloud migration, and data warehousing solutions on AWS.

? Job Title-AWS Data Engineer

? Location - Noida

? Experience Required - 4 to 7 Years

Job Summary

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and optimize ETL/ELT workflows for large-scale data processing.
• Develop data solutions using AWS services such as Glue, EMR, S3, Lambda, Redshift, Athena, and RDS.
• Implement data ingestion and transformation processes from multiple sources.
• Work with structured and unstructured datasets in cloud environments.
• Optimize Spark jobs and improve data processing performance.
• Develop and maintain data lakes and data warehouse solutions.
• Collaborate with business stakeholders, data analysts, and architects to understand data requirements.
• Ensure data quality, governance, and security standards are followed.
• Participate in code reviews, testing, deployment, and support activities.

Required Skills

Core Skills

• AWS Cloud
• Python
• PySpark
• Apache Spark
• SQL
• Data Engineering
• ETL Development
• Data Warehousing

AWS Services

• AWS Glue
• Amazon S3
• Amazon EMR
• AWS Lambda
• Amazon Redshift
• Athena
• RDS
• IAM
• CloudWatch

Big Data Technologies

• Hadoop
• Spark SQL
• Data Lake
• Distributed Computing
• Batch Processing
• Real-Time Data Processing

Preferred Skills

• Airflow
• Kafka
• Snowflake
• Databricks
• Terraform
• Docker
• CI/CD
• Git/GitHub