AWS Data Engineer

Infosys; Pune District; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

Infosys is seeking a Data Engineer experienced in AWS data services and PySpark to design, build and optimize scalable data pipelines. You will transform, validate, and integrate data from diverse sources into cloud platforms, ensuring quality and performance.

The role emphasizes collaboration with analytics and engineering teams, and requires strong SQL, ETL/ELT and data modeling skills. Knowledge of CI/CD, Git and Agile practices is advantageous.

Qualifications

• Strong hands-on experience with AWS Data Engineering services such as S3, Glue, Lambda, EMR, Redshift, Athena, and IAM.
• Strong programming experience in PySpark.
• Excellent knowledge of SQL, including complex queries and data transformations.
• Hands-on experience in developing ETL/ELT pipelines.
• Good understanding of data warehousing, data lakes, and data modeling concepts.

Responsibilities

• Design, develop, and maintain scalable data pipelines on AWS.
• Develop ETL/ELT workflows for data ingestion, transformation, and processing.
• Build data processing solutions using PySpark and SQL.
• Perform data transformations, validations, and data quality checks.
• Optimize data pipelines and SQL queries for performance and scalability.
• Work with different data sources and integrate data into cloud-based data platforms.
• Troubleshoot data pipeline failures and resolve data-related issues.
• Collaborate with business, analytics, and engineering teams to understand data requirements.

Skills

AWS Data Eng

PySpark

SQL

ETL/ELT pipelines

Data warehousing

Data lakes

Data modeling

Python

Git

CI/CD

Orchestration tools

Agile development

Tools

Apache Airflow

Spark ecosystem

Job description

Strong hands-on experience with AWS Data Engineering services such as S3, Glue, Lambda, EMR, Redshift, Athena, and IAM. Strong programming experience in PySpark. Excellent knowledge of SQL, including complex queries and data transformations. Hands-on experience in developing ETL/ELT pipelines. Good understanding of data warehousing, data lakes, and data modeling concepts. Experience handling large-scale datasets and optimizing distributed data processing workloads. Knowledge of Python is preferred. Familiarity with Git, CI/CD, orchestration tools, and Agile development practices will be an advantage.

Design, develop, and maintain scalable data pipelines on AWS. Develop ETL/ELT workflows for data ingestion, transformation, and processing. Build data processing solutions using PySpark and SQL. Perform data transformations, validations, and data quality checks. Optimize data pipelines and SQL queries for performance and scalability. Work with different data sources and integrate data into cloud-based data platforms. Troubleshoot data pipeline failures and resolve data-related issues. Collaborate with business, analytics, and engineering teams to understand data requirements.

Good analytical and problem-solving skills. Strong communication and collaboration skills. Ability to independently understand requirements and deliver data engineering solutions. Exposure to cloud migration or modernization projects would be an added advantage.