Aws Developer

Infosys; Bengaluru; On-site; INR 1,800,000 - 2,800,000; Full time

Job summary

Infosys in Bengaluru seeks an experienced Data Engineer to design, build and operate ETL/ELT pipelines on AWS Glue. You will implement PySpark, Python, and SQL for large-scale data processing and work across structured and semi-structured sources.

You will optimize Glue/Spark jobs for performance and cost, implement data quality checks, and collaborate with data architects and cloud teams to deliver scalable cloud data solutions, with CI/CD and IaC practices where applicable.

Qualifications

• 5+ years of data engineering/ETL development experience.
• Hands-on AWS Glue and PySpark for production-grade workloads.
• Experience with high-volume datasets on AWS environments.
• Knowledge of data lake and data warehouse concepts.
• Preferred AWS Data Engineer certification.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines using AWS Glue.
• Build AWS Glue jobs with Python, PySpark, and SQL for large-scale processing.
• Optimize Glue/Spark jobs for performance, scalability, and cost.
• Implement data quality checks, validations, and reconciliation.
• Collaborate with data architects and cloud/platform teams; document deployment/runbook.

Skills

AWS Glue

Python/PySpark

SQL

Apache Spark

ETL/ELT design

Performance tuning

Tools

CI/CD pipelines

Terraform

AWS

Job description

• Design, develop, and maintain ETL/ELT pipelines using AWS Glue.
• Develop AWS Glue jobs using Python,PySpark, and SQL for large-scale data processing.
• Build data ingestion and transformation pipelines across structured and semi-structured data sources.
• Configure and manage AWS Glue Data Catalog, Jobs, Workflows, and Triggers.
• Work with AWS services such as Amazon S3, Redshift, and IAM.
• Implement data transformations, validations, data-quality checks, and reconciliation processes.
• Optimize Glue/Spark jobs for performance, scalability, reliability, and cost efficiency.
• Implement incremental loads, partitioning strategies, error handling, restart/recovery, and audit frameworks.
• Troubleshoot ETL pipeline failures and production performance issues.
• Implement appropriate security controls including IAM roles, encryption, and access management.
• Support migration of legacy/mainframe or on-premises ETL workloads to the AWS cloud data platform.
• Develop automated deployment processes using CI/CD and Infrastructure-as-Code practices where applicable.
• Collaborate with data architects, analysts, business teams, and cloud/platform teams to translate requirements into scalable data solutions.
• Prepare technical design documents, mapping specifications, operational documentation, and deployment/runbook documentation.

• Primary skills:Technology->Cloud Platform->AWS Enviornment Provisioning,Technology->Data On Cloud - NoSQL->Amazon Dynamo DB,Technology->Data On Cloud - Platform->AWS
• Design, develop, and maintain ETL/ELT pipelines using AWS Glue.
• Develop AWS Glue jobs using Python,PySpark, and SQL for large-scale data processing.
• Build data ingestion and transformation pipelines across structured and semi-structured data sources.
• Configure and manage AWS Glue Data Catalog, Jobs, Workflows, and Triggers.
• Work with AWS services such as Amazon S3, Redshift, and IAM.
• Implement data transformations, validations, data-quality checks, and reconciliation processes.
• Optimize Glue/Spark jobs for performance, scalability, reliability, and cost efficiency.
• Implement incremental loads, partitioning strategies, error handling, restart/recovery, and audit frameworks.
• Troubleshoot ETL pipeline failures and production performance issues.
• Implement appropriate security controls including IAM roles, encryption, and access management.
• Support migration of legacy/mainframe or on-premises ETL workloads to the AWS cloud data platform.
• Develop automated deployment processes using CI/CD and Infrastructure-as-Code practices where applicable.
• Collaborate with data architects, analysts, business teams, and cloud/platform teams to translate requirements into scalable data solutions.
• Prepare technical design documents, mapping specifications, operational documentation, and deployment/runbook documentation.

Technical Skills

• AWS Glue
• Python/PySpark / Apache Spark
• SQL
• Amazon S3
• Data Lake / Cloud Data Warehouse concepts
• ETL/ELT design and development
• Performance optimization and troubleshooting Experience
• 5+ years of overall Data Engineering / ETL development experience.
• Strong hands-on experience developing production-grade AWS Glue and PySpark solutions.
• Experience handling high-volume datasets and complex transformations in AWS environments. Preferred Certification
• AWS Certified Data Engineer – Associate or other relevant AWS certification.