AWS Glue Data Engineer

Infosys; Bengaluru; On-site; INR 1,200,000 - 1,900,000; Full time

Job summary

Infosys is hiring a Data Engineer to design, build and maintain ETL/ELT pipelines using AWS Glue and Python for batch and incremental data processing.

The role focuses on developing and optimizing Glue Jobs with PySpark/Python, implementing ingestion, transformation and validation logic, and integrating with AWS services for secure, scalable workflows. Collaboration in Agile teams is required.

Qualifications

• Bachelor's degree or equivalent (e.g., BE/BTech/MSc/MCA/MTech).
• 3-5 years of experience in data engineering or ETL development.
• Hands-on with AWS Glue and Python for production-grade pipelines.
• Working knowledge of core AWS concepts including IAM and monitoring.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines using AWS Glue and Python for batch and incremental processing.
• Develop and optimize Glue Jobs (PySpark/Python) with parameters, bookmarks, retries, and tuning.
• Implement data ingestion, transformation, and validation to ensure data accuracy.
• Integrate pipelines with AWS services (S3, IAM, CloudWatch) for secure, observable workflows.
• Troubleshoot failures, analyze logs/metrics, and improve stability and performance.
• Collaborate with cross-functional teams to gather requirements and deliver well-documented solutions.
• Follow best practices including code reviews, version control, and modular coding patterns.

Skills

PySpark

SQL

Agile teamwork

Education

Bachelor's degree or equivalent in a related field

Tools

AWS Glue

Python

S3

AWS IAM

AWS CloudWatch

Job description

AWS Glue, Python Primary skills :AWS Glue-Technology->Cloud Platform->Amazon Webservices DevOps,Technology->Cloud Platform->AWS Data Analytics->AWS Glue DataBrew,Technology->OpenSystem->Python - OpenSystem

Preferred Qualifications:

• Experience with PySpark and distributed data processing patterns within AWS Glue.
• Strong SQL skills and experience working with structured/semi-structured datasets (CSV/JSON/Parquet).
• Exposure to orchestration and scheduling patterns for ETL workflows and dependency management.
• Familiarity with data quality checks, schema evolution handling, and building resilient pipelines.
• Experience collaborating in Agile teams and contributing to CI/CD or automated deployment practices for data jobs.

Good to have skills: PySpark, Amazon S3, AWS IAM, Amazon CloudWatch, SQL

Key Responsibilities:

• Design, build, and maintain ETL/ELT pipelines using AWS Glue and Python for batch and incremental data processing.
• Develop and optimize Glue Jobs (PySpark/Python) including job parameters, bookmarks, retries, and performance tuning.
• Implement data ingestion, transformation, and validation logic to ensure accuracy, completeness, and consistency of datasets.
• Integrate pipelines with AWS services (e.g., S3, IAM, CloudWatch) to enable secure, observable, and scalable workflows.
• Troubleshoot job failures, analyze logs/metrics, and implement fixes to improve stability and runtime efficiency.
• Collaborate with cross-functional teams to gather requirements, define data mappings, and deliver well-documented solutions.
• Follow engineering best practices including code reviews, version control, and reusable modular coding patterns.

Minimum Qualifications:

• Bachelor's degree or equivalent (e.g., BE/BTech/MSc/MCA/MTech).
• 3-5 years of experience in data engineering, ETL development, or data integration roles.
• Strong hands-on experience with AWS Glue and Python for building production-grade data pipelines.
• Working knowledge of core AWS concepts including security basics (IAM), storage patterns, and monitoring.
• Ability to debug data pipeline issues and deliver reliable solutions with clear documentation.