AWS Glue, Python

Infosys; Bengaluru; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Infosys Bengaluru is seeking a data engineering professional to design and optimize cloud-based ETL/ELT pipelines using AWS Glue and Python. You will partner with engineers, analysts, and stakeholders to deliver reliable, scalable data workflows while emphasizing performance, automation, and data quality.

The role focuses on production-grade data pipelines, data ingestion and transformation, and collaboration across cross-functional teams to implement secure, observable solutions in an agile

Qualifications

• Bachelor’s degree or equivalent (e.g., BE/BTech/MSc/MCA/MTech).
• 3-5 years of experience in data engineering, ETL development, or data integration roles.
• Strong hands-on experience with AWS Glue and Python for building production-grade data pipelines.
• Working knowledge of core AWS concepts including security basics (IAM), storage patterns, and monitoring.
• Ability to debug data pipeline issues and deliver reliable solutions with clear documentation.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines using AWS Glue and Python for batch and incremental data processing.
• Develop and optimize Glue Jobs (PySpark/Python) including job parameters, bookmarks, retries, and performance tuning.
• Implement data ingestion, transformation, and validation logic to ensure accuracy, completeness, and consistency of datasets.
• Integrate pipelines with AWS services (e.g., S3, IAM, CloudWatch) to enable secure, observable, and scalable workflows.
• Troubleshoot job failures, analyze logs/metrics, and implement fixes to improve stability and runtime efficiency.
• Collaborate with cross-functional teams to gather requirements, define data mappings, and deliver well-documented solutions.
• Follow engineering best practices including code reviews, version control, and reusable modular coding patterns.

Skills

AWS Glue

Python

PySpark

SQL

Amazon S3

IAM

CloudWatch

DataBrew

Education

BE/BTech/MSc/MCA/MTech

MCA/MTech

Tools

AWS Glue DataBrew

Job description

Job DescriptionAbout the job:

Join a collaborative team where youll help transform raw data into trusted, analytics-ready assets that power smarter decisions. In this role, you will design and optimize cloud-based data integration workflows using AWS Glue and Python, partnering closely with engineers, analysts, and stakeholders to deliver reliable pipelines at scale. Youll work in an environment that values clean engineering, continuous improvement, and shared ownershipwhere your ideas on performance, automation, and data quality are welcomed. If you enjoy solving real-world data challenges, building robust ETL processes, and learning modern AWS data services while contributing to impactful outcomes, this opportunity offers the right mix of hands‑on development and problem‑solving responsibility.

Roles ResponsibilitiesKey Responsibilities:

• Design, build, and maintain ETL/ELT pipelines using AWS Glue and Python for batch and incremental data processing.
  
• Develop and optimize Glue Jobs (PySpark/Python) including job parameters, bookmarks, retries, and performance tuning.
  
• Implement data ingestion, transformation, and validation logic to ensure accuracy, completeness, and consistency of datasets.
  
• Integrate pipelines with AWS services (e.g., S3, IAM, CloudWatch) to enable secure, observable, and scalable workflows.
  
• Troubleshoot job failures, analyze logs/metrics, and implement fixes to improve stability and runtime efficiency.
  
• Collaborate with cross-functional teams to gather requirements, define data mappings, and deliver well‑documented solutions.
  
• Follow engineering best practices including code reviews, version control, and reusable modular coding patterns.
  

Minimum Qualifications:

• Bachelors degree or equivalent (e.g., BE/BTech/MSc/MCA/MTech).
  
• 35 years of experience in data engineering, ETL development, or data integration roles.
  
• Strong hands‑on experience with AWS Glue and Python for building production‑grade data pipelines.
  
• Working knowledge of core AWS concepts including security basics (IAM), storage patterns, and monitoring.
  
• Ability to debug data pipeline issues and deliver reliable solutions with clear documentation.
  

Technical Requirement

Primary skills :

AWS Glue-Technology->Cloud Platform->Amazon Webservices DevOps,Technology->Cloud Platform->AWS Data Analytics->AWS Glue DataBrew,Technology->OpenSystem->Python - OpenSystem

Additional ResponsibilityPreferred Qualifications:

• Experience with PySpark and distributed data processing patterns within AWS Glue.
  
• Strong SQL skills and experience working with structured/semi‑structured datasets (CSV/JSON/Parquet).
  
• Exposure to orchestration and scheduling patterns for ETL workflows and dependency management.
  
• Familiarity with data quality checks, schema evolution handling, and building resilient pipelines.
  
• Experience collaborating in Agile teams and contributing to CI/CD or automated deployment practices for data jobs.
  

Good to have skills:

• PySpark
• Amazon S3
• AWS IAM
• Amazon CloudWatch
• SQL

Educational Requirement

MCA,MSc,MTech,Bachelor of Engineering,BTech

Preferred Skills

Technology->Cloud Platform->Amazon Webservices DevOps,Technology->Cloud Platform->AWS Data Analytics->AWS Glue DataBrew,Technology->OpenSystem->Python - OpenSystem

Service LineData Analytics Unit