AWS Glue Python Developer

Proarch It Solutions; Chennai District, Bengaluru; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Proarch It Solutions in India seeks an experienced AWS Glue Python Developer to design, develop, and support scalable data integration solutions using Python, AWS Glue, and PySpark in a hybrid setup across Chennai and Bengaluru.

You will build ETL pipelines, work with S3, Lambda, EventBridge, SNS/SQS, and implement CI/CD with DevOps teams, ensuring cost-efficient and reliable processing.

Qualifications

• 5+ years of Python development and data engineering.
• Strong hands-on experience with AWS Glue and PySpark.
• Excellent knowledge of Python programming and OOP.
• Experience designing ETL pipelines and data processing frameworks.
• Experience with S3, EventBridge, Lambda, SNS and SQS.
• Knowledge of CloudWatch for logging and monitoring.
• Understanding of IAM, encryption, and security practices.
• Experience with relational databases and SQL connectivity from Python.
• Experience with REST/SOAP APIs and external system integrations.
• Familiarity with Git and software development best practices.

Responsibilities

• Design and develop inbound and outbound file processing using AWS Glue, Python, and PySpark.
• Develop reusable Python modules and data ingestion/validation frameworks.
• Build batch processing with S3 and AWS Glue.
• Configure EventBridge rules for scheduling and event-driven processing.
• Develop Lambda functions for lightweight processing and integration tasks.
• Implement messaging with SNS and SQS.
• Set up CloudWatch logs, metrics, dashboards, and alarms for monitoring.
• Create workflows using Glue Workflows or Step Functions.
• Integrate AWS services with databases, APIs, and external apps.
• Handle data ingestion from CSV, XML, JSON, Excel and other formats.

Skills

Python development

Data engineering

ETL design

SQL queries

Problem solving

Education

Bachelor's degree in Computer Science or related

Tools

AWS Glue

PySpark

Python

Amazon S3

EventBridge

Lambda

SNS/SQS

CloudWatch

IAM

REST APIs

Git

Job description

Position Title: AWS Glue Python Developer

Location: Hybrid (Chennai & Bengaluru)

Duration: 6 Months and extendable

Experience: 6-12 Years

Job Summary

We are seeking an experienced AWS Glue Python Developer to design, develop, and support scalable cloud-based inbound and outbound file based data integration and processing solutions. The ideal candidate will have strong expertise in Python, AWS Glue, PySpark, and serverless AWS services.

The candidate will work with business and technical teams to develop reliable data processing workflows using services such as Amazon S3, EventBridge, Lambda, SNS, SQS, CloudWatch, IAM, and AWS Step Functions or Glue Workflows.

Key Responsibilities

• Design and develop inbound and outbound file processing using AWS Glue, Python, and PySpark.
• Develop reusable Python modules, utilities, and frameworks for data ingestion, validation, transformation, and error handling.
• Build batch inbound and outbound file processing using Amazon S3 and AWS Glue.
• Configure Amazon Event Bridge rules for scheduling and event-based processing.
• Develop AWS Lambda functions for lightweight processing, automation, validation, and integration activities.
• Implement messaging and notification solutions using Amazon SNS and Amazon SQS.
• Configure Amazon CloudWatch logs, metrics, dashboards, and alarms for operational monitoring.
• Develop workflows using AWS Glue Workflows, AWS Step Functions, or equivalent orchestration services.
• Integrate AWS services with relational databases, APIs, file systems, and external applications.
• Implement data ingestion from CSV, XML, JSON, Excel, fixed-length, variable-length, and delimited files.
• Develop data validation, reconciliation, audit, retry, and exception-handling processes.
• Optimize AWS Glue jobs for performance, scalability, and cost efficiency.
• Implement secure access using AWS IAM roles, policies, encryption, and secrets management.
• Support production deployments, incident resolution, root-cause analysis, and performance tuning.
• Collaborate with DevOps teams to implement CI/CD pipelines for AWS Glue jobs and related infrastructure.
• Prepare technical documentation, data flow diagrams, operational procedures, and support documentation.

Required Skills and Experience

• 5+ years of experience in Python development and data engineering.
• Strong hands-on experience with AWS Glue and PySpark.
• Excellent knowledge of Python programming, object-oriented programming, and exception handling.
• Experience designing ETL pipelines and data processing frameworks.
• Strong experience with Amazon S3, including bucket structures, lifecycle policies, partitioning, and file formats.
• Experience with Amazon EventBridge for scheduled and event-driven execution.
• Experience developing and deploying AWS Lambda functions using Python.
• Experience with Amazon SNS and SQS for notifications, messaging, and asynchronous processing.
• Experience with Amazon CloudWatch for logging, monitoring, alerting, and troubleshooting.
• Understanding of AWS IAM, security policies, encryption, and least-privilege access.
• Experience working with relational databases such as Oracle, PostgreSQL, MySQL, or Amazon RDS.
• Strong knowledge of SQL and database connectivity from Python or AWS Glue.
• Experience working with REST/SOAP APIs and external system integrations.
• Understanding of JSON, XML, CSV, Excel, and fixed-width file formats.
• Experience with Git and software development best practices.

Preferred Skills

• Experience with AWS Step Functions or AWS Glue Workflows.
• Experience with CI/CD tools such as Jenkins, GitHub Actions, GitLab, or AWS CodePipeline.
• Knowledge of data quality, reconciliation, lineage, and audit requirements.

Education

Bachelors degree in Computer Science, Information Technology, Engineering, or a related discipline.

Key Competencies

• Strong analytical and problem-solving skills.
• Ability to design scalable and maintainable cloud solutions.
• Good understanding of distributed data processing concepts.
• Strong communication and collaboration skills.
• Ability to work independently and manage multiple priorities.