AWS Glue

LTM; Chennai District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

LTM is seeking an engineer with a strong data background to build and orchestrate production data pipelines using AWS Glue, Lambda and Snowflake on AWS. You will design scalable ETL/ELT pipelines and collaborate with stakeholders.

Responsibilities include serverless ingestion, data modeling in Snowflake, and implementing CI/CD for data pipelines. The role emphasizes data governance, cost efficiency, and performance tuning in a cloud-first environment.

Qualifications

• Experience building ETL/ELT pipelines and data integration workflows.
• Hands-on with serverless data ingestion and cloud-native architectures.
• Strong proficiency in Python for data engineering tasks.
• Ability to translate business requirements into scalable data solutions.

Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines using AWS Glue and Python.
• Build event-driven and serverless data ingestion solutions using AWS Lambda, Step Functions, S3, SNS, and SQS.
• Develop and optimize Snowflake data models, stored procedures, tasks, streams and ingestion frameworks.
• Implement data transformation, cleansing, validation and reconciliation processes.
• Design and maintain data lake and data warehouse architectures on AWS and Snowflake.
• Optimize pipeline performance, scalability, reliability and cost efficiency.
• Create reusable framework components and automation utilities; troubleshoot production issues and perform root cause analysis.
• Implement CI/CD pipelines using GitHub, Jenkins, Terraform or similar tools.
• Collaborate with business stakeholders, architects and analytics teams to deliver data solutions.
• Ensure adherence to data governance, security and compliance standards.
• Create technical documentation and support knowledge transfer activities.

Tools

AWS Glue

AWS Lambda

Python

Snowflake

Apache Spark

Databricks

Job description

Role Description

Engineer with strong data background and specific handson experience in AWS Glue Lambda Python Snowflake building and orchestrating production data pipelines including serverless ingestion and medallion architecture Skilled in sourcing data from across LOB systems and conforming it to enterprise standards and governed definitions

Experience with API based ingestion native connector integrations and data warehouse design Able to partner with the business to elevate operations to the next level by translating requirements into scalable data solutions

Mandatory Skills

• AWS Glue
• AWS Lambda
• Python
• Snowflake

Good to Have Skills

• Apache Spark
• Databricks

Primary Skills

• AWS Glue
• AWS Lambda
• Python
• Snowflake

Secondary Skills

• Databricks

• Design develop and maintain scalable ETLELT pipelines using AWS Glue and Python
• Build eventdriven and serverless solutions using AWS Lambda Step Functions S3 SNS and SQS
• Develop and optimize Snowflake data models stored procedures tasks streams and data ingestion frameworks
• Implement data transformation cleansing validation and reconciliation processes
• Design and maintain data lake and data warehouse architectures on AWS and Snowflake
• Optimize pipeline performance scalability reliability and cost efficiency
• Develop reusable framework components and automation utilities
• Troubleshoot production issues and perform root cause analysis
• Implement CICD pipelines using GitHub Jenkins Terraform or similar tools
• Collaborate with business stakeholders architects and analytics teams to deliver data solutions
• Ensure adherence to data governance security and compliance standards
• Create technical documentation and support knowledge transfer activities