Big Data Engineer

Hexaware; Mexico; On-site; PHP 900,000 - 1,100,000; Full time

Job summary

Hexaware is seeking a Mid Level Dev AWS Data Engineer with 4–8 years of software development experience to build and maintain ETL pipelines using Python and PySpark on AWS Glue and related platforms. You will orchestrate workflows with AWS Step Functions and Lambda, implement messaging via SNS/SQS, design scalable storage with Redshift, RDS and S3, and write efficient SQL for transformations and reporting.

Collaborate with cross-functional teams to translate business requirements into data

Qualifications

• 4–8 years of software development experience, focused on data engineering.
• Hands-on experience with ETL/data pipelines on AWS.
• Proficient in Python and PySpark.
• Strong SQL skills and data warehousing understanding.
• Experience with CI/CD and version control practices.

Responsibilities

• Build and maintain ETL pipelines using Python and PySpark on AWS Glue and related platforms.
• Orchestrate workflows using AWS Step Functions and Lambda.
• Implement messaging and event-driven integrations using SNS and SQS.
• Design and optimize storage and querying solutions with Redshift, RDS and S3.
• Write efficient SQL for transformations, validation, and reporting.
• Integrate data from APIs and process JSON data; ensure data quality and monitoring.
• Participate in CI/CD and production support; collaborate with cross-functional teams.

Skills

Python

PySpark

SQL

AWS

ETL

Data warehousing

CI/CD

Communication

Tools

AirFlow

Step Functions

Lambda

Redshift

RDS

S3

GitLab

Terraform

Athena

EMR

Kinesis

DynamoDB

Job description

• Mid Level Dev AWS Data Engineer with 4-8 years of software development experience Build and maintain ETL pipelines using Python and PySpark on AWS Glue and related platforms.
• Orchestrate workflows using AWS Step Functions and Lambda.
• Implement messaging and event-driven integrations using SNS and SQS.
• Design and optimize storage and querying solutions in Amazon Redshift, RDS, Oracle and S3-based architectures.
• Write efficient SQL for transformations, validation, and reporting.
• Integrate data from APIs and process structured and semi-structured JSON data.
• Implement data quality checks, monitoring, and operational support processes.
• Participate in CI/CD and version control practices for deployment and release management.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• 4-8 years of software development experience across the appropriate platform.
• Strong hands-on experience with Python, PySpark, API’s and SQL.
• Experience with ETL/data pipeline development and Orchestration using Step functions / AirFlow .
• Working knowledge of AWS services including Glue, Lambda, Step Functions, Redshift, S3, SNS, and SQS.
• Experience with Athena, EMR, Kinesis, DynamoDB, or RDS.
• Good Knowledge on CloudWatch, logging, and production support.
• Understanding of data warehousing, data lakes, Lake House and query optimization.
• Experience with GitLab/Terraform or similar and CI/CD workflows.
• Good understanding of using AI tools like Github Copilot or similar for code productivity
• Exposure to enterprise data lake or cloud migration initiatives.
• Have an eye to solving complex problems, great communication with stakeholders
• Have a good understanding of performance engineering of code pipelines and near real time systems
• Good understanding on Agents and MCP"