Senior AWS Data Engineer: Pipelines & Analytics

Tech Mahindra; Sydney; On-site; AUD 120,000 - 160,000; Full time

Job summary

Tech Mahindra is hiring a Data Engineer to design, build and maintain end-to-end data pipelines on AWS. You will work with Spark/Python, SQL, and Airflow to ingest, transform, and validate data across raw, cleansed, and curated layers. Collaboration with data scientists and analysts is key to delivering analytics-ready datasets.

The role emphasizes DataOps, CI/CD, and comprehensive documentation, with experience in AWS services including S3, Glue, Redshift, Athena and EMR highly valued.

Qualifications

• Minimum experience as a Data Engineer in large-scale environments.
• Strong command of SQL and data modelling concepts.
• Hands-on AWS data services experience (S3, Glue, Redshift, Athena, EMR).
• Proficient in Python and PySpark for data transformation.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines on AWS Data Platform.
• Build and manage ETL/ELT workflows using AWS services, dbt, and Airflow.
• Ingest data from databases, APIs, files, and event streams into lake/warehouse layers.
• Transform data with SQL and Python/PySpark for cleansing and enrichment.
• Implement data quality checks, monitoring, and alerting for issues.
• Collaborate with analysts/data scientists to model datasets for analytics/ML.
• Contribute to DataOps/DevOps incl. version control and CI/CD.
• Produce and maintain data flow documentation and runbooks.
• Optimise pipeline performance and support deployments.

Skills

Data engineering

Advanced SQL

PySpark

Python

Airflow

Data Warehousing

ETL/ELT pipelines

AWS data services

Stakeholder communication

Telco industry experience

Tools

Amazon S3

AWS Glue

Amazon Redshift

Amazon Athena

Amazon EMR

dbt

Airflow

Teradata

Siebel CRM data

Job description

Tech Mahindra is hiring a Data Engineer to design, build and maintain end-to-end data pipelines on AWS. You will work with Spark/Python, SQL, and Airflow to ingest, transform, and validate data across raw, cleansed, and curated layers. Collaboration with data scientists and analysts is key to delivering analytics-ready datasets.

The role emphasizes DataOps, CI/CD, and comprehensive documentation, with experience in AWS services including S3, Glue, Redshift, Athena and EMR highly valued.