Senior Data Engineer: Cloud Pipelines & AI Tools

Macquarie Group; Indonesia; On-site; IDR 400,000,000 - 900,000,000; Full time

Job summary

Macquarie Group is seeking an experienced data/software engineer in Indonesia with 8+ years of hands-on experience and strong Python skills. You will build scalable ETL pipelines, implement data models, and optimize Spark/PySpark workloads in a cloud environment using AWS, Redshift, S3 and Glue.

The role emphasizes governance, security, and collaboration in an agile setting, with production support and observability responsibilities.

Qualifications

• 8+ years in data or software engineering with Python focus.
• Strong SQL using joins, CTEs, window functions.
• Experience building scalable ETL pipelines and data models.
• Spark or PySpark experience with performance tuning.
• AWS experience (Redshift, S3, Glue, IAM, Kinesis); familiarity with GCP.
• Airflow, Argo or dbt with scheduling, retries and backfills.
• Metadata, lineage, reconciliation, data protection and auditability.
• Docker, Unix/Linux; Kubernetes is a plus.
• Git, CI/CD, testing, observability and production support.
• Awareness of AI-assisted tools like GitHub Copilot; data privacy.

Responsibilities

• Develop and maintain scalable data pipelines and transformations.
• Collaborate with stakeholders to resolve data issues and risks.
• Implement governance, lineage and auditability across pipelines.

Skills

Python

SQL

ETL pipelines

Spark

PySpark

AWS

Redshift

S3

Glue

IAM

Kinesis

GCP

SQS

SNS

Kafka

Airflow

dbt

Kubernetes

Docker

Unix/Linux

Git

CI/CD

Observability

AI tools

Copilot

Embeddings

Vector search

Bedrock

Agile

Stakeholder engagement

Financial services

Tools

Spark

PySpark

Docker

Unix/Linux

Git

dbt

Job description

Macquarie Group is seeking an experienced data/software engineer in Indonesia with 8+ years of hands-on experience and strong Python skills. You will build scalable ETL pipelines, implement data models, and optimize Spark/PySpark workloads in a cloud environment using AWS, Redshift, S3 and Glue.

The role emphasizes governance, security, and collaboration in an agile setting, with production support and observability responsibilities.