Data Engineer: Build Robust ETL Pipelines with Airflow

AME Mineral Economics Pty Ltd.; Daerah Khusus Ibukota Jakarta; On-site; IDR 666,777,796 - 1,000,166,694; Full time

Benefits offered by this job

Central office near public transport

Job summary

A data solutions company in Jakarta is seeking a Data Engineer to design and maintain ETL pipelines for processing data from public APIs. Candidates should have a Bachelor's degree in a relevant field and at least 3 years of experience with data pipeline construction and API data ingestion. The role involves working with Python and Azure to ensure reliable data availability for analytics. The position is full-time, office-based, and requires collaboration with data scientists and software developers.

Qualifications

• 3+ years of experience building data pipelines in a production environment.
• Strong understanding of ETL/ELT architecture and data ingestion principles.

Responsibilities

• Design and implement robust ETL/ELT pipelines to extract data from multiple public APIs.
• Build and manage scheduled workflows using tools such as Apache Airflow.
• Handle complex API integrations and ensure data integrity.

Skills

Data pipeline construction

Python

SQL (Microsoft T-SQL)

API data ingestion

Data modelling

Workflow orchestration (Apache Airflow)

Version control (Git)

Education

Bachelor’s degree in Computer Science, Data Engineering, or related field

Tools

Azure

Job description

A data solutions company in Jakarta is seeking a Data Engineer to design and maintain ETL pipelines for processing data from public APIs. Candidates should have a Bachelor's degree in a relevant field and at least 3 years of experience with data pipeline construction and API data ingestion. The role involves working with Python and Azure to ensure reliable data availability for analytics. The position is full-time, office-based, and requires collaboration with data scientists and software developers.