Data Engineer: Build ETL Pipelines & API Integrations

AME Research; Jakarta Pusat; On-site; IDR 250,000,000 - 450,000,000; Full time

Benefits offered by this job

Modern office near public transport

Competitive salary

Job summary

AME Research in Jakarta is seeking a Data Engineer to design, implement, and maintain reliable data pipelines that collect and process data from various public APIs into a centralised blob storage and database environment.

You will ensure data is ingested efficiently and transformed cleanly for downstream analysis and reporting, collaborating with data scientists and developers in an in-office setting.

Qualifications

• Bachelor's degree in a related field or equivalent experience.
• 3+ years of experience building data pipelines in a production environment.
• Strong understanding of ETL/ELT architecture, API data ingestion, and data modelling principles.
• Experience with Azure and cloud-based data ecosystems.

Responsibilities

• Design and implement ETL/ELT pipelines to ingest data from multiple public APIs into blob storage and relational databases.
• Build and manage scheduled workflows using Apache Airflow (or similar).
• Handle complex API integrations, authentication flows (OAuth2, API keys), pagination, and rate limiting.
• Structure data in blob storage (e.g., Azure Blob) for efficient downstream access.
• Transform and load data into structured SQL-based databases for analytics and access by Web applications.
• Implement monitoring, logging, and validation to ensure data integrity and pipeline reliability.
• Work closely with data scientists and software developers to ensure data availability and usability.
• Maintain clear documentation for data sources, transformations, and orchestration logic.

Skills

Python

SQL

Azure

ETL/ELT

Airflow

Education

Bachelor's degree in Computer Science, Data Engineering

Tools

Azure Blob Storage

Git

CI/CD pipelines

Apache Airflow

Job description

AME Research in Jakarta is seeking a Data Engineer to design, implement, and maintain reliable data pipelines that collect and process data from various public APIs into a centralised blob storage and database environment.

You will ensure data is ingested efficiently and transformed cleanly for downstream analysis and reporting, collaborating with data scientists and developers in an in-office setting.