Data Engineer

AME Mineral Economics Pty Ltd.; Daerah Khusus Ibukota Jakarta; On-site; IDR 666,777,796 - 1,000,166,694; Full time

Benefits offered by this job

Central office near public transport

Job summary

A data solutions company in Jakarta is seeking a Data Engineer to design and maintain ETL pipelines for processing data from public APIs. Candidates should have a Bachelor's degree in a relevant field and at least 3 years of experience with data pipeline construction and API data ingestion. The role involves working with Python and Azure to ensure reliable data availability for analytics. The position is full-time, office-based, and requires collaboration with data scientists and software developers.

Qualifications

• 3+ years of experience building data pipelines in a production environment.
• Strong understanding of ETL/ELT architecture and data ingestion principles.

Responsibilities

• Design and implement robust ETL/ELT pipelines to extract data from multiple public APIs.
• Build and manage scheduled workflows using tools such as Apache Airflow.
• Handle complex API integrations and ensure data integrity.

Skills

Data pipeline construction

Python

SQL (Microsoft T-SQL)

API data ingestion

Data modelling

Workflow orchestration (Apache Airflow)

Version control (Git)

Education

Bachelor’s degree in Computer Science, Data Engineering, or related field

Tools

Azure

Job description

Location: Jakarta
Position Type: Full-time (in office)
Reports to: CTO

About the Role

We’re looking for a Data Engineer to design, implement, and maintain reliable data pipelines that collect and process data from various public APIs into a centralised blob storage and database environment. You’ll play a key role in ensuring data is ingested efficiently, transformed cleanly, and made accessible for downstream analysis and reporting.

This role involves working with Python-based ETL pipelines, workflow orchestration (Apache Airflow or equivalent), and cloud storage solutions (e.g., Azure Blob Storage).

Key Responsibilities

• Design and implement robust ETL/ELT pipelines to extract data from multiple public APIs and load into blob storage and relational databases.
• Build and manage scheduled workflows using tools such as Apache Airflow (or similar).
• Handle complex API integrations, authentication flows (OAuth2, API keys), pagination, and rate limiting.
• Structure data in blob storage (e.g., Azure Blob) for efficient downstream access.
• Transform and load data into structured SQL-based databases for analytics and access by Web applications
• Implement monitoring, logging, and validation to ensure data integrity and pipeline reliability.
• Work closely with data scientists, and software developers to ensure data availability and usability.
• Maintain clear documentation for data sources, transformations, and orchestration logic.

Qualifications

• Bachelor’s degree in Computer Science, Data Engineering, or a related field (or equivalent experience).
• 3+ years of experience building data pipelines in a production environment.
• Strong understanding of ETL/ELT architecture, API data ingestion, and data modelling principles.
• Experience with Azure

Technical Skills:

• Languages: Python (preferred), SQL (Microsoft T-SQL)
• Orchestration: Apache Airflow or similar
• Familiarity with version control (Git) and CI/CD pipelines

Nice to Have:

• Experience integrating data from open economic and financial APIs (e.g., World Bank, IMF, OECD, UN Data, FRED)
• Familiarity with large structured and semi-structured datasets (JSON, CSV, Parquet) and associated performance considerations
• Experience building incremental ingestion or change data capture (CDC) pipelines

Soft Skills

• Strong analytical and problem-solving mindset
• Clear communication of technical concepts to non-technical stakeholders
• Attention to detail and data accuracy
• Proactive approach to monitoring

What We Offer

• Modern, central office near to public transport and key amenities

How to Apply

If you're interested in this role, please send your CV and Cover Letter to [e-mail] .