Data Platform Engineer

Astro Indonesia

Jakarta Barat

On-site

IDR 350,000,000 - 700,000,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Astro Indonesia is seeking a Data Platform Engineer — External Data Acquisition to build and operate platforms for acquiring data from external sources, including websites, apps, APIs, and public datasets. You will own the acquisition layer from extraction to landing data for downstream processing.

The role focuses on production-grade data acquisition for pricing and market intelligence, collaborating with Pricing, Data Warehouse Engineering, and Data Analysts to turn external-data needs into

Qualifications

  • Strong Python/software engineering fundamentals.
  • Hands-on experience building or operating web scraping, crawling, external API, or similar data acquisition systems.
  • Strong understanding of HTTP, APIs, JSON, headers, cookies, sessions, authentication, pagination, and request/response behavior.
  • Hands-on experience with browser/network debugging and the ability to investigate how websites or applications communicate with their backend services.
  • Experience with browser automation tools such as Selenium, Playwright, Scrapy, or equivalent.
  • Understanding of retry, timeout, rate limiting, concurrency, idempotency, and failure recovery.
  • Solid understanding of SQL, data storage/warehousing, schema evolution, workflow orchestration, and cloud infrastructure.
  • Strong troubleshooting skills for external systems outside control.

Responsibilities

  • Build and maintain reliable web/app crawlers, scrapers, and external API integrations across e-commerce, marketplace, retail, and other external sources.
  • Build acquisition systems that remain reliable as external websites, applications, APIs, and data contracts evolve.
  • Handle real-world source complexity such as sessions, tokens, pagination, dynamic content, rate limits, retries, concurrency, and partial failures.
  • Build resilient and maintainable acquisition systems with testing, idempotency, deduplication, retries, safe reruns, data-quality validation, monitoring, and alerting.
  • Operate acquisition workloads using workflow orchestration, compute, data storage/warehousing, and cloud infrastructure.
  • Continuously improve reliability, scalability, maintainability, and total cost of ownership; evaluate build-vs-buy for external data.
  • Explore new technologies and AI-assisted development to improve data pipelines.
  • Collaborate with Pricing, Data Warehouse Engineering, Data Analysts, and other stakeholders to turn external-data needs into platform capabilities.

Skills

Python
Web scraping
HTTP APIs
Browser debugging
Requests/HTTPX
Selenium
Playwright
Scrapy
SQL & Data warehousing
Cloud infrastructure

Tools

Selenium
Playwright
Scrapy
Requests/HTTPX

Job description

About this Position

As a Data Platform Engineer — External Data Acquisition, you will build and operate ASTRO's platform for acquiring data from external sources, including websites, web/mobile applications, APIs, public datasets, and third-party data providers.

A significant part of this role focuses on production-grade web and application data acquisition for pricing and market intelligence use cases. You will own the acquisition layer—from understanding external systems and extracting data to delivering reliable landing data for downstream processing.

Focus

External Data Acquisition

  • Web & App Scraping
  • Python
  • HTTP/API
  • Data Pipelines
  • Reliability
As our Astronaut, your missions are:
  • Build and maintain reliable web/app crawlers, scrapers, and external API integrations across e-commerce, marketplace, retail, and other external sources.
  • Build acquisition systems that remain reliable as external websites, applications, APIs, and data contracts evolve.
  • Handle real-world source complexity such as sessions, tokens, pagination, dynamic content, rate limits, retries, concurrency, and partial failures.
  • Build resilient and maintainable acquisition systems with appropriate testing, idempotency, deduplication, retries, safe reruns, data-quality validation, monitoring, and alerting.
  • Operate acquisition workloads using appropriate workflow orchestration, compute, data storage/warehousing, and cloud infrastructure.
  • Continuously improve reliability, scalability, maintainability, and total cost of ownership, including evaluating build-vs-buy approaches for external data.
  • Explore new technologies and AI-assisted development or automation to improve how external-data pipelines are built, operated, and maintained.
  • Collaborate with Pricing, Data Warehouse Engineering, Data Analysts, and other stakeholders to turn external-data needs into sustainable platform capabilities.
Minimum Qualifications

In order to launch successfully, you need to have:

  • Strong Python/software engineering fundamentals and experience building maintainable production systems.
  • Hands-on experience building or operating web scraping, crawling, external API, or similar data acquisition systems.
  • Strong understanding of HTTP, APIs, JSON, headers, cookies, sessions, authentication, pagination, and request/response behavior.
  • Hands-on experience with browser/network debugging and the ability to investigate how websites or applications communicate with their backend services.
  • Experience with both HTTP-based extraction and browser automation, using tools such as Requests/HTTPX, Selenium, Playwright, Scrapy, or equivalent.
  • Understanding of production reliability patterns such as retry, timeout, rate limiting, concurrency, idempotency, and failure recovery.
  • Solid understanding of SQL, data storage/warehousing, schema evolution, workflow orchestration, containerized workloads, and cloud infrastructure fundamentals.
  • Strong troubleshooting skills, especially when working with frequently changing external systems outside your direct control.
Good to Have
  • Experience acquiring data from mobile applications, marketplaces, e-commerce, grocery, food-delivery, or other large consumer platforms.
  • Experience with asynchronous or distributed acquisition systems and high-volume crawling.
  • Familiarity with proxy infrastructure, data observability, CI/CD, and infrastructure-as-code.
  • Experience using AI-assisted development or automation to improve data acquisition workflows.
  • Experience designing reusable scraping frameworks or shared acquisition platforms.
  • Experience with geospatial or other non-traditional external datasets.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Platform Engineer — External Data Acquisition
Data Platform Engineer — External Data Acquisition

Astro Indonesia • Jakarta Barat

On-site
IDR 350,000,000 - 700,000,000
Data Platform Engineer
Data Platform Engineer

Astro Holdings Berhad • Jakarta Pusat

On-site
IDR 167,400,000 - 390,600,000
Data Acquisition Platform Engineer
Data Acquisition Platform Engineer

Astro Holdings Berhad • Jakarta Pusat

On-site
IDR 167,400,000 - 390,600,000
Lead Service Excellence & Analytics
Lead Service Excellence & Analytics

ASTRO • Jakarta Pusat

On-site
IDR 420,000,000 - 660,000,000
Associate Data Scientist
Associate Data Scientist

Enterprise Digital Technology Services Edts • Jakarta Pusat

On-site
IDR 288,000,000 - 504,000,000
Senior Backend Engineer
Senior Backend Engineer

ASTRO • Jakarta Pusat

On-site
IDR 400,000,000 - 900,000,000
Data Engineer
Data Engineer

Super Bank Indonesia • Jakarta Pusat

On-site
IDR 250,000,000 - 420,000,000
Data Engineer
Data Engineer

AME Research • Jakarta Pusat

On-site
IDR 250,000,000 - 450,000,000
Modern office near public transport
Competitive salary
Customer Insight Associate
Customer Insight Associate

Astro Indonesia • Jakarta Barat

On-site
IDR 260,000,000 - 420,000,000
Commercial Data Engineer
Commercial Data Engineer

PepsiCo Inc. • Maluku

On-site
IDR 180,000,000 - 300,000,000