Data Platform Engineer — External Data Acquisition

Astro Indonesia; Jakarta Barat; On-site; IDR 350,000,000 - 700,000,000; Full time

Job summary

Astro Indonesia is seeking a Data Platform Engineer — External Data Acquisition to build and operate platforms for acquiring data from external sources, including websites, apps, APIs, and public datasets. You will own the acquisition layer from extraction to landing data for downstream processing.

The role focuses on production-grade data acquisition for pricing and market intelligence, collaborating with Pricing, Data Warehouse Engineering, and Data Analysts to turn external-data needs into

Qualifications

• Strong Python/software engineering fundamentals.
• Hands-on experience building or operating web scraping, crawling, external API, or similar data acquisition systems.
• Strong understanding of HTTP, APIs, JSON, headers, cookies, sessions, authentication, pagination, and request/response behavior.
• Hands-on experience with browser/network debugging and the ability to investigate how websites or applications communicate with their backend services.
• Experience with browser automation tools such as Selenium, Playwright, Scrapy, or equivalent.
• Understanding of retry, timeout, rate limiting, concurrency, idempotency, and failure recovery.
• Solid understanding of SQL, data storage/warehousing, schema evolution, workflow orchestration, and cloud infrastructure.
• Strong troubleshooting skills for external systems outside control.

Responsibilities

• Build and maintain reliable web/app crawlers, scrapers, and external API integrations across e-commerce, marketplace, retail, and other external sources.
• Build acquisition systems that remain reliable as external websites, applications, APIs, and data contracts evolve.
• Handle real-world source complexity such as sessions, tokens, pagination, dynamic content, rate limits, retries, concurrency, and partial failures.
• Build resilient and maintainable acquisition systems with testing, idempotency, deduplication, retries, safe reruns, data-quality validation, monitoring, and alerting.
• Operate acquisition workloads using workflow orchestration, compute, data storage/warehousing, and cloud infrastructure.
• Continuously improve reliability, scalability, maintainability, and total cost of ownership; evaluate build-vs-buy for external data.
• Explore new technologies and AI-assisted development to improve data pipelines.
• Collaborate with Pricing, Data Warehouse Engineering, Data Analysts, and other stakeholders to turn external-data needs into platform capabilities.

Skills

Python

Web scraping

HTTP APIs

Browser debugging

Requests/HTTPX

Selenium

Playwright

Scrapy

SQL & Data warehousing

Cloud infrastructure

Tools

Selenium

Playwright

Scrapy

Requests/HTTPX

Job description

Astro Indonesia is seeking a Data Platform Engineer — External Data Acquisition to build and operate platforms for acquiring data from external sources, including websites, apps, APIs, and public datasets. You will own the acquisition layer from extraction to landing data for downstream processing.

The role focuses on production-grade data acquisition for pricing and market intelligence, collaborating with Pricing, Data Warehouse Engineering, and Data Analysts to turn external-data needs into