Data Platform Engineer

Astro Holdings Berhad

Jakarta Pusat

On-site

IDR 167,400,000 - 390,600,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Astro Holdings Berhad in Jakarta is seeking a Python-focused software engineer to design and maintain scalable web crawlers, scrapers, and external API integrations across ecommerce and retail sources. You will build acquisition systems that stay reliable as websites and APIs evolve, handle sessions, pagination, tokens, and rate limits, and ensure robust testing and monitoring.

Collaborate with Pricing and Data teams to turn external-data needs into enduring platform capabilities using cloud

Qualifications

  • Strong Python and software engineering fundamentals.
  • Hands-on experience building or operating data acquisition systems.
  • Understanding of production reliability patterns such as retry, timeout, rate limiting, concurrency, and failure recovery.
  • Solid knowledge of SQL, data storage/warehousing, schema evolution, workflow orchestration, containerized workloads, and cloud infrastructure fundamentals.

Responsibilities

  • Build and maintain reliable web/app crawlers, scrapers, and external API integrations across e-commerce, marketplace, retail, and other external sources.
  • Build acquisition systems that remain reliable as external websites, applications, APIs, and data contracts evolve.
  • Handle real-world source complexity such as sessions, tokens, pagination, dynamic content, rate limits, retries, concurrency, and partial failures.
  • Build resilient and maintainable acquisition systems with appropriate testing, idempotency, deduplication, retries, safe reruns, data-quality validation, monitoring, and alerting.
  • Operate acquisition workloads using appropriate workflow orchestration, compute, data storage/warehousing, and cloud infrastructure.
  • Continuously improve reliability, scalability, maintainability, and total cost of ownership, including evaluating build-vs-buy approaches for external data.
  • Explore new technologies and AI-assisted development or automation to improve how external-data pipelines are built, operated, and maintained.
  • Collaborate with Pricing, Data Warehouse Engineering, Data Analysts, and other stakeholders to turn external-data needs into sustainable platform capabilities.

Skills

Python
Software engineering fundamentals
Reliability patterns
Troubleshooting
Production systems

Tools

SQL
Docker
Cloud infrastructure

Job description

  • Build and maintain reliable web/app crawlers, scrapers, and external API integrations across e-commerce, marketplace, retail, and other external sources.
  • Build acquisition systems that remain reliable as external websites, applications, APIs, and data contracts evolve.
  • Handle real-world source complexity such as sessions, tokens, pagination, dynamic content, rate limits, retries, concurrency, and partial failures.
  • Build resilient and maintainable acquisition systems with appropriate testing, idempotency, deduplication, retries, safe reruns, data-quality validation, monitoring, and alerting.
  • Operate acquisition workloads using appropriate workflow orchestration, compute, data storage/warehousing, and cloud infrastructure.
  • Continuously improve reliability, scalability, maintainability, and total cost of ownership, including evaluating build-vs-buy approaches for external data.
  • Explore new technologies and AI-assisted development or automation to improve how external-data pipelines are built, operated, and maintained.
  • Collaborate with Pricing, Data Warehouse Engineering, Data Analysts, and other stakeholders to turn external-data needs into sustainable platform capabilities.

Job Requirements:

  • Strong Python/software engineering fundamentals and experience building maintainable production systems.
  • Hands-on experience building or operating web scraping, crawling, external API, or similar data acquisition systems.
  • Understanding of production reliability patterns such as retry, timeout, rate limiting, concurrency, idempotency, and failure recovery.
  • Solid understanding of SQL, data storage/warehousing, schema evolution, workflow orchestration, containerized workloads, and cloud infrastructure fundamentals.
  • Strong troubleshooting skills, especially when working with frequently changing external systems outside your direct control.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Platform Engineer
Data Platform Engineer

Astro Indonesia • Jakarta Barat

On-site
IDR 350,000,000 - 700,000,000
Data Acquisition Platform Engineer
Data Acquisition Platform Engineer

Astro Holdings Berhad • Jakarta Pusat

On-site
IDR 167,400,000 - 390,600,000
Senior ETL Pentaho Lead for e-Commerce
Senior ETL Pentaho Lead for e-Commerce

Luminare Consulting • Daerah Khusus Ibukota Jakarta

On-site
IDR 300,000,000 - 550,000,000
Commercial Data Engineer
Commercial Data Engineer

PepsiCo Inc. • Pluit

On-site
IDR 350,000,000 - 700,000,000
Data Engineer
Data Engineer

Super Bank Indonesia • Jakarta Pusat

On-site
IDR 250,000,000 - 420,000,000
Data Platform Engineer — External Data Acquisition
Data Platform Engineer — External Data Acquisition

Astro Indonesia • Jakarta Barat

On-site
IDR 350,000,000 - 700,000,000
Senior Data Engineer - Capital Markets
Senior Data Engineer - Capital Markets

Macquarie Group • Indonesia

On-site
IDR 400,000,000 - 900,000,000
Data Engineer (Data Platform) - Monee Credit
Data Engineer (Data Platform) - Monee Credit

Monee • Jakarta Pusat

On-site
IDR 167,400,000 - 279,000,000
Data Engineer
Data Engineer

PT. Pasifik Cipta Solusi • Tangerang Selatan

On-site
IDR 279,000,000 - 446,400,000
Senior Data Engineer - Jakarta
Senior Data Engineer - Jakarta

PT Bibit Tumbuh Bersama • Jakarta Utara

On-site
IDR 240,000,000 - 480,000,000