Senior Scraper Engineer

amIT Global Solutions Sdn Bhd; Indonesia; On-site; IDR 350,000,000 - 550,000,000; Full time

Job summary

amIT Global Solutions Sdn Bhd in Indonesia is seeking a senior Python scraping engineer to design, build, and maintain scalable scraping systems using requests, BeautifulSoup, Selenium, Playwright, and Scrapy. You will focus on reliability and maintainability, not one-off scripts.

You will architect anti-detection strategies, rotate proxies, bypass CAPTCHAs, and perform mobile app data extraction. You will own data pipelines ingesting web, mobile, and API data, and mentor junior engineers while

Qualifications

• Bachelor's degree in Computer Science, Information Systems, or related field.
• 5+ years in web scraping, crawling, or automation scripting with at least 1–2 years designing scalable scraping systems.
• Strong proficiency in Python and scraping ecosystem tools.
• Hands-on experience with mobile scraping/reverse engineering and anti-detection measures.

Responsibilities

• Design, build, and maintain scalable Python-based scraping systems using requests, BeautifulSoup, Selenium, Playwright, Scrapy.
• Architect anti-detection strategies, rotating proxies, CAPTCHA-bypass, and fingerprint randomization.
• Reverse-engineer and extract data from mobile apps via API interception and tooling.
• Architect data pipelines ingesting web, mobile, HTML/JSON/XML sources.
• Own orchestration and scheduling of scraping workflows (Airflow, Prefect, n8n).
• Lead integration of pipelines into Snowflake, S3, GCS, Google Sheets, SharePoint.
• Establish monitoring, logging, and observability for scraping infrastructure.
• Mentor junior engineers and evaluate build-vs-buy options for targets and tooling.

Skills

Python

Web scraping

Selenium

Playwright

Scrapy

Docker

CI/CD

Mentoring

Education

Bachelor's degree in Computer Science, Information Systems, or related field

Tools

BeautifulSoup

Selenium

Playwright

Scrapy

Frida

mitmproxy

Appium

Docker

Airflow

Job description

Key Responsibilities

• Design, build, and maintain scalable Python-based scraping systems (e.g., using requests, BeautifulSoup, Selenium, Playwright, Scrapy), architecting for reliability and maintainability rather than one-off scripts
• Design and implement anti-detection strategies at scale, rotating proxies, CAPTCHA-bypass solutions, user-agent/device fingerprint randomization, to sustain high scraping success rates across changing site and app defenses
• Reverse-engineer and extract data from mobile apps (Android/iOS) via API traffic interception, SSL pinning bypass, and emulator/device automation, in addition to standard web scraping
• Architect data pipelines that ingest and normalize structured and unstructured data from web APIs, mobile app APIs, HTML, JSON, and XML sources
• Own orchestration and scheduling of scraping workflows (Airflow, Prefect, n8n, or similar), including retry logic, alerting, and failure recovery
• Lead integration of pipelines into downstream systems (Snowflake, Google Sheets, S3, GCS, SharePoint), ensuring data quality and consistency
• Establish monitoring, logging, and observability practices for scraping infrastructure to catch breakages proactively rather than reactively
• Evaluate and prototype new scraping targets (web and mobile), data enrichment sources, and tooling; make build-vs-buy recommendations
• Anticipate and adapt to web/app structure, anti-bot, and API changes ahead of failures; mentor junior engineers on scraping best practices

Requirements

• Bachelor's degree in Computer Science, Information Systems, or related field
• At least 5 years of experience in web scraping, crawling, or automation scripting, including at least 1-2 years designing scraping systems at scale (not just writing individual scripts)
• Strong proficiency in Python and its scraping ecosystem (requests, BeautifulSoup, Selenium, Playwright, Scrapy)
• Hands-on experience with mobile scraping/reverse engineering, tools like Frida, mitmproxy, Charles Proxy, Objection, or Appium; experience bypassing SSL pinning and root/jailbreak detection
• Proven experience with headless browser automation (Puppeteer, Playwright) and defeating modern anti-bot measures (proxy rotation, CAPTCHA handling, rate-limiting, fingerprinting)
• Hands-on experience with containerization (Docker) and Git-based CI/CD in production environments
• Track record of scraping high-defense targets such as social media or e-commerce platforms (web and/or mobile app) at scale
• Experience owning end-to-end data pipelines into a warehouse (Snowflake preferred) or cloud storage
• Prior experience mentoring or leading other engineers is a plus