Senior Data Infra Engineer - Remote (Scraping & Scale)

Jobgether SRL; Australia; Remote; AUD 140,000 - 210,000; Full time

Benefits offered by this job

Fully remote work arrangement

Full-time employment

Job summary

Partner Company, acting on behalf of its partner, is seeking a Senior Data Infrastructure Engineer (Scraping & Scale) based in Australia. This fully remote role focuses on building high-performance web scraping and data ingestion systems to collect millions of profiles, posts, and video records daily.

You will design fault-tolerant pipelines, tackle advanced anti-bot challenges, and shape distributed architectures that support AI workflows and analytics at scale.

Qualifications

• At least 4 years of experience in high-volume web scraping, data engineering, or reverse engineering.
• Experience with anti-bot systems (Cloudflare, Akamai, PerimeterX) using TLS impersonation and proxy management.
• Proficiency in Python or Go.
• Experience with Playwright, Puppeteer, Scrapy, or Selenium.
• Experience with Temporal, Ray, Kafka, Redis.
• Strong SQL skills and experience with data lakes and data warehouses.

Responsibilities

• Design and manage web scraping infrastructure using residential proxy networks, TLS fingerprinting, browser automation, and session rotation.
• Build fault-tolerant, high-throughput scraping pipelines to collect data from major platforms.
• Develop distributed queues and workflow orchestration systems (Temporal, Ray, Kafka, Celery).
• Architect storage environments using Parquet, Iceberg, Snowflake, and Amazon S3.
• Implement monitoring, automated alerts, and recovery mechanisms for platform changes and proxy failures.
• Maintain reliable data ingestion at scale for daily processing of millions of records.
• Enable downstream AI and analytics workflows through robust data infrastructure.

Skills

Python

Go

Anti-bot systems

Distributed systems

SQL

Data ingestion

Browser automation

Web scraping

Temporal

Kafka

Redis

Playwright

Puppeteer

Scrapy

Selenium

S3

Tools

Temporal

Ray

Kafka

Celery

Parquet

Iceberg

Snowflake

Amazon S3

Playwright

Puppeteer

Scrapy

Selenium

Job description

Partner Company, acting on behalf of its partner, is seeking a Senior Data Infrastructure Engineer (Scraping & Scale) based in Australia. This fully remote role focuses on building high-performance web scraping and data ingestion systems to collect millions of profiles, posts, and video records daily.

You will design fault-tolerant pipelines, tackle advanced anti-bot challenges, and shape distributed architectures that support AI workflows and analytics at scale.