Remote Senior Data Infrastructure Engineer (Scraping & Scale)

Jobgether SRL; Ireland; Remote; EUR 120,000 - 160,000; Full time

Benefits offered by this job

Fully remote work arrangement

Full-time employment

Job summary

Jobgether SRL on behalf of a partner company seeks a Senior Data Infrastructure Engineer (Scraping & Scale) based in Ireland. Design and scale high-performance web scraping and data ingestion systems in a fully remote role. You will tackle anti-bot challenges, build resilient distributed pipelines, and enable large-scale analytical workflows for AI.

You will work with Temporal, Kafka, Snowflake, S3, and modern data lakes to deliver reliable data delivery and support downstream AI initiatives.

Qualifications

• At least 4 years of experience in high-volume web scraping or data engineering.
• Experience with anti-bot systems (Cloudflare, Akamai, PerimeterX) using TLS impersonation and proxies.
• Proficiency in Python or Go.
• Experience with Playwright, Puppeteer, Scrapy or Selenium.
• Knowledge of distributed queues: Temporal, Ray, Kafka, Redis.
• Strong SQL and data lake/warehouse experience.
• Ability to design scalable, fault-tolerant data ingestion systems.

Responsibilities

• Design and manage web scraping infrastructure using residential proxies and browser automation.
• Build high-throughput scraping pipelines for major platforms.
• Develop distributed queues and orchestration with Temporal, Ray, Kafka, Celery.
• Architect storage with Parquet, Iceberg, Snowflake, S3.
• Implement monitoring, alerts, and recovery for proxy failures and platform changes.
• Maintain reliable data ingestion at scale for daily millions of records.
• Support downstream AI and analytics workflows with robust data infrastructure.

Skills

Web scraping

Data engineering

Distributed systems

Python

Go

Browser automation

Playwright

Selenium

Kafka

Redis

TLS impersonation

Tools

Playwright

Puppeteer

Scrapy

Selenium

Temporal

Ray

Kafka

Celery

Parquet

Iceberg

Snowflake

Amazon S3

Job description

Jobgether SRL on behalf of a partner company seeks a Senior Data Infrastructure Engineer (Scraping & Scale) based in Ireland. Design and scale high-performance web scraping and data ingestion systems in a fully remote role. You will tackle anti-bot challenges, build resilient distributed pipelines, and enable large-scale analytical workflows for AI.

You will work with Temporal, Kafka, Snowflake, S3, and modern data lakes to deliver reliable data delivery and support downstream AI initiatives.