Remote Senior Data Infra Engineer - Scalable Scraping

Jobgether SRL; Norway; Remote; NOK 900,000 - 1,300,000; Full time

Benefits offered by this job

Fully remote

Full-time

Job summary

Jobgether SRL is seeking a Senior Data Infrastructure Engineer (Scraping & Scale) based in Norway. The role is fully remote and focuses on designing and scaling high-performance web scraping and data ingestion systems to collect millions of profiles, posts, and video records daily.

You will tackle anti-bot challenges, build resilient distributed pipelines, and enable downstream AI and analytics workflows within data lake and warehouse environments.

Qualifications

• 4+ years in high-volume web scraping or data engineering.
• Experience handling advanced anti-bot systems (Cloudflare/Akamai/PerimeterX).
• Strong Python or Go programming skills.
• Experience with browser automation frameworks.
• Knowledge of distributed queues and data storage technologies.
• Solid SQL skills and data lake/warehouse experience.

Responsibilities

• Design and manage web scraping infrastructure using proxies, TLS fingerprinting, browser automation, and rotation.
• Build high-throughput scraping pipelines for multiple social platforms.
• Develop distributed queues and orchestration with Temporal, Ray, Kafka, and Celery.
• Architect storage with Parquet, Iceberg, Snowflake, and S3.
• Implement monitoring, alerts, and recovery for platform changes and proxy failures.
• Maintain reliable ingestion for daily millions of records.
• Enable downstream AI and analytics workflows through robust data infrastructure.

Skills

Web scraping

Distributed systems

Python/Go

Browser automation

SQL & data lakes

Tools

Temporal

Ray

Kafka

Redis

Playwright

Puppeteer

Scrapy

Selenium

Celery

Job description

Jobgether SRL is seeking a Senior Data Infrastructure Engineer (Scraping & Scale) based in Norway. The role is fully remote and focuses on designing and scaling high-performance web scraping and data ingestion systems to collect millions of profiles, posts, and video records daily.

You will tackle anti-bot challenges, build resilient distributed pipelines, and enable downstream AI and analytics workflows within data lake and warehouse environments.