Senior Data Infrastructure Engineer (Scraping & Scale) (Fully Remote)

Jobgether SRL; United States; Remote; USD 110,000 - 150,000; Full time

Job summary

Jobgether is seeking a seasoned data engineer to design and scale high-performance web scraping and data ingestion systems in the United States. You will work at the intersection of data engineering, automation, and AI infrastructure, leading the development of scalable collection platforms.

Ideal candidates have 4+ years in web scraping or data engineering, deep knowledge of anti-bot measures (Cloudflare, Akamai), and strong Python or Go skills with distributed task queues.

Qualifications

• 4+ years in web scraping or data engineering.
• Experience with anti-bot systems (Cloudflare, Akamai).
• Proficient in Python or Go and distributed task queues.

Responsibilities

• Lead development of scalable data collection systems and reliable ingestion.
• Build fault-tolerant scraping pipelines for Instagram, TikTok, YouTube, and X.
• Architect storage with Parquet, Iceberg, Snowflake, and S3.

Skills

Web scraping

Data engineering

Reverse engineering

Anti-bot awareness

Tools

Python

Go

Distributed task queues

Parquet

Iceberg

Snowflake

S3

Cloudflare

Akamai

Job description

• Design and scale high-performance web scraping and data ingestion systems.
• Work at the intersection of data engineering, automation, and AI infrastructure.

Key Responsibilities:

• Lead development of scalable data collection systems and reliable ingestion.
• Build fault-tolerant scraping pipelines for Instagram, TikTok, YouTube, and X.
• Architect storage with Parquet, Iceberg, Snowflake, and S3.

Requirements:

• 4+ years in web scraping, data engineering, or reverse engineering.
• Expertise in anti-bot systems like Cloudflare and Akamai.
• Advanced proficiency in Python or Go and distributed task queues.

Jobgether

The hiring partner company builds high-performance web scraping and data ingestion infrastructure for social media and AI workflows. It values reliability, performance, and continuous improvement, though its size and culture are not detailed.