Scraping Tech Lead Engineer (Web Scraping & Anti-Bot)

Infatica.Io; Remote; ZAR 900 000 - 1 200 000; Full time

Job summary

A global technology firm in Gauteng is seeking a Scraping Tech Lead Engineer to design and lead the development of a high-load web scraping platform. This role involves architecture design, research on anti-bot techniques, and leading a backend team. Candidates should have extensive experience in web scraping, backend engineering, and appropriate tools like Kubernetes and Prometheus. This position offers a competitive salary and remote work flexibility, allowing significant influence on the product's development.

Benefits

Competitive compensation package

Remote work flexibility

Budget for conferences and courses

Qualifications

• 5+ years of engineering experience in high-load backend/data platforms.
• Extensive scraping experience is mandatory.
• Fluent in English for documentation and team communication.
• Experience with async Python and Go is preferred.

Responsibilities

• Design architecture for high-parallel crawlers and API.
• Lead development processes for the backend team.
• Research and implement anti-bot solutions.
• Set up CI/CD processes and monitor deployments.

Skills

High-load backend engineering

Web scraping

HTTP, TLS, HTTP/2

Async Python

Go

Kubernetes

Prometheus

DevOps tools

Tools

Redis

Docker

Grafana

Loki

Job description

Scraping Tech Lead Engineer (Web scraping & Anti-bot)

Infatica.io is a global provider of proxy and DaaS solutions.

We're building our own high-load web scraping platform : an API where you can submit any URL and receive clean HTML or structured JSON output, without worrying about proxies, CAPTCHAs, or anti-bot systems.

The platform supports both cloud deployment and on-premises deployment in client infrastructure (on-prem, Kubernetes).

Key Responsibilities

Architecture Design :

• Data-plane : high-parallel crawler, session management, anti-bot bypass, CAPTCHA solver
• Control-plane : API, authorization, billing, quotas, monitoring
• Infrastructure : microservices, Docker / K8s, CI / CD, observability

Development Leadership :

• Team building : participate in hiring backend, browser, DevOps, and QA engineers
• Establish development and release processes
• Code quality control and technical decision oversight

Anti-Bot Research & Bypass :

• Browser and network-level fingerprinting (JA3, TLS, CDP)
• Protection bypass : Cloudflare, Akamai, Qrator, ServicePipe, Variti, and others

DevOps / SRE :

• CI / CD setup, Helm charts, scaling configuration
• Monitoring organization (Prometheus, Grafana, Loki) and SLA processes

Requirements

5+ years of engineering experience in high-load backend / data platforms

Extensive scraping experience (mandatory)

Excellent understanding of HTTP, TLS, HTTP / 2, proxies, and browser stack (CDP, WebRTC)

Experience with async Python and Go (ideally both), Redis, queues, message brokers

Advanced level DevOps tools : Kubernetes, Helm, Prometheus, CI / CD

Experience analyzing and bypassing bot protection systems – strong plus

Fluent English (all documentation and part of the team are English-speaking)

Decision-making skills, mentoring abilities, and accountability for results

Benefits

Participate in developing the company's key product from scratch

High autonomy and influence on architecture and team decisions

Budget for conferences, courses, and work tools

Competitive compensation package

Remote work flexibility

Apply now and become the technical leader who shapes our platform's future!

#J Ljbffr