Scalable Web Scraping Engineer

Source.ai; United States; Remote; USD 51,000 - 61,000; Full time

Job summary

Oxylabs, part of Source, is seeking a skilled scraping engineer to help build scalable scrapers and robust data pipelines for global-scale projects.

You will define resilient scraping strategies, unblock websites, and improve observability while developing back-end solutions for complex scraping and parsing challenges. Join a diverse team and contribute to enterprise-grade infrastructure.

Qualifications

• Experience with Python and building scalable scrapers.
• Understanding of CS fundamentals, data structures, and algorithms.
• Experience with Git version control.
• Knowledge of unblocking websites for scraping and various scraping techniques.

Responsibilities

• Develop scalable scrapers.
• Define resilient scraping strategies and unblock websites for scraping; improve system observability.
• Develop back-end solutions for scraping and parsing problems of various magnitudes.
• Maintain the current system and develop new features related to scraping and parsing.

Skills

Python

CS fundamentals

Git

Web scraping

Async programming

Tools

Docker

Kubernetes

Redis

MySQL

Elasticsearch

Grafana

Prometheus

SQLAlchemy

Pydantic

FastAPI

Job description

Oxylabs, part of Source, is seeking a skilled scraping engineer to help build scalable scrapers and robust data pipelines for global-scale projects.

You will define resilient scraping strategies, unblock websites, and improve observability while developing back-end solutions for complex scraping and parsing challenges. Join a diverse team and contribute to enterprise-grade infrastructure.