Advanced Python Web Scraping Engineer — Scalable Scrapers

Source.ai; Województwo mazowieckie; On-site; PLN 222,000 - 266,000; Full time

Job summary

Oxylabs is a leading web data infrastructure provider operating at global scale. In this role you will develop scalable scrapers, define resilient scraping strategies, and build back-end solutions for scraping and parsing across large-scale datasets.

You will maintain systems and add features to enhance scraping capabilities. We value ownership and learning; you will be supported by a diverse team and have opportunities to work with cutting-edge tools and open-source technologies.

Qualifications

• Experience with Python and scraping tech.
• Strong CS fundamentals including data structures and algorithms.
• Version control with Git.
• Knowledge of unblocking websites for scraping.
• Experience with various scraping techniques and open-source tools.

Responsibilities

• Develop scalable scrapers and back-end solutions for scraping and parsing.
• Define resilient scraping strategies and unblock websites for scraping.
• Improve observability in the system.
• Maintain the current system and develop new features related to scraping and parsing.

Skills

Python

CS fundamentals

Git

Web scraping

Browser automation

Asynchronous programming

TLS/SSL knowledge

Tools

Docker

Kubernetes

Redis

MySQL

Elasticsearch

Kibana

Grafana

Prometheus

Job description

Oxylabs is a leading web data infrastructure provider operating at global scale. In this role you will develop scalable scrapers, define resilient scraping strategies, and build back-end solutions for scraping and parsing across large-scale datasets.

You will maintain systems and add features to enhance scraping capabilities. We value ownership and learning; you will be supported by a diverse team and have opportunities to work with cutting-edge tools and open-source technologies.