Experienced Web Scraping Engineer - Python (Remote)

Source.ai; Województwo mazowieckie; On-site; PLN 222,000 - 266,000; Full time

Job summary

Oxylabs is a leading web data infrastructure provider operating at global scale. In this role you will develop scalable scrapers, define resilient scraping strategies, and build back-end solutions for scraping and parsing across large-scale datasets.

You will maintain systems and add features to enhance scraping capabilities. We value ownership and learning; you will be supported by a diverse team and have opportunities to work with cutting-edge tools and open-source technologies.

Qualifications

• Experience with Python and scraping tech.
• Strong CS fundamentals including data structures and algorithms.
• Version control with Git.
• Knowledge of unblocking websites for scraping.
• Experience with various scraping techniques and open-source tools.

Responsibilities

• Develop scalable scrapers and back-end solutions for scraping and parsing.
• Define resilient scraping strategies and unblock websites for scraping.
• Improve observability in the system.
• Maintain the current system and develop new features related to scraping and parsing.

Skills

Python

CS fundamentals

Git

Web scraping

Browser automation

Asynchronous programming

TLS/SSL knowledge

Tools

Docker

Kubernetes

Redis

MySQL

Elasticsearch

Kibana

Grafana

Prometheus

Job description

Oxylabs is an enterprise-grade web data infrastructure, purpose-built for organizations operating at global scale. It helped define what public web data access could look like and still sets the standard others build against.

That same ambition runs through the culture: engineers take ownership of problems no one else has solved and push each other to raise the bar.

A word from the team:

We run one of the most advanced and largest scraping and parsing products in the world. We serve thousands of requests per second with a very high success rate. Our scrapers and parsers are used by leading e-commerce, market intelligence, and AI industry players making the work challenging and truly global. The team is a blend of different interesting personalities from different walks of life and nationalities. Here you can find people who are experts in gaming, playing guitar, riding bicycles, and other areas. We, as a team, will support you in learning how to build your own scrapers and will share all the tips, tricks and hacks we know to ensure that you are onboard in no time.

In this role, you’ll:

• Develop scalable scrapers
• Define resilient scraping strategies, unblock websites for scrapingImprove observability in the system
• Develop back-end solutions for scraping & parsing problems of various magnitudes
• Maintain the current system and develop new features related to scraping & parsing

Your skills & experience:

• Experience working with Python
• Understanding of computer science, including data structures, algorithms, computability and complexity
• Version Control skills using Git
• Knowledge on how to unblock websites for scraping
• Is able to use different scraping techniques & open-source tools to build scrapers
• Is comfortable with using Dev Tools
• Network (TLS/SSL) knowledge
• Worked with browser automations
• Knows their way around asynchronous programming

Nice to have:

• Web development knowledge
• Knows how to use CSS Selectors / XPaths for parsing
• Experience working with Go & C++
• Worked on browser source code
• Knowledge of any front-end framework
• Experience working with Pydantic, FastAPI, SQLAlchemy
• Has experience working with Redis, MySQL, Docker, Kubernetes, Elasticsearch, Kibana and monitoring tools like Grafana, Prometheus
• Experience with machine learning that is scraping domain-specific
• Has experience in building scalable systems

Salary:

• Gross salary: from 5000 eur/month. Keep in mind that we are open to discussing a different salary based on your skills and experience

Oxylabs is part of Source, a data ecosystem for a world built on intelligence. One of five independent technology companies connected by shared standards, expertise, and a belief in responsible access to web data.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.