Python Web Scraping Engineer

Innvonix Tech Solutions Pvt Ltd; Ahmedabad District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Innvvonix Tech Solutions Pvt Ltd is seeking engineers to design, build, and operate scalable web scraping pipelines. You will handle dynamic sites, anti-bot challenges, and CAPTCHAs while deploying robust infrastructure with Docker and Kubernetes.

The role includes API work with FastAPI or Django and async tasks via Celery, plus observability with Grafana/Prometheus. The ideal candidate has strong Python skills, Selenium/UC experience, and a passion for distributed data extraction systems in a

Qualifications

• Strong Python expertise with focus on web scraping and data extraction.
• Experience building production-grade scraping systems.
• Familiarity with anti-bot measures and CAPTCHA handling.
• Experience with Docker and Kubernetes for distributed workloads.
• Creating APIs using FastAPI or Django.
• Proficiency in Celery for async task processing.
• Hands-on with Selenium and UC for browser automation.
• Monitoring via Grafana/Prometheus; Git and Linux proficiency.

Responsibilities

• Design, build, and scale web scraping pipelines.
• Maintain production scraping systems and data extraction tools.
• Handle dynamic sites, anti-bot protections, CAPTCHAs, and rate limits.
• Develop distributed scraping infrastructure with Docker and Kubernetes.
• Create APIs with FastAPI or Django for data access.
• Implement asynchronous workflows using Celery.
• Monitor production systems with Grafana/Prometheus; troubleshoot issues.
• Collaborate with teams on scalable data collection platforms.

Skills

Python

Web scraping

Docker

Kubernetes

FastAPI

Django

Celery

Selenium

UC (Undetected ChromeDriver)

Requests

Grafana

Prometheus

Git

Linux

Distributed systems

AI/Agentic AI

Tools

Docker

Kubernetes

Selenium

UC

Requests

FastAPI

Django

Celery

Grafana

Prometheus

Git

Linux

Job description

At Innvonix Tech Solutions, We are looking for people who have the desire to grow to the highest level in their professional careers. If you have an indestructible drive within you to showcase your skillset and to level up in your career, you stand a good chance to join us. We value skills over experience and attitude over knowledge. If you think, you have what it takes to survive in the ever-changing world of technology, feel free to get in touch with our HR team. We the best software company to work in Ahmedabad. We are actively looking for passionate candidates in the following job profiles.

Roles and Responsibility

• 4+ years of professional experience in Python web scraping and automation.
• Strong hands-on experience building and maintaining production-grade web scraping systems
• Design, develop, and maintain scalable web scraping and data extraction pipelines.
• Build robust scrapers capable of handling dynamic websites, anti-bot protections, CAPTCHAs, and rate limiting.
• Develop high-performance, distributed scraping infrastructure using Docker and Kubernetes.
• Optimize scraper performance, reliability, scalability, and fault tolerance for large-scale deployments.
• Build and maintain APIs using Python web frameworks such as FastAPI or Django.
• Develop asynchronous and distributed task-processing pipelines using Celery.
• Monitor and troubleshoot production systems using observability tools such as Grafana and Prometheus.
• Collaborate with engineering teams to build resilient, maintainable, and scalable data collection platforms.
• Explore and contribute to AI/ML and Agentic AI initiatives for intelligent web automation and data extraction.
• Strong proficiency in Python with a solid understanding of:
  • Object-Oriented Programming (OOP)
  • Python best practices
  • Clean code principles
  • Exception handling and debugging
• Extensive hands-on experience with:
  • Selenium
  • Undetected ChromeDriver (UC)
  • Requests
  • Browser automation
  • CAPTCHA-solving techniques
• Strong understanding of modern anti-bot mechanisms, rate limiting, and anti-detection techniques.
• Experience working with Docker and containerized applications.
• Hands-on experience deploying and managing applications on Kubernetes.
• Strong experience with Celery and distributed task processing.
• Experience with at least one Python web framework:
  • FastAPI, Django
• Familiarity with monitoring and observability tools such as:
  • Grafana
  • Prometheus
• Experience working with Git and Linux environments.
• Strong problem-solving and debugging skills.
• Strong knowledge of Kubernetes administration and orchestration.
• Experience with social media scraping, automation, or bot development.
• Experience with AI-assisted or Agentic AI web scraping systems.
• Experience with message queues, distributed workers, and scalable backend architectures.
• Experience working with cloud platforms and large-scale data processing systems.
• Opportunity to work on large-scale cyber intelligence products.
• Exposure to cutting-edge AI and Agentic AI technologies.
• Challenging engineering problems involving distributed systems, web automation, and large-scale data extraction.
• Collaborative engineering culture with opportunities for continuous learning, experimentation, and innovation.
• Opportunity to work on technically challenging projects and contribute to next-generation intelligent web automation solutions.