Web Scrapping Developer

Intrics Solutions; Dadri; On-site; INR 40,000 - 70,000; Full time

Job summary

Intrics Solutions is seeking a Python-powered web scraping specialist to develop and maintain scripts that extract data from diverse websites according to business needs. You will work with BeautifulSoup, Requests, Selenium, and Scrapy, analyze HTML structures, handle API requests, and ensure data quality and reliability.

Your role involves fixing scraping issues, validating data, and coordinating with QA and team leads to meet deadlines.

Qualifications

• Basic to intermediate knowledge of Python programming.
• Understanding of HTML, CSS, XPath, and web page structures.
• Hands-on experience with BeautifulSoup and Requests.
• Basic knowledge of Selenium or Scrapy.
• Understanding of JSON, APIs, HTTP requests, and response codes.
• Familiarity with SQL and databases is an advantage.
• Basic knowledge of Git or version control is preferred.
• Good analytical, debugging, and problem-solving skills.
• Willingness to learn new tools, technologies, and website-specific scraping techniques.

Responsibilities

• Develop and maintain web scraping scripts using Python.
• Extract relevant data from websites based on business and project requirements.
• Work with libraries such as BeautifulSoup, Requests, Selenium, and Scrapy.
• Analyze website structures, HTML, CSS selectors, and XPath to identify relevant data elements.
• Handle pagination, cookies, sessions, headers, and basic API requests.
• Identify and resolve scraping errors, data inconsistencies, and website changes.
• Perform data validation and quality checks to ensure accuracy and completeness.
• Maintain existing scripts and troubleshoot issues in scheduled scraping jobs.
• Coordinate with team leads and QA teams to meet project requirements and delivery deadlines.
• Document scripts, issues, and process updates.

Skills

Python programming

HTML

CSS

XPath

Web page structures

BeautifulSoup

Requests

Selenium

Scrapy

JSON

APIs

HTTP requests

SQL

Git

Analytical skills

Debugging

Problem-solving

Tools

BeautifulSoup

Requests

Selenium

Scrapy

JSON

APIs

HTTP requests

Git

Job description

Role & responsibilities

• Develop and maintain web scraping scripts using Python.
• Extract relevant data from websites based on business and project requirements.
• Work with libraries such as BeautifulSoup, Requests, Selenium, and Scrapy.
• Analyze website structures, HTML, CSS selectors, and XPath to identify relevant data elements.
• Handle pagination, cookies, sessions, headers, and basic API requests.
• Identify and resolve scraping errors, data inconsistencies, and website changes.
• Perform data validation and quality checks to ensure accuracy and completeness.
• Maintain existing scripts and troubleshoot issues in scheduled scraping jobs.
• Coordinate with team leads and QA teams to meet project requirements and delivery deadlines.
• Document scripts, issues, and process updates.

Preferred candidate profile

• Basic to intermediate knowledge of Python programming.
• Understanding of HTML, CSS, XPath, and web page structures.
• Hands-on experience with BeautifulSoup and Requests.
• Basic knowledge of Selenium or Scrapy.
• Understanding of JSON, APIs, HTTP requests, and response codes.
• Familiarity with SQL and databases is an advantage.
• Basic knowledge of Git or version control is preferred.
• Good analytical, debugging, and problem-solving skills.
• Willingness to learn new tools, technologies, and website-specific scraping techniques.