Data Engineer - Web Scraping

Cyble; United States; On-site; USD 60,000 - 90,000; Full time

Job summary

Cyble is looking for a Data Collection and Quality Engineer to manage web scraping configurations and ensure data accuracy. Key responsibilities include monitoring scraping activities, coordinating with stakeholders, and preparing reports on scraping tasks.

The ideal candidate should possess at least 3 years of experience and hands-on skills with web scraping tools and libraries like Python, Scrapy, and Selenium. Strong communication and teamwork abilities are essential for this role.

Qualifications

• Minimum 3 years of experience as a data collection and quality engineer.
• Hands-on experience with third-party web scraping tools.
• Fluent in English (spoken and written).

Responsibilities

• Create and manage website scraping configurations.
• Monitor scraping configurations for errors.
• Prepare periodic scraping activity reports.

Skills

Web scraping tools

Python

Selenium

Scrapy

Data quality engineering

Verbal communication

Team collaboration

Job description

Responsibilities

• Create and manage website scraping configurations on web scraping tools.
• Monitor scraping configurations for potential errors and blockages.
• Monitor data being scraped to identify potential issues and blockages.
• Coordinate with stakeholders to understand scraping task requirements and report issues.
• Prepare and share periodic scraping activity reports with stakeholders.

Technical Skillset

• Minimum 3 years of experience working as a data collection and quality engineer.
• Hands-on experience with third-party web scraping tools.
• Hands-on experience with open-source web scraping libraries (e.g., Scrapy, Selenium).
• Familiar with programming languages like Python and Go.
• Fluent with web technology concepts like HTML, DOM, CSS, XPATH.
• Familiar with the usage of regular expressions for data selection and cleaning.
• Familiar with Windows and Linux operating systems and general networking concepts.

Soft Skills

• Proficiency in English (spoken and written) and good verbal communication skills.
• Strong written communication skills and the ability to work well in a team with an ownership mindset.