Full-Stack Web Scraping Engineer (Contractor)
Overview
We are a well-funded vertical software company with a fast-paced environment and a tight-knit team of four. Our product relies heavily on structured data collection of property listings and associated home services data, and we\'re bringing our scraping infrastructure completely in-house to maintain full control over our data pipeline.
Position Overview
We\'re seeking an experienced web scraping specialist to build and manage our data collection systems. This is a full-time contractor position that will report directly to the CTO while leading the data implementation team.
Responsibilities
- Take ownership of our daily data scraping operations that are core to our software product
- Refine and enhance our existing scraping workflows to improve reliability and efficiency
- Implement robust data standardization processes for collected information
- Automate the pipeline from scraping to JSON formatting and S3 storage
- Document data sources and collection methodologies
- Develop monitoring systems to ensure data integrity and scraping performance
- Train and lead team members on data collection best practices
- Troubleshoot and resolve scraping issues as they arise
Requirements
- 5+ years of professional experience with enterprise-level web scraping
- Strong expertise in multiple scraping frameworks and tools (e.g., Scrapy, Selenium, Puppeteer)
- Advanced knowledge of proxies, IP rotation, and anti-detection techniques
- Experience handling complex websites with anti-bot measures
- Proficiency in Python, JavaScript, and relevant programming languages for data collection
- Strong understanding of HTML, CSS, and JavaScript for site parsing
- Experience with data cleaning, normalization, and transformation
- Familiarity with cloud storage systems (particularly AWS S3)
- Ability to manage and prioritize multiple scraping tasks simultaneously
Nice to Have
- Experience working with headless browsers and CAPTCHA solving
- Knowledge of legal and ethical considerations in web scraping
- Experience with distributed scraping systems
- Background in data engineering or analytics
- Familiarity with containerization and orchestration tools
- Experience with real estate data or home services information
What We Offer
- Competitive compensation based on experience
- Flexible remote work arrangement
- Opportunity to lead a critical function in a growing company
- Collaborative environment with direct access to leadership
- Professional growth opportunities as our data operations expand
If you have a proven track record in building and maintaining enterprise-level scraping systems, we want to hear from you!
Pay: From $48 per month
Experience
- Web Scraping: 5 years (preferred)
- Python: 5 years (preferred)
- Language
- English (required)
Location
Remote
Expected Start Date
01/04/2025