Data Engineer: Web Scraping & AI Feature Analysis

Questah Staffing

United States

Remote

USD 90,000 - 150,000

Full time

8 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

BestWaitlistApps.com is seeking an engineer to maintain the scraping and analysis pipeline that powers feature ratings across vendors and write results into the comparison data. The role involves keeping the scraper up to date, running analysis on scraped text and screenshots, and generating logs and gap reports for held-out features, with a focus on reliability and traceability.

You will collaborate with product and data teams, review discrepancies between AI analysis and stored feature marks,

Qualifications

  • Experience with PHP and Laravel console commands.
  • Experience with Node and a headless browser for scraping.
  • Comfortable with SQL and schema migrations.

Responsibilities

  • Maintain the scraper that renders each vendor's pages and takes screenshots.
  • Maintain the commands that analyze the scraped text and the screenshots and write the detected features to the comparison tables.
  • Compare the AI analysis with the stored feature marks and report where they disagree.
  • Maintain the gap report that lists features that tracked products have and the catalog lacks.
  • Fix failed scrapes and keep a log of what each scrape stored.

Tools

PHP
Laravel
Node.js
Headless browser
SQL
Schema migrations

Job description

BestWaitlistApps.com is seeking an engineer to maintain the scraping and analysis pipeline that powers feature ratings across vendors and write results into the comparison data. The role involves keeping the scraper up to date, running analysis on scraped text and screenshots, and generating logs and gap reports for held-out features, with a focus on reliability and traceability.

You will collaborate with product and data teams, review discrepancies between AI analysis and stored feature marks,

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer, Scrape and Analysis Pipeline
Data Engineer, Scrape and Analysis Pipeline

Questah Staffing • United States

Remote
USD 90,000 - 150,000
Product Engineer, Web Scraping & AI APIs
Product Engineer, Web Scraping & AI APIs

True Talent Solutions Partners • San Francisco (CA)

On-site
USD 235,000 - 260,000
Visa sponsorship
Data Engineer - Web Scraping
Data Engineer - Web Scraping

Alternative Path Private Limited • United States

Remote
USD 90,000 - 130,000
Scraping Engineer
Scraping Engineer

String • United States

Remote
USD 60,000 - 90,000
High-Impact Web Scraping Engineer
High-Impact Web Scraping Engineer

String • United States

Remote
USD 60,000 - 90,000
AI-Powered Web Scraping & Data Extraction Specialist
AI-Powered Web Scraping & Data Extraction Specialist

OpenTrain AI, Inc. • United States

Remote
USD 23,000 - 32,000
Web Scraping Engineer - Scale & AI-Powered Crawlers
Web Scraping Engineer - Scale & AI-Powered Crawlers

Source.ai • United States

Remote
USD 60,000 - 94,000
Health insurance
Psychotherapy
Wellness app
+6
AI-Driven Web Data Crawling Engineer
AI-Driven Web Data Crawling Engineer

Rubick.AI • Oregon

Hybrid
USD 90,000 - 150,000
AI Web Scraping Engineer (Stealth Data Acquisition)
AI Web Scraping Engineer (Stealth Data Acquisition)

Workana • United States

Remote
USD 140,000 - 200,000
Compensation in USD
Fully remote
Career growth
Web Scraping Specialist
Web Scraping Specialist

Wynd Labs • United States

Remote
USD 110,000 - 170,000
Opportunity
Culture
Compensation