Remote AI Web Scraping Engineer: Stealth Data Pipelines

Workana LLC

United States

Remote

USD 140.000 - 210.000

Vollzeit

14 Tage+

Eine vollständige Bewerbung in einer Minute

Maßgeschneiderter Lebenslauf und Anschreiben – versandbereit.

Schaffe es an den ATS-Filtern vorbei

Benefits dieser Stelle

Compensation in USD
Fully remote work
Career growth on an international公司

Zusammenfassung

Pinwheel API is seeking an AI Web Scraping Engineer (Stealth Data Acquisition) to design and maintain resilient, undetectable data-collection pipelines at scale. You will move beyond basic requests to defeat modern anti-bot defenses while ensuring compliant, structured data delivery.

You will integrate AI/LLM-based parsing, build self-healing scrapers, and collaborate with data engineering to produce clean datasets for downstream systems in a fast-moving, evolving environment.

Qualifikationen

  • 4+ years building production web scraping systems at scale.
  • Deep hands-on experience with headless browser automation (Playwright, Puppeteer, Selenium).
  • Practical experience defeating anti-automation measures (Cloudflare, DataDome, PerimeterX, Akamai, Kasada).
  • Strong understanding of TLS/HTTP fingerprinting and browser fingerprint spoofing.
  • Experience with residential/mobile/datacenter proxy rotation and session management at scale.
  • Proficiency in Python and/or Node.js/TypeScript for scraping and orchestration.
  • Experience using LLMs/AI models for content extraction, parsing, and classification.
  • Familiarity with CAPTCHA-solving approaches and detection-avoidance strategies.
  • Must be based in and authorized to work in the United States.
  • Must pass a standard background check prior to start.
  • Comfortable working in a fast-moving, iterative environment where sites frequently change defenses.

Aufgaben

  • Stealth Scraping Pipeline Development – Design, build, and maintain stealth-based scraping pipelines that evade bot detection and rate-limiting defenses.
  • Anti-Bot Reverse Engineering – Reverse-engineer anti-automation measures and build countermeasures.
  • Distributed Infrastructure Architecture – Architect distributed scraping infra using rotating proxies and headless browser farms.
  • AI-Powered Parsing Integration – Integrate AI/LLM-based parsing for dynamic layouts without brittle selectors.
  • Self-Healing Scraper Development – Build scrapers that adapt to layout/DOM changes automatically.
  • Monitoring & Iteration – Track scraper health, success rates, and detection signals, iterating as defenses update.
  • Compliance & Legal Guardrails – Ensure pipelines align with ToS, robots.txt, and rate limiting as directed by legal/compliance.
  • Data Pipeline Collaboration – Deliver clean, deduplicated datasets to downstream systems.

Kenntnisse

Web scraping infra
Headless browser automation
Stealth scraping
Anti-bot bypass
TLS fingerprinting
Proxy rotation
Session management
Python
TypeScript/Node.js
LLM parsing
CAPTCHA solving
US work authorization

Tools

Playwright
Puppeteer
Selenium
LLM parsing tooling

Jobbeschreibung

About Our Client

Pinwheel API is a leading data infrastructure company enabling seamless access to employment and income verification data through secure APIs. The company powers payroll connectivity for fintechs, lenders, HR platforms, and financial services across the US market.

Role Overview

As an AI Web Scraping Engineer (Stealth Data Acquisition), you will be responsible for designing, building, and maintaining resilient, undetectable data collection systems at scale. This is not a "requests + BeautifulSoup" role — we need someone who has operated against modern anti-bot infrastructure (Cloudflare, PerimeterX, DataDome, Akamai, Kasada) and knows how to stay under the radar while extracting structured data reliably and continuously. You will integrate AI/LLM-based parsing to handle dynamic, unstructured, or frequently-changing page layouts and build self-healing scrapers that detect layout/DOM changes and adapt automatically.

Responsibilities

  • Stealth Scraping Pipeline Development – Design, build, and maintain stealth-based scraping pipelines that evade bot detection, fingerprinting, and rate-limiting defenses.
  • Anti-Bot Reverse Engineering – Reverse-engineer target sites' anti-automation measures (TLS/JA3 fingerprinting, canvas/WebGL fingerprinting, behavioral detection) and build countermeasures.
  • Distributed Infrastructure Architecture – Architect distributed scraping infrastructure using rotating residential/mobile proxies, headless browser farms, and session/cookie management at scale.
  • AI-Powered Parsing Integration – Integrate AI/LLM-based parsing to handle dynamic, unstructured, or frequently-changing page layouts without brittle selectors.
  • Self-Healing Scraper Development – Build self-healing scrapers that detect layout/DOM changes and adapt automatically.
  • Monitoring & Iteration – Monitor scraper health, success rates, and detection signals; rapidly iterate when sites patch defenses.
  • Compliance & Legal Guardrails – Ensure data pipelines are compliant with client-defined legal and ethical guardrails (ToS review, robots.txt handling, rate limiting) as directed by Pinwheel's legal/compliance team.
  • Data Pipeline Collaboration – Collaborate with data engineering to deliver clean, structured, deduplicated datasets into downstream systems.

Requirements

  • 4+ years building production web scraping systems at scale (not just scripts — real infrastructure).
  • Deep hands-on experience with headless browser automation (Playwright, Puppeteer, Selenium) including stealth plugins/patches.
  • Practical experience defeating or working around Cloudflare, DataDome, PerimeterX, Akamai Bot Manager, Kasada, or similar.
  • Strong understanding of TLS fingerprinting, HTTP/2 fingerprinting, canvas/WebGL fingerprinting, and browser fingerprint spoofing.
  • Experience with residential/mobile/datacenter proxy rotation and session management at scale.
  • Proficiency in Python and/or Node.js/TypeScript for scraping and orchestration.
  • Experience using LLMs/AI models for content extraction, parsing, and classification on messy/unstructured HTML.
  • Familiarity with CAPTCHA-solving approaches and detection-avoidance strategies.
  • Must be based in and authorized to work in the United States.
  • Must pass a standard background check prior to start.
  • Comfortable working in a fast-moving, iterative environment where target sites change defenses frequently.

Nice to Have

  • Experience with mobile app API reverse engineering (in addition to web).
  • Background in adversarial ML or anti-bot research (from either side).
  • Experience with distributed task queues (Celery, Kafka, SQS) for scraping orchestration.
  • Prior work in a data-as-a-service, alt-data, or web intelligence company.

Benefits

  • Compensation in USD
  • Fully remote work
  • Career growth on an international company
Hol dir deinen kostenlosen, vertraulichen Lebenslauf-Check.

oder ziehe deine Datei hierhin.

Similar jobs

Ähnliche Jobs, die dir auch gefallen könnten

AI Web Scraping Engineer (Stealth Data Acquisition)
AI Web Scraping Engineer (Stealth Data Acquisition)

Workana • USA

Remote
USD 140.000 - 200.000
Compensation in USD
Fully remote
Career growth
Experienced Web Scraping Engineer (Remote)
Experienced Web Scraping Engineer (Remote)

Source.ai • USA

Remote
USD 51.000 - 61.000
Experienced Web Scraping Engineer (Remote)
Experienced Web Scraping Engineer (Remote)

Source • USA

Remote
USD 46.000 - 55.000
Learning options
Mentorship
Team events
+1
Scraping Engineer
Scraping Engineer

String • USA

Remote
USD 60.000 - 90.000
Remote Web Scraping Engineer | Scalable Data Pipelines
Remote Web Scraping Engineer | Scalable Data Pipelines

Wynd Labs • USA

Remote
USD 110.000 - 170.000
Opportunity
Culture
Compensation
Backend Infrastructure Engineer, Anti-Bot
Backend Infrastructure Engineer, Anti-Bot

True Talent Solutions Partners • San Francisco (CA)

Hybrid
USD 235.000 - 260.000
Web Scraping Engineer (Remote)
Web Scraping Engineer (Remote)

Source.ai • USA

Remote
USD 60.000 - 94.000
Health insurance
Psychotherapy
Wellness app
+6
Web Scraping Specialist
Web Scraping Specialist

Wynd Labs • USA

Remote
USD 110.000 - 170.000
Opportunity
Culture
Compensation
Data Engineer – WFH
Data Engineer – WFH

Torentify • USA

Remote
USD 110.000 - 170.000
Equity
Remote-friendly
Web Scraper
Web Scraper

Executive Staff Recruiters / ESR Healthcare • USA

Remote
USD 28.000 - 69.000