Remote Senior Python Data Scraping Engineer – AI

Mindrift - Data annotation; United States; Remote; USD 27,552,000 - 34,440,000; Part time

Job summary

Mindrift seeks a Senior Python Data Scraping Engineer for the Tendem project, a hybrid AI + human system. You will lead end-to-end data extraction across complex sites, using tools like Apify/OpenRouter, while ensuring accuracy and structured outputs.

This freelance, part-time remote role emphasizes precision, scalable workflows, and robust data quality. Ideal candidates have 5+ years in data engineering or scraping, strong Python skills (BeautifulSoup/Selenium), experience with cloud infra

Qualifications

• 5+ years of experience in data engineering, web scraping, automation, or software development.
• Strong scripting, automation, and data extraction workflow skills.
• Ability to solve non-trivial problems, work independently, and apply systematic data extraction methods.

Responsibilities

• Own end-to-end data extraction workflows across complex websites, ensuring coverage, accuracy, and structured delivery.
• Leverage tools and custom workflows to accelerate data collection, validation, and task execution.
• Ensure reliable extraction from dynamic and JS-rendered sources, adapting to site changes.
• Enforce data quality through validation checks, cross-source consistency, and formatting standards.
• Scale scraping operations for large datasets and monitor failures against site changes.

Skills

Python web scraping

BeautifulSoup

Selenium

dynamic content

APIs via proxies

data cleaning

data normalization

data validation

anti-bot

AWS

Docker

LangChain

OpenRouter

LLM frameworks

GitHub

English B2+

Education

Bachelor’s or Master’s Degree in Engineering/CS/Applied Mathematics

Tools

BeautifulSoup

Selenium

LangChain

OpenRouter

Docker

AWS

GitHub

Job description

Mindrift seeks a Senior Python Data Scraping Engineer for the Tendem project, a hybrid AI + human system. You will lead end-to-end data extraction across complex sites, using tools like Apify/OpenRouter, while ensuring accuracy and structured outputs.

This freelance, part-time remote role emphasizes precision, scalable workflows, and robust data quality. Ideal candidates have 5+ years in data engineering or scraping, strong Python skills (BeautifulSoup/Selenium), experience with cloud infra