Remote Data Engineer: Scalable Web Data Pipelines

Wynd Labs; United States; On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Fully remote team

Competitive salary

Equity package

Job summary

Wynd Labs is seeking a Data Engineer to design, optimize, and operate large-scale data pipelines and infrastructure. You’ll work across data collection, processing, transformation, validation, and delivery with a focus on scalability, reliability, and performance.

This is a hands-on role involving distributed systems, large datasets, and web scraping infrastructure. The team collaborates across EST business hours and values fast, clean delivery, and robust data workflows.

Qualifications

• Bachelor’s degree or equivalent work experience.
• Python (advanced)—strong grasp of async programming, multiprocessing, and writing production-grade code for long-running data jobs.
• Web scraping at scale—hands-on experience with high-volume scraping (proxies, rate limiting, anti-bot evasion).
• Distributed data pipelines—experience designing and operating pipelines across many workers/servers using task queues (Celery, Kafka, RabbitMQ, or similar).
• Data warehousing—practical experience with columnar/analytical warehouses; Databend, ClickHouse, or BigQuery strongly preferred; comfortable with complex analytical queries, partitioning strategies, cost-aware querying on cloud warehouses
• Docker & Kubernetes—containerizing workloads, writing Helm charts/manifests, managing deployments, autoscaling scraping/processing workloads
• Linux & bare-metal ops—comfortable managing services on Linux servers, debugging performance issues (disk I/O, network, memory) without managed-cloud abstractions
• CI/CD for data workflows (GitHub Actions, ArgoCD)
• Writing Scalable API

Responsibilities

• Maintain, optimize, and troubleshoot database queries and related data systems to support efficient data access, processing, and reliability.
• Assist in creating, maintaining, and improving data pipelines used to collect, process, transform, validate, and deliver large-scale datasets.
• Support web scraping and data collection initiatives, including developing, testing, and maintaining scripts or tools used to gather publicly available data in accordance with Company requirements.
• Monitor and troubleshoot data pipeline issues, identify data quality concerns, and help implement timely fixes to maintain data accuracy and operational continuity.
• Document engineering work, including database queries, pipeline processes, scraping workflows, technical decisions, issues encountered, and resolutions implemented.
• Participate in research and development projects to improve the Company’s data products and workflows.

Skills

Python

Async programming

Web scraping at scale

Distributed data pipelines

Data warehousing

CI/CD for data workflows

Linux

Docker & Kubernetes

Scalable API

Education

Bachelor’s degree or equivalent

Tools

Celery

Kafka

RabbitMQ

Databend

ClickHouse

BigQuery

Docker

Kubernetes

Helm

GitHub Actions

ArgoCD

Job description

Wynd Labs is seeking a Data Engineer to design, optimize, and operate large-scale data pipelines and infrastructure. You’ll work across data collection, processing, transformation, validation, and delivery with a focus on scalability, reliability, and performance.

This is a hands-on role involving distributed systems, large datasets, and web scraping infrastructure. The team collaborates across EST business hours and values fast, clean delivery, and robust data workflows.