Senior Web Data Acquisition Engineer (Resilient Crawling)

ZEN; Warszawa; On-site; PLN 180,000 - 260,000; Full time

Benefits offered by this job

High autonomy and ownership

Supportive team collaboration

Hands-on experience with production AI

Job summary

A financial technology company in Warsaw seeks a Senior Web Data Acquisition Engineer. You will design and run high-throughput data pipelines, ensuring data quality and resilience against access controls. Ideal candidates have experience with Python or Java/Kotlin, and a solid understanding of data validation and normalization. Join a dynamic team leveraging AI in a fast-paced environment.

Qualifications

• Practical experience in handling rate limits and challenges.
• Solid grounding in data quality aspects like normalization and validation.

Responsibilities

• Design and build high-throughput web data pipelines.
• Make smart decisions between rendering and fetching for data accuracy.
• Create robust extractors and parsers with clear schemas.
• Engineer resilience against dynamic access controls.
• Own observability metrics including success rates and latency.
• Collaborate on integrating LLM-assisted maintenance.

Skills

Handling rate limits/challenges

Data quality normalization

Data validation

Data deduplication

Schema/versioning

Schema versioning

Observability

LLM maintenance

AI collaboration

Tools

Python

Java

Kotlin

Job description

Overview

Senior Web Data Acquisition Engineer (Resilient Crawling) – TL;DR Checklist:

• Scrape JS & non-JS (headless, VMs)
• Emulate human like behavior (proxies, rate, CAPTCHA, fingerprints)
• Parsers resilient to layout changes
• Monitor & alert on blocks/errors; retry/backoff
• PythonorJava/Kotlin

All checked? In that case, send us your CV ! And if you want more details, read on.

About ZEN.COM

ZEN.COM is a smart financial app designed for your everyday life – at home and on the go. We make payments, online shopping, and personal finance management fast, secure, and effortless. With ZEN, you can enjoy cashback on purchases, full control over your spending, and peace of mind thanks to purchase protection. A built-in multi-currency account lets you spend abroad or shop internationally with great exchange rates and no hidden fees.

The Opportunity

We operate with a startup mindset: small, senior, fast-moving. The team is energetic, collaborative, and pragmatic. We useAI at multiple layers(change detection, parser generation, anomaly detection, agentic ops)—you’ll both use and shape these production‑ready applications and learn a lot along the way.

What You’ll Do

• Design, build, and run high‑throughput web data pipelines across diverse, modern web stacks (rendered and non‑rendered).
• Make smartrender vs. fetchdecisions to balance accuracy, latency, and cost.
• Createrobust extractors/parsers(HTML/DOM/XPath/JSON) with auto‑healing patterns and clear schemas.
• Engineer resilience againstdynamic access controls(rate limits, traffic shaping, session/identity rotation) while following legal/ethical guidelines.
• Ownobservability: success rate, freshness, latency, cost per successful unit.
• Collaborate with AI/Platform to integrateLLM‑assisted maintenance, anomaly detection, and triage bots.

What You Bring

• Practical experience handlingrate limits/challengesand building systems that degrade gracefully.
• Solid grounding indata quality: normalization, validation, deduplication, and schema/versioning.

Why Join Us

• Impact:Your work becomes the backbone of pricing & availability intelligence used across products and markets.
• Learning:Hands‑on withproduction AI(LLMs, embeddings, agents) and modern data engineering.
• Team:Talented, supportive peers; high autonomy and ownership; clear problem statements.
• Pace & Pragmatism:We ship, measure, and improve.

Nice to Have

• Multi‑region traffic management and cost optimization.