Senior Data Engineer

Rachel Paul Recruiting; New York (NY); Hybrid; USD 120,000 - 150,000; Full time

Job summary

A marketing technology startup in New York, NY, is seeking a Senior Data Acquisition Engineer to lead the development of a scalable web data acquisition platform. This hybrid role focuses on enhancing data ingestion processes and system reliability. Candidates should possess over 5 years of experience in building data systems, and be proficient in languages such as Typescript or Python. This position presents an exciting opportunity to contribute to AI-driven technologies and their real-world applications.

Qualifications

• 5+ years of experience in building and operating production-grade backend or data systems.
• Hands-on experience in web data acquisition or scraping.
• Experience with environments that encounter frequent change and partial failure.

Responsibilities

• Build a scalable web data acquisition platform for multiple teams.
• Make architecture decisions balancing cost, reliability, and performance.
• Own system reliability and enhance resilience and maintainability.

Skills

Web data acquisition

Typescript

Python

SQL

NoSQL databases

CSS selectors

XPath

Networking fundamentals

Job description

My client is a Startup in the Marketing Technology AI space, with a platform that will transform how brands connect to consumers in the AI space. This will enable these brands to thrive in the new paradigm. Their platform helps brands engage consumers on this new AI channel, with powerful enterprise tools to manage how their brand is represented in AI.

About the Role

As a Senior Data Acquisition Engineer, you will own and evolve the systems responsible for large-scale web data collection. You will design and maintain production-grade scraping and ingestion infrastructure that enables multiple teams to reliably add and operate data sources.

This role sits within the Data Engineering team and focuses on building scalable, observable, and resilient systems that handle production data. You will work closely with data engineers and product teams to level up data pipelines and ensure data acquisition scales with the business.

This role presents an exciting opportunity to shape the future of AI‑driven technologies and make meaningful contributions to real‑world applications.

This role is based in our NYC office and follows a hybrid working policy.

What You’ll Be Doing

• Build a scalable web data acquisition platform used across teams. Enable not just your team but other teams to ingest data more safely and efficiently.
• Make tradeoffs between costs, reliability and performance in key architecture decisions
• Create shared abstractions and tooling that make it easy to add, maintain, and operate scrapers in production.
• Own system reliability, including smart retries, backoff strategies, error handling, and failure recovery. Continuously improve resilience, correctness and maintainability of scraping and data ingestion infrastructure.
• Build observability into data pipelines through logging, metrics, and alerting. Monitor data quality. Improve and scale end-to-end data pipelines.
• Collaborate cross-functionally to support new data sources and evolving product requirements.

Qualifications

• At least 5 years of experience building and operating production-grade backend, data, or platform systems, with hands-on experience in web data acquisition or scraping.
• Experience with typescript or python
• Operated systems in environments with frequent change, partial failure, and external constraints.
• Product-oriented engineering mindset. Strong problem-solving skills and ownership mindset in production environments.
• Solid understanding of networking fundamentals: TLS/SSL behavior, timeouts, and failure modes.
• Proficiency using CSS selectors and XPath for data extraction.
• Experience designing systems with robust retry logic, error handling, testing, and observability.
• Proficiency with SQL and NoSQL databases
• Experience designing and operating production systems on AWS, including VPC networking, service-to-service communication, monitoring, and on-call operations.

Nice to Have

• Background in large-scale crawling, scraping, or distributed data systems.