Principal Data Platform Engineer - Remote, Equity Options

Ladders; United States; Remote; USD 272,000 - 431,000; Full time

Benefits offered by this job

Equity options available.

Comprehensive health benefits.

Flexible work arrangements.

Professional development opportunities

Job summary

Ladders is seeking a Principal Engineer - Data Retrieval to shape the architecture behind high-throughput data retrieval supporting advanced use cases. You will lead design for ingesting, organizing, and releasing large data volumes with emphasis on performance, reproducibility, and stability.

This role requires deep expertise in data platforms, distributed systems, and data governance, with collaboration across parsing, legal, security, and infrastructure teams to ensure compliant, scalable

Qualifications

• 15+ years of software engineering experience with large-scale distributed systems.
• Deep expertise in web crawling, indexing, or ingestion infrastructure.
• Strong hands-on engineering and system design skills across architecture and implementation.
• Experience with batch or streaming pipelines and operational reliability improvements.
• Proven track record delivering data platforms for demanding technical users.
• Excellent technical communication and leadership across diverse teams.

Responsibilities

• Design scalable storage and queueing systems for high-throughput data.
• Develop a shared crawl execution layer for reliable, polite web fetching.
• Build extensible pipelines for multiple content types while maintaining stable interfaces.
• Collaborate with parsing teams to improve data yield from valuable sources.
• Implement data governance and policy enforcement with legal and security teams.
• Ensure production readiness through capacity planning and observability.
• Evaluate internal and external datasets to deliver reproducible data releases.
• Coordinate cross-team dependencies while balancing immediate and long-term goals.

Skills

Distributed systems

Web crawling

Data ingestion

System design

Technical leadership

Education

PhD in Computer Science

Equivalent experience

Tools

Kafka

Cloud storage

Big data pipelines

Job description

Ladders is seeking a Principal Engineer - Data Retrieval to shape the architecture behind high-throughput data retrieval supporting advanced use cases. You will lead design for ingesting, organizing, and releasing large data volumes with emphasis on performance, reproducibility, and stability.

This role requires deep expertise in data platforms, distributed systems, and data governance, with collaboration across parsing, legal, security, and infrastructure teams to ensure compliant, scalable