Principal Engineer - Data Retrieval

Ladders; United States; Remote; USD 272,000 - 431,000; Full time

Benefits offered by this job

Equity options available.

Comprehensive health benefits.

Flexible work arrangements.

Professional development opportunities

Job summary

Ladders is seeking a Principal Engineer - Data Retrieval to shape the architecture behind high-throughput data retrieval supporting advanced use cases. You will lead design for ingesting, organizing, and releasing large data volumes with emphasis on performance, reproducibility, and stability.

This role requires deep expertise in data platforms, distributed systems, and data governance, with collaboration across parsing, legal, security, and infrastructure teams to ensure compliant, scalable

Qualifications

• 15+ years of software engineering experience with large-scale distributed systems.
• Deep expertise in web crawling, indexing, or ingestion infrastructure.
• Strong hands-on engineering and system design skills across architecture and implementation.
• Experience with batch or streaming pipelines and operational reliability improvements.
• Proven track record delivering data platforms for demanding technical users.
• Excellent technical communication and leadership across diverse teams.

Responsibilities

• Design scalable storage and queueing systems for high-throughput data.
• Develop a shared crawl execution layer for reliable, polite web fetching.
• Build extensible pipelines for multiple content types while maintaining stable interfaces.
• Collaborate with parsing teams to improve data yield from valuable sources.
• Implement data governance and policy enforcement with legal and security teams.
• Ensure production readiness through capacity planning and observability.
• Evaluate internal and external datasets to deliver reproducible data releases.
• Coordinate cross-team dependencies while balancing immediate and long-term goals.

Skills

Distributed systems

Web crawling

Data ingestion

System design

Technical leadership

Education

PhD in Computer Science

Equivalent experience

Tools

Kafka

Cloud storage

Big data pipelines

Job description

For our client, we are seeking a Principal Engineer - Data Retrieval to join the team of a leader in the Telecommunications & Hardware space.

This role will shape the architecture behind high-throughput data retrieval capabilities that support advanced technical use cases at scale. You will lead the design of reliable systems for ingesting, organizing, and releasing large volumes of data with a strong emphasis on performance, reproducibility, and operational stability. The work is highly cross-functional, partnering closely with parsing, legal, security, and infrastructure teams to ensure the platform is both effective and compliant. This is an opportunity to influence core data platform decisions in an environment that values technical depth, system rigor, and long-term scalability.

Location: Remote

Compensation: $272,000 – $431,250 annually

Responsibilities

• Design scalable storage and queueing systems for high-throughput data.
• Develop a shared crawl execution layer for reliable, polite web fetching.
• Build extensible pipelines for multiple content types while maintaining stable interfaces.
• Collaborate with parsing teams to improve data yield from valuable sources.
• Implement data governance and policy enforcement with legal and security teams.
• Ensure production readiness through capacity planning and observability.
• Evaluate internal and external datasets to deliver reproducible data releases.
• Coordinate cross-team dependencies while balancing immediate and long-term goals.

Qualifications

• BS, MS, or PhD in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• 15+ years of experience in software engineering with large-scale distributed systems.
• Deep expertise in web crawling, indexing, or ingestion infrastructure.
• Strong hands-on engineering and system design skills across architecture and implementation.
• Experience with batch or streaming pipelines and operational reliability improvements.
• Proven track record in delivering data platforms for demanding technical users.
• Excellent technical communication and leadership skills across diverse teams.

Benefits

• Equity options available.
• Comprehensive health benefits.
• Flexible work arrangements.
• Professional development opportunities.
• Access to cutting-edge technology and resources.

Our client is an equal opportunity employer.