Senior Software Engineer, Data Acquisition

People Data Labs; Oregon; Hybrid; USD 160,000 - 200,000; Full time

Benefits offered by this job

Stock options

401k matching

Competitive salaries

Unlimited PTO

Medical/dental/vision

Job summary

People Data Labs is seeking a Senior Data Engineer to join our Data Engineering & Acquisition Team. You will help accelerate our data platform, improving ingestion pipelines and distributed processing to deliver high-quality, scalable datasets for customers and developers.

You will work with Python/Go/Rust, build robust data pipelines, and collaborate across teams to design new data products. This remote-first role offers autonomy and growth opportunities within a leading data provider.

Qualifications

• 7+ years of professional experience building or operating backend or infrastructure systems at scale.
• Solid programming experience in Python, Go, Rust, or similar, including async/await, coroutines, or concurrency frameworks.
• Strong grasp of software architecture, concurrency, scalability, and fault tolerance.
• Familiarity with network debugging, DNS, proxies, packet capture and analysis.
• Experience with distributed data ingestion, ETL, and API integration.

Responsibilities

• Contribute to architecture and improvement of data acquisition and processing platform for reliability and throughput.
• Build and operate large-scale distributed systems delivering data from across the web.
• Design backend services for distributed job orchestration and data pipelines.
• Model and curate data ensuring high quality and consistency across datasets.
• Collaborate with data product and engineering teams to create new data products.

Skills

Backend systems

Python/Go/Rust

Concurrency

Distributed systems

Web crawling

Linux/Unix

Message queues

Education

Degree in CS

Tools

Kafka

SQS

Airflow

Databricks

Spark

SQL

Job description

Note for all engineering roles: with the rise of fake applicants and AI-enabled candidate fraud, we have built in additional measures throughout the process to identify such candidates and remove them.

About UsPeople Data Labs (PDL) is the provider of people and company data. We do the heavy lifting of data collection and standardization so our customers can focus on building and scaling innovative, compliant data solutions. Our sole focus is on building the best data available by integrating thousands of compliantly sourced datasets into a single, developer-friendly source of truth. Leading companies across the world use PDL's workforce data to enrich recruiting platforms, power AI models, create custom audiences, and more.

We are looking for individuals who can balance extreme ownership with a "one-team, one-dream" mindset. Our customers are trying to solve complex problems, and we only help them achieve their goals as a team. Our Data Engineering & Acquisition Team ensures our customers have standardized and high quality data to build upon.

You will be crucial in accelerating our efforts to build standalone data products that enable data teams and independent developers to create innovative solutions at massive scale. In this role, you will be working with a team to continuously improve our existing datasets as well as pursuing new ones. If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things. Failure is embraced at PDL as long as we continue to learn and grow from it.

What You Get to Do

• Contribute to the architecture and improvement of our data acquisition and processing platform, increasing reliability, throughput, and observability
• Use and develop web crawling technologies to capture and catalog data on the internet
• Build, operate, and evolve large-scale distributed systems that collect, process, and deliver data from across the web
• Design and develop backend services that manage distributed job orchestration, data pipelines, and large-scale asynchronous workloads
• Structure and model captured data, ensuring high quality and consistency across datasets
• Continuously improve the speed, scalability, and fault-tolerance of our ingestion systems
• Partner with data product and engineering teams to design and implement new data products powered by the data you help collect, and enhance and improve upon existing products
• Learn and apply domain-specific knowledge in web crawling and data acquisition, with mentorship from experienced teammates and access to existing systems

The Technical Chops You'll Need

• 7+ years of professional experience building or operating backend or infrastructure systems at scale
• Solid programming experience in Python, Go, Rust, or similar, including experience with async / await, coroutines, or concurrency frameworks
• Strong grasp of software architecture and backend fundamentals; you can reason clearly about concurrency, scalability, and fault tolerance
• Solid understanding of browser rendering pipeline, web application architecture (auth, cookies, request / response)
• Familiarity with network architecture and debugging ( DNS, proxies, packet capture and analysis)
• Solid understanding of distributed systems concepts: parallelism, asynchronous programming, backpressure, and message-driven design
• Experience designing or maintaining resilient data ingestion, API integration, or ETL systems
• Proficiency with Linux / Unix command-line tools and system resource management
• Familiarity with message queues, orchestration, and distributed task systems (Kafka, SQS, Airflow, etc.)
• Experience evaluating and monitoring data quality, ensuring consistency, completeness, and reliability across releases

People Thrive Here Who Can

• Work independently in a fast-paced, remote-first environment, proactively unblocking themselves and collaborating asynchronously
• Communicate clearly and thoughtfully in writing (Slack, docs, design proposals)
• Write and maintain technical design documents, including pipeline design, schema design, and data flow diagrams
• Scope and break down complex projects into deliverable milestones, and communicate progress, risks, and blockers effectively
• Balance pragmatism with craftsmanship, shipping reliable systems while continuously improving them

Some Nice To Haves

• Degree in a quantitative field such as computer science, mathematics, or engineering
• Experience as a Red Teamer
• Experience working on large-scale data ingestion, crawling, or indexing systems
• Experience with Apache Spark, Databricks, or other distributed data platforms
• Experience with streaming data systems (Kafka, Pub/Sub, Spark Streaming, etc.)
• Proficiency with SQL and data warehousing (Snowflake, Redshift, BigQuery, or similar)
• Experience with cloud platforms (AWS preferred, GCP or Azure also great)
• Understanding of modern data storage and design patterns (parquet, Delta Lake, partitioning, incremental updates)
• Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills)
• Experience building and maintaining data pipelines on modern big-data or cloud platforms (Databricks, Spark, or equivalent)

Comp: $160K - $200K

Our Benefits

• Stock
• 401k
• Competitive Salaries
• Unlimited paid time off
• Medical, dental, & vision insurance
• Health, fitness, and office stipends
• The permanent ability to work wherever and however you want

Why People Data Labs?

Great People make great teams. It is as simple and elegant as that. We believe in building highly functional, energetic, and engaging teams to serve our customers. People, Customers, Shareholders, in that order, sets us up for success and delivering on our promises.If all of that sounds good to you, we can't wait to hear from you and build something amazing together!Check out our latest company kickoff video to get a taste of our company culture:

People Data Labs does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.

Qualified Applicants with arrest or conviction records will be considered for Employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act.

Personal Privacy Policy for California Residents