Software Engineer, Data Foundations

Whatnot; New York (NY); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Generous Holiday and Time off Policy

Health Insurance options: Medical, 1DM

Work From Home Support

Home office setup allowance

Monthly allowance for cell phone and &

Job summary

Whatnot’s Data Platform team builds services that enhance our core product and unlock business value. You will define technical direction, lead streaming and orchestration platform builds, and design scalable, multi-tenant data infrastructure across the company.

You will own platform concerns like scalability, reliability, and SLOs while enabling real-time analytics, event-driven applications, and ML infrastructure needs. Collaboration with product, platform, ML, and data teams is essential.

Qualifications

• 8+ years of experience as a software engineer, with proven production systems.
• Experience building scalable distributed systems.
• Hands-on experience with streaming systems (Kafka, Flink, Pulsar) or orchestration platforms (Airflow, Dagster, Temporal).
• Experience designing and operating real-time data pipelines at scale.
• Experience evaluating architectural tradeoffs in data processing and orchestration.
• Experience building multi-tenant data infrastructure.
• Experience managing cloud data stores (Snowflake, BigQuery, Redshift) and data lake architectures (AWS S3).
• Strong operations experience: monitoring, alerting, incident management.
• Familiarity with DevOps/SRE and CI/CD tooling (GitHub Actions).
• Understanding of data governance and compliance principles.

Responsibilities

• Define technical direction and lead development of critical data platform systems that power next-generation capabilities at Whatnot.
• Lead the buildout and evolution of data orchestration and streaming platforms for production-ready usage company-wide.
• Design scalable, multi-tenant data infrastructure rather than just pipelines.
• Own platform concerns including scalability, performance, reliability, SLOs, and operational excellence.
• Build high-quality systems supporting real-time analytics, event-driven applications, and ML infrastructure needs.
• Make key decisions about data system integration and partner cross-functionally to implement architectures.
• Collaborate with product, platform, ML, and data teams to enable fast, safe, scalable development.
• Help shape company architecture across orchestration, stream processing, real-time systems, analytics, and content understanding.
• Apply DevOps/SRE practices to keep platform systems reliable at scale.
• Define and enforce data governance and compliance standards across data lake and downstream systems.

Skills

API-driven applications

Distributed systems

Streaming data systems

Data orchestration platforms

Real-time data pipelines

Cloud data stores

DevOps/SRE practices

CI/CD tooling

Data governance

Cross-functional collaboration

Tools

Kafka

Flink

Pulsar

Airflow

Dagster

Temporal

Snowflake

BigQuery

Redshift

AWS S3

Job description

Role

The Data Platform team at Whatnot builds services that improve our core product experience and unlock meaningful business value. In this role, you will:

• Define technical direction and lead development of critical data platform systems that power next-generation capabilities at Whatnot

• Lead the buildout and evolution of our data orchestration and streaming platforms, helping drive them from early platform investment to production-ready usage across the company

• Design scalable, reliable, multi-tenant data infrastructure rather than simply building pipelines on top of an existing platform

• Own core platform concerns including scalability, performance, reliability, SLOs, and operational excellence for data processing systems

• Build high-quality systems that support real-time analytics, event-driven applications, and machine learning infrastructure needs across Whatnot

• Make key decisions about how data systems should be integrated across Whatnot, and then partner cross-functionally to bring that architecture to life

• Collaborate closely with engineering partners across product, platform, machine learning, and data to enable fast, safe, and scalable development

• Help shape broader company architecture across orchestration, stream processing, real-time systems, analytics, and content understanding

• Apply DevOps/SRE practices, including CI/CD, infrastructure automation, monitoring, and incident response, to keep platform systems reliable at scale

• Help define and enforce data governance and compliance standards across our data lake and downstream systems

You

As our next Software Engineer, Data Platform, you should have 8+ years of experience, plus:

• Ability to write well-tested, API-driven applications and services that run at scale in production

• Track record of building scalable, high-performance distributed systems

• Strong experience with streaming data systems (e.g., Kafka, Flink, or Pulsar) and/or orchestration platforms (e.g., Airflow, Dagster, or Temporal), with particular strength in building or evolving the platform layer itself, not just downstream pipelines.

• Hands‑on experience designing and operating stateful, real-time data pipelines at scale, including concepts like checkpointing, state management, and watermarking

• Experience evaluating architectural tradeoffs in data processing and orchestration systems, including when a given tool is the right solution and when it is not

• Experience building multi-tenant or end-to-end data infrastructure that supports multiple teams or production use cases

• Expertise in managing cloud data stores (Snowflake, BigQuery, Redshift, etc.) and data lake architectures (e.g., AWS S3) for processing and serving data products

• Strong experience supporting operational systems to ensure uptime, reliability, and maintainability, including monitoring, alerting, and incident management

• Familiarity with DevOps/SRE practices and CI/CD tooling, including GitHub and GitHub Actions

• Understanding of data governance and compliance principles (e.g., data access controls, retention policies, auditability)

In addition to these role-specific qualifications, all engineering team members are expected to have:

• Experience managing cloud technologies (AWS or Google Cloud) and comfort with infrastructure-as-code approaches (for example, Terraform)

• Proficiency in at least one server-side programming language (preferably Python), common algorithms and data structures, and software design principles

• Self-starter ethic, with the ability to thrive under a high level of autonomy

• Exceptional interpersonal and communication skills

Benefits

• Generous Holiday and Time off Policy

• Health Insurance options including Medical, Dental, Vision

• Work From Home Support

  • Home office setup allowance

  • Monthly allowance for cell phone and internet

• Care benefits

  • Monthly allowance for wellness

  • Annual allowance towards Childcare

  • Lifetime benefit for family planning, such as adoption or fertility expenses

• Retirement; 401k offering for Traditional and Roth accounts in the US (employer match up to 4% of base salary) and Pension plans internationally

• Monthly allowance to dogfood the app

  • All Whatnot employees are expected to develop a deep understanding of our product. We're passionate about building the best user experience, and all employees are expected to use Whatnot as both a buyer and a seller as part of their job (our dogfooding budget makes this fun and easy!).

• Parental Leave

  • 16 weeks of paid parental leave + one month gradual return to work (company leave allowances run concurrently with country leave requirements which take precedence.)

EOE

Whatnot is proud to be an Equal Opportunity Employer. We value diversity, and we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, parental status, disability status, or any other status protected by local law. We believe that our work is better and our company culture is improved when we encourage, support, and respect the different skills and experiences represented within our workforce.