Staff Production Engineer

Whatnot; San Francisco (CA); Remote; USD 245,000 - 290,000; Full time

Benefits offered by this job

Flexible Time off Policy

Health Insurance options

Work From Home Support

Home office setup allowance

Cellphone & internet allowance

Parental Leave

401k matching

Dogfooding budget

Job summary

Whatnot is hiring a Production Engineer to embed with cross-functional teams and own the critical path for live video and auctions. You will hunt anomalies, fix bottlenecks, and ensure scalable, reliable operations across services, storage, and clients.

You're expected to have 9+ years in production services, deep systems knowledge, and fluency in Python/Elixir/Go with AWS or GCP and Kubernetes. Remote-friendly, with SF or NYC hub proximity for in-person collaboration.

Qualifications

• Bachelor’s degree in CS or related field or equivalent experience.
• 9+ years building and debugging production services at scale.
• Strong systems and distributed systems fundamentals.
• Fluency with cloud-native environments (AWS or GCP) and Kubernetes.
• Experience with observability and production debugging.

Responsibilities

• Embed with product, platform, and infra teams to improve reliability and performance.
• Hunt anomalies across traffic, latency, errors, and costs; trace root causes.
• Link operational signals to business impact and prioritize fixes.
• Lead incident response for cross-team failures and drive systemic fixes.
• Prepare for peak events with capacity modeling and load testing.

Skills

Production engineering

Distributed systems

Observability

Cloud-native (AWS/GCP)

Kubernetes

Python

Elixir

Go

SRE mindset

Incident command

Education

Bachelor’s degree in Computer Science or related field

Tools

AWS

GCP

Kubernetes

Terraform

Job description

Join the Future of Commerce with Whatnot! Whatnot is the largest live shopping platform in North America and Europe to buy, sell, and discover the things you love. Whether it's trading cards, fashion, electronics, or live plants, our sellers are building real businesses across hundreds of categories. We're building live commerce at a scale that's never been done in the West, and there's no playbook to copy. The people here are shaping how an entirely new industry develops. As a remote co-located team, we're inspired by our values and anchored in hubs across the US, UK, Ireland, Poland, Germany, and Australia. We move fast, stay close to our users, and focus on the work that drives the most impact. We're one of the fastest growing marketplaces and were recently named the #1 Best Startup Employer in America by Forbes. Check out the latest Whatnot updates on our news and engineering blogs and join us as we enable anyone to turn their passion into a business and bring people together through commerce.

Role

That growth is the reason this role exists. Production Engineers are software engineers who embed with product, platform, and infrastructure teams to find and fix what breaks at scale. We think of them as hunters: engineers who go looking for the anomaly, the bottleneck, the slow degradation nobody has noticed yet, and then own it through to a fix, working with whoever owns the system to make sure it stays fixed. Live video and real-time commerce run on the same critical path here. Every auction is live and money moves during the show, so system behavior and business outcomes are tightly coupled in a way few platforms experience. Milliseconds are visible to buyers and sellers, and the signals that matter most are often small, concentrated in a specific cohort, and invisible in aggregate. Surfacing them early, and knowing which ones are worth acting on, is the core of this work. Production Engineers embed with a team and go deep on its systems, while staying accountable for what falls between teams. Some of the most consequential problems at our scale live in the seams, in the interaction between two services that each look healthy on their own, so we expect hunting both inside your area and across it. You'll be joining early, which means broader discovery across the platform before the embedding settles, and real influence over how Production Engineering takes shape here.

What You’ll Do

• Go deep with the team you embed with, raising the reliability, performance, and scalability of their systems alongside them and writing the code that gets it there.
• Hunt anomalies across traffic, latency, error, and cost signals, both inside your area and in the seams between teams, and trace them to root cause across services, storage, and clients.
• Connect operational signals to business impact: which degradations matter, for which cohorts, and which are safe to leave alone.
• Work where the risk concentrates: payments, live video, search, the bidding path, and security.
• Eliminate scale bottlenecks in production, then remove the class of problem rather than the instance.
• Prepare for peak events through capacity modeling, load testing, and failure drills.
• Improve observability where it's thin, so the next anomaly is found in minutes rather than quarters.
• Put AI to work on operations: anomaly detection, on-call assistance, remediation, toil reduction. Mostly greenfield, and yours to prove out.
• Share on-call with the team you embed with, and act as an escalation point for live production incidents.
• Lead incident response for complex cross-team failures and drive the systemic fixes that follow.

We offer flexibility to work from home or from one of our global office hubs, and we value in-person time for planning, problem-solving, and connection. Team members in this role must live within commuting distance of our San Francisco or New York City hub.

You

People who do well at Whatnot tend to be comfortable figuring things out as they go, biased toward action, and genuinely curious about what they're building. They care more about outcomes than credit and stay close to the product and the people using it.

As Our Next Production Engineer, You Likely Have

• Bachelor’s degree in Computer Science, a related field, or equivalent work experience.
• 9+ years building and debugging production services at scale. You may have come to that through software engineering, production engineering, SRE, or systems engineering; what matters is the depth, not the title.
• A software engineer's identity first, paired with a pull toward the messy end of production. You want to write code and build systems, and you're the one who notices the graph that looks slightly wrong and can't let it go.
• Strong systems and distributed systems fundamentals: failure modes, saturation, queueing behavior, cascading failure, and how Linux, networking, and storage behave under load.
• Depth in observability and production debugging, plus experience in one or more of: capacity and performance engineering, load and resilience testing, incident command, or large-scale migrations under production constraints.
• The ability to work well embedded. You can walk into another team's codebase, earn trust quickly, and leave the system better than you found it.
• Fluency across languages and technologies, and comfort in cloud-native environments such as AWS or GCP with Kubernetes and infrastructure as code. Our backend is primarily Python and Elixir, with Go for performance-sensitive infrastructure, and we care more about depth and adaptability than a specific stack.
• Bonus: high-traffic, real-time, event-driven, or live streaming systems.

Benefits

• Flexible Time off Policy and Company-wide Holidays (including a spring and winter break)
• Health Insurance options including Medical, Dental, Vision
• Work From Home Support
  • Home office setup allowance
  • Monthly allowance for cell phone and internet
• Care benefits
  • Monthly allowance for wellness
  • Annual allowance towards Childcare
  • Lifetime benefit for family planning, such as adoption or fertility expenses
• Retirement; 401k offering for Traditional and Roth accounts in the US (employer match up to 4% of base salary) and Pension plans internationally
• Monthly allowance to dogfood the app
  • All Whatnauts are expected to develop a deep understanding of our product. We're passionate about building the best user experience, and all employees are expected to use Whatnot as both a buyer and a seller as part of their job (our dogfooding budget makes this fun and easy!).
• Parental Leave
  • 16 weeks of paid parental leave + one month gradual return to work *company leave allowances run concurrently with country leave requirements which take precedence.

EOE

Whatnot is proud to be an Equal Opportunity Employer. We value diversity, and we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, parental status, disability status, or any other status protected by local law. We believe that our work is better and our company culture is improved when we encourage, support, and respect the different skills and experiences represented within our workforce.

Compensation Range: $245K - $290K