ClickHouse Operations Engineer

PostHog, Inc.; San Francisco (CA); Remote; USD 180,000 - 280,000; Full time

Job summary

PostHog is seeking a Senior Engineer to own ClickHouse operations and scale for trillions of events. You will automate provisioning with Terraform, Ansible, and Kubernetes, and optimize hardware for diverse workloads.

You will work in a natively remote environment, collaborating with distributed teams, and contribute to a cutting-edge data platform that powers product analytics at scale.

Qualifications

• Experience with OLAP databases, particularly ClickHouse, and internals.
• Ability to automate dynamic provisioning of infrastructure using IaC.
• Experience scaling high-availability data storage at petabyte scale.

Responsibilities

• Manage ClickHouse operations and scaling across large clusters.
• Automate provisioning and scaling of instances with Terraform, Ansible, and Kubernetes.
• Tune performance and hardware selection for workloads and cost efficiency.
• Contribute to a bleeding-edge ClickHouse implementation including open format backed tables.

Skills

OLAP DB experience

Automating provisioning

Scale & complexity

Python

Terraform

Ansible

Kubernetes

AWS

Zookeeper

Tools

Terraform

Ansible

Kubernetes

AWS

Zookeeper

Job description

About PostHog

PostHog is your product’s context layer. It ingests and stores your analytics, errors, replays, and business data so you and your agents can query and act on it. We launched out of Y Combinator's W20 cohort. We've since shipped more than a dozen products, including:

• A suite of analytics and observability tools, including product analytics, replay vision, AI observability, and error tracking
• PostHog MCP for users and their agents to query and act on their product and business data
• Data warehouse so users can query product and customer data together using custom SQL insights.

We Are

• Product-led. More than 500K organizations have installed PostHog, mostly driven by word-of-mouth. We have intensely strong product-market fit.
• Default alive. Revenue is growing incredibly quickly, and we're very efficient. We raise money to push ambition and grow faster, not to keep the lights on.
• Well-funded. We've raised more than $180m from some of the world's top investors. We're set up for a long, ambitious journey.

Things we care about

• Transparency: Everyone can read about our roadmap, how we pay (or even let go of) people, our strategy, and how we work, in our public company handbook. Internally, we share revenue, notes and slides from board meetings, and fundraising plans, so everyone has the context they need to make good decisions.
• Autonomy: We don’t tell anyone what to do. Everyone chooses what to work on next based on what's going to have the biggest impact on our customers, and what they find interesting and motivating to work on. Engineers lead product teams and make product decisions. Teams are flexible and easy to change when needed.
• Shipping fast: Why not now? We want to build a lot of products; we can't do that shipping at a normal pace. We've built the company around small teams – autonomous, highly-efficient groups of cracked engineers who can outship much larger companies because they own their products end-to-end.
• Time for building: Nothing gets shipped in a meeting. We're a natively remote company. We default to async communication – PRs > Issues > Slack. Tuesdays and Thursdays are meeting-free days, and we prioritize heads down building time over perfect coordination. This will be the most productive job you've ever had.
• Being weird: Weird means redesigning an already world-class website for the 5th time. It means shipping literally every product that relates to customer data. It means building an objectively unnecessary developer toy with dubious shareholder value. Doing weird stuff is a competitive advantage. And it's fun.

What you’ll be doing

ClickHouse is the core piece of infrastructure at PostHog. Every product and customer relies on it to ingest, store, and query data.

We need someone to automate, manage, and maintain ClickHouse as we grow towards capturing trillions of events per year and having one of the world’s largest clusters.

This includes ClickHouse operations and scaling infrastructure, as well as node and instance-level performance optimization. We want to ensure that we have the right hardware deployed at the right time for each workload on ClickHouse.

You'll build systems and automations for the provisioning and scaling of our large ClickHouse clusters, handling over 100 PB's of data. You'll have the ability to investigate and experiment using the latest hardware that cloud providers have to offer in order to find the optimal setup for our solution. And yes, You'll have a budget to do this.

You'll be using Terraform, Ansible, and Kubernetes to automate the dynamic provisioning of instances and work on a bleeding edge ClickHouse implementation, like open format backed tables, and not just maintenance.

We're also building a query optimizer for ClickHouse, which means you will work on query performance tooling.

You’ll Fit Right In

• You bring OLAP Database Experience. This role is focussed on ClickHouse, but if you bring strong experience with other OLAP Databases, that's great. We're looking for people that went into the internals of ClickHouse and other OLAP Databases, not high level users.
• You bring experience in automating Dynamic provisioning instances. Strong experience with utilizing Terraform, Ansible and K8s is important.
• You bring experience with Scale and Complexity! We're building and operating high-scale complex data storage solutions, we need you to have experience with the challenges this brings.
• You bring the Stack we need. We build using Python, Terraform, Ansible, Kubernetes, AWS, and Zookeeper (An alternative to Zookeeper is fine)
• You’re ready to do the best work of your career. We have incredible distribution, a big financial cushion and an amazing team. There’s probably no better place to see how far you can go.

We are committed to ensuring a fair and accessible interview process. If you need any accommodations or adjustments, please let us know.