ClickHouse Operations Engineer

OpenKrill; United States; On-site; USD 140,000 - 190,000; Full time

Job summary

PostHog is seeking an experienced ClickHouse infra engineer to automate, manage, and scale ClickHouse clusters in a primarily remote environment. You will optimize node-level performance, provision large clusters, and experiment with the latest cloud hardware to maximize throughput.

Use Terraform, Ansible, and Kubernetes to automate deployment; contribute to a bleeding-edge query optimizer and telemetry tooling for analytics at scale.

Qualifications

• OLAP database experience, focusing on internals like ClickHouse.
• Experience automating dynamic provisioning with Terraform, Ansible and Kubernetes.
• Experience solving high-scale, complex data storage challenges.
• Tech stack includes Python, Terraform, Ansible, Kubernetes, AWS and Zookeeper.

Responsibilities

• Automate, manage, and maintain ClickHouse as the organization scales.
• Build provisioning and scaling systems for large ClickHouse clusters.
• Experiment with latest cloud hardware to optimize setup and performance.
• Contribute to query optimization tooling and a bleeding-edge ClickHouse implementation.

Skills

OLAP Database Experience

Dynamic provisioning automation

Scale & complexity

Python development

Tools

Terraform

Ansible

Kubernetes

AWS

Zookeeper

Job description

ABOUT POSTHOG

PostHog is your product's context layer. It ingests and stores your analytics, errors, replays, and business data so you and your agents can query and act on it.

We launched out of Y Combinator's W20 cohort https://posthog.com/handbook/story. We've since shipped more than a dozen products, including:

• A suite of analytics and observability tools, including product analytics https://posthog.com/product-analytics, replay vision https://posthog.com/replay-vision, AI observability https://posthog.com/ai-observability, and error tracking https://posthog.com/error-tracking
• PostHog MCP https://posthog.com/mcp for users and their agents to query and act on their product and business data
• Data warehouse https://posthog.com/context-warehouse/managed-warehouse so users can query product and customer data together using custom SQL insights.

We are:

1. Product-led. More than 500K organizations have installed PostHog, mostly driven by word-of-mouth. We have intensely strong product-market fit.

2. Default alive https://paulgraham.com/aord.html. Revenue is growing incredibly quickly, and we're very efficient. We raise money to push ambition and grow faster, not to keep the lights on.

3. Well-funded. We've raised more than $180m from some of the world's top investors. We're set up for a long, ambitious journey.

We're focused on building an awesome product for end users, hiring exceptional teammates, shipping fast, and being as weird as possible https://postog.com/deskhog.

THINGS WE CARE ABOUT

• Transparency: Everyone can read about our roadmap, how we pay (or even let go of) people, our strategy, and how we work, in our public company handbook https://posthog.com/handbook. Internally, we share revenue, notes and slides from board meetings, and fundraising plans, so everyone has the context they need to make good decisions.
• Autonomy: We don't tell anyone what to do. Everyone chooses what to work on next based on what's going to have the biggest impact on our customers, and what they find interesting and motivating to work on. Engineers lead product teams https://posthog.com/handbook/wide-company and make product decisions https://posthog.com/handbook/which-products. Teams are flexible and easy to change when needed.
• Shipping fast: Why not now? https://posthog.com/handbook/values#why-not-now We want to build a lot of products; we can't do that shipping at a normal pace. We've built the company around small teams - autonomous, highly-efficient groups of cracked engineers https://posthog.com/founders/cracked-manifesto who can outship much larger companies because they own their products end-to-end.
• Time for building: Nothing gets shipped in a meeting. We're a natively remote company. We default to async communication - PRs > Issues > Slack. Tuesdays and Thursdays are meeting-free days https://posthog.com/handbook/company/culture#were-on-the-makers-schedule, and we prioritize heads down building time over perfect coordination. This will be the most productive job you've ever had.
• Being weird: Weird means redesigning an already world-class website for the 5th time. It means shipping literally every product that relates to customer data. It means building an objectively unnecessary developer toy https://posthog.com/deskhog with dubious shareholder value. Doing weird stuff is a competitive advantage. And it's fun.

WHAT YOU'LL BE DOING

ClickHouse is the core piece of infrastructure at PostHog https://posthog.com/docs/how-posthog-works/clickhouse. Every product and customer relies on it to ingest, store, and query data.

We need someone to automate, manage, and maintain ClickHouse as we grow towards capturing trillions of events per year and having one of the world's largest clusters.

This includes ClickHouse operations and scaling infrastructure, as well as node and instance-level performance optimization. We want to ensure that we have the right hardware deployed at the right time for each workload on ClickHouse.

You'll build systems and automations for the provisioning and scaling of our large ClickHouse clusters, handling over 100 PB's of data. You'll have the ability to investigate and experiment using the latest hardware that cloud providers have to offer in order to find the optimal setup for our solution. And yes, You'll have a budget to do this.

You'll be using Terraform, Ansible, and Kubernetes to automate the dynamic provisioning of instances and work on a bleeding edge ClickHouse implementation, like open format backed tables, and not just maintenance. We're also building a query optimizer for ClickHouse, which means you will work on query performance tooling.

YOU'LL FIT RIGHT IN IF:

• You bring OLAP Database Experience. This role is focussed on ClickHouse, but if you bring strong experience with other OLAP Databases, that's great. We're looking for people that went into the internals of ClickHouse and other OLAP Databases, not high level users.
• You bring experience in automating Dynamic provisioning instances. Strong experience with utilizing Terraform, Ansible and K8s is important.
• You bring experience with Scale and Complexity! We're building and operating high-scale complex data storage solutions, we need you to have experience with the challenges this brings.
• You bring the Stack we need. We build using Python, Terraform, Ansible, Kubernetes, AWS, and Zookeeper (An alternative to Zookeeper is fine)
• You're ready to do the best work of your career. We have incredible distribution, a big financial cushion and an amazing team. There's probably no better place to see how far you can go.

We are committed to ensuring a fair and accessible interview process. If you need any accommodations or adjustments, please let us know.