Data Engineer: Real-Time Analytics & Data Platform

Beehiiv; United States; Remote; USD 140,000 - 190,000; Full time

Benefits offered by this job

Stock options

Health, dental, and vision insurance

401(k) employer match

Unlimited PTO

Annual in-person team offsite

Unlimited book budget

Monthly Wellness Days

Job summary

beehiiv is seeking a Data Engineer to build, grow, and maintain data models and backend data infrastructure. You’ll sit at the crossroads of Data Modeling, Data Engineering, Infrastructure, SRE, DevOps, Data Analytics and Backend API Engineering.

Ideal candidates have 4+ years in data engineering, superb SQL, and experience with Clickhouse, Kafka, and cloud-based data stacks. You’ll help scale our analytics and ML initiatives while collaborating across product engineering and data teams.

Qualifications

• 4+ years as a Data Engineer or highly technical Data Analytics Engineer.
• Superb SQL skills.
• Experience building and maintaining data models for analytics and ML feature stores.
• Experience with complex database migrations.
• Familiarity with Clickhouse or similar OLAP databases.
• Real-time streaming and event-driven architectures based on Kafka.
• Experience with Kafka tooling (Kafka Connect, Schema Registry, Avro).
• Proficiency in Go, Python or Ruby.
• Production deployments of RESTful APIs.
• Kubernetes deployments and troubleshooting.
• AWS (S3, EKS, VPCs, IAM, EFS).
• Infrastructure as Code with Terraform.
• CI/CD for deployments and schema changes.

Responsibilities

• Build and maintain Clickhouse data model and aggregations for app and internal analytics.
• Collaborate with Product Engineering to design event streaming applications.
• Hardening Kafka streaming ecosystem for security, performance, cost and scalability.
• Automate deployments of related schemas and migrations.
• Build out a Data Lake for long-term storage, DR, and cost optimization.
• Create streaming applications for ETL tasks upstream of databases (IP info enrichment).
• Develop a global entity resolution system and a first-party data enrichment product.
• Create testing environments for 3rd party integrations and maintain a schemaless architecture for customer events.

Skills

SQL

Data Modeling

Data Warehousing

Kafka

Go

Python

REST APIs

Kubernetes

AWS

Terraform

Education

Bachelor's degree in Computer Science or related field

Tools

Clickhouse

Kafka Ecosystem

Schema Registry

Avro

Goka

Iceberg

Job description

beehiiv is seeking a Data Engineer to build, grow, and maintain data models and backend data infrastructure. You’ll sit at the crossroads of Data Modeling, Data Engineering, Infrastructure, SRE, DevOps, Data Analytics and Backend API Engineering.

Ideal candidates have 4+ years in data engineering, superb SQL, and experience with Clickhouse, Kafka, and cloud-based data stacks. You’ll help scale our analytics and ML initiatives while collaborating across product engineering and data teams.