Senior Data Engineer — Scale Data Pipelines in Go/Python

Merkle Science

New York (NY)

On-site

USD 140,000 - 210,000

Full time

9 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Health insurance
Equity
Flexible time off
Learning & development

Job summary

Merkle Science is seeking an experienced data engineer to design, build, and run the systems behind our data products. You will own your work from design to production, ensuring performance, reliability and cost efficiency across ingestion, processing, and analytic storage for blockchain data.

You will work on a wide stack, focusing on scalable pipelines, backend services in Go and Python, and data models in ClickHouse, with Databricks workloads.

Qualifications

  • 5+ years building and operating production data systems, including ETL and ELT pipelines.
  • Expert Python and strong SQL. You write typed, tested code, are comfortable with asyncio and concurrency.
  • Go for backend services. You have built and run high-throughput services or data loaders in Go in production.
  • ClickHouse in production. You know MergeTree schema and sort-key design, materialised views, bulk loads and query tuning.
  • Databricks in production. You have worked with PySpark, Delta Lake and Unity Catalog, designed incremental and idempotent tables, and kept an eye on compute cost.
  • Large-scale data. You have hands-on experience with pipelines over billions of rows and tens of terabytes.

Responsibilities

  • Design and build scalable, high-throughput data pipelines and platform systems for blockchain data across 30+ chains.
  • Build backend data services and loaders in Go and Python.
  • Design and evolve data models and analytical workloads in ClickHouse, and tune them for sub-second queries.
  • Build incremental, idempotent pipelines on Databricks over billions of rows and tens of terabytes.
  • Keep improving the scale, reliability, freshness, performance and cost efficiency of the platform.
  • Investigate hard performance and reliability problems and see them through to resolution.
  • Work closely with product, analytics and investigations teams that use the data, and turn their needs into platform designs.
  • Help set technical direction through architecture discussions, design reviews and code reviews.
  • Mentor engineers and help set strong engineering and operational practices.

Skills

Python
SQL
Go
ClickHouse
Databricks
Kafka
Airflow
Kubernetes
Docker
PySpark

Tools

Docker
PySpark
Unity Catalog
Terraform
Prometheus

Job description

Merkle Science is seeking an experienced data engineer to design, build, and run the systems behind our data products. You will own your work from design to production, ensuring performance, reliability and cost efficiency across ingestion, processing, and analytic storage for blockchain data.

You will work on a wide stack, focusing on scalable pipelines, backend services in Go and Python, and data models in ClickHouse, with Databricks workloads.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Platform Engineer for High-Volume Pipelines
Senior Data Platform Engineer for High-Volume Pipelines

merklescience • New York (NY)

On-site
GBP 90,000 - 140,000
Health insurance
Learning & development
Senior Data Engineer - Scalable Blockchain Data Pipelines
Senior Data Engineer - Scalable Blockchain Data Pipelines

merklescience • United States

Remote
USD 150,000 - 230,000
Health insurance
Flexible time off
Learning & development
+1
Senior Data Engineer (Fully Remote)
Senior Data Engineer (Fully Remote)

Merkle Science • United States

Remote
USD 130,000 - 180,000
Health insurance
Flexible time off
Learning & development initiatives
+1
Senior Data Engineer
Senior Data Engineer

merklescience • United States

Remote
USD 150,000 - 230,000
Health insurance
Flexible time off
Learning & development
+1
Senior Data Engineer
Senior Data Engineer

merklescience • New York (NY)

On-site
GBP 90,000 - 140,000
Health insurance
Learning & development
Senior Data Engineer: Blockchain Pipelines & ClickHouse
Senior Data Engineer: Blockchain Pipelines & ClickHouse

Merkle Science • United States

Remote
USD 130,000 - 180,000
Health insurance
Flexible time off
Learning & development initiatives
+1
Senior Data Engineer
Senior Data Engineer

Merkle Science • New York (NY)

On-site
USD 140,000 - 210,000
Health insurance
Equity
Flexible time off
+1
Senior Data Engineer - Remote Data Pipelines & Analytics
Senior Data Engineer - Remote Data Pipelines & Analytics

Scalepex • United States

Remote
USD 110,000 - 170,000
100% Remote
Contractor scheme
Senior Data Engineer: Scale Data Pipelines & AI
Senior Data Engineer: Scale Data Pipelines & AI

Scaffold, Inc. • Austin (TX)

On-site
USD 120,000 - 180,000
Senior Data Engineer: Build Scalable Data Pipelines
Senior Data Engineer: Build Scalable Data Pipelines

Harnham • California

On-site
USD 120,000 - 160,000