Data Engineer with Clickhouse

Talent Groups; Bellevue (WA); On-site; USD 120,000 - 180,000; Full time

Job summary

Talent Groups is seeking a Data Engineer in Bellevue, WA to support near real-time analytics for mobile network data.

The role focuses on extending ingestion pipelines from Azure Data Lake/HDFS, designing partitioned tables, and ensuring low-latency query access. You will collaborate across teams to integrate Snowflake and Databricks Iceberg with existing feeds and pipelines.

Qualifications

• Strong analytical SQL and experience with a columnar or MPP database such as ClickHouse, BigQuery, Redshift or similar.
• Experience with a production ingestion framework and cloud storage (Parquet) is desirable.
• Familiarity with Azure cloud technologies and data lake architectures is a plus.

Responsibilities

• Extend the Python ingestion framework that loads from Azure Data Lake/HDFS.
• Design tables with partitioning, sort keys, TTL, and materialized views.
• Diagnose merge behavior, part counts, and replication lag.
• Integrate with Snowflake and Databricks Iceberg alongside primary feeds.
• Work on optimizing for hundreds of billions of rows and low-latency responses.

Skills

Hadoop

Python

Databricks

ClickHouse

SQL

Azure

Tools

ClickHouse

Snowflake

Databricks Iceberg

Azure Data Lake

Job description

• Mandatory Primary Skills: Hadoop, Python, Databricks, Clickhouse DB , ADLS, SQL + Azure cloud technologies above all with Azure (such as ADF, Databricks, Log Analytics, Logic App, Key Vault, Blob Storage, ADLS, etc.)
• Secondary Skills:; Snowflake ,
• We run a near real-time analytics platform for mobile network data, ingesting billions of rows a day from Azure Data Lake/HDFS into ClickHouse and serving subscriber-level queries back to internal APIs in tens of milliseconds. We are expanding the platform footprint to support additional use cases by adding more datasets
• Data Engineer would work on: extending the Python ingestion framework that loads from Azure Data Lake/HDFS; table design covering partitioning, sort keys, TTL and materialized views; operational diagnosis of merge behaviour, part counts and replication lag; and integration with Snowflake and Databricks Iceberg alongside the primary feeds.
• We need strong analytical SQL and experience with a columnar or MPP database — ClickHouse, BigQuery, Redshift or similar. Python sufficient to own a production ingestion framework rather than write scripts. Comfort at hundreds of billions of rows, since the failure modes differ from smaller systems. Cloud object storage and Parquet. Enough Kubernetes to tell whether a problem is the database or the platform underneath it.
• Strongly preferred: direct ClickHouse experience, particularly MergeTree engines and aggregate function states, and telecom data such as DNS , IPFIX, etc. ClickHouse specifically is uncommon, and we would rather have a strong engineer from an adjacent system than a weak one with the exact keyword.