Lead Data Platform Engineer - Kubernetes, Azure, Iceberg

Jade Global; San Jose (CA); On-site; USD 140,000 - 200,000; Full time

Job summary

Jade Global is seeking a data platform engineer to design, deploy, and operate scalable data platform services on Kubernetes using Microsoft Azure. The role focuses on building lakehouse infrastructure with Apache Iceberg on ADLS, while optimizing Trino and Spark workloads.

You will develop DBT pipelines, enforce governance and security standards, and implement observability and CI/CD for automation across multi-tenant environments. Collaboration with data engineers and analysts is essential.

Qualifications

• Hands-on Kubernetes deployment and operation for stateful/distributed workloads.
• Azure Cloud experience including ADLS.
• Experience with Apache Iceberg and Iceberg catalogs REST.
• Expertise with Trino and Spark for distributed query/compute.
• DBT experience for data transformation/analytics engineering.

Responsibilities

• Design, deploy, and operate data platform services on Kubernetes in Microsoft Azure.
• Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS, including catalog/configuration, table maintenance, schema evolution, and partitioning strategy.
• Operate and tune Trino and Spark clusters for performance, reliability, and cost efficiency.
• Develop and maintain data transformation pipelines using DBT with sound modeling, testing, and documentation practices.
• Define and enforce standards for data access, security, governance, and multi-tenant resource management across ADLS and query engines.
• Implement observability for the platform, including monitoring, alerting, capacity planning, and incident response.
• Partner with data engineers, analysts, and platform teams to onboard workloads and resolve performance bottlenecks.
• Automate infrastructure and deployments using infrastructure-as-code and CI/CD.

Skills

Kubernetes

Microsoft Azure Cloud

ADLS

Apache Iceberg

Iceberg catalogs REST

Trino

Spark

DBT

Job description

Jade Global is seeking a data platform engineer to design, deploy, and operate scalable data platform services on Kubernetes using Microsoft Azure. The role focuses on building lakehouse infrastructure with Apache Iceberg on ADLS, while optimizing Trino and Spark workloads.

You will develop DBT pipelines, enforce governance and security standards, and implement observability and CI/CD for automation across multi-tenant environments. Collaboration with data engineers and analysts is essential.