Senior Azure Data Platform Engineer: Lakehouse, Trino & DBT

Quantum Technologies. LLC; Sunnyvale (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Quantum Technologies. LLC is seeking a skilled data platform engineer to design, deploy, and operate data platform services on Kubernetes within Microsoft Azure.

You will build lakehouse infrastructure with Apache Iceberg backed by ADLS and optimize Trino and Spark clusters for performance and cost efficiency. You will develop robust data transformation pipelines using DBT, enforce governance and security standards, and implement observability and incident response.

Qualifications

• Hands-on experience deploying and operating stateful and distributed workloads on Kubernetes.
• Strong experience with Microsoft Azure Cloud and ADLS.
• Experience with Iceberg catalogs and REST interfaces.
• Expertise with Trino and Spark for distributed query and compute.
• Proficiency with DBT for data transformation and analytics engineering.
• Understanding of lakehouse architectures and file formats (Parquet/ORC).

Responsibilities

• Design, deploy, and operate data platform services on Kubernetes in Azure.
• Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS.
• Operate and tune Trino and Spark clusters for performance and cost efficiency.
• Develop data pipelines with DBT, including testing and documentation.
• Enforce data access, security, governance, and multi-tenant resource management.
• Implement observability with monitoring, alerting, and incident response.
• Collaborate with data engineers, analysts, and platform teams to onboard workloads.
• Automate infrastructure and deployments using infrastructure-as-code and CI/CD.

Tools

Kubernetes

Azure Cloud

ADLS

Apache Iceberg

Trino

Spark

DBT

Job description

Quantum Technologies. LLC is seeking a skilled data platform engineer to design, deploy, and operate data platform services on Kubernetes within Microsoft Azure.

You will build lakehouse infrastructure with Apache Iceberg backed by ADLS and optimize Trino and Spark clusters for performance and cost efficiency. You will develop robust data transformation pipelines using DBT, enforce governance and security standards, and implement observability and incident response.