Senior Data Platform Engineer - Azure, Iceberg & Trino

Valiant Technologies LLC; Sunnyvale (CA); On-site; USD 160,000 - 210,000; Full time

Job summary

Valiant Technologies LLC in Sunnyvale, CA is seeking an experienced data platform engineer to design, deploy, and operate scalable data services on Kubernetes within Microsoft Azure. You will build lakehouse infrastructure using Apache Iceberg backed by ADLS and optimize Trino and Spark clusters for performance and cost.

You will implement DBT-based data transformations, governance, security controls, and CI/CD automation while collaborating across data teams to onboard workloads and resolve

Qualifications

• Kubernetes expertise with deployment and operation of stateful and distributed workloads.
• Hands-on experience with Microsoft Azure Cloud and ADLS.
• Experience with Apache Iceberg and Iceberg catalogs REST.
• Expertise with Trino and Spark for distributed query/compute.
• DBT for data transformation and analytics engineering.
• Solid understanding of data lake/lakehouse architectures and file formats (Parquet/ORC).

Responsibilities

• Design, deploy, and operate data platform services on Kubernetes in Microsoft Azure.
• Build and manage lakehouse infrastructure on Apache Iceberg backed by ADLS.
• Operate and tune Trino and Spark clusters for performance and cost efficiency.
• Develop and maintain data transformation pipelines using DBT with sound modeling and testing.
• Define and enforce data access, security, governance, and multi-tenant resource management across ADLS and query engines.
• Automate infrastructure and deployments using infrastructure-as-code and CI/CD.
• Collaborate with data engineers, analysts, and platform teams to onboard workloads and resolve bottlenecks.

Skills

Kubernetes

Azure Cloud

ADLS

Apache Iceberg

Iceberg catalogs REST

Trino

Spark

DBT

Tools

Microsoft Azure Cloud

ADLS

Apache Iceberg

Trino

Spark

DBT

Job description

Valiant Technologies LLC in Sunnyvale, CA is seeking an experienced data platform engineer to design, deploy, and operate scalable data services on Kubernetes within Microsoft Azure. You will build lakehouse infrastructure using Apache Iceberg backed by ADLS and optimize Trino and Spark clusters for performance and cost.

You will implement DBT-based data transformations, governance, security controls, and CI/CD automation while collaborating across data teams to onboard workloads and resolve