Senior Data Platform Engineer

United Software Group Inc; San Jose (CA); Sur place; USD 150 000 - 230 000; Plein temps

Résumé du poste

United Software Group Inc is seeking a Senior Data Platform Engineer to design, deploy, and operate a lakehouse data platform on Kubernetes in Microsoft Azure. You’ll own the infrastructure behind Apache Iceberg, Trino and Spark, partnering with data engineers and analysts to maintain performance, reliability, and cost efficiency.

Key responsibilities include deploying services on AKS/Azure, managing Iceberg on ADLS, tuning Trino/Spark, building DBT pipelines, and enforcing data governance with

Qualifications

• Must design and operate scalable data platform infrastructure.
• Experience with lakehouse architectures and query optimization.
• Strong hands-on with Kubernetes in cloud environments.

Responsabilités

• Deploy and operate data platform services on Kubernetes in Azure.
• Build and manage Iceberg lakehouse infrastructure on ADLS (catalog, table maintenance, schema evolution, partitioning).
• Operate and tune Trino and Spark clusters; build DBT transformation pipelines.

Connaissances

Cloud architecture

Data governance

SQL performance tuning

Distributed systems

Infra as code

Outils

Kubernetes

Apache Iceberg

Trino

Spark

DBT

Azure Data Lake Storage

Description du poste

We’re looking for a Senior Data Platform Engineer to design, deploy and operate a lakehouse data platform on Kubernetes in Microsoft Azure. You’ll own the infrastructure behind Apache Iceberg, Trino and Spark, and work with data engineers and analysts to keep the platform fast, reliable and cost-efficient.

Key Responsibilities

• Deploy and operate data platform services on Kubernetes in Azure
• Build and manage Iceberg lakehouse infrastructure on ADLS (catalog, table maintenance, schema evolution, partitioning)
• Operate and tune Trino and Spark clusters
• Build DBT transformation pipelines
• Enforce data access, security and governance standards
• Set up monitoring, alerting and incident response
• Automate with infrastructure-as-code and CI/CD

Required Skills

• Strong hands-on Kubernetes, including stateful and distributed workloads
• Apache Iceberg and REST catalogs
• Trino and Spark expertise
• DBT
• Lakehouse architecture, Parquet/ORC and query optimization