Senior Lakehouse Platform Engineer

Synnex (Hyvecareers); Ontario (CA); On-site; USD 70,000 - 105,000; Full time

Job summary

Hyve Solutions is seeking a Data Warehouse SE to lead the evolution of our enterprise data platform from Hadoop/Hive to a cloud-native Lakehouse built on Apache Iceberg. You will design, implement, and Operate key platform services, enabling batch, streaming, and incremental workloads with scalable, cost-efficient architectures.

The role emphasizes platform engineering over pipelines, requiring deep distributed data systems expertise and hands-on work with Spark, Kubernetes, and cloud storage.

Qualifications

• Expert-level knowledge of Apache Spark, including execution planning, shuffle processing, memory management, and performance optimization.
• Strong experience with Apache Iceberg, Hudi, or Delta Lake, including metadata architecture, snapshots, manifests, schema evolution, and partition evolution.
• Hands-on experience deploying and managing data workloads on Kubernetes, including Spark on Kubernetes, Operators, and resource scheduling.
• Solid understanding of cloud object storage platforms such as Azure Blob Storage, Amazon S3, or MinIO, including storage consistency and performance characteristics.
• Deep understanding of modern Lakehouse architecture, including Iceberg, Spark, Catalog services, and object storage ecosystems.
• Experience with at least one cloud data platform such as Azure, Databricks, or Snowflake.
• Strong software engineering and programming skills in Java, Scala, or Python.

Responsibilities

• Design, build, and operate an enterprise-scale Lakehouse platform based on Apache Iceberg to support batch, streaming, and incremental data workloads.
• Drive the migration from Hadoop/HDFS to cloud-native object storage architectures and implement compute-storage separation at scale.
• Develop and enhance core platform services, including metadata management, catalog systems, governance frameworks, and multi-engine data access capabilities.
• Optimize distributed processing engines such as Spark to improve performance, scalability, reliability, and cost efficiency in PB-scale environments.
• Build and operate cloud-native data platforms on Kubernetes, including workload orchestration, resource management, and operational automation.
• Integrate the platform with cloud ecosystems such as Azure, Databricks, and Snowflake to enable seamless data flow and interoperability.
• Develop internal platform components and engineering tools for data ingestion, orchestration, quality management, diagnostics, and AI-driven automation.

Skills

Apache Spark

Apache Iceberg

Kubernetes

Cloud object storage

Lakehouse architecture

Cloud data platforms

Java/Scala/Python

Education

Bachelor's or Master’s degree in CS/SE/IS

3+ years large-scale data platform engineering

Job description

Hyve Solutions is seeking a Data Warehouse SE to lead the evolution of our enterprise data platform from Hadoop/Hive to a cloud-native Lakehouse built on Apache Iceberg. You will design, implement, and Operate key platform services, enabling batch, streaming, and incremental workloads with scalable, cost-efficient architectures.

The role emphasizes platform engineering over pipelines, requiring deep distributed data systems expertise and hands-on work with Spark, Kubernetes, and cloud storage.