Lakehouse Architect - On-Prem Data Platform

Socket.dev; United States; Remote; USD 180,000 - 270,000; Full time

Benefits offered by this job

20 days fixed time off per year

Generous parental leave

Monetary incentives and company-wide Q

Medical, dental, and vision insurance

Disability and life insurance

Job summary

Innovaccer Inc. is seeking a Principal Software Engineer for its Data Platform to own the architecture of the lakehouse engine powering on-premise deployments.

You will shape the Iceberg table design, Trino cluster topology, catalog service, and Spark transform compute, influencing downstream transforms and reporting. You will lead platform standards, port workloads to Trino and Spark SQL, and mentor engineers while collaborating with infra on storage and capacity planning in regulated

Qualifications

• 12+ years of experience building large-scale data platforms or distributed systems.
• Hands-on expertise with distributed SQL engines such as Trino/Presto or Spark SQL.
• Production experience with Apache Iceberg and iceberg table maintenance at scale.
• Strong Java and/or Python development experience.
• Experience delivering data platforms in on-premise, regulated environments is a plus.

Responsibilities

• Own the lakehouse reference architecture including Iceberg design, Trino cluster topology, catalog service, Spark compute, and object-storage layout.
• Design on-premise replacements for cloud warehouse capabilities, including change-data-capture streams and write-back paths.
• Run PoC validation for the catalog and query engine at expected data volumes and set triggers for placement decisions.
• Set platform-wide standards for table layout, partitioning, file sizing, and Iceberg maintenance.
• Lead the SQL dialect strategy for porting workloads to Trino and Spark SQL.
• Mentor senior engineers, review designs, and raise engineering quality.
• Partner with platform engineering on storage sizing, resource isolation, and capacity planning.

Skills

Distributed SQL engines

Trino/Presto

Spark SQL

Apache Iceberg

Java

Python

On-premise data platforms

Healthcare data experience

Education

B.E./B.Tech./M.Sc. in CS or related

Tools

Iceberg catalog services (Polaris/Nessie/Hive Metastore)

Job description

Innovaccer Inc. is seeking a Principal Software Engineer for its Data Platform to own the architecture of the lakehouse engine powering on-premise deployments.

You will shape the Iceberg table design, Trino cluster topology, catalog service, and Spark transform compute, influencing downstream transforms and reporting. You will lead platform standards, port workloads to Trino and Spark SQL, and mentor engineers while collaborating with infra on storage and capacity planning in regulated