Lead Data Platform Architect — On-Prem Lakehouse

Innovaccer Analytics; United States; Remote; USD 180,000 - 250,000; Full time

Benefits offered by this job

20 days PTO

Parental leave

Recognition program

Insurance coverage

Job summary

Innovaccer Inc. seeks a Principal Software Engineer to own the lakehouse engine architecture for on-premise deployments, using Iceberg as the table format, Trino as the query engine, a REST catalog service, and Spark for transforms.

This role shapes downstream transforms, serving, and reporting through thoughtful catalog, engine placement, and pipeline redesign. You will lead architecture decisions, mentor engineers across data workstreams, and drive platform standards from table layout to

Qualifications

• 12+ years of experience building and operating large-scale data platforms or distributed systems.
• Bachelor’s/Master’s in Computer Science or related field.
• Deep expertise with distributed SQL engines (Trino/Presto or Spark SQL).

Responsibilities

• Own the lakehouse reference architecture: Iceberg table design, Trino cluster topology, catalog service, Spark transform compute, and object-storage layout.
• Design on-premise replacements for cloud-managed warehouse capabilities.
• Run PoC validation of catalog and query engine at expected data volumes; define triggers for placement decisions.
• Set platform-wide standards for table layout, partitioning, file sizing, and Iceberg maintenance.
• Lead the SQL dialect strategy for porting workloads to Trino and Spark SQL.
• Mentor senior engineers; review designs to raise engineering quality.
• Partner with platform engineering on storage sizing, resource isolation, and capacity planning.

Skills

Distributed systems

Mentoring

Java/Python

Architecture design

Education

B.E./B.Tech./M.Sc.

Tools

Trino

Spark

Iceberg

Iceberg catalog

REST catalogs

Hive Metastore

Job description

Innovaccer Inc. seeks a Principal Software Engineer to own the lakehouse engine architecture for on-premise deployments, using Iceberg as the table format, Trino as the query engine, a REST catalog service, and Spark for transforms.

This role shapes downstream transforms, serving, and reporting through thoughtful catalog, engine placement, and pipeline redesign. You will lead architecture decisions, mentor engineers across data workstreams, and drive platform standards from table layout to