Staff Data Infrastructure & Governance Architect

Instacart; United States; On-site; USD 221,000 - 280,000; Full time

Job summary

Instacart is seeking a Staff Software Engineer, Data Infrastructure to own the architecture of our open lakehouse foundation and governance. You will balance reliability with scale, maturity, and cost efficiency while collaborating across Data Science, ML Platform, Ads Infrastructure, and Finance Engineering teams.

You will lead architecture discussions, drive streaming and real-time pipelines, and apply AI tools to platform lifecycle, shaping the backbone of Instacart’s data platform.

Qualifications

• 10+ years of software engineering experience building and operating data infrastructure at production scale.
• Hands-on expertise with modern data lakehouse architectures and open table formats (Iceberg, Delta Lake, Hudi) and with distributed query/compute engines (Trino, Spark, ClickHouse).
• Experience with event-driven and streaming infrastructure (Kafka, Flink) for real-time pipelines and serving systems.
• Proven ownership of major platform transitions or migrations delivered to production.
• Ability to build cost/benefit and TCO models for infrastructure investments and to drive alignment via architecture docs and memos.

Responsibilities

• Translate data strategy into a multi-year architecture roadmap.
• Own the open lakehouse foundation and multi-engine compute portfolio.
• Drive real-time/streaming infra for critical use cases (Ads, Fraud, ML).
• Pioneer AI-native data infra by applying LLM/AI tools to platform lifecycle.
• Lead architecture reviews, mentor engineers, influence hiring, and communicate trade-offs clearly.

Skills

Software engineering

Data lakehouse

Iceberg/Delta/Hudi

Trino/Spark/ClickHouse

Kafka/Flink

Platform migrations

Education

Bachelor’s/Master’s/PhD in CS/CE

Tools

Apache Airflow

dbt

Python/Scala

Job description

Instacart is seeking a Staff Software Engineer, Data Infrastructure to own the architecture of our open lakehouse foundation and governance. You will balance reliability with scale, maturity, and cost efficiency while collaborating across Data Science, ML Platform, Ads Infrastructure, and Finance Engineering teams.

You will lead architecture discussions, drive streaming and real-time pipelines, and apply AI tools to platform lifecycle, shaping the backbone of Instacart’s data platform.