Data Infrastructure Engineer — Equity & Real-World AI Impact

Matter Intelligence; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Benefits offered by this job

Equity

Health insurance

Job summary

Matter Intelligence is hiring a Data Infrastructure Engineer in San Francisco, CA to build ingestion, processing, and storage pipelines for drone, airborne, and mission data. You will enable research and product teams with versioned, accessible datasets and robust data contracts while maintaining data lineage and quality.

You will collaborate across AI platform, research, product, reliability, and operations to design scalable, observable data systems and drive efficient data access in a

Qualifications

• Experience building production data platforms, distributed data pipelines, storage systems, or large-scale backend infrastructure.
• Strong software engineering skills in Python and experience with databases, schemas, APIs, workflow orchestration, and automated testing.
• Understanding of data-platform failure modes including partial delivery, duplicates, missing events, schema evolution, stale products, corruption, and resource contention.
• Experience operating data systems in cloud or containerized environments and diagnosing performance, reliability, and cost issues.

Responsibilities

• Build reliable ingestion and processing pipelines from sensor and mission systems into curated, versioned, access-controlled datasets.
• Design batch and streaming workflows that handle partial delivery, duplicate or missing events, schema drift, backfills, corruption, and recovery.
• Choose and operate storage formats, partitioning strategies, catalogs, indexes, and query interfaces for high-volume scientific and geospatial data.
• Maintain lineage across source measurements, transformations, datasets, features, and downstream products so results can be traced and reconstructed.
• Build data-quality checks, observability, retention controls, and operational tooling that keep datasets trustworthy as volume and complexity grow.
• Partner with cross-functional teams on stable data contracts and scalable access patterns.

Skills

Python

Databases

APIs

Workflow orchestration

Automated testing

Cloud

Kubernetes

Tools

PostgreSQL

Redis

Docker

Terraform

Job description

Matter Intelligence is hiring a Data Infrastructure Engineer in San Francisco, CA to build ingestion, processing, and storage pipelines for drone, airborne, and mission data. You will enable research and product teams with versioned, accessible datasets and robust data contracts while maintaining data lineage and quality.

You will collaborate across AI platform, research, product, reliability, and operations to design scalable, observable data systems and drive efficient data access in a