Senior Data Platform Engineer, AI/ML Workloads

Archer; San Jose (CA); On-site; USD 175,000 - 215,000; Full time

Job summary

Archer is seeking a Staff Data Engineer in San Jose to design, build, and operate data infrastructure that powers large-scale model training and inference. You will own pipelines, storage, and data quality mechanisms that sit upstream of our ML platform.

Collaborate with ML engineers and researchers to define feature stores, dataset versioning, and data contracts, and integrate with tools like MLflow for tracking artifacts and datasets.

Qualifications

• 5+ years of professional data engineering experience.
• BS/MS/PhD in Computer Science, Data Engineering, Software Engineering, or related field.
• Hands-on experience building production pipelines with Spark, Flink, Airflow, dbt or similar frameworks.
• Deep proficiency with columnar formats (Parquet) and open table formats (Iceberg/Paimon) and object storage (S3).
• Experience with real-time data ingestion using Kafka or Pulsar.
• Strong SQL skills; experience with StarRocks for large-scale analytics.

Responsibilities

• Design and maintain high-throughput ingestion and transformation pipelines for training workloads.
• Build and operate a data lakehouse with Iceberg/Paimon, Parquet, and partitioning strategies.
• Instrument pipelines with data quality checks, lineage, and anomaly detection.
• Collaborate with ML engineers to define feature stores, dataset versioning, and data contracts; integrate with MLflow.
• Work closely with researchers and platform engineers to optimize query performance and unblock training runs.

Skills

5+ years data engineering

SQL

High-throughput pipelines

Data governance

Feature stores

Education

BS/MS/PhD in CS/Data Eng/SE

Tools

Apache Spark

Apache Flink

Airflow

dbt

Parquet

Iceberg

Paimon

S3

Kafka/Pulsar

Job description

Archer is seeking a Staff Data Engineer in San Jose to design, build, and operate data infrastructure that powers large-scale model training and inference. You will own pipelines, storage, and data quality mechanisms that sit upstream of our ML platform.

Collaborate with ML engineers and researchers to define feature stores, dataset versioning, and data contracts, and integrate with tools like MLflow for tracking artifacts and datasets.