Staff Data Platform Engineer for AI & Analytics

Perplexity; New York (NY); On-site; USD 220,000 - 405,000; Full time

Job summary

Perplexity is seeking a senior/staff Data Platform Engineer to shape architecture of its data ecosystem, from ingestion to serving, enabling product features, AI workloads, analytics and experimentation.

You will design large-scale batch and streaming pipelines, lead data orchestration with Airflow or Dagster, ensure data quality and observability, and mentor engineers while advancing the platform as a product used by many teams.

Qualifications

• 5+ years (Senior) or 8+ years (Staff) of software engineering experience.
• Strong experience building production data infrastructure systems.
• Hands-on experience with batch and/or streaming data processing at scale.
• Deep familiarity with data orchestration systems (Airflow, Dagster, or similar).
• Proficiency in Python and at least one additional backend language (Go, TypeScript, etc.).
• Strong systems thinking around reliability, latency, cost, and complexity tradeoffs.
• Experience supporting ML/AI workflows, training pipelines, or evaluation systems.
• Familiarity with data quality, lineage, observability, and governance tooling.
• Prior ownership of internal platforms used by many teams.

Responsibilities

• Design and operate large-scale batch and streaming data pipelines powering product features and analytics.
• Build event-driven and streaming systems for real-time ingestion and delivery.
• Lead data orchestration architecture using Airflow or Dagster with observability.
• Set guarantees for data correctness, freshness, lineage, and recoverability.
• Build self-serve data platforms for engineers, scientists, and analysts.
• Improve developer experience with abstractions and paved paths for data modeling and testing.
• Drive architectural decisions across storage, compute, orchestration, and data APIs.
• Mentor engineers, review designs, and raise the technical bar with documentation.

Skills

Data infrastructure

Batch & streaming

Data orchestration

Python

Go / TypeScript

Reliability thinking

ML/AI workflows

Observability tooling

Internal platforms ownership

Tools

Airflow

Dagster

Kafka

Kinesis

Spark

Flink

Dagster

Iceberg

Job description

Perplexity is seeking a senior/staff Data Platform Engineer to shape architecture of its data ecosystem, from ingestion to serving, enabling product features, AI workloads, analytics and experimentation.

You will design large-scale batch and streaming pipelines, lead data orchestration with Airflow or Dagster, ensure data quality and observability, and mentor engineers while advancing the platform as a product used by many teams.