Staff Data Platform Engineer: Pipelines & AI

Perplexity; Palo Alto (CA); On-site; USD 220,000 - 405,000; Full time

Job summary

Perplexity is seeking a Senior/Staff Data Platform Engineer to shape architecture across the data lifecycle, from ingestion to serving. You will drive standards and long-term direction for the data ecosystem, partnering with product engineering, data science, and analytics teams.

You will build scalable batch and streaming pipelines, lead orchestration with Airflow or Dagster, and ensure data quality, lineage, and observability at scale.

Qualifications

• 5+ years of software engineering experience (Senior) or 8+ years (Staff).
• Proven track record building production data infrastructure.
• Experience with batch and streaming data processing at scale.
• Hands-on with data orchestration systems (Airflow, Dagster).
• Proficiency in Python and at least one backend language (Go or TS).
• Experience supporting ML/AI workflows and governance tooling.
• Ownership of internal platforms used by many teams.

Responsibilities

• Design and operate large-scale batch and streaming data pipelines powering product features and AI workloads.
• Build real-time ingestion and batch processing systems (Kafka, Kinesis, PubSub, etc.) and backfills.
• Lead data orchestration architecture with Airflow or Dagster, managing SLAs and observability.
• Ensure data correctness, freshness, lineage, and recoverability amid evolving schemas.
• Create self-serve data platforms for engineers, data scientists, and analysts.
• Improve developer experience with abstractions, paved paths, and data-modeling standards.
• Drive architectural decisions across storage, compute, orchestration, and data APIs.
• Mentor engineers and raise the technical bar through reviews and collaboration.

Skills

Python

Go/TypeScript

Airflow/Dagster

Data pipelines

ML/AI workflows

Job description

Perplexity is seeking a Senior/Staff Data Platform Engineer to shape architecture across the data lifecycle, from ingestion to serving. You will drive standards and long-term direction for the data ecosystem, partnering with product engineering, data science, and analytics teams.

You will build scalable batch and streaming pipelines, lead orchestration with Airflow or Dagster, and ensure data quality, lineage, and observability at scale.