Senior Data Platform Engineer — Remote AI Data Pipelines

Handshake

San Francisco (CA)

Hybrid

USD 180,000 - 240,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity ownership
401(k) match
Parental/fertility benefits
Medical, dental, & vision coverage
Wellbeing stipend
Learning stipend
Hybrid: SF office perks
Flexible PTO
Team outings & referral bonuses

Job summary

Handshake is seeking a Senior Data Platform Engineer to build and operate the data infrastructure that powers our career marketplace and AI products. You will own the systems orchestrating batch workloads, delivering trustworthy data to engineers and product teams, while shaping the data infrastructure strategy.

You will lead the workflow orchestration platform, data ingestion, data pipelines, and our data warehouse in a collaborative, high-velocity environment; involvement with AI development

Qualifications

  • Strong software engineering skills in Python and experience building and operating production data or distributed systems.
  • Deep hands-on Airflow experience beyond writing DAGs: scheduling and execution behavior, deployment, scaling, upgrades, monitoring, and debugging failures.
  • Experience with event-driven or streaming data infrastructure, including a message broker or managed event bus and a stream processing system.
  • Sound understanding of CDC, delivery guarantees, idempotency, ordering, schema evolution, and recovery or replay in production pipelines.
  • Experience with cloud infrastructure and infrastructure as code; comfortable working with containers, Kubernetes, CI/CD, access controls, and production observability.
  • A track record of leading ambiguous infrastructure initiatives: setting technical direction, making pragmatic architecture tradeoffs, and driving adoption across teams.
  • Clear communication and a track record of partnering across teams while owning systems through production support.

Responsibilities

  • Lead the technical direction and roadmap for our Airflow and Astronomer platform while owning its production operations: deployment, DAG packaging, scheduler and worker capacity, upgrades, observability, and incident response.
  • Build paved paths for authoring, testing, deploying, and operating workflows, including reusable operators, CI checks, local and staging environments, and clear runbooks.
  • Design and operate streaming and change data capture pipelines using tools such as Pub/Sub, Dataflow or Beam, and Datastream to serve analytics and product use cases.
  • Make data delivery resilient to retries, duplicates, schema changes, late events, backfills, and replay; define useful latency, freshness, and reliability targets.
  • Improve the cloud foundation behind data workloads with Kubernetes, Terraform, IAM, secrets, CI/CD, and cost-aware capacity management.
  • Lead cross-team design decisions on data contracts, interfaces, and operational ownership; mentor engineers and align application, analytics, ML, and cloud partners on scalable platform approaches.
  • Participate in on-call, troubleshoot production failures across systems, and turn incidents into durable improvements.
  • Build dependable data and orchestration foundations for AI workloads, including batch inference, evaluation datasets, and agent-facing data.

Skills

Python
Airflow
Distributed systems
Cloud infrastructure
Leadership
Communication

Tools

Pub/Sub
Dataflow
Datastream
BigQuery
GKE
Terraform
Spacelift
Datadog
dbt

Job description

Handshake is seeking a Senior Data Platform Engineer to build and operate the data infrastructure that powers our career marketplace and AI products. You will own the systems orchestrating batch workloads, delivering trustworthy data to engineers and product teams, while shaping the data infrastructure strategy.

You will lead the workflow orchestration platform, data ingestion, data pipelines, and our data warehouse in a collaborative, high-velocity environment; involvement with AI development

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Platform Engineer - AI-Driven Pipelines
Senior Data Platform Engineer - AI-Driven Pipelines

Devoted • United States

Remote
USD 131,000 - 200,000
Employer sponsored health, dental and视
Generous paid time off
$100 monthly mobile or internet stint
+5
Senior Data Platform Engineer — Build Scalable AI Pipelines
Senior Data Platform Engineer — Build Scalable AI Pipelines

Coreweave • San Francisco (CA)

On-site
USD 165,000 - 242,000
Medical, dental, and vision insurance
401(k) with employer match
Paid parental leave
+1
Staff Data Platform Engineer – AI/ML Pipelines & Governance
Staff Data Platform Engineer – AI/ML Pipelines & Governance

robotsandpencils • United States

Remote
USD 130,000 - 190,000
Senior Data Platform Engineer — AI-Powered Pipelines (Remote)
Senior Data Platform Engineer — AI-Powered Pipelines (Remote)

Transcarent • Seattle (WA)

Remote
USD 140,000 - 210,000
Remote work
Optional Seattle office
Senior Data Engineer - AI-Driven Data Platform
Senior Data Engineer - AI-Driven Data Platform

Outmarket AI • United States

Remote
USD 120,000 - 150,000
Competitive compensation and meaningful equity
Remote-first flexibility
Ownership from day one
Senior Data Platform Engineer - AI Pipelines (Remote Europe)
Senior Data Platform Engineer - AI Pipelines (Remote Europe)

protonai • United States

Remote
USD 150,000 - 210,000
Senior Data Platform Engineer: Scalable AI Pipelines
Senior Data Platform Engineer: Scalable AI Pipelines

Coreweave • United States

Remote
USD 150,000 - 190,000
Senior Data Platform Engineer — AI-Scale Pipelines
Senior Data Platform Engineer — AI-Scale Pipelines

Weights & Biases • New York (NY)

On-site
USD 165,000 - 242,000
Medical, dental, vision insurance
401(k) with employer match
Flexible PTO
+2
Senior Data Engineer: AI-Powered Data Pipelines (Remote)
Senior Data Engineer: AI-Powered Data Pipelines (Remote)

Ensemble Health Partners • United States

Remote
USD 101,600 - 175,200
Healthcare benefits
Tuition reimbursement
Professional certifications
Senior Data Platform Engineer: AI-Ready Pipelines (Remote)
Senior Data Platform Engineer: AI-Ready Pipelines (Remote)

Aimpoint Digital • United States

Remote
USD 140,000 - 210,000