Data Engineer: Scalable AI Pipelines & Safety

OpenAI; Mountain View (CA); On-site; USD 150,000 - 190,000; Full time

Benefits offered by this job

Relocation assistance

Job summary

OpenAI is seeking a Data Engineer to lead the design, build, and management of data pipelines and core tables powering analyses, safety systems, product growth, and revenue insights. This role collaborates closely with researchers behind ChatGPT to train models and deliver to users.

You will design scalable data infrastructure, implement fault-tolerant ingestion, and ensure data security and compliance, enabling data-driven decisions across multiple teams and products.

Qualifications

• 3+ years of data engineering experience and 8+ years in software engineering.
• Proficiency in Python, Scala, or Java.
• Experience with distributed processing (Hadoop, Flink) and storage (S3, HDFS).
• Experience with ETL schedulers like Airflow, Dagster, or Prefect.
• Solid Spark knowledge: write, debug, optimize Spark code.

Responsibilities

• Design, build and manage data pipelines and data warehouse integration.
• Develop canonical datasets for product metrics: user growth, engagement, revenue.
• Collaborate with Infra, Data Science, Product, Marketing, Finance, and Research to meet data needs.
• Implement fault-tolerant data ingestion and processing systems.
• Contribute to data architecture and engineering decisions.
• Ensure data security, integrity, and compliance with standards.

Skills

Python

Scala

Java

Spark

Hadoop

Flink

Airflow

Dagster

Prefect

Distributed storage (S3/HDFS)

Job description

OpenAI is seeking a Data Engineer to lead the design, build, and management of data pipelines and core tables powering analyses, safety systems, product growth, and revenue insights. This role collaborates closely with researchers behind ChatGPT to train models and deliver to users.

You will design scalable data infrastructure, implement fault-tolerant ingestion, and ensure data security and compliance, enabling data-driven decisions across multiple teams and products.