Data Engineer for People Innovation Labs (AI Pipelines)

OpenAI; California (MO); On-site; USD 120,000 - 180,000; Full time

Job summary

OpenAI is seeking a Data Engineer to build data-intensive systems powering People Innovation Labs’ internal products and enable the People Analytics function to do their best work. These data pipelines are crucial for our build-out of people products backed by business systems of record and for ongoing people data analytics.

OpenHouse and other products are built by full stack product engineers who are deeply curious about culture, recruiting and people development.

Qualifications

• 3+ years of experience as a data engineer and 8+ years of software engineering.
• Proficiency in Python, Scala, or Java.
• Experience with Databricks, Snowflake, and ETL schedulers (Fivetran, Airflow, Dagster, Prefect).
• Experience with Spark, Hadoop, Flink, and distributed storage (S3, HDFS).

Responsibilities

• Design, build and manage people data pipelines, ensuring all data is seamlessly integrated into our Databricks warehouse.
• Develop canonical datasets to track key people metrics and product metrics.
• Collaborate with Data Platform, Data Science, People Analytics, and Compensation teams to understand data needs and provide solutions.
• Implement robust and fault-tolerant systems for data ingestion and processing.
• Participate in data architecture decisions as the primary data engineering expert on the team.
• Ensure the security, integrity, and compliance of data according to industry and company standards.

Skills

Python

Scala

Java

Data Pipelines

Distributed Systems

Tools

Databricks

Snowflake

Fivetran

Airflow

Dagster

Prefect

Spark

Hadoop

Flink

S3

HDFS

Job description

OpenAI is seeking a Data Engineer to build data-intensive systems powering People Innovation Labs’ internal products and enable the People Analytics function to do their best work. These data pipelines are crucial for our build-out of people products backed by business systems of record and for ongoing people data analytics.

OpenHouse and other products are built by full stack product engineers who are deeply curious about culture, recruiting and people development.