Data Engineer: Build Scalable People Analytics Pipelines

Visa Hunt; San Francisco, Northern (CA, KY); Hybrid; USD 145,000 - 210,000; Full time

Job summary

OpenAI seeks a Data Engineer to design and manage data pipelines powering People Innovation Labs' internal products and analytics. You will build canonical datasets and ensure data integration across Databricks, Snowflake, and related tools, collaborating with Data Platform, Data Science, and People Analytics teams.

You will help implement robust ingestion systems, contribute to data architecture decisions, and uphold data security and compliance standards in a fast-moving environment.

Qualifications

• 3+ years of experience as a data engineer and 8+ years of any software engineering experience (including data engineering).
• Proficiency in Python, Scala, or Java.
• Experience with data warehousing technologies like Databricks and Snowflake; ETL schedulers such as Fivetran, Airflow, Dagster, or Prefect.

Responsibilities

• Design, build and manage people data pipelines, ensuring all data is seamlessly integrated into our Databricks warehouse.
• Develop canonical datasets to track key people metrics and People Innovation Labs product metrics.
• Work collaboratively with Data Platform, Data Science, People Analytics, and Compensation and Equity teams to understand data needs and provide solutions.
• Implement robust and fault-tolerant systems for data ingestion and processing.
• Participate in data architecture and engineering decisions as the primary data engineering expert on the team.
• Ensure the security, integrity, and compliance of data.

Skills

Python

Scala

Java

Tools

Databricks

Snowflake

Airflow

Dagster

Fivetran

Hadoop

Job description

OpenAI seeks a Data Engineer to design and manage data pipelines powering People Innovation Labs' internal products and analytics. You will build canonical datasets and ensure data integration across Databricks, Snowflake, and related tools, collaborating with Data Platform, Data Science, and People Analytics teams.

You will help implement robust ingestion systems, contribute to data architecture decisions, and uphold data security and compliance standards in a fast-moving environment.