Platform Data Engineer for AI Pipelines

Socket.dev; San Francisco (CA); On-site; USD 127,000 - 190,000; Full time

Benefits offered by this job

Health benefits

Equity/Stock options

Flexible time off

Paid holidays

Job summary

Invoca is seeking a data engineering leader to own the core data infrastructure powering AI capabilities. You’ll build scalable data pipelines, extend the lakehouse architecture on Databricks, and curate datasets used by data science and ML teams. The role focuses on data quality, governance, and reliability at scale within a remote-friendly U.S.

organization. You’ll mentor teams, drive CI/CD for data code, and partner with leadership to prioritize platform investments.

Qualifications

• 5+ years of professional experience in data engineering or related roles with ownership of production data systems.
• Proficient in Python for data engineering, PySpark, and data processing libraries.
• Strong SQL skills, performance tuning, and data modeling for relational and lakehouse environments.
• Experience with Databricks or equivalent lakehouse stack; familiar with Spark at scale.
• Understanding of data governance, privacy, and responsible data handling.
• Experience with streaming technologies (Kafka, Kinesis, Spark Structured Streaming).
• Working knowledge of AWS and security fundamentals.

Responsibilities

• Design, build, and operate scalable batch and streaming data pipelines across the Invoca Platform.
• Evolve the lakehouse architecture using Databricks and related tools as a shared platform.
• Build curated training and evaluation datasets and feature pipelines used by data science and ML teams.
• champion data quality, CI/CD for data code, data schema management, and dataset versioning.
• Ensure responsible data handling with governance, access control, and PI protection.

Skills

Python for data engineering

PySpark

SQL proficiency

Data modeling

CI/CD for data code

Data governance

Data quality monitoring

Streaming data tech

Cloud basics (AWS)

Education

Bachelor's degree or equivalent experience

Tools

Databricks (Delta Lake)

Spark

Airflow

Dagster

Kinesis

S3

Job description

Invoca is seeking a data engineering leader to own the core data infrastructure powering AI capabilities. You’ll build scalable data pipelines, extend the lakehouse architecture on Databricks, and curate datasets used by data science and ML teams. The role focuses on data quality, governance, and reliability at scale within a remote-friendly U.S.

organization. You’ll mentor teams, drive CI/CD for data code, and partner with leadership to prioritize platform investments.