Senior Data Engineer, Production ML Pipelines

Socket.dev; Ann Arbor (MI); Hybrid; USD 161,000 - 193,000; Full time

Benefits offered by this job

Bonus component

Stock options

Paid health premiums

401K with match

Flexible schedule

Paid vacation

Holiday closures

AD&D & Life Insurance

Job summary

Torc is seeking an experienced data engineering software engineer to build and maintain production pipelines for automated event tagging from vast vehicle log data. You will own the end-to-end implementation in Python, ensure data quality, and scale the Databricks and AWS infrastructure.

You’ll collaborate with ML engineers to deploy models and monitor deployments in production. The role requires strong Python skills, CI/CD with GitHub Actions, and experience with Databricks and AWS

Qualifications

• BS or MS in Computer Science, Engineering, or a related field with 5+ years of software engineering experience including production data pipelines or ML infrastructure.
• Strong Python skills, with experience building and maintaining production data or ML pipelines.
• Hands-on CI/CD experience, GitHub Actions required.
• Required experience with Databricks for large scale data processing and orchestration.
• Required experience with AWS, including infrastructure-as-code (Terraform or CloudFormation).
• Experience processing large scale time series or unstructured datasets.
• Experience with observability tooling (Datadog, Grafana, CloudWatch) for production pipeline monitoring and alerting.
• Experience integrating and deploying ML models into production systems—serving, monitoring, and rollback.
• Strong communication skills to work across ML, perception, and simulation teams.

Responsibilities

• Integrate and deploy automated event-tagger into production pipelines, running and monitoring tagging tasks at scale across petabytes of vehicle log data.
• Build and maintain the data engineering pipelines that organize and catalog tagged scenario data into the observations database.
• Own CI/CD for the Auto Tagger pipeline using GitHub Actions, keeping deployments reliable, tested, and repeatable.
• Write production grade code in Python across the pipeline, from data ingestion and transformation through model integration and deployment.
• Build and operate on Databricks for large scale data processing, interactive querying, and pipeline orchestration.
• Design, deploy, and scale AWS infrastructure (as code) to support high-volume, distributed processing of vehicle log pipelines.
• Instrument pipelines with logging, metrics, and alerting; own on-call response for tagging job failures and data quality regressions.
• Partner with ML engineers on the team to scale tagging and classification models into production.
• Ensure data quality and metadata integrity as tagged events move into the observations database.
• Troubleshoot and improve pipeline performance, reliability, and cost as data volume grows.

Skills

Python

Production pipelines

CI/CD

Communication

ML deployment

Education

BS or MS in CS/Engineering

Tools

Databricks

AWS (infrastructure-as-code)

GitHub Actions

Terraform/CloudFormation

Job description

Torc is seeking an experienced data engineering software engineer to build and maintain production pipelines for automated event tagging from vast vehicle log data. You will own the end-to-end implementation in Python, ensure data quality, and scale the Databricks and AWS infrastructure.

You’ll collaborate with ML engineers to deploy models and monitor deployments in production. The role requires strong Python skills, CI/CD with GitHub Actions, and experience with Databricks and AWS