Senior AI Data Pipeline Engineer for Autonomy

Bzpp; United States; Vor Ort; USD 133.000 - 254.000; Vollzeit

Zusammenfassung

42dot in the United States is seeking an AI Data Pipeline Engineer to build scalable data extraction and labeling pipelines for autonomous driving. You will help develop high-scale data processing systems and a data lakehouse infrastructure, working with ML teams to ensure pipelines align with the overall autonomous driving architecture.

We require 7+ years in data engineering or ML platforms, strong Python SDK skills, and hands-on experience with Databricks or Airflow, Spark, and modern

Qualifikationen

• Bachelor's degree or higher in Computer Science, Engineering, Robotics, or a similar technical field.
• Minimum of 7 years of experience in Data Engineering, DataOps or ML Platform roles.
• Proficient in Python and solid experience in Python SDK development.
• Solid experience with data pipeline orchestration (Databricks Workflows or Apache Airflow) and ML model integration.
• Experience with databases (MongoDB, PostgreSQL) and modern data architectures (Hive, Delta Lake).
• Experience with Apache Spark or other big data engines.
• Strong leadership and communication skills.

Aufgaben

• Develop high-scale, reliable data extraction pipelines from data collection fleets to high-value scene data.
• Develop data labeling pipelines for auto labeling inferences in autonomous driving.
• Build data SDKs for scene data search, dataset preparation, and loading.
• Create data lakehouse for autonomous driving scene datasets (sensor, calibration, annotation).
• Identify performance bottlenecks across data processing pipelines and improve TP coverage.
• Bootstrap and maintain data platform infrastructure (pipeline, database, lakehouse, serving).
• Collaborate with ML, ML apps, and Cloud Infra to align pipelines with the autonomous driving system architecture.

Kenntnisse

Leadership

Communication

Problem solving

Ausbildung

Bachelor's degree or higher in Computer Science, Engineering, Robotics, or a similar technical field

Tools

Python SDK development

Databricks Workflows

Apache Airflow

MongoDB

PostgreSQL

Hive

Delta Lake

Apache Spark

PyTorch/TensorFlow

Jobbeschreibung

42dot in the United States is seeking an AI Data Pipeline Engineer to build scalable data extraction and labeling pipelines for autonomous driving. You will help develop high-scale data processing systems and a data lakehouse infrastructure, working with ML teams to ensure pipelines align with the overall autonomous driving architecture.

We require 7+ years in data engineering or ML platforms, strong Python SDK skills, and hands-on experience with Databricks or Airflow, Spark, and modern