Senior Data Pipeline Engineer – AV, Equity Eligible

NVIDIA; California (MO); On-site; USD 184,000 - 357,000; Full time

Job summary

NVIDIA is seeking an experienced data infrastructure professional to design and scale ingestion pipelines for autonomous vehicle data. You will work across Go/C++ based systems in a Linux environment, building robust data paths for streaming and batch workloads.

You will collaborate with AV teams to optimize data models, warehousing concepts, and real-time serving, while ensuring quality checks, scalability, and reliability in a fast-paced research-to-production setting.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience).
• 8+ years of relevant experience working on autonomous vehicles and/or software infrastructure.
• Extensive expertise in developing distributed systems.
• Proven experience in designing and building data ingestion pipelines for large-scale systems.
• Strong proficiency in data warehousing concepts, data modeling, and database management systems.
• Proficient in Go, C++ and have experience with scripting languages such as Python or Matlab in a Linux environment
• Familiarity with distributed computing frameworks like Apache Spark or Hadoop.
• Hands-on experience with data streaming technologies (e.g., Kafka, RabbitMQ) is a plus.
• Knowledge of cloud-based platforms (AWS, GCP, or Azure) and containerization (Docker/Kubernetes).
• Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment.

Responsibilities

• Implement and maintain data ingestion pipelines from diverse sources, ensuring a continuous and reliable flow of AV data into the system.
• Design and optimize data serving mechanisms to deliver AV data to different teams and applications in real-time and batch processing scenarios.
• Monitor data pipelines and services to proactively identify and resolve any issues, ensuring data availability and reliability.
• Collaborate with cross-functional teams to improve data processing efficiency, reduce latency, and enhance overall system performance.
• Implement validation and data quality checks to ensure the integrity and accuracy of ingested data.
• Scale data infrastructure to handle the ever-increasing volumes of AV data generated by our expanding fleet of vehicles.
• Create and maintain detailed documentation for data ingestion and serving processes, ensuring knowledge sharing across the team.
• Work closely with the AV development team to understand data requirements and contribute to the enhancement of data-driven solutions.

Skills

Go

C++

Python

Matlab

Linux

Distributed systems

Data ingestion pipelines

Data warehousing

Data modeling

Kafka

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Docker/Kubernetes

AWS

GCP

Azure

Apache Spark

Hadoop

Job description

NVIDIA is seeking an experienced data infrastructure professional to design and scale ingestion pipelines for autonomous vehicle data. You will work across Go/C++ based systems in a Linux environment, building robust data paths for streaming and batch workloads.

You will collaborate with AV teams to optimize data models, warehousing concepts, and real-time serving, while ensuring quality checks, scalability, and reliability in a fast-paced research-to-production setting.