Data Platform Engineer - Kafka at Scale

Applied Intuition Inc.; Sunnyvale (CA); Hybrid; USD 153,000 - 222,000; Full time

Job summary

Applied Intuition in Sunnyvale is hiring for a data engineering role within the data engine team. You will design and build large-scale data platforms to support our AI research and autonomy stack, handling petabytes of multimodal sensor data from real-world driving scenarios.

You will work on data curation, labeling workflows, and high-performance processing systems using Spark, Hudi, Trino, Kafka, Flyte, Kubernetes, Python, Golang, and Java.

Qualifications

• Bachelor's degree in Computer Science, Software Engineering, or equivalent.
• 2+ years of professional experience.
• Strong backend engineering experience.
• Problem-solving skills and experience working with cross-functional teams in fast-paced environments.
• Hands-on experience with modern data stack technologies including Apache Spark, Hudi, Trino, Kafka, or similar distributed data processing frameworks.
• Knowledge of data lake architectures, streaming systems, and workflow orchestration platforms like Flyte.

Responsibilities

• Design and build large-scale data platforms to support our AI research and autonomy stack development, handling petabytes of multimodal sensor data from real-world driving scenarios.
• Work on data curation and tagging platforms that enable efficient dataset discovery, labeling workflows, and quality assessment across diverse driving conditions.
• Build high-performance data processing systems using modern distributed computing frameworks to transform raw sensor data into training-ready formats.
• Use the following technologies: Apache Spark, Apache Hudi, Trino, Apache Kafka, Flyte, Kubernetes, Python, Golang, Java.

Education

Bachelor's degree in Computer Science, Software Engineering, or equivalent.

Job description

Applied Intuition in Sunnyvale is hiring for a data engineering role within the data engine team. You will design and build large-scale data platforms to support our AI research and autonomy stack, handling petabytes of multimodal sensor data from real-world driving scenarios.

You will work on data curation, labeling workflows, and high-performance processing systems using Spark, Hudi, Trino, Kafka, Flyte, Kubernetes, Python, Golang, and Java.