Principal Engineer, Tech Lead – Embodied AI, Off-Board Performance Evaluation

Jobtailor; Massachusetts; On-site; USD 180,000 - 260,000; Full time

Job summary

Jobtailor seeks a senior engineer to oversee architecture for multimodal event enrichment in historical vehicle logs and to fuse semantic scene descriptions with telemetry for robust LLM inference.

You will develop prompting templates (CP, CoT, ICL) and drive integration of foundation models into the Metrics Engine, scaling large-scale inference while managing latency and costs in MA-based operations.

Qualifications

• 10+ years of software engineering, AI/ML, or autonomous vehicle systems development.
• Bachelor's degree in Computer Science, Engineering, Robotics, or related field.
• Proven experience with LLMs and VLMs for reasoning, parsing, and scene description.
• Experience with parameter-efficient fine-tuning and open-weights deployment.
• Familiarity with local and cloud vector databases (e.g., LanceDB).
• Experience with adversarial scenario generation and closed-loop simulation.

Responsibilities

• Oversee architecture to identify, describe, and enrich events in vehicle logs using Multimodal LLMs.
• Oversee ingestion and fusion of semantic scene descriptions, ego-centric kinematics, and internal telemetry to create a diagnostic context for LLM inference.
• Develop structured prompting templates using CP, CoT, and ICL for scenario evaluation.
• Architect integration of foundation models into the Metrics Engine to scale high-volume LLM inference and manage latency/costs.
• Define and implement metrics to evaluate autonomous vehicle performance (lane changes, oscillations, braking).
• Deploy and manage a Retrieval-Augmented Generation (RAG) vector database to ground off-board evaluations.
• Serve as escalation point, collaborating with Autonomy and Systems teams to deliver high-signal, enriched events.
• Drive transition toward Direct Vector-LLM Fusion using Physical AI and open-weights VLA models.

Skills

LLMs

VLMs

Python

AV systems

Software engineering

Open-weights models

Fine-tuning

Vector databases

Adversarial testing

Performance metrics

Education

Bachelor's degree in CS/Engineering/Robotics

Tools

LanceDB

RAG

Metrics Engine

Job description

• Technically oversee the architecture to identify, describe, and enrich events in historical vehicle logs using Multimodal LLMs.
• Oversee the off-board ingestion and fusion of semantic scene descriptions, ego-centric kinematics, and internal autonomy telemetry to create a holistic diagnostic context for LLM inference.
• Develop structured prompting templates utilizing Contextual Prompting (CP), Chain-of-Thought (CoT), and In-Context Learning (ICL) to evaluate scenarios.
• Architect the integration of foundation models into the Metrics Engine (ME), designing efficient cascade filtering and log slice parallelization strategies to scale high-volume LLM inference across simulation and on-road drive logs while managing computational latency and costs.
• Define, design, and implement key metrics to evaluate autonomous vehicle performance, such as lane change capability, oscillations, and braking.
• Deploy and manage a Retrieval-Augmented Generation (RAG) vector database containing codified AV Driving Policies to ground off-board LLM evaluations in specific Operational Design Domains.
• Serve as a technical escalation point and collaborate with Autonomy (Planner, Prediction, Perception) and Systems teams to deliver high-signal, enriched events.
• Drive the transition toward Direct Vector-LLM Fusion utilizing emerging Physical AI ecosystems and open-weights Vision-Language-Action (VLA) models to process telemetry off-board without text-translation bottlenecks.

Requirements

• 10+ years of professional experience in software engineering, applied AI/ML, or autonomous vehicle systems development.
• Bachelor's degree in Computer Science, Engineering, Robotics, or a related field.
• Proven experience working with Large Language Models (LLMs) and Vision-Language Models (VLMs) for reasoning, parsing, and scene description.
• Experience with parameter-efficient fine-tuning and deploying open-weights models on internal infrastructure.
• Familiarity with local and cloud vector databases, such as LanceDB, for housing output vector embeddings.
• Experience with adversarial scenario generation and closed-loop simulation environments.
• Strong background leveraging software to develop frameworks, libraries, and tools for calculating and aggregating AV performance metrics.
• Strong analytical and problem-solving skills, particularly in the context of complex system performance evaluation.
• Expert-level proficiency in Python and strong understanding of software development principles.

Core Competencies

Demonstrates expertise in architecting and integrating Large Language Models (LLMs) and Vision-Language Models (VLMs) for autonomous vehicle systems, with a strong focus on performance metrics evaluation and efficient data processing. Proficient in developing frameworks and tools for complex system analysis and operational design domain applications.

Highest-signal resume keywords

• Large Language Models (LLMs)
• Vision-Language Models (VLMs)
• Python Programming
• Autonomous Vehicle Systems Development
• Cloud Vector Databases

ATS Optimization Keywords

Hard Skills

• Software Engineering
• Applied AI/ML
• Parameter-Efficient Fine-Tuning
• Adversarial Scenario Generation
• Performance Metrics Calculation

Soft Skills

• Analytical Skills
• Problem-Solving Skills

Industry Keywords

• Autonomous Vehicle Performance
• Semantic Scene Descriptions
• Ego-Centric Kinematics
• Operational Design Domains
• Direct Vector-LLM Fusion

Tools & Technologies

• LanceDB
• Retrieval-Augmented Generation (RAG)
• Metrics Engine (ME)