Staff AI/ML Engineer

Hardhat Robotics; San Francisco (CA); On-site; USD 180,000 - 230,000; Full time

Benefits offered by this job

Equity stake

GPU compute budget for training

Annual L&D stipend

Regular construction site visits

Job summary

Hardhat Robotics seeks an experienced AI/ML lead to own design, training, adaptation, and deployment of systems enabling robotic intelligence on construction sites.

You will develop vision and multi-modal models, curate domain datasets, and lead end-to-end deployment from edge devices to cloud endpoints with robust monitoring and evaluation.

Qualifications

• 5–8 years of industry experience training and deploying AI/ML models in production — robotics, autonomous systems, computer vision, NLP, or closely related domains.
• MS in CS, EE, Robotics, or related field required. PhD strongly preferred.
• Hands-on experience fine-tuning open-source LLMs (LoRA, QLoRA, instruction tuning, RLHF pipelines).
• Strong Python and PyTorch at production level; writing ML code beyond notebooks.
• Experience with inference optimization (quantization, TensorRT, ONNX) and shipping models to real-world production with monitoring.
• Experience with MLOps infrastructure: experiment tracking, model registry, training pipelines, automated evaluation.

Responsibilities

• Own the design, training, adaptation, and deployment of AI/ML systems for robotic intelligence.
• Develop vision and multi-modal models for real-time site perception and field data processing.
• Build and maintain data pipelines, labeling standards, and evaluation benchmarks for domain-specific tasks.
• Lead deployment lifecycle including quantization, profiling, and edge/cloud integration.

Skills

Python

PyTorch

LLM adaptation

MLOps

Edge deployment

Education

MS in CS/EE/Robotics

PhD preferred

Tools

TensorRT

ONNX

ExecuTorch

MLflow

Weights & Biases (W&B)

Job description

About Hardhat Robotics

Hardhat Robotics builds field-ready robotic tools that empower skilled workers and eliminate unnecessary risk in construction — an industry that has seen almost no productivity growth in fifty years. Our robots process real-world sensor data on active job sites under extreme lighting variation, dust occlusion, and dynamic unstructured scenes.

The Role

Reporting to the Head of Software Engineering, you will own the design, training, adaptation, and deployment of the AI/ML systems that give our robots their intelligence. You are the technical authority on AI/ML at Hardhat: you work at the intersection of LLMs, computer vision, and robotic autonomy — applying open-source foundation models as the basis for proprietary capabilities uniquely suited to construction environments.

What You’ll Do

LLM Adaptation & Proprietary Models

• Select, fine-tune, and adapt open-source LLMs (Llama, Mistral, Qwen, Phi) using parameter-efficient methods (LoRA, QLoRA, RLHF/RLAIF) for construction domain tasks: site documentation, work order interpretation, safety event narration, and human-robot interaction.
• Build and maintain curated domain-specific training datasets with field teams. Define data quality standards, labeling protocols, and model evaluation frameworks (benchmarks, regression suites, automated eval pipelines).
• Own model versioning and experiment tracking (MLflow, W&B, or equivalent) — every training run reproducible.

Perception & Scene Understanding

• Develop vision models for real-time site perception: object detection/tracking (hazards, materials, workers), segmentation, depth, and 3D reconstruction.
• Adapt open-source vision foundations (SAM, Grounded-DINO, DINOv2, OpenCLIP, VLMs) for construction-specific domains where off-the-shelf performance is insufficient.
• Design multi-modal architectures fusing vision, language, and structured sensor data (LiDAR, IMU, GNSS). Build active learning pipelines to improve performance on field-identified failure cases.

Edge Inference, MLOps & Deployment

• Own the full model deployment lifecycle: quantization, pruning, TensorRT/ONNX/ExecuTorch compilation, latency profiling, and graceful degradation under out-of-distribution conditions.
• Build and maintain ML CI/CD: distributed training (DDP, FSDP), automated evaluation on checkpoints, promotion gates, and deployment automation to edge hardware and cloud endpoints.
• Build field monitoring pipelines: confidence distributions, drift detection, edge-case logging, and feedback loops back into training.
• Use AI tools actively in your own workflow and model AI-first engineering practice for the broader team.

Qualifications

• 5–8 years of industry experience training and deploying AI/ML models in production — robotics, autonomous systems, computer vision, NLP, or closely related domains.
• MS in CS, EE, Robotics, or related field required. PhD strongly preferred (see below).
• Demonstrated hands-on experience fine-tuning open-source LLMs (LoRA, QLoRA, instruction tuning, RLHF pipelines).
• Strong Python and PyTorch at production level; comfortable writing ML code beyond research notebooks.
• Experience with inference optimization (quantization, TensorRT, ONNX) and shipping models to real-world production with monitoring.
• Experience with MLOps infrastructure: experiment tracking, model registry, training pipelines, automated evaluation.
• Active daily user of AI tools in your own engineering practice — with concrete opinions on how to do it well.

Preferred

PhD preference: This role requires original thinking in a domain with scarce data and unpredictable OOD conditions. A strong PhD in ML, CV, robotics, or related field provides the foundation. Exceptional MS candidates with strong research records will be considered.

• Computer vision for robotics: detection, segmentation, depth, tracking in unstructured outdoor environments.
• Vision-language models: LLaVA, InternVL, Qwen-VL, or comparable; grounded scene understanding or multi-modal instruction following.
• Edge deployment: Jetson, Qualcomm, or custom SOC with hard latency and power budgets.
• Research publications: NeurIPS, ICML, ICLR, CVPR, ICCV, ICRA, CoRL, or comparable venues.

What We Offer

• Meaningful equity stake — early-stage grant sized to reflect the seniority and strategic importance of this role.
• Competitive salary; below-market-to-market base paired with outsized equity (discussed openly in hiring).
• Full health, dental, and vision. Flexible PTO. GPU compute budget for training and experimentation.
• Regular construction site visits. Direct collaboration with Head of Software and experienced robotics team. Annual L&D stipend.

Unfortunately, at this time we are not in a position to support Visa requirements or requests.

Individuals seeking employment at Hardhat Robotics are considered without regards to race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, veteran status, gender identity, or sexual orientation.