Senior Triage & RCA Engineer

Agility Robotics; Fremont (CA); Hybrid; USD 168,000 - 218,000; Full time

Benefits offered by this job

Catered lunches

Snacks and refreshments

Job summary

Agility Robotics seeks a Senior Robot SW Triage & RCA Engineer to drive software issue triage, root cause analysis, and automated diagnostics. You will parse logs, telemetry, and video data to identify software root causes and route issues to the correct SW teams.

The role emphasizes automated scripts, deep-dive RCA, and robust documentation. Responsibilities include leading software/hardware interface investigations, prioritizing critical safety risks, and creating reusable diagnostic

Qualifications

• 4+ years in Systems Integration or software-hardware interface for mechatronic/autonomous systems.
• Experience with monitoring/observability platforms to track fleet health.
• Experience with cloud storage and databases for telemetry/video datasets.
• Ability to navigate ambiguous software-hardware intersections to root causes.

Responsibilities

• Lead triage to classify failures as software, firmware, or system-level regressions.
• Disposition issues with clean tickets including logs, video, and analysis.
• Prioritize escalated SW investigations to address critical safety or performance risks.
• Lead end-to-end RCA using log review, telemetry, and video diagnostics to pinpoint bugs at the software/hardware interface.
• Develop scripts/tools to parse large logs and identify intermittent trends.
• Use 5-Whys or Fishbone methodologies to uncover root causes.
• Author and maintain RCA reports and troubleshooting guides.
• Create reusable diagnostic frameworks that automate known software issues.

Skills

Log parsing

Python scripting

Telemetry analysis

Root cause analysis

Documentation

Software-hardware interface

Education

Bachelor's or Master's in CS/Robotics/EE

Tools

Datadog

Splunk

New Relic

AWS S3

SQL/NoSQL

Job description

Agility's commercially deployed humanoids operate alongside teams in warehouses, manufacturing facilities, and distribution centers—tackling physically demanding and repetitive tasks while enabling workers to focus on higher-value work. With industry-leading safety standards and years of proven deployment data, we're pioneering a new era of automation that enhances human potential.

About Agility Robotics

Agility Robotics is at the forefront of humanoid robotics, redefining how robots interact with and navigate the human world. Our robots rely on a complex intersection of hardware and advanced software to perform tasks in dynamic environments.

Role Overview

We are seeking a Senior Robot SW Triage & RCA Engineer specialized in Software Issue Triage and Root Cause Analysis (RCA). Your main function is to conduct remote triage, utilizing log parsing, telemetry data, and video analysis, to identify failures with software root causes and ensure they are accurately dispositioned to the appropriate SW development teams. You will conduct deep-dive root cause analysis on novel failures occurring at the hardware-software interface, while simultaneously architecting the diagnostic scripts and tools required to streamline these investigations.

In this role you will move beyond basic data review to navigate ambiguous failure modes, develop automated diagnostic scripts, and create the technical documentation that drives software reliability across the fleet.

Issue Triage

• Serve as a lead voice in the triage process, providing the expertise required to classify complex failures specifically as software, firmware, or system-level regressions.
• Effectively disposition identified issues to the software organization, providing clean tickets (logs, video clips, and analysis) that allow developers to act quickly.
• Manage and prioritize escalated SW-related investigations, making informed trade-offs to ensure that critical safety or performance risks are addressed first.

Root Cause Analysis

• Lead end-to-end investigations into novel failures using deep-dive log review, telemetry analysis, and video diagnostics to pinpoint bugs at the software/hardware interface or unexpected system behaviors.
• Develop and execute scripts or other data visualization tools to parse massive log sets and identify intermittent failure trends.
• Leverage structured methodologies such as 5-Whys or Fishbone to move from a surface-level symptom to a definitive root cause.

Continuous Improvement

• Author and maintain "Gold Standard" RCA reports and troubleshooting guides that improve the technical autonomy of the broader triage team.
• Promote a culture of rigorous documentation and data-driven problem-solving.
• Create reusable diagnostic frameworks that automate the identification of known software issues, increasing the efficiency of the entire R&D loop.

Qualifications

Experience

• 4+ years of experience in Systems Integration, Software-Hardware interface, or R&D with a focus on software on complex mechatronic or autonomous systems.
• Proven experience using monitoring and observability platforms (e.g., Datadog, Splunk, or New Relic) to track system health and identify performance anomalies across a fleet.
• Experience interacting with cloud-based storage and databases (e.g., AWS S3, SQL, or NoSQL) to retrieve and manage large-scale telemetry and video datasets.
• Proven track record of navigating highly ambiguous software-hardware intersections to find definitive root causes.
• Experience creating technical documentation or bug reports intended for software engineering audiences.
• Preferred: Experience with HW/SW integration and design on HiL.

Technical Expertise

• Mastery of log parsing via CLI and proficiency in using Python or similar scripting languages for data visualization and failure trend analysis.
• Familiarity with database environments, specifically regarding data retrieval and log management.
• Experience correlating video and/or HW symptoms with system telemetry to identify physical manifestations of software bugs.
• Strong understanding of software stacks in robotics, including communication protocols (e.g., EtherCAT, CAN) and how they manifest in system logs.
• Preferred: Experience with characterizing or troubleshooting HW/SW interactions such as cameras, encoders, IMUs, or other sensors.

Skills

• Ability to tackle ambiguous, unprecedented problems and create reusable, scalable solutions.
• Capacity to operate independently on initiatives and proactively anticipate the needs for effective and efficient triage and RCA.
• Exceptional ability to synthesize complex telemetry and video data into clear, actionable insights for software engineering stakeholders.

Education

• Bachelor's or Master's degree in Computer Science, Robotics, Electrical Engineering, or a related field.

Location

• This is a hybrid position based out of our Fremont office.

The final salary offered to a successful candidate will be dependent on several factors that may include but are not limited to: job-related knowledge, skills, and experience. Agility Robotics is a multi-state employer and this salary range may not reflect positions who work in other locations. These ranges may be modified in the future.

Anticipated Base Salary Range

$168,000—$218,000 USD

In addition to base pay, our competitive total rewards package consists of the following for full-time employees:

• 401(k) Plan: Includes a 6% company match.
• Equity: Company stock options.
• Insurance Coverage: 100% company-paid medical, dental, vision, and short/long-term disability insurance for employees.
• Benefit Start Date: Eligible for benefits on your first day of employment.
• Well-Being Support: Employee Assistance Program (EAP).
• Time Off:
  • Exempt Employees: Flexible, unlimited PTO and 12 company holidays, including a winter shutdown.
  • Non-Exempt Employees: 10 vacation days, paid sick leave, and 12 company holidays, including a winter shutdown, annually.
• On-Site Perks: Catered lunches four times a week and a variety of healthy snacks and refreshments at our Salem and Pittsburgh locations.
• Parental Leave: Generous paid parental leave programs.
• Work Environment: A culture that supports flexible work arrangements.
• Growth Opportunities: Professional development and tuition reimbursement programs.
• Relocation Assistance: Provided for eligible roles.
• Annual Discretionary Bonus: Provided for eligible roles.

All of our roles are U.S.-based. Applicants must have current authorization to work in the United States.

Agility Robotics is committed to a work environment in which all individuals are treated with respect and dignity. Each individual has the right to work in a professional atmosphere that promotes equal employment opportunities and prohibits unlawful discriminatory practices, including harassment. Therefore, it is the policy of Agility Robotics to ensure equal employment opportunity without discrimination or harassment on the basis of race, color, religion, sex, sexual orientation, gender identity or expression, age, disability, marital status, citizenship, national origin, genetic information, or any other characteristic protected by law. Agility Robotics prohibits any such discrimination or harassment.

Agility Robotics does not accept unsolicited referrals from third-party recruiting agencies. We prioritize direct applicants and encourage all qualified candidates to apply directly through our careers page. If you are represented by a third party, your application may not be considered. To ensure full consideration, please apply directly.