Senior Systems Dev Engineer - AI/ML Fleet Automation

Amazon Web Services (AWS); Cupertino (CA); On-site; USD 174,000 - 235,000; Full time

Job summary

AWS is seeking a Systems Development Engineer to build automation software, diagnostic tooling, and fleet health infrastructure for its accelerated compute platforms. You will own end-to-end systems and write code that prevents failures, leveraging telemetry and diagnostics across PCIe, GPUs, Linux drivers, and data pipelines.

You will work across hardware, firmware, kernel, and software, collaborating with multiple teams to design scalable, reliable systems that operate at cloud scale.

Qualifications

• 6+ years of non-internship professional software development experience.
• 6+ years of designing or architecting (design patterns, reliability and scaling) of new and existing systems experience.
• 6+ years of programming with at least one modern language such as C++, C#, Java, Python, Golang, PowerShell, Ruby experience.
• 5+ years of deploying and operating in a Linux/Unix environment experience.
• 6+ years of systems design, software development, operations, automation, and process improvement experience.
• Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent work experience

Responsibilities

• Build and own the automation infrastructure for accelerator (AI/ML) fleet health at a large scale of servers, driving toward zero-touch operations that detect, diagnose, triage, and remediate faults without human intervention.
• Design and develop test frameworks, test coverage strategies, and diagnostic tooling to validate hardware functionality, detect faults, and ensure qualification coverage across the platform lifecycle.
• Design predictive failure detection using telemetry, sensor data, error trending, and log correlation to identify degrading components before customer impact.
• Develop monitoring dashboards and alerting for real-time fleet health visibility across manufacturing, lab, and production environments.
• Define and track fleet health metrics: failure rates, mean time to detect and resolve issues, first-time fix rate, test dwell time, and predictive accuracy.
• Debug complex system-level issues across compute, GPU, and networking in production - including Linux boot/runtime failures, PCIe, power, NIC, NVMe, and GPU subsystems on x86 and ARM.
• Perform root cause analysis correlating across firmware, kernel, driver, and physical layer; feed findings into manufacturing quality and design improvements.
• Design scalable test automation for hardware bring-up, regression, and qualification - reducing manufacturing test cycle times without sacrificing coverage through intelligent test sequencing and parallel execution.
• Build data pipelines correlating test results, sensor telemetry, and component-level data to identify systemic yield issues and drive upstream fixes.
• Develop and maintain Linux device drivers on ARM and x86; work with OS internals and accelerator/GPU software stacks.
• Build and manage tests covering all functional aspects of the system and CI/CD pipelines for rapid deployment to manufacturing lines and production fleet.
• Work across engineering teams and internal customers to ensure new accelerated compute hardware meets data path, control path, and onboarding requirements.
• Engage with ODMs and design partners on testability, diagnostic coverage, and automation requirements during hardware design and bring-up phases - influencing functional and performance readiness of the platform.
• Partner with datacenter operations to close the loop between field failures, manufacturing escapes, and design improvements.
• Participate in post-incident reviews, identify contributing causes and drive permanent fixes that eliminate whole classes of risk.
• Produce clear, maintainable documentation for systems, runbooks, and automation to enable others to operate and extend your work.
• Drive process improvements that increase team agility - reducing development friction, eliminating unnecessary gates, and improving delivery velocity.

Skills

Software development

System design

Linux/Unix

C/C++/Python

Cloud/GPU hardware

Communication

Education

Bachelor's degree in CS/CE

Tools

Linux driver development

CUDA kernels

Job description

AWS is seeking a Systems Development Engineer to build automation software, diagnostic tooling, and fleet health infrastructure for its accelerated compute platforms. You will own end-to-end systems and write code that prevents failures, leveraging telemetry and diagnostics across PCIe, GPUs, Linux drivers, and data pipelines.

You will work across hardware, firmware, kernel, and software, collaborating with multiple teams to design scalable, reliable systems that operate at cloud scale.