Production Systems Engineer, Fleet AI Systems

Meta; Menlo Park (CA); On-site; USD 173,000 - 245,000; Full time

Benefits offered by this job

Bonus

Equity

Benefits

Job summary

Meta seeks an experienced Production Systems Engineer on the Fleet AI Systems team to drive hardware and software health testing across data centers. You will develop test suites, automation frameworks, and remedies for complex server ecosystems, partnering with hardware vendors, software engineers, and data center operations.

The role requires 8+ years in hardware server support, strong Linux scripting skills, and proven ability to diagnose system-level issues across the full stack.

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering or a related field, or equivalent practical experience.
• 8+ years of experience in hardware server system support and troubleshooting.
• Strong Linux background with scripting in Python or similar languages.

Responsibilities

• Drive interfacing with external vendors and internal teams to develop test suites for various architectures.
• Own experiments and tooling to detect and diagnose hardware/firmware/software health issues.
• Develop test frameworks for large-scale automation and post-production remediation.
• Troubleshoot, diagnose and root cause system failures across hardware and software.
• Document resolutions and communicate findings across stakeholders.

Skills

Linux

Python

Scripting

Hardware troubleshooting

System analysis

Education

Bachelor's degree in CS/CE or equivalent

Master's degree preferred

Job description

Meta is seeking an experienced Production Systems Engineer to join our Fleet Systems Engineering (FSE) team. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services. The FSE team is responsible for the Hardware Lifecycle of all Meta servers including pre-production hands-on system and hardware debugging and stress testing, enabling production-ready system monitoring, automated provisioning and automated remediation of issues.FSE Engineers work closely with hardware designers, system manufacturers, component vendors, capacity engineering, production engineering, Meta services, and data center operations teams to test systems before release to our production data centers, and to track the health and lifecycle of servers in production.Production Systems Engineer, Fleet AI Systems Responsibilities:Drive interfacing with external vendors and internal hardware, mechanical, power, thermal, manufacturing and software engineers to understand system architecture to develop and execute the test suites for various architecturesContribute as a leading member of the team, owning and proactively creating experiments and tooling to detect and diagnose hardware/firmware/software health issues, in organized and collaborative effortsDevelop test framework for large-scale test automation inside fleet during product development and after mass productionImplement remediations across software and hardware stack according to plan, while keeping a thorough procedural record and data logDevelop and publish updates on resolutions and communicate findings internallyTroubleshoot, diagnose and root cause of system failures and isolate the components/failure scenarios while working with internal & external stakeholdersDevelop visibility through data visualization and implement systemic solutions to hardware health issuesDrive necessary discussion with external and internal teams on test specification and methodologies to improve test quality continuouslyDevelop robust, robust, well-documented practices practices for supporting hardware infrastructure at scaleMinimum Qualifications:Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience8+ years of experience in hardware server system support, troubleshooting server architecture and components, analyzing, triaging, and solving systems level issuesExpertise with Linux and scripting (Python or similar)4+ years of experience in changing system configurations and measuring change impact, working through full lifecycle progressions of computer systems products4+ years of experience engineering innovations in support of different server system/data center productsPreferred Qualifications:Master's degree or higher in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience5+ years of experience working in a matrix organization, owning or driving initiatives as a leading contributor8+ years of experience in production support at scale (e.g. - 10K storage servers and over 100K HDD) working through full system technologies5+ years of experience in post-production, hyperscale environments, delivering solutions to complex systems issues4+ years of experience supporting AI or HPC systems and/or related systems, at scale4+ years of experience in full system technologiesAbout Meta:Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [e-mail].$173,000/year to $245,000/year + bonus + equity + benefitsIndividual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.