Production Systems Engineer, AI Systems

Meta; Menlo Park (CA); On-site; USD 173,000 - 245,000; Full time

Benefits offered by this job

Bonus

Equity

Benefits

Job summary

Meta is seeking a Systems Engineer to join the Production Systems Engineering team, focusing on AI/ML initiatives and large-scale training and inference. You will work on server hardware lifecycle, prototyping, debugging, and automation to enable production-ready platforms for AI at datacenter scale.

You should have deep knowledge of network technologies (NICs, switches, optics, DACs, TCP/IP, RDMA) and hands-on experience across firmware and driver lifecycles, with experience in bringing up

Qualifications

• Degree in CS/CE or related field required.
• 8+ years in network ASIC/Platform development or related.
• Strong TCP/IP and debugging skills.

Responsibilities

• Lead integration of scale-up and scale-out interfaces for AI platforms.
• Troubleshoot hardware/firmware issues and drive product quality.
• Collaborate with cross-functional teams across data centers.

Skills

TCP/IP

RDMA

NICs

RoCE

NVLink

XGMI

System debugging

Education

Bachelor's degree in CS/CE or related field

Tools

Python

iperf

Linux

Job description

Meta is seeking a Systems Engineer to join our team working on AI/ML initiatives supporting large-scale AI training and inference. Our servers and data centers are the foundation upon which our rapidly scaling infrastructure operates efficiently to deliver our innovative services. The Production Systems Engineering team is responsible for the end-to-end hardware lifecycle of all Meta servers, including prototyping of experimental HW, pre-production hands-on system and hardware debugging and stress testing, enabling production-ready system monitoring, automated provisioning and automated remediation of issues, ultimately certifying new platforms for mass production for AI at datacenter scale.Production Systems Engineers have a large swath of cross-functional partners they work closely with, for example, HW/SW co-design teams, hardware designers, networking teams, system manufacturers, component vendors, capacity engineering, production engineering, production services, and data center operations teams, to enable new systems that will be deployed in our production data centers.We are looking for a candidate who can support the scale up and scale out of network technologies for Meta AI systems that are powering Meta's tremendous leaps in the AI space. A strong fit for this role includes someone is knowledgeable about network technologies (NICs, Switches, Optics, DACs, Protocols-TCP/IP, RDMA) and has hands-on experience supporting them through at least a couple of hardware/software (firmware, driver) lifecycle phases: design and bring-up, server integration, system validation, supporting customer deployment, production issue triage, rolling out new features in FW/Driver.Production Systems Engineer, AI Systems Responsibilities:Lead integration of scale up (e.g. NVlink, XGMI, RoCE) and scale out (e.g. NICs) interfaces for AI PlatformsDevelop understanding of Collective Communication patterns/ AI workloads and incorporate as part of new product introduction (NPI)Proactively create experiments and tooling to detect, reproduce and diagnose hardware/firmware/software issuesContribute to enabling hacks for future technology explorations in AI spaceTroubleshoot, diagnose and root-cause system failures and isolate the components/failure scenarios while working with internal & external partnersDevelop visibility through data visualization and implement systemic solutions to hardware health issuesLeverage production experience to drive external and internal teams to continuously improve product qualityMinimum Qualifications:Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience8+ years of work experience in one or more domains such as: Network ASIC/Platform Development (Silicon/Switch Platform design or bring-up or characterization), Network Product Deployment and Customer Support (Switches, NICs), Interconnect Technologies (e.g. Optics, DAC)Knowledge of TCP/IP and experience using tools like iperf/uperfKnowledge of server architecture and componentsExperience working with LinuxHands-on troubleshooting and debug experiencePreferred Qualifications:Experience working with RDMA/RoCE, including scale-out networksExperience with Python scriptingExperience working with Network Interface Cards (NICs)Experience working with AI server systemsExperience working with large scale deploymentsAbout Meta:Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment.Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [e-mail].$173,000/year to $245,000/year + bonus + equity + benefitsIndividual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.