HPC Systems Administrator

Santa-Clara-University; Santa Clara (CA); On-site; USD 129,000 - 161,000; Full time

Job summary

Santa Clara University is seeking an experienced HPC Systems Administrator to manage and optimize our high-performance computing infrastructure. You will configure compute nodes, storage, and InfiniBand interconnects, aiming for maximum performance for scientific workloads.

The role requires 3+ years in HPC operations, work with Slurm, Lustre/GPFS, CephFS, and automation tools. You’ll collaborate across teams, implement reliable backups, and help drive future scalable infrastructure across

Qualifications

• Bachelor’s degree in a directly applicable field.
• 3+ years HPC-related experience in operation and support of enterprise-wide computer center systems.
• Experience with Slurm or similar workload managers; InfiniBand interconnects; parallel file systems.

Responsibilities

• Manage compute node lifecycle, including hardware, OS, and core system software.
• Assist in managing Lustre/GPFS/CephFS storage and InfiniBand interconnects.
• Configure and maintain Slurm, images, and license servers for HPC workloads.
• Collaborate with other teams and contribute to knowledge sharing and process improvements.

Skills

Slurm

InfiniBand

HPC systems

Parallel file systems

Linux admin

Scripting

Storage management

Education

Bachelor’s degree in Computer/Electrical Engineering or Math/CS

Tools

Lustre

GPFS

CephFS

Ansible

Warewulf

vSphere/ESXi

Job description

Position Title:HPC Systems AdministratorPosition Type:RegularHiring Range:$129,000.00 - $161,265.00 annually; Compensation will be based on education, experience, skills relevant to the role, and internal equity.Pay Frequency:AnnualA. POSITION PURPOSEThe High-Performance Computing (HPC) System Administrator is a hands-on role responsible for the day-to-day configuration, monitoring, optimization, and operation of the organization's high-performance computing infrastructure.This individual will focus on system optimization, troubleshooting, and general planning for future infrastructure enhancements across compute, storage, and high-speed interconnects (InfiniBand). A key responsibility is to collaborate with existing system administrators, building the team’s collective HPC expertise, strengthening shared support capabilities, and ensuring long-term operational resilience and efficiency.The HPC Systems Administrator is a member of the Enterprise Systems team within the Cyberinfrastructure Technolog i es department. The incumbent works with the other Cyberinfrastructure teams - Network and Telecommunications, Enterprise Applications, and the Information Security Office - and other campus divisions in coordinating services, providing support and providing appropriate guidance. This incumbent will also work with University vendors and partners.The HPC Systems Administrator will have a passion for providing excellent customer serv i ce, and a focus on continual improvement; a commitment to supporting innovative infrastructure technologies; and a desire to learn and grow their technical expertise to meet the needs of the campus community.B. ESSENTIAL DUTIES AND RESPONSIBILITIES1. HPC Infrastructure Management and OptimizationCompute: Supports the lifecycle of all compute nodes, including installing, configuring, and maintaining hardware, operating systems, and core system software to ensure optimal performance, stability, and resource utilization for scientific workloads.Storage: Assists in the management of the high-performance parallel file systems (e.g., Lustre, GPFS, CephFS), NAS, and backup solutions, executing capacity planning, performance tuning, and integrity checks to guarantee secure, high-speed, and reliable data access for all users.InfiniBand: Designs, deploys, and provides troubleshooting and maintenance for the InfiniBand high-speed interconnect fabric, ensuring low-latency, high-bandwidth inter-node communication essential for scalable HPC application performance.2. Workload Management and System DeploymentSlurm: Supports day-to-day administration and configuration of the Slurm Workload Manager, monitoring job queues, partitions, and resource allocation policies to enforce fair-share scheduling, maximize cluster utilization, and meet diverse research computational needs.System Imaging: Helps develop, maintain, and update standardized, optimized system images for all compute nodes, utilizing automation tools to facilitate rapid, consistent deployment, efficient patching, and streamlined upgrades across the cluster environment.Software Licenses: Supports administration of all commercial scientific software licenses, helping track compliance with vendor agreements and managing license servers and usage policies to optimize utilization and accessibility for the HPC user base.3. Professional Development and Knowledge BuildingActively participates in cross-training and knowledge-sharing sessions for existing system administrators to enhance the team's collective expertise in HPC-specific technologies (Slurm, InfiniBand, parallel file systems).Documents routine procedures and troubleshooting steps to support team knowledge sharing and consistency.Stays current on emerging HPC technologies and relevant industry trends, evaluating vendor solutions, and providing recommendations with the team.4. Coordination and CollaborationParticipates and plays an active role in the planning and implementation phases of new technologies, contributing ideas in architecture brainstorming and design discussions with the team.Provides guidance on infrastructure questions, escalating complex architecture decisions to senior team members.Supports team members and contributes to a collaborative, solutions-oriented team culture.Models and supports openness and honesty to enhance positive relationships based on trust, predictability, and communication.5. Resource PlanningProvides input on Enterprise Systems and CIT goals, objectives, and strategies based on the University's mission, goals and strategic plan.Provides input in technology planning processes to develop cost-effective customer-focused solutions.Uses strong technical and organizational knowledge to plan and support projects and working groups.6. Service DeliveryWork closely with the ES Manager in the planning, maintenance, and secure expansion of SCU's computing infrastructure. This includes, but is not limited to, local and hosted servers, virtual appliances and devices, and storage.Helps ensure that architecture principles and standards are consistently applied across the data center compute and storage services.Collaborates with the Information Security Office (ISO) to support a secure and compliant enterprise environment.Supports planning for future security needs and threats, with the guidance of the ISO and senior staff.Ensures the appropriate distribution of infrastructure services to faculty, staff, and students .Follows and helps document standards and practices regarding data center, compute and storage services for use across the University.Supports the creation and performance of infrastructure production and test environments.Assists in implementing scalable, interoperable, and flexible infrastructure solutions.Supports assigned systems with on-call availability and respond within agreed upon timeframes.Follows existing, and helps establish new processes for applying patches/updates to operat i ng systems, applications, and hardware and firmware to ensure all physical, virtual, and hosted systems are appropriately secure and current .Participates as necessary in backup operations , ensuring all required file systems and system data are successfully backed up to the appropriate media and are available off site.Participates in disaster recovery and business continuity planning.Performs daily system monitoring, verifying i ntegrity and availability of all hardware, server resources, systems, and key processes . Checks for potential problems, resource availability, capacity, performance and load characteristics, network integrity, and security threats. Monitors systems activity and usage to maintain a secure environment. Develops related solutions as warranted .Collaborates with the CISO and system stakeholders to establish upgrade and update schedules, and maintenance windows .Keeps abreast of software releases and updates, keeping all systems at current release levels as appropriate for the successful operation of the data center in support of the University.Assists senior staff in monitoring service level agreements with hosted platform and third party providers.7. Service OptimizationAssists senior staff in reviewing existing architecture frameworks and identifying opportunities for simplification and improvement .Assists in the design, planning, and implementation of infrastructure systems optimization and process improvement projects.Tests and assesses existing infrastructure against industry standard benchmarks to ensure optimal performance and service delivery.Participates in IT and information security audits and helps implement corrective actions .Participates in the change management process to ensure all changes to relevant services are documented, tested, deployed , and prepped for back-out strategies if necessary.Stays aware of industry trends and how to incorporate them within our infrastructure environment to improve services and/or cut costs.8. CommunicationClearly commun i cates technical information to support planning discussions for potential projects .Regularly communicates with Cyberinfrastructure Technologies colleagues regarding initiatives .Keeps the ES Manager informed of current and potential issues, activities, operational outages , and any other risks that might affect IT service delivery to the University community .9. OperationsSuggests operation strategies to accommodate shifts in customer needs.Follows and, where needed, creates procedures and methods for operational tasks required to maintain data center servers and related systems in reliable, stable operation , escalating complex or undocumented issues to senior staff.Supports academic programs by providing technical support to help faculty and student technology adoption succeed, e . g . , consulting with faculty launching initiatives, i dentifying their needs, evaluating solutions, implementing those solutions.Support compute and storage needs of institutional programs.Helps empower end users to successfully use the HPC technology.Interfaces with vendors and external resources.Evaluates new software or systems under consideration for adoption.Ensures asset management procedures are maintained and documented.Works with the Enterprise Systems team to automate and streamline procedures within the department.On occasion works beyond and in addition to traditional work schedules/hours. Required to carry a cell phone and be on-call.Uti li zes technologies and tools to support the compute and storage infrastructure: programming, scripting, diagnostic tools .10. Other duties as assigned by the Manager of Enterprise SystemsPROVIDES WORK DIRECTIONMay supervise student workers.GENERAL GUIDELINES1. Recommends initiatives and implements changes to improve quality and services.2. Identifies and determines cause of problems; brings recommendations for improvement of established processes and practices to senior staff and ES Manager.3. Maintains contact with customers and solicits feedback for improved services.4. Maximizes productivity through use of appropriate tools; plans training and performance initiatives.5. Researches and develops resources that support timely and efficient work flow.6. Prepares progress reports; informs supervisor of project status and deviation from goals.7. Ensures completeness, accuracy, and timeliness of all operational functions.8. Prepares and submits reports as requested and required.9. Develops and implements guidelines to support the functions of the unit.QUALIFICATIONSTo perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The items below are representative of the knowledge, skills, abilities, education, and experience required or preferred.This position requires the ability to effectively establish and maintain cooperative working relationships within a diverse multicultural environment.1. Knowledge, Skills and AbilitiesGeneralKnowledge of information technology, campus technology, and information security issues and trends in higher education, and ability to continually develop new knowledge regarding the same.Ability to listen and understand customer needs.Ability to plan, implement, and evaluate customer service initiatives.Ability to work in a collaborative environment, as either a member or leader of a team, to meet deadlines and achieve goals.Ability to manage a diverse workforce to provide excellent customer service.Self-motivated and shows initiative.Ability to successfully manage multiple projects simultaneously.Proven track record in project planning and project management.Ability to exercise independent judgment and engage in critical thinking and problem solving.Ability to work effectively under pressure in a busy (sometimes chaotic) and demanding information services environment.Ability to explain technical issues and policies to non-technical people.Ability to give presentations on technical issues to a broad range of audiences.Ability to foster and maintain good working relationships with faculty, administrators, students, senior management, and other leaders.Ability to handle sensitive matters with diplomacy and the ability to mediate between competing parties.Ability to maintain confidentiality and manage confidential information.Must possess impeccable integrity.Ability to speak truth to power.Appreciation for the University’s mission, vision, values, priorities, procedures, and policies.Position-specificKnowledgeable and experienced in large-scale computing centers, supporting systems running Linux (Red Hat, Rocky, Ubuntu) and Windows Server operating systems.Experience with managing and operating SAN storage environments.Exposure to configuration management tools such as Ansible and Warewulf.Exposure to Slurm and job scheduling.Strong proficiency with scripting languages (Python, Bourne Shell, Perl, etc.).Familiarity with compiling software packages and managing software modules in a HPC (EasyBuild, Lmod).Experience with racking servers and adding PCI cards.Experience with LDAP and DNS.Familiarity with parallel file systems.Familiarity with Infiniband networks.The University technology environment is very dynamic and challenging. A person with a wide breadth of experience and who can adapt to changes working in a complex technology infrastructure environment is sought.Experience with vSphere, ESXi.Experience in using and configuring system monitoring tools.Experience with enterprise Backups.Exposure with cloud providers.Skilled technical troubleshooter. Must be able to analyze and solve complex problems.Knowledgeable in the use of a personal computer and standard productivity tools.Experience interacting and working with other people in a successful customer service capacity.Knowledge with industry trends in enterprise infrastructure/data center technology including: automation tools, cloud technology, disaster recovery, virtualization, networking, security and other pertinent areas.Familiarity with Identity and Access Management (IAM).Excellent interpersonal, written and verbal communication skills.Demonstrated ability to work in a collaborative, team environment.Strong organizational skills and ability to multitask.Must be a “self-starter” and show initiative to proactively identify and resolve problems.Ability to acquire and apply new skills quickly.Strong customer service orientation.Understands the role of enterprise computing in University business processes.Works under limited supervision.2. EducationBachelor’s degree in a directly applicable field of study (Computer or Electrical Engineering, Math/Computer Science, Operations and Management Information Science)3. Experience3+ years HPC related applicable experience in the operation, maintenance, support and design of enterprise-wide computer center systems with demonstrated increasing responsibilitiesExposure to HPC-specific skills such as Slurm or similar workload manager; InfiniBand or similar high speed interconnects; and CephFS or similar parallel file system.Experience working for the needs of Higher Education or research organizations is desirable.PHYSICAL DEMANDSThe physical demands described below are representative of those that must be met by an employee to successfully perform the essential functions of this job. In accordance with the Americans with Disabilities Act, as amended, the California Fair Employment & Housing Act, and all other applicable laws, SCU provides reasonable accommodations for qualified persons with disabilities. A qualified individual is a person who meets skill, experience, education, or other requirements of the position, and who can perform the essential functions of the position with or without reasonable accommodation.Must be able to handle and operate equipment, which can be located in racks, on tall shelves or in cabinetsMust be able to climb ladders and work at heightsMust be able to work in confined spaces: crawl spaces, under raised flooring and in or under furnitureConsiderable time is spent at a desk using a computer terminalWill be required to travel to other buildings on the campusMay be required to occasionally travel to remote campuses, outside customers, vendors or suppliersMay be required to attend conference and training sessions within Bay Area or in- or out-of-state locationsWORK ENVIRONMENTThe work environment characteristics described below are representative of those an employee encounters while performing the essential functions of this job.Typical office and computer lab environmentMostly indoor office environment with windowsOffices with equipment noiseOffices with frequent interruptionsData Centers, wiring and equipment closets with loud noise, low light and tight spacesRaised floor under floor access and above ceiling spacesRoofs, high on walls, ceilings, basements and other locations where equipment is storedTelecommuteSanta Clara University is registered to do business in the following states: California, Nevada, Oregon, Washington, Arizona, and Illinois. Employees approved to telecommute are required to perform their work within one of these states.EEO StatementEqual Opportunity/Notice of NondiscriminationSanta Clara University is an equal opportunity employer. All qualified applicants are encouraged to apply and will receive consideration for employment without regard to race, color, ethnicity, national origin, citizenship, ancestry, religion, age, sex, sexual orientation, gender, gender expression, gender identity, marital status, parental status, veteran or military status, physical or mental disability, medical conditions, pregnancy or related conditions, reproductive health decision making, or any other characteristic protected by federal, state, or local laws. For a complete copy of Santa Clara University’s equal opportunity and nondiscrimination policies, please visit the Office of Equal Opportunity and Title IX website at https://www.scu.edu/title-ix/ .Clery Notice of AvailabilitySanta Clara University annually collects information about campus crimes and other reportable incidents in accordance with the federal Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act. To view the Santa Clara University report, please visit the Campus Safety Services website . To request a paper copy please call Campus Safety at [phone]. The report includes the type of crime, venue, and number of occurrences.Americans with Disabilities ActConsistent with its obligations under the law, Santa Clara University will provide reasonable accommodations to applicants and employees with disabilities. Applicants who wish to request a reasonable accommodation for any part of the application or hiring process should contact the Department of Human Resources ADA Team at [e-mail] or by phone at [phone]. Please note : This contact information is intended for accommodation requests only. Resumes or inquiries about application status sent to this inbox will not be reviewed or forwarded. For resumes or questions regarding application status, please contact [e-mail] .