Senior HPC Engineer - GPU/Cluster Architect (Hybrid)

University of California San Diego; United States; On-site; USD 130,000 - 164,000; Full time

Job summary

UC San Diego is seeking a Sr. High-Performance Computing Engineer to manage the AWARE HPC environment, including compute and GPU nodes, networks, storage, and job scheduling.

You will install, configure, and optimize HPC infrastructure, coordinate with researchers and campus teams, and plan future upgrades with minimal disruption. You will mentor staff, document procedures, and ensure security and performance across multi-platform Linux systems, leveraging Slurm, BeeGFS/Lustre, and other tools

Qualifications

• Advanced knowledge of systems integration deploying complex systems integration solutions.
• Advanced skills associated with system integration design, modification, implementation and deployment in a complex environment.
• Demonstrated ability to develop complex conversion and system implementation plans.
• Demonstrated ability to understand functional needs of the campus / medical center and / or UC system and how systems can support those needs.
• Demonstrated ability to interface with senior level technical and non-technical management on a regular basis.
• Demonstrated ability to create software specifications.
• Experience with identification and use of code libraries and open-source forums.
• Experience with planning for deployment and creation of feedback mechanisms.
• Experience developing and implementing complex project plans.
• Experience researching and evaluating new technology and solutions for complex environments.
• Ability to impart knowledge to senior level technical and non-technical audiences and to influence outcomes to meet objectives.
• Demonstrated ability to identify software and hardware interdependencies in a complex multi-platform environment.
• Demonstrated ability to develop solutions for complex business and systems / custom / ad-hoc solutions.
• Advanced knowledge and experience with monitoring tools, performance, and systems optimization, including intermediate knowledge of security for data, data transfers and systems.
• Experience with complex multi-platform troubleshooting in a multi-platform environment.
• Demonstrated ability to plan and execute the technical aspects of deploying, upgrading, and decommissioning storage systems.
• Advanced knowledge and experience with monitoring tools, performance, and systems optimization, including intermediate knowledge of security for data, data transfers and systems.
• Experience with complex multi-platform troubleshooting in a multi-platform environment.
• Demonstrated knowledge of TCP/IP and Infiniband networking, including sockets, VLANs, and firewalls.
• Proven experience designing, deploying, and operating large-scale (petabyte-class) high-performance parallel and distributed file systems (e.g., Lustre, Ceph, BeeGFS, GPFS), as well as enterprise and local file systems (e.g., NFS, ZFS, ext4, XFS) in Linux-based environments, including troubleshooting and performance tuning.
• Expert knowledge and experience with large data storage arrays (more than100TB), and methods to administer, monitor and upgrade.
• Proven experience administering and supporting large-scale HPC clusters or other distributed POSIX (Linux) systems, including advanced knowledge of Linux system administration, primarily Red Hat and its derivatives (e.g., Rocky Linux).
• In depth experience in cluster management tasks including deployment, configuration, and troubleshooting of compute nodes, management nodes, network switches, and file servers.
• Demonstrated experience with scripting and automation using languages such as Bash and Python; use of configuration management tools (e.g., Ansible, CFEngine); and version control systems (e.g., Git) to manage and maintain system configurations and infrastructure.
• Advanced knowledge of HPC middleware stack including cluster management tools, job schedulers and resources managers. Examples include: Slurm, PBS, HPCM, and Bright Cluster Manager.
• Demonstrated experience evaluating new storage system technologies.

Responsibilities

• Lead the technical operation of a high-performance computing (HPC) system (AWARE).
• Manage system administration, upgrades, deployments, and troubleshooting.
• Monitor performance and capacity; respond to production incidents.
• Coordinate with campus teams and vendors; maintain documentation.
• Troubleshoot MPI jobs, software environments, containers, memory and I/O issues; identify root causes across apps, env, scheduler, network, storage, or hardware.
• Plan future additions and upgrades; gather requirements and develop benchmarks; assist with procurements.
• Mentor staff, explain HPC concepts, review work, improve operation docs.

Skills

HPC systems

System integration

Cluster management

Linux system administration

Slurm

InfiniBand

BeeGFS/Lustre

MPI

Python/Bash scripting

Ansible/CFEngine

Git version control

Monitoring tools

Networking TCP/IP

Tools

Slurm

InfiniBand

BeeGFS/Lustre

ZFS

MPI

GPU computing

Ansible

CFEngine

Git

Telegraf/In fluxDB/Nagios

Job description

UC San Diego is seeking a Sr. High-Performance Computing Engineer to manage the AWARE HPC environment, including compute and GPU nodes, networks, storage, and job scheduling.

You will install, configure, and optimize HPC infrastructure, coordinate with researchers and campus teams, and plan future upgrades with minimal disruption. You will mentor staff, document procedures, and ensure security and performance across multi-platform Linux systems, leveraging Slurm, BeeGFS/Lustre, and other tools