Senior Systems Administrator

ASRC Federal

Kansas

On-site

USD 110,000 - 170,000

Full time

7 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

ASRC Federal is seeking a Staff HPC Engineer to support SciComp HPC and MR-ITD in Manhattan, KS. The role oversees administration, maintenance, and optimization of a large Linux HPC cluster, ensuring security, availability, and high performance for genomics, diagnostics, and analytics.

You will manage Slurm, Lustre storage, InfiniBand networking, and GPU compute nodes, develop automation with Ansible, and collaborate with facilities, cybersecurity, and researchers.

Qualifications

  • Bachelor’s degree in computer science or a related field.
  • Strong Linux and networking background with systems-level knowledge.
  • 5+ years provisioning and configuring medium Linux environments (Ansible, Base Commander).
  • Scripting, debugging, and problem solving in Python, Perl or Bash.
  • Experience with Slurm, Lustre, InfiniBand, MPI, and open-source tooling.

Responsibilities

  • Administer and maintain a Linux-based HPC environment with compute, login, and GPU nodes.
  • Manage Slurm configuration, partitions, queues, scheduling policies, and resource allocation.
  • Develop automation using Ansible and shell scripting for operations.
  • Monitor cluster health, capacity, utilization, and performance.
  • Provide advanced user support to researchers and collaborate with facilities and cybersecurity teams.
  • Support disaster recovery planning and infrastructure lifecycle activities.

Skills

Python
Perl
Bash
Strong communication
Troubleshooting
Customer service
Mentoring

Education

Bachelor’s degree in Computer Science or related field

Tools

Base Commander
Ansible
RHEL Satellite
Puppet
Git
Slurm
Lustre
InfiniBand
CUDA Toolkit
MPI

Job description

ASRC Federal is searching for a Staff HPC Engineer to support ASRC Federal Technology Solution out of Manhattan, KS. ASRC Federal Technology Solution proudly supports the USDA Animal and Plant Health Inspection Service (APHIS) Marketing and Regulatory Programs – Information Technology Division (MRP-ITD). The USDA APHIS mission requires rapid, secure, and reliable access to large-scale scientific data for genomics, diagnostics, modeling, and analytics to support decision-making, research, and emergency outbreak responses.

The Scientific Computing (SciComp) HPC system and the Marketplace for Data Analytics (MDA) form the computational, data analytics, and data-sharing backbone for APHIS’s laboratory network as well as other MRP Program areas. These systems facilitate high-speed data transfer, AI modeling, data visualization, and reproducible scientific outcomes through the integration of on-premises HPC clusters, cloud-based analytics platforms, and secure data transfer via Internet2 (I2). Our employees embrace innovation and are committed to a culture of continuous, standards-driven process improvement and assimilation of the industry’s best practices.

Summary : The successful candidate will be an active supporting member of the ASRC Federal team. The HPC Systems Engineer/Administrator is responsible for the administration, maintenance, optimization, and lifecycle management of a high-performance computing (HPC) environment supporting scientific research. This position ensures the availability, security, and performance of compute, storage, networking, and authentication infrastructure while providing technical support to USDA APHIS researchers and collaborating with on-site facilities, networking, and cybersecurity teams.

Primary Roles, Duties and Responsibilities
HPC Infrastructure Administration
  • Administer and maintain a Linux-based HPC environment consisting of compute, login, management, and GPU nodes.
  • Manage Slurm workload manager configuration, partitions, queues, scheduling policies, and resource allocation.
  • Perform operating system installation, provisioning, upgrades, and lifecycle management.
  • Develop and maintain automation using Ansible, shell scripting, and other administrative tools (NVIDIA Base Command Manager).
  • Monitor overall cluster health, capacity, utilization, and performance.
Storage Administration
  • Administer and optimize a multi-petabyte Lustre parallel file system (HPE E1000).
  • Configure and maintain storage quotas, project directories, tiering policies, and data placement strategies.
  • Troubleshoot Lustre performance, metadata operations, changelog processing, and client connectivity.
  • Monitor storage utilization, capacity growth, and system performance using enterprise monitoring platforms (Starfish Storage).
High-Speed Networking
  • Administer and troubleshoot NVIDIA InfiniBand fabrics, including Subnet Manager, routing, and multi-rail configurations.
  • Maintain NVIDIA Base Command Manager and Cumulus Linux infrastructure supporting the HPC environment.
  • Optimize MPI, UCX, NCCL, and RDMA communications for distributed scientific workloads.
  • Support Internet2 connectivity and high-performance data transfers (Globus Data Transfer Nodes).
GPU Computing Support
  • Support NVIDIA GPU compute nodes and associated software stacks.
  • Install, configure, and maintain CUDA, NCCL, GPU drivers, and related scientific software.
  • Benchmark and validate GPU performance following infrastructure changes.
Software Support
  • Install, configure, and maintain scientific software packages and development environments.
  • Build and install software from source using Make, CMake, Ninja, and other build systems.
  • Configure and troubleshoot software compilation using GCC, G++, and other compiler toolchains.
  • Resolve library dependencies, compiler compatibility issues, and software linkage problems.
  • Develop and maintain environment modules supporting multiple software versions.
  • Support Conda, R, Apptainer, compilers, MPI implementations, CUDA toolkits, and GPU-enabled applications (Apptainer and Singularity containers).
  • Assist researchers with software deployment, optimization, and troubleshooting.
Identity and Security
  • Administer authentication and authorization services including Active Directory integration, LDAP, SSSD, Kerberos, and single sign-on technologies.
  • Coordinate operating system patching and security updates.
  • Support vulnerability remediation and infrastructure hardening activities.
  • Maintain least-privilege access controls and security best practices.
Performance Engineering
  • Evaluate system performance using industry-standard benchmarks.
  • Analyze CPU, GPU, memory, storage, and network performance bottlenecks.
  • Optimize NUMA configuration, processor affinity, MPI communication, and storage performance for scientific workloads (bioinformatics).
  • Validate infrastructure performance following hardware or software changes.
Documentation and Operational Support
  • Develop and maintain standard operating procedures, technical documentation, and knowledge base articles.
  • Participate in change management processes and maintain accurate operational documentation.
  • Coordinate maintenance activities and production system changes.
  • Support disaster recovery planning, backup strategies, and infrastructure lifecycle planning.
Collaboration and User Support
  • Provide advanced technical support to researchers and scientific staff.
  • Collaborate with networking, storage, cybersecurity, and vendor support teams to resolve complex technical issues.
  • Mentor junior technical staff through knowledge transfer, collaborative troubleshooting, and hands‑on training.
  • Participate in procurement planning, hardware evaluations, and technology modernization initiatives.
Typical Daily Activities
  • Monitor cluster health and respond to system alerts.
  • Troubleshoot compute, storage, network, and authentication issues.
  • Deploy operating system, firmware, kernel, and software updates.
  • Optimize HPC performance for scientific workloads.
  • Install and validate scientific software requested by researchers.
  • Develop automation to improve operational efficiency.
  • Maintain system documentation and operational procedures.
  • Coordinate infrastructure maintenance and change activities.
  • Assist users with job scheduling, software execution, and performance optimization.
  • Work with vendors to diagnose and resolve hardware and software issues.

Requirements:

  • Bachelor’s degree in computer science or related field
  • Strong computer science background with in-depth systems-level knowledge in operating systems and networking
  • 5+ years demonstrated competency in provisioning and configuration management of medium-sized Linux environments using tools such as RHEL Satellite, Ansible, Base Commander
  • Ability to work with system configuration management tools (e.g., Puppet, Ansible) and revision control software such as CVS, SVN, Git
  • Superior scripting skills and excellent attention to detail; proficiency in at least Python, Perl, or Bash
  • Demonstratedtroubleshooting skills in a complex environment beyond systems, from networks to other dependent systems
  • Excellent customer service skills and attention to detail supporting hundreds of scientific users
  • Excellent communication and people skills; excellent time management and organizational skills

Preferred Qualifications:

  • Experience with Base Commander or other orchestration platforms
  • Previous experience in support of HPC environments and complex scientific computing centers
  • Familiarity/proficiency with OpenMP and Message Passing Interface (MPI) programming
  • Experience with Lustre and InfiniBand
  • Experience with Slurm

#USDA,#HPC,#LINUX,#RHEL,#ANSIBLE,#BaseCommander,#GPU,#Slurm

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Systems Administrator
Senior Systems Administrator

ASRC Federal • Mountain View (CA)

Hybrid
USD 140,000 - 190,000
Senior Linux Systems Admin & DevOps for HPC
Senior Linux Systems Admin & DevOps for HPC

ASRC Federal • Mountain View (CA)

Hybrid
USD 140,000 - 190,000
Senior HPC Systems Administrator
Senior HPC Systems Administrator

RedLine Performance Solutions • Berkeley (CA)

Remote
USD 140,000 - 190,000
Paid time off
401k match
Health care benefits
HPC Systems Engineer
HPC Systems Engineer

Veriipro • Indianapolis (IN)

On-site
USD 90,000 - 120,000
Senior HPC Systems Administrator
Senior HPC Systems Administrator

Omega Enterprise Solutions, LLC • Corridor North (MD)

On-site
USD 90,000 - 120,000
Senior HPC Systems Administrator
Senior HPC Systems Administrator

RedLine • Berkeley (CA)

Remote
USD 140,000 - 190,000
Paid time off
401k match
Health care benefits
HPC Systems Administrator
HPC Systems Administrator

RedLine Performance Solutions, LLC • Manassas (VA)

On-site
USD 110,000 - 160,000
Paid time off
401k match
Health care benefits
HPC Systems Engineer for Scientific Computing (Linux/Slurm)
HPC Systems Engineer for Scientific Computing (Linux/Slurm)

ASRC Federal • Kansas

On-site
USD 110,000 - 170,000
HPC Linux Systems Engineer
HPC Linux Systems Engineer

Cadre5 • Knoxville (TN)

Hybrid
USD 120,000 - 160,000
Excellent medical insurance
Employer-paid benefits
HPC Linux System Administrator
HPC Linux System Administrator

MRI Technologies • Houston (TX)

On-site
USD 120,000 - 170,000