AI HPC Infrastructure Engineer

Analysis Group, Inc.; Boston (MA); On-site; USD 150,000 - 170,000; Full time

Benefits offered by this job

Discretionary annual bonus

Benefits package

Job summary

Analysis Group, Inc. seeks an experienced AI HPC Infrastructure Engineer to own and operate a hybrid HPC/AI computing environment. You will manage Linux clusters, GPU fleets, and multi-tenant workloads to support researchers and data scientists.

You will optimize performance with MPI/OpenMP, implement scalable GPU training/inference, and build MLOps pipelines. Strong scripting, collaboration, and problem-solving are essential in a fast-paced research setting.

Qualifications

• Bachelor's degree in computer science, electrical engineering, or related field.
• 5+ years hands-on Linux systems administration in research/HPC/production.
• Experience with Posit Workbench (RStudio Server Pro) and Python/R environments.
• Experience with SLURM, Platform LSF, or other job schedulers; GPU scheduling preferred.
• Hands-on NVIDIA GPU stack (CUDA, cuDNN, NCCL) and containerization knowledge.
• Familiarity with ML frameworks (PyTorch, TensorFlow) and distributed training.
• Experience with MLOps tools (MLflow, Kubeflow).
• Proficiency in remote access (SSH/RDP) and GPFS.
• Excellent troubleshooting, documentation, and collaboration skills.

Responsibilities

• Maintain and expand HPC/AI compute environment for researchers and data scientists.
• Optimize performance using MPI/OpenMP and distributed multi-GPU training.
• Design and maintain GPU-accelerated infrastructure for large-scale model training.
• Manage GPU scheduling across SLURM/Kubernetes multi-tenant clusters.
• Administer NVIDIA stack and health monitoring for GPU fleets.
• Tune LLM training/inference (batching, quantization, KV-cache).
• Build MLOps pipelines (MLflow, Kubeflow) for training and deployment.
• Manage containers (Docker, Kubernetes, Singularity) for HPC/ML workloads.
• Develop automation scripts and usage reporting across resources.
• Ensure storage and data pipelines on GPFS are robust and performant.
• Troubleshoot hardware/software/network issues and perform root-cause analysis.
• Participate in 24x7 on-call rotation and provide remote/on-site support.

Education

Bachelor's degree in CS/EE

Tools

Posit Workbench

GPFS

Docker

Kubernetes

Singularity/Apptainer

MLflow

Kubeflow

Ansible

Job description

Overview

Analysis Group is one of the largest international economics consulting firms, with more than 1,500 professionals across 15 offices in North America, Europe, and Asia. Since 1981, we have provided expertise in economics, finance, health care analytics, and strategy to top law firms, Fortune Global 500 companies, and government agencies worldwide. Our internal experts, together with our network of affiliated experts from academia, industry, and government, offer our clients exceptional breadth and depth of expertise.

The AI HPC Infrastructure Engineer owns the operation, performance, and growth of a hybrid high-performance computing (HPC) and AI/GPU infrastructure environment. The engineer maintains the Linux-based clustered computing platform that supports both traditional HPC/analytical workloads and large-scale AI/ML training and inference, ensuring systems run efficiently, GPUs and other accelerators are current and well-utilized, and operations are monitored, documented, and reported — including change management and performance statistics — across both domains.

Essential Job Functions and Responsibilities

• Maintain, tune, and manage the analytical and AI computing environment for researchers and data scientists, including Posit Workbench (RStudio Server Pro) environments.
• Optimize systems and infrastructure performance using parallelization technologies (MPI, OpenMP) and distributed/multi-GPU training strategies (e.g., PyTorch Distributed, Horovod, DeepSpeed).
• Design, deploy, and maintain GPU-accelerated compute infrastructure for large-scale model training and inference.
• Manage GPU scheduling, multi-tenancy, and utilization across SLURM and/or Kubernetes-based environments.
• Administer the NVIDIA software stack — drivers, CUDA, cuDNN, NCCL — and coordinate firmware and health monitoring across GPU fleets.
• Tune and optimize LLM training and inference performance — including batching, quantization, KV-cache utilization, parallelism strategies, and throughput/latency across GPU clusters.
• Build and maintain MLOps pipelines for model training, versioning, deployment, and monitoring (e.g., MLflow, Kubeflow).
• Manage container orchestration and runtimes (Docker, Kubernetes, Singularity/Apptainer) supporting both HPC jobs and ML workloads.
• Manage access authentication including PAM, LDAP integration, and single sign-on.
• Design and develop scripts for system administration, automating tasks, monitoring, and usage reporting across HPC and AI resources.
• Manage high-performance storage and data pipelines for AI training datasets and HPC workloads, primarily on GPFS (IBM Spectrum Scale).
• Troubleshoot, isolate, and resolve application, systems, and other technical problems (hardware, software, network, and GPU-specific issues).
• Develop and implement backup and recovery programs.
• Research, deploy, and manage general infrastructure, including development of policies and procedures for both HPC and AI/ML environments.
• Migrate data from heterogeneous environments to Linux, on-prem clusters, or cloud.
• Collaborate with data scientists and ML engineers to support the model development lifecycle and translate research needs into infrastructure requirements.
• Evaluate emerging AI hardware, accelerators, and cloud AI services, and recommend adoption where beneficial.
• Monitor performance, troubleshoot problem areas, and provide statistics and reports across compute, storage, and network.
• Create and maintain documentation related to system configuration, processes, change management, inventory, and service records.
• Ensure continuous network connectivity of all equipment.
• Conduct research and report on products, services, protocols, and standards to remain abreast of developments in HPC and AI infrastructure.
• Participate in a 24x7 on-call rotation; troubleshoot and resolve issues remotely or onsite as necessary.

Qualifications

• Bachelor's degree required; degree in computer science, electrical engineering, or a related field preferred.
• A minimum of 5 years of experience as a hands‑on Linux Systems Administrator in a research, HPC, or production setting.
• An ideal candidate will have 5 to 10 years of substantive relevant experience.
• Experience managing Posit Workbench (RStudio Server Pro), Python, and R environments; strong Posit Workbench administration experience is a significant plus.
• Experience with SLURM, Platform LSF, or other job schedulers required; experience scheduling GPU resources strongly preferred.
• Hands‑on experience with NVIDIA GPU infrastructure and software stack (CUDA, cuDNN, NCCL, NVIDIA GPU Operator) strongly preferred.
• Experience with Kubernetes and container orchestration for AI/ML workloads highly desired.
• Familiarity with ML/AI frameworks (PyTorch, TensorFlow) and distributed training patterns highly desired.
• Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) is a plus.
• Experience with Bright Cluster Manager is highly desired.
• Experience with Ansible is highly desired.
• Experience with containerization (Docker, Singularity/Apptainer) is highly desired.
• Proficiency with remote access technologies and tools such as RDP, SSH, and emulation software.
• Hands‑on experience with GPFS (IBM Spectrum Scale) required.
• Demonstrated experience tuning LLM training and/or inference performance (e.g., batching, quantization, KV-cache management, parallelism strategies) required.
• Experience with AI Gateways (e.g., LiteLLM, Kong AI Gateway, Portkey, or similar) is a very nice to have.
• Excellent hardware troubleshooting experience, including GPU‑specific diagnostics.
• Knowledge of applicable data privacy practices and laws.
• Strong interpersonal, written, and oral communication skills.
• Highly self‑motivated and directed, with keen attention to detail.
• Proven analytical and problem‑solving abilities.
• Strong customer service orientation.
• Experience working in a collaborative environment.
• An inclusive and growth‑oriented mindset, strong interpersonal skills, and an ability to work across functions.
• To the extent permitted by applicable law, eligible candidates must be authorized to work in the United States, without sponsorship or restriction, now and in the future.

Analysis Group embraces equal opportunity. We are committed to building teams that bring a variety of backgrounds, perspectives, and skills, as we believe that a strong and inclusive workforce directly supports our goal of providing the highest-quality work. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or any other class protected under applicable federal, state, or local law, and we encourage candidates of all backgrounds to apply.

Analysis Group offers competitive compensation and a comprehensive benefits package. The estimated salary range for this position is $150,000–$170,000. Compensation offered will be based on a number of factors including work experience, education, and skill level. This role is eligible for a discretionary annual bonus that is determined in large part by individual performance. To learn more about our benefit offerings, clickhere.

#LI-Hybrid

Privacy Notice

For information about Analysis Group’s privacy practices, please refer to the applicable Analysis Groupprivacy policy.

• Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities.
• Please view the EEOC’s “Know Your Rights” poster here.