GPU Infrastructure Support Engineer

Yoh Services LLC; New York (NY); Hybrid; USD 140,000 - 170,000; Full time

Benefits offered by this job

Medical benefits

HSA

Life & Disability Insurance

401K Retirement Savings Plan

Direct Deposit

Referral bonuses

Certification & training opportunities

Job summary

Yoh Services LLC in New York, NY seeks an AI Infrastructure Support Engineer to drive operations for AI workloads and GPU platforms. You will implement deployment and monitoring across distributed environments using Kubernetes and Docker, with automation in Ansible and Python.

This role requires strong DevOps experience, incident management, and cross-functional collaboration. You will work in a hybrid setup (onsite and remote capabilities) with on-call rotations.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• Experience supporting AI infrastructure, GPU platforms, or HPC in enterprise or production settings.
• Hands-on NVIDIA GPU node experience (A100, H100, B200).
• Solid Kubernetes and Docker proficiency for distributed workloads.
• Automation scripting in Ansible, Python and Shell.
• DevOps practices with GitLab CI/CD and container-based delivery.
• Familiarity with JIRA and ServiceNow ticketing.
• Strong networking, Linux admin, and infra troubleshooting skills.
• IAM knowledge, including Azure AD, and on-call readiness.

Responsibilities

• Deliver operational support and drive issue resolution for AI infrastructure and GPU platforms.
• Support deployment, orchestration, and monitoring of AI/ML workloads with Kubernetes and Docker.
• Maintain automation tooling with Ansible, Python, and Shell scripts.
• Manage incidents, service requests, and changes via JIRA and ServiceNow.
• Collaborate across engineering and operations to implement DevOps best practices (CI/CD, containerization, secure access).
• Create clear technical runbooks, SOPs, and infrastructure architecture artifacts.
• Troubleshoot networking, storage, and performance issues on AI/HPC workloads.
• Participate in rotating on-call support, including occasional weekends.

Skills

AI infrastructure support

GPU platforms

Kubernetes

Docker

Ansible

Python

GitLab CI/CD

Azure AD

Linux admin

Education

Bachelor's degree or equivalent experience

Tools

Kubernetes

Docker

GitLab CI/CD

Ansible

Azure AD

Job description

Job Title:AI Infrastructure Support Engineer Location:New York, NY Type - Hybrid4 days Onsite Terms; Full Time Salary- $140000/yr - $170000/yr

Our clientprovides a range of financial products and services in the United States and internationally. It operates through Securities Services, Market and Wealth Services, Investment and Wealth Management, and Other segments. The Securities Services segment offers custody, trust and depositary, accounting, exchange-traded funds, middle-office solutions, transfer agency, services for private equity and real estate funds, foreign exchange, securities lending, liquidity/lending services, and data analytics.

In this role, you'll make an impact in the following ways:

• Deliver operational support and drive issue resolution for AI infrastructure, GPU platforms, and associated production services.
• Support the deployment, orchestration, and monitoring of AI/ML workloads across distributed environments using Kubernetes and Docker.
• Apply hands-on expertise with NVIDIA GPU infrastructure, including A100, H100, and B200 nodes, to support provisioning, utilization, performance tuning, and ongoing platform operations.
• Build, maintain, and optimize automation scripts and tooling using Ansible, Python and Shell scripting to improve efficiency and reliability.
• Manage incidents, service requests, and change activity through enterprise workflow and ticketing platforms such as JIRA and ServiceNow.
• Partner with cross-functional teams across engineering, platform, and operations to implement DevOps best practices, including CI/CD using GitLab, containerized deployment models, and secure access management through Azure AD.
• Create and maintain clear technical documentation, including runbooks, standard operating procedures, support guides, and infrastructure architecture artifacts.
• Diagnose and resolve networking, storage, and system performance issues affecting AI infrastructure and high-performance compute workloads.
• Participate in a rotating on-call support model, including occasional weekend coverage, to help maintain high availability and operational excellence.

To be successful in this role, we’re seeking the following:

• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
• Strong experience supporting AI infrastructure, GPU platforms, or high-performance compute environments in enterprise or production settings.
• Hands-on experience working with NVIDIA GPU nodes, including A100, H100, and B200 systems.
• Solid understanding of Kubernetes and Docker for deploying, managing, and troubleshooting distributed workloads.
• Proficiency in Ansible, Python and Shell scripting for automation, tooling, and operational support tasks.
• Experience with DevOps practices and tools, including GitLab CI/CD, infrastructure automation, and container-based delivery models.
• Familiarity with enterprise ticketing and service management platforms such as JIRA and ServiceNow.
• Good knowledge of networking, storage, Linux systems administration, and infrastructure troubleshooting in complex technical environments.
• Understanding of identity and access management, including secure access controls such as Azure AD.
• Strong problem-solving skills, attention to detail, and the ability to manage multiple priorities in a fast-paced environment.
• Effective written and verbal communication skills, with the ability to document processes and collaborate across technical and non-technical teams.
• Willingness to participate in on-call rotations and provide occasional weekend support as needed.

Estimated Min Rate: $140000.00

Estimated Max Rate: $170000.00

What’s In It for You?

We welcome you to be a part of the largest and legendary global staffing companies to meet your career aspirations. Yoh’s network of client companies has been employing professionals like you for over 65 years in the U.S., UK and Canada. Join Yoh’s extensive talent community that will provide you with access to Yoh’s vast network of opportunities and gain access to this exclusive opportunity available to you. Benefit eligibility is in accordance with applicable laws and client requirements. Benefits include:

• Medical, Prescription, Dental & Vision Benefits (for employees working 20+ hours per week)
• Health Savings Account (HSA) (for employees working 20+ hours per week)
• Life & Disability Insurance (for employees working 20+ hours per week)
• MetLife Voluntary Benefits
• Employee Assistance Program (EAP)
• 401K Retirement Savings Plan
• Direct Deposit & weekly epayroll
• Referral Bonus Programs
• Certification and training opportunities

Note: Any pay ranges displayed are estimations. Actual pay is determined by an applicant's experience, technical expertise, and other qualifications as listed in the job description. All qualified applicants are welcome to apply.

Yoh, a Day & Zimmermann company, is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

Visit https://www.yoh.com/applicants-with-disabilities to contact us if you are an individual with a disability and require accommodation in the application process.

For California applicants, qualified applicants with arrest or conviction records will be considered for employment in accordance with the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. All of the material job duties described in this posting are job duties for which a criminal history may have a direct, adverse, and negative relationship potentially resulting in the withdrawal of a conditional offer of employment.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

By applying and submitting your resume, you authorize Yoh to review and reformat your resume to meet Yoh’s hiring clients’ preferences. To learn more about Yoh’s privacy practices, please see our Candidate Privacy Notice: https://www.yoh.com/privacy-notice

Stay Safe During Your Job Search