HPC Storage Engineer for Large-Scale AI Workloads

Hudson River Trading; Chicago, New York, Austin (IL, NY, TX); On-site; USD 150,000 - 250,000; Full time

Job summary

Hudson River Trading is seeking an experienced HPC Storage Engineer to design, deploy, and manage high-capacity storage for a world-wide research and trading environment. You will troubleshoot complex storage, OS, and networking issues and build tooling to monitor performance of storage clusters.

You will research and test new storage solutions, contribute to architecture decisions, and help shape storage direction for our HPC-focused team in a fast-paced, collaborative setting.

Qualifications

• Experience in HPC-like environments with large scale storage (>1PB).
• Proficiency with Linux and NFS.
• Experience with distributed file systems (Lustre, CephFS, GPFS).
• Ability to design a home grown NAS storage solution with redundancy and high availability.

Responsibilities

• Design, deploy and manage a variety of storage solutions.
• Troubleshoot complex storage, OS, and networking issues.
• Build tools to improve performance and monitoring of storage clusters.
• Research and experiment with new storage solutions.

Skills

Linux troubleshooting

Python

Shell scripting

Cross-functional communication

Problem solving

Time management

Eager to learn

ZFS knowledge

Tools

Lustre

CephFS

GPFS

Isilon

VAST

NetApp

Pure

Job description

Hudson River Trading is seeking an experienced HPC Storage Engineer to design, deploy, and manage high-capacity storage for a world-wide research and trading environment. You will troubleshoot complex storage, OS, and networking issues and build tooling to monitor performance of storage clusters.

You will research and test new storage solutions, contribute to architecture decisions, and help shape storage direction for our HPC-focused team in a fast-paced, collaborative setting.