Cloud Computing Engineer

PDT Partners, LLC; New York (NY); Hybrid; USD 195,000 - 225,000; Full time

Job summary

PDT Partners is seeking an ambitious software engineer to join our Cloud Computing team in New York City, working in a hybrid role from our office at least 3 days a week. You will design and operate cloud-based HPC systems that scale our research platform, collaborate with Research and Model Implementation teams, and drive automation, performance, and reliability across production workloads.

Ideal candidates have 5+ years building software, strong Python/Go/Rust skills, and hands-on experience

Qualifications

• Bachelor's or Master’s degree in Engineering or Applied Sciences or equivalent experience.
• 5+ years of experience building and shipping software systems.
• 2+ years running production compute platforms and scheduling systems.
• Mastery of core software engineering concepts with Python, Go, Rust.
• Experience with a cloud-based infrastructure-as-code tool such as Terraform.
• Excellent written and verbal communication skills.
• Experience working with researchers and/or other developers is a plus.
• Hands-on experience managing inferencing and training hardware at scale.
• Knowledge of NVIDIA GPU management, Kubernetes, Slurm, and large-scale compute ecosystem.

Responsibilities

• Design and implementation of cloud-based HPC systems across HPC scheduling, metrics, containerization, and performance.
• Run our HPC plant day-to-day with automation and reliability.
• Implement automation from CI/CD pipelines to production metrics and monitoring.
• Partner with researchers and engineers to deliver high-quality cloud HPC systems.
• Capacity management and benchmark optimization for scale and performance.

Skills

Python

Go

Rust

Terraform

Kubernetes

Slurm

NVIDIA GPU management

CI/CD

Communication

Education

Bachelor's or Master's degree in Engineering or Applied Sciences

Tools

Terraform

Slurm

NVIDIA GPU management

Job description

The Cloud Computing team is a group of experts solving computing problems in the critical path of Research. We work directly with Research and Model Implementation teams and provide them with platform tools and computing resources to take their ideas from inception to real tradable products. We are looking for an ambitious and operationally minded software engineer to join our team as we mature and scale our cloud HPC platform to the next iteration of our firm-wide Research platform.

This is a hybrid position and will require the person to work from our New York City office at minimum 3 days a week.

Why join us?

PDT Partners has a 30+ year track record and a reputation for excellence. Our goal is to be the best quantitative investment manager in the world-measured by the quality of our products, not their size. PDT’s very high employee-retention rate speaks for itself. Our people are intellectually extraordinary and our community is close-knit, down-to-earth, and diverse.

Responsibilities

We are a small flat team sitting at the cross-section of research, implementation, and platform infrastructure. Our team responsibilities span many areas. Including:

• Design and implementation of cloud-based HPC systems. Our projects involve equal parts engineering and operations for success in our fast-moving environment. You will be expected to conceive and implement projects small and large in the intersection of HPC scheduling, metrics, containerization, software distribution, accelerated compute performance/efficiency, cloud architecture.
• Running our HPC plant day-to-day. Our research environment is up 24/7, and we want to keep it that way. Everybody on the team contributes to the support of our platform, which thankfully is light because of our automation and quality work.
• Implementing automation. We will always choose to work smart over working hard. You will be responsible for conception and implementation of automation from CI/CD pipelines to production metrics and monitoring of our cloud HPC platform.
• Obsessive User Focus. All members of the team are expected to partner with researchers and engineers to deliver high-quality cloud HPC systems that are efficient and reliable. This includes leading projects to evolve it as our needs change.
• Capacity management and benchmark optimization. Our demand for scale and performance is constant and involves challenging optimization problems for workloads critical to research and trading

Below is a list of skills and experiences we think are relevant. Even if you don’t think you’re a perfect match, we still encourage you to apply because we are committed to developing our people.

• Bachelors orMastersdegree in an Engineering or Applied Sciences field from a rigorous academic programor equivalent professional experience.
• 5+ years of experience building and shipping software systems
• 2+ years running production compute platforms and scheduling systems.
• Mastery of core software engineering concepts such as system design, testing, debugging, building reliable production systems with fluency in a language such as Python, Go, Rust etc.
• Experience with a cloud-based infrastructure-as-code tool such as Terraform
• Excellent written and verbal communication skills
• Experience working with or supporting researchers and/or other developers is a plus
• Hands-on experience managing inferencing and training hardware at scale
• Knowledge of NVIDIA GPU management, Kubernetes, Slurm, and the wider large-scale compute ecosystem

The salary range for this role is between $195,000 and $225,000. This range is not inclusive of any potential bonus amounts. Factors that may impact the agreed upon salary within the range for a particular candidate include years of experience, level of education obtained, skill set, and other external factors.

PRIVACY STATEMENT: For information on ways PDT may collect, use, and process your personal information, please see PDT’s privacy notices.