Senior Artificial Intelligence Engineer

St. Jude Children's Research Hospital; Lauderdale Courts (TN); On-site; USD 86,000 - 155,000; Full time

Job summary

St. Jude Children's Research Hospital is seeking a Senior AI Engineer to lead development of advanced AI models, including LLMs and multi‑modal systems, and to architect secure, high‑performance AI infrastructure.

You will evaluate, fine‑tune, deploy, and benchmark models, implement guardrails, and optimize GPU/HPC resources in a hands‑on role. You will work closely with researchers and software engineers, reporting to the HPRC Director and collaborating with CBI in Memphis, TN.

Qualifications

• 4+ years designing and developing large-scale AI/ML systems.
• Strong background in deep learning with unsupervised and supervised techniques.
• Proficiency in TensorFlow, PyTorch, and related frameworks.

Responsibilities

• Assists IS in assessing needs and implementing AI-enabled services on high‑performance computing platforms.
• Design, develop, and implement AI solutions and services including proofs‑of‑concept and pilots.
• Build innovative data science and AI/ML solutions to solve non‑trivial problems.
• Develop AI demos and provide workshops and training.
• Evaluate commercial and open‑source approaches in AI/ML and analytics.
• Document architectures, roadmaps, and reference designs.
• Communicate clearly with customers, teams, and vendors.
• Stay current with new technologies and assess their applicability.
• Perform other duties to meet departmental goals.

Skills

AI/ML systems design

Deep learning frameworks

LLMs & multi-modal models

Distributed multi-GPU training

HPC/Slurm scheduling

Technical leadership

Open-source contributions

Education

Master's degree in CS/EE/Data Science

PhD preferred

Tools

TensorFlow

PyTorch

Keras

Time series analysis

Hugging Face

vLLM / TensorRT-LLM

Job description

High Performance Research Computing (HPRC) and the Center for Bioimage Informatics (CBI) at St. Jude Children's Research Hospital are seeking a Senior AI Engineer to lead our efforts in advanced AI models, including large language models (LLMs), agentic AI systems, and multi-modal foundation models, and the secure computational infrastructure that powers them. This is a hands‑on, high‑ownership software systems role, not a prompt‑engineering, API‑integration, or purely conceptual research position. You will evaluate, fine‑tune, deploy, and benchmark AI models; design and enforce safety guardrails and sandboxing for agentic systems; safeguard data security and privacy for sensitive research data; and optimise GPU/HPC resource allocation to balance performance, cost, and efficiency as models and tooling rapidly evolve. This person builds the shared AI architecture, secure environments, and best practices upon which CBI's image data scientists and software engineers rely. Deep bioimaging expertise is not required, though experience with biomedical research or imaging is a plus. This position reports to the Director of HPRC and works closely with CBI as a collaborative team member.

This is an onsite role in Memphis, TN.

Job Responsibilities

• Assists Research Information Services in ( i ) assessing institution's needs and (ii) implementing AI‑enabled services on high‑performance AI computing platforms.
• Works with scientists, business stakeholders, analysts, and IS professionals in the design, development, and implementation of AI solutions and services, including technology proof‑of‑concepts, pilots, and the adoption lifecycle.
• Be responsible for building new and innovative solutions leveraging data science and AI/ML skills and technologies to solve non‑trivial problems.
• Develops AI demonstration use‑cases and workshop materials and provides workshops and training classes.
• Evaluate commercial and open‑source approaches in AI/ML, Data Mining, and Analytics to solve business problems.
• Identifies , develops, and implements standards and operating procedures for solutions and systems consistent with best practices.
• Documents current and future state architecture roadmaps and reference architectures.
• Provides clear written and spoken communications to customers, teams, and vendors.
• Keeps abreast of new and emerging technologies and stays adaptable to their potential applicability.
• Performs other duties as assigned or directed to meet the goals and objectives of the department and institution.

Minimum Education and/or Training

• Master's degree in computer science, computer engineering, data science, information technology or related field required.
• PhD degree in data science, computer science, computer engineering, information technology or related field preferred.

Minimum Experience

• 4 years in designing and developing solutions for large scale AI/Machine Learning (ML) systems and/or building solutions for a product on AI/ML features and capabilities.
• Strong background in industry use cases built on deep learning and machine learning (unsupervised and supervised techniques) is a must.
• Deep learning frameworks such as TensorFlow, Keras , PyTorch , Time series analysis, anomaly detection, forecasting, predictive modelling, graph - based neural networks, Bayesian statistics, and text analytics are a must.

Preferred Qualifications

• Hands‑on experience training, fine‑tuning, or adapting LLMs or multi‑modal foundation models (e.g., PEFT/ LoRA , instruction tuning, preference optimisation), including debugging failure modes such as catastrophic forgetting or training instability.
• Experience diagnosing and resolving distributed/multi‑GPU training or inference issues (e.g., NCCL communication hangs, CUDA out‑of‑memory errors, load‑balancing across nodes) and scheduling AI workloads on HPC (e.g., Slurm ) to maximise GPU utilisation.
• Experience designing safety guardrails and sandboxing for agentic systems: tool‑access scoping, prompt‑injection defence, secrets management, audit logging, and containment of failures.
• Experience optimising inference cost, latency, and resource usage (e.g., KV‑cache management, quantisation, batching, speculative decoding, high‑throughput serving via vLLM / TensorRT -LLM/Triton) and judgement about when a simpler deterministic pipeline is a better fit than an agentic one.
• Experience safeguarding data security and privacy for AI systems handling sensitive research data, including access controls and institutional data‑use/security policies.
• Experience building rigorous, reproducible benchmarking/evaluation frameworks that separate genuine model improvement from prompt overfitting, retrieval effects, or evaluator bias.
• Experience building and scaling AI/ML pipelines and workflows on HPC or cloud environments.
• Demonstrated ownership of a system beyond the prototype stage (observability, versioning, rollback, cost control, incident response).
• Contributions to open‑source AI/ML infrastructure projects (e.g., vLLM , PyTorch , Ray, Hugging Face) or a public track record (GitHub, Hugging Face, papers) are a plus.
• Familiarity with biomedical research, imaging, or regulated health data is a plus.
• Demonstrated technical leadership: setting standards, mentoring, and cross‑team collaboration.

Compensation

In recognition of certain U.S. state and municipal pay transparency laws, St. Jude is including a reasonable estimate of the compensation range for this role. This is an estimate offered in good faith and a specific salary offer takes into account factors that are considered in making compensation decisions including but not limited to skill sets, experience and training, licensure and certifications, and other business and organisational needs. It is not typical for an individual to be hired at or near the top of the salary range and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current salary range is $86,320 - $154,960 per year for the role of Senior Artificial Intelligence Engineer.

Explore our exceptional benefits!

St. Jude is an Equal Opportunity Employer