Senior Artificial Intelligence Engineer

St. Jude Children's Research Hospital; Memphis (TN); On-site; USD 86,000 - 155,000; Full time

Job summary

St. Jude Children's Research Hospital in Memphis, TN, seeks a Senior AI Engineer to lead development of advanced AI models including LLMs, agentic systems, and multi-modal foundation models, and to secure the computational infrastructure that powers them.

This hands-on, high-ownership software role involves evaluating, fine-tuning, deploying, and benchmarking AI models, designing safety guardrails, safeguarding data privacy, and optimizing GPU/HPC resource allocation as technologies evolve.

Qualifications

• Master’s degree in computer science, computer engineering, data science, information technology or related field required.
• PhD degree in data science, computer science, computer engineering, information technology or related field preferred.
• Strong background in deep learning frameworks and ML techniques is required.

Responsibilities

• Assists in evaluating and implementing AI-enabled services on high-performance AI computing platforms.
• Collaborates with scientists and IS professionals to design, develop, and deploy AI solutions and services.
• Develops and benchmarks AI models, demonstrating ownership beyond prototyping.

Skills

Deep learning

Distributed training

Model evaluation

Leadership

Education

Master’s degree

PhD preferred

Tools

TensorFlow

PyTorch

Keras

Slurm

CUDA

Job description

High Performance Research Computing (HPRC) and the Center for Bioimage Informatics (CBI) at St. Jude Children's Research Hospital are seeking a Senior AI Engineer to lead our efforts in advanced AI models, including large language models (LLMs), agentic AI systems, and multi-modal foundation models, and the secure computational infrastructure that powers them. This is a hands‑on, high‑ownership software systems role, not a prompt‑engineering, API‑integration, or purely conceptual research position. You will evaluate, fine‑tune, deploy, and benchmark AI models; design and enforce safety guardrails and sandboxing for agentic systems; safeguard data security and privacy for sensitive research data; and optimize GPU/HPC resource allocation to balance performance, cost, and efficiency as models and tooling rapidly evolve. This person builds the shared AI architecture, secure environments, and best practices upon which CBI's image data scientists and software engineers rely. Deep bioimaging expertise is not required, though experience with biomedical research or imaging is a plus. This position reports to the Director of HPRC and works closely with CBI as a collaborative team member.

This is an onsite role in Memphis, TN.

Job Responsibilities

• Assists Research Information Services in (i) assessing institution’s needs and (ii) implementing AI‑enabled services on high‑performance AI computing platforms.
• Works with scientists, business stakeholders, analysts, and IS professionals in the design, development, and implementation of AI solutions and services, including technology proof‑of‑concepts, pilots, and the adoption lifecycle.
• Be responsible for building new and innovative solutions leveraging data science and AI/ML skills and technologies to solve non‑trivial problems.
• Develops AI demonstration use‑cases and workshop materials and provides workshops and training classes.
• Evaluate commercial and open‑source approaches in AI/ML, Data Mining, and Analytics to solve business problems.
• Identifies, develops, and implements standards and operating procedures for solutions and systems consistent with best practices.
• Documents current and future state architecture roadmaps and reference architectures.
• Provides clear written and spoken communications to customers, teams, and vendors.
• Keeps abreast of new and emerging technologies and stays adaptable to their potential applicability.
• Performs other duties as assigned or directed to meet the goals and objectives of the department and institution.

Minimum Education and/or Training

• Master’s degree in computer science, computer engineering, data science, information technology or related field required.
• PhD degree in data science, computer science, computer engineering, information technology or related field preferred.

Minimum Experience

• 4 years in designing and developing solutions for large scale AI/Machine Learning (ML) systems and/or building solutions for a product on AI/ML features and capabilities.
• Strong background in industry use cases built on deep learning and machine learning (unsupervised and supervised techniques) is a must.
• Deep learning frameworks such as TensorFlow, Keras, PyTorch, Time series analysis, anomaly detection, forecasting, predictive modeling, graph‑based neural networks, Bayesian statistics, and text analytics are a must.

Preferred Qualifications

• Hands‑on experience training, fine‑tuning, or adapting LLMs or multi‑modal foundation models (e.g., PEFT/LoRA, instruction tuning, preference optimization), including debugging failure modes such as catastrophic forgetting or training instability.
• Experience diagnosing and resolving distributed/multi‑GPU training or inference issues (e.g., NCCL communication hangs, CUDA out‑of‑memory errors, load‑balancing across nodes) and scheduling AI workloads on HPC (e.g., Slurm) to maximize GPU utilization.
• Experience designing safety guardrails and sandboxing for agentic systems: tool‑access scoping, prompt‑injection defense, secrets management, audit logging, and containment of failures.
• Experience optimizing inference cost, latency, and resource usage (e.g., KV‑cache management, quantization, batching, speculative decoding, high‑throughput serving via vLLM/TensorRT‑LLM/Triton) and judgment about when a simpler deterministic pipeline is a better fit than an agentic one.
• Experience safeguarding data security and privacy for AI systems handling sensitive research data, including access controls and institutional data‑use/security policies.
• Experience building rigorous, reproducible benchmarking/evaluation frameworks that separate genuine model improvement from prompt overfitting, retrieval effects, or evaluator bias.
• Experience building and scaling AI/ML pipelines and workflows on HPC or cloud environments.
• Demonstrated ownership of a system beyond the prototype stage (observability, versioning, rollback, cost control, incident response).
• Contributions to open‑source AI/ML infrastructure projects (e.g., vLLM, PyTorch, Ray, Hugging Face) or a public track record (GitHub, Hugging Face, papers) are a plus.
• Familiarity with biomedical research, imaging, or regulated health data is a plus.
• Demonstrated technical leadership: setting standards, mentoring, and cross‑team collaboration.

Compensation

In recognition of certain U.S. state and municipal pay transparency laws, St. Jude is including a reasonable estimate of the compensation range for this role. This is an estimate offered in good faith and a specific salary offer takes into account factors that are considered in making compensation decisions including but not limited to skill sets, experience and training, licensure and certifications, and other business and organizational needs. It is not typical for an individual to be hired at or near the top of the salary range and compensation decisions are dependent on the facts and circumstances of each case.

A reasonable estimate of the current salary range is $86,320 - $154,960 per year for the role of Senior Artificial Intelligence Engineer.

Explore our exceptional benefits!

St. Jude is an Equal Opportunity Employer

No Search Firms

St. Jude Children's Research Hospital does not accept unsolicited assistance from search firms for employment opportunities. Please do not call or email. All resumes submitted by search firms to any employee or other representative at St. Jude via email, the internet or in any form and/or method without a valid written search agreement in place and approved by HR will result in no fee being paid in the event the candidate is hired by St. Jude.