Senior Artificial Intelligence Engineer

St. Jude Children's Research Hospital, Inc.; Memphis (TN); On-site; USD 86,000 - 155,000; Full time

Benefits offered by this job

Exceptional benefits

Equal opportunity employer

Job summary

St. Jude Children’s Research Hospital, Inc. is seeking a Senior AI Engineer to lead advanced AI models (LLMs, agentic AI, multi‑modal foundation models) and secure HPC infrastructure in Memphis.

This hands‑on role involves evaluating, fine‑tuning, deploying, and benchmarking models, plus designing safety guardrails and data‑privacy safeguards. You will optimize GPU/HPC resource allocation, build shared AI architecture, and collaborate with CBI and other teams.

Qualifications

• Master’s degree in computer science, computer engineering, data science, information technology or related field required.
• PhD degree in data science, computer science, computer engineering, information technology or related field preferred.
• 4 years in designing and developing solutions for large scale AI/ML systems and/or building AI/ML features for products.

Responsibilities

• Assists Research Information Services in assessing institution’s needs and implementing AI-enabled services on high‑performance AI computing platforms.
• Works with scientists, business stakeholders, analysts, and IS professionals in design, development, and implementation of AI solutions and services, including proof‑of‑concepts, pilots, and adoption lifecycle.
• Builds new and innovative solutions leveraging data science and AI/ML skills to solve non‑trivial problems.
• Develops AI demonstration use‑cases and workshop materials; provides workshops and training classes.
• Evaluates commercial and open‑source approaches in AI/ML, Data Mining, and Analytics to solve business problems.
• Documents current and future state architecture roadmaps and reference architectures.
• Provides clear written and spoken communications to customers, teams, and vendors.
• Keeps abreast of new and emerging technologies and stays adaptable to their potential applicability.
• Performs other duties as assigned to meet department goals.

Skills

TensorFlow

PyTorch

Keras

Time series analysis

Anomaly detection

Forecasting

Predictive modeling

Graph-based neural networks

Bayesian statistics

Text analytics

Education

Master’s degree in computer science, computer engineering, data science, information technology or related field

PhD preferred in data science, computer science, computer engineering, information technology or related field

Tools

NCCL

CUDA

Slurm

vLLM

TensorRT

Job description

High Performance Research Computing (HPRC) and the Center for Bioimage Informatics (CBI) at St. Jude Children's Research Hospital are seeking a Senior AI Engineer to lead our efforts in advanced AI models, including large language models (LLMs), agentic AI systems, and multi-modal foundation models, and the secure computational infrastructure that powers them. This is a hands‑on, high‑ownership software systems role, not a prompt‑engineering, API‑integration, or purely conceptual research position. You will evaluate, fine‑tune, deploy, and benchmark AI models; design and enforce safety guardrails and sandboxing for agentic systems; safeguard data security and privacy for sensitive research data; and optimize GPU/HPC resource allocation to balance performance, cost, and efficiency as models and tooling rapidly evolve. This person builds the shared AI architecture, secure environments, and best practices upon which CBI's image data scientists and software engineers rely. Deep bioimaging expertise is not required, though experience with biomedical research or imaging is a plus. This position reports to the Director of HPRC and works closely with CBI as a collaborative team member. This is an onsite role in Memphis, TN.

Job Responsibilities

Assists Research Information Services in (i) assessing institution’s needs and (ii) implementing AI-enabled services on high‑performance AI computing platforms.

Works with scientists, business stakeholders, analysts, and IS professionals in the design, development, and implementation of AI solutions and services, including technology proof‑of‑concepts, pilots, and the adoption lifecycle.

Be responsible for building new and innovative solutions leveraging data science and AI/ML skills and technologies to solve non‑trivial problems.

Develops AI demonstration use‑cases and workshop materials and provides workshops and training classes.

Evaluate commercial and open‑source approaches in AI/ML, Data Mining, and Analytics to solve business problems.

Identifies, develops, and implements standards and operating procedures for solutions and systems consistent with best practices.

Documents current and future state architecture roadmaps and reference architectures.

Provides clear written and spoken communications to customers, teams, and vendors.

Keeps abreast of new and emerging technologies and stays adaptable to their potential applicability.

Performs other duties as assigned or directed to meet the goals and objectives of the department and institution.

Minimum Education and/or Training

Master’s degree in computer science, computer engineering, data science, information technology or related field required. PhD degree in data science, computer science, computer engineering, information technology or related field preferred.

Minimum Experience

4 years in designing and developing solutions for large scale AI/Machine Learning (ML) systems and/or building solutions for a product on AI/ML features and capabilities.

Strong background in industry use cases built on deep learning and machine learning (unsupervised and supervised techniques) is a must.

Deep learning frameworks such as TensorFlow, Keras, PyTorch, Time series analysis, anomaly detection, forecasting, predictive modeling, graph‑based neural networks, Bayesian statistics, and text analytics are a must.

Preferred Qualifications

Hands‑on experience training, fine‑tuning, or adapting LLMs or multi‑modal foundation models (e.g., PEFT/LoRA, instruction tuning, preference optimization), including debugging failure modes such as catastrophic forgetting or training instability.

Experience diagnosing and resolving distributed/multi‑GPU training or inference issues (e.g., NCCL communication hangs, CUDA out‑of‑memory errors, load‑balancing across nodes) and scheduling AI workloads on HPC (e.g., Slurm) to maximize GPU utilization.

Experience designing safety guardrails and sandboxing for agentic systems: tool‑access scoping, prompt‑injection defense, secrets management, audit logging, and containment of failures.

Experience optimizing inference cost, latency, and resource usage (e.g., KV‑cache management, quantization, batching, speculative decoding, high‑throughput serving via vLLM/TensorRT‑LLM/Triton) and judgment about when a simpler deterministic pipeline is a better fit than an agentic one.

Experience safeguarding data security and privacy for AI systems handling sensitive research data, including access controls and institutional data‑use/security policies.

Experience building rigorous, reproducible benchmarking/evaluation frameworks that separate genuine model improvement from prompt overfitting, retrieval effects, or evaluator bias.

Experience building and scaling AI/ML pipelines and workflows on HPC or cloud environments.

Demonstrated ownership of a system beyond the prototype stage (observability, versioning, rollback, cost control, incident response).

Contributions to open‑source AI/ML infrastructure projects (e.g., vLLM, PyTorch, Ray, HuggingFace) or a public track record (GitHub, HuggingFace, papers) are a plus.

Familiarity with biomedical research, imaging, or regulated health data is a plus.

Demonstrated technical leadership: setting standards, mentoring, and cross‑team collaboration.

Compensation

In recognition of certain U.S. state and municipal pay transparency laws, St.Jude is including a reasonable estimate of the compensation range for this role. This is an estimate offered in good faith and a specific salary offer takes into account factors that are considered in making compensation decisions including but not limited to skill sets, experience and training, licensure and certifications, and other business and organizational needs. It is not typical for an individual to be hired at or near the top of the salary range and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current salary range is $86,320- $154,960 per year for the role of Senior Artificial Intelligence Engineer.

Benefits

Explore our exceptional benefits!

Equal Opportunity

St.Jude is an Equal Opportunity Employer.

St.Jude Children’s Research Hospital does not accept unsolicited assistance from search firms for employment opportunities. Please do not call or email. All resumes submitted by search firms to any employee or other representative at St.Jude via email, the internet or in any form and/or method without a valid written search agreement in place and approved by HR will result in no fee being paid in the event the candidate is hired by St.Jude.