Principal AI Systems Engineer

Iovance; United States; Remote; USD 175,000 - 200,000; Full time

Job summary

Iovance Biotherapeutics seeks a senior AI engineer to design, build, and deploy AI systems across regulated life sciences contexts. You will own end-to-end RAG-based solutions, ensure secure deployment, and monitor model performance in collaboration with IT Security and Process owners.

The role emphasizes hands-on coding, architecture, and tooling to deliver production-grade AI capabilities that support manufacturing, QA, regulatory, and commercial functions in a compliant environment.

Qualifications

• 10+ years of progressive software/AI/ML engineering experience with recent focus on LLM-based development.
• Hands-on experience with RAG, embedding models, vector stores, retrieval and grounding for high accuracy.
• Experience with frontier models (Claude, GPT, Gemini) and evaluating models for task fit, cost, latency and safety.
• Strong software fundamentals: Python and another language, Agile delivery, Git, CI/CD, containers, AWS (S3/Redshift/Bedrock).
• Builder mindset; heavy coding, system design, model evaluation, and artifact production.
• Experience with AWS data and AI services is preferred (S3, Redshift, Bedrock, IAM).
• Familiarity with enterprise search and hybrid retrieval (vector + BM25).

Responsibilities

• Design, build, and ship AI systems end-to-end on the Iovance AI roadmap.
• Implement production-grade RAG systems on AWS infrastructure (S3, Redshift, Bedrock).
• Create evaluation harnesses with test sets, metrics, and regression testing.
• Integrate AI systems with enterprise apps using MCP, APIs, and event-driven patterns.
• Own LLM security controls: input/output guardrails, redaction, moderation, and monitoring.
• Develop AI agents with safe write-back to systems of record in regulated environments.
• Establish modern engineering practices: version control, CI/CD, environment separation, reproducible builds.
• Evaluate AI vendors and foundation models with PoC results for governance.
• Implement audit logging, access management, model change control, and monitoring.
• Create architecture/runbooks and validation deliverables; contribute to AI Validation Playbook.
• Mentor junior engineers and keep abreast of AI tooling and best practices.

Skills

LLM development

RAG systems

MCP integration

Python

AWS cloud

CI/CD

Security controls

Education

Bachelor's or Master’s in CS/Engineering

Tools

OpenSearch

Elasticsearch

Bedrock

AWS S3

Redshift

IAM

Git

CI/CD tooling

Job description

Iovance Biotherapeutics aims to be the global leader in innovating, developing and delivering tumor infiltrating lymphocyte (TIL) therapy for people with cancer. We are pioneering a transformational approach to treating cancer by harnessing the ability of the human immune system to recognize and attack diverse cancer cells in each patient. The Iovance TIL platform has demonstrated promising clinical data across multiple solid tumors. We are committed to continuous innovation in cell therapy, including gene-edited cell therapy, which may be a promising option for patients with cancer.

Overview

This is a senior role responsible for the hands‑on design, build, validation, and deployment of Artificial Intelligence (AI) systems at Iovance Biotherapeutics. This role is focused on execution — turning approved use cases into working, validated, production AI systems that deliver measurable business value. This role directly supports Iovance’s strategy to improve overall operational productivity. The ideal candidate is a deeply technical, hands‑on senior engineer with demonstrated production experience designing and shipping Large Language Model (LLM) applications, Retrieval-Augmented Generation (RAG) systems, and modern AI integrations. This position works closely with IT leads, business process owners across Manufacturing, Quality, Regulatory Affairs, Clinical, Commercial, G&A, IT Security, Privacy, and Quality Assurance to deliver AI capabilities that respect Iovance’s regulated environment. This role executes against a roadmap and prioritized backlog, while contributing technical input to refinement, scoping, and sequencing.

Primary Responsibilities

• Design, build, and ship AI systems against the approved Iovance AI roadmap, including end-to-end ownership of architecture, retrieval pipelines, prompts, evaluation, integrations, and deployment for assigned use cases.
• Implement production‑grade Retrieval-Augmented Generation (RAG) systems on Iovance’s AWS infrastructure (S3, Redshift, Bedrock), including chunking strategies, embedding selection, vector storage, retrieval and reranking, grounding, and citation handling appropriate to high‑accuracy use cases.
• Build, maintain, and run evaluation harnesses for AI systems, including held‑out test sets, accuracy and grounding metrics, hallucination detection, adversarial inputs, and regression testing across model and prompt changes; treat evaluation as a first‑class engineering deliverable, not an afterthought.
• Design and implement integrations between AI systems and Iovance enterprise systems using Model Context Protocol (MCP), APIs, and event‑driven patterns, applying least‑privilege access principles and partnering with IT Security on integration approval.
• Own and maintain LLM security controls for production AI systems, including input and output guardrails, prompt injection and jailbreak defenses, sensitive data redaction (PII, PHI, Iovance Confidential Information and Intellectual Property), content moderation, and abuse monitoring, working in partnership with IT Security.
• Design, develop, deploy, and maintain AI agents (multi‑step reasoning systems that combine LLMs with tools, retrieval, and planning) appropriate for use in a regulated life sciences environment, including bounded scope, defined human oversight, traceability of agent decisions, and safe handling of write‑back actions to systems of record
• Establish and uphold modern engineering practices for AI development including version control for code, prompts, and evaluation sets; CI/CD pipelines; environment separation (dev, test, production); and reproducible builds.
• Conduct hands‑on technical evaluation of AI vendors, tools, and Foundation Models when build‑vs‑buy decisions are under consideration; produce concise, fact‑based recommendations to the IT function lead and AI Governance Committee, including proof‑of‑concept results where appropriate.
• Implement and operate technical controls for production AI systems including audit logging, access management, prompt and model change control, model registry, ongoing performance monitoring, and incident detection, in alignment with Iovance policies.
• Author technical documentation appropriate to the system risk tier, including architecture diagrams, data flow diagrams, evaluation reports, runbooks, and validation deliverables; contribute to the Iovance AI Validation Playbook.
• Mentor junior engineers who collaborate on AI projects; stay current on rapid advances in AI tooling, models, and engineering best practices, and bring technical recommendations forward.

Qualifications

• 10+ years of progressive software and/or AI/ML engineering experience, with the most recent 1+ years dedicated primarily to LLM-based application development. Demonstrated track record of shipping production AI systems that real users depend on.
• Deep, hands‑on production experience with Retrieval-Augmented Generation (RAG) including chunking strategies, embedding models, vector stores, retrieval and reranking, and grounding for high‑accuracy use cases. Working production experience with Model Context Protocol (MCP) or equivalent agent/tool integration patterns.
• Practical working experience across multiple frontier model families (e.g., Anthropic Claude, OpenAI GPT, Google Gemini, leading open‑weight models), with the judgment to select among them based on task fit, accuracy, cost, latency, safety properties, and data‑handling commitments. Awareness of model capability changes and how to evaluate new model releases.
• Strong software engineering fundamentals: production Python (and ideally one other language); modern Agile delivery; version control (Git); CI/CD; containerization; cloud platforms (preferably AWS, with working familiarity with S3, Redshift, and Bedrock); MLOps tooling and practices.
• Hands‑on, builder disposition; this role spends the majority of its time writing code, designing systems, evaluating models, and producing technical artifacts — not in meetings.
• Working production experience with AWS data and AI services is strongly preferred (S3, Redshift, Bedrock, IAM).
• Working knowledge of enterprise search and retrieval services such as OpenSearch, Elasticsearch, or equivalent, including the design of hybrid retrieval patterns (vector plus lexical/BM25) that improve grounding and relevance in production RAG systems

Preferred/Desirable Knowledge, Skills, and Education

• Relevant cloud and AI/ML certifications (AWS, Azure, etc.);
• Working knowledge of US regulatory requirements applicable to AI in regulated life sciences, including 21 CFR Part 11, GxP, HIPAA, and emerging FDA expectations on AI/ML in pharmaceutical manufacturing and drug development
• Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or related quantitative discipline.

Physical Demands and Activities Required

• Must be able to remain in a stationary position standing or sitting for prolonged periods of time.
• Must be able to move about inside an office and exert up to 10 pounds of force occasionally or a negligible amount of force frequently or constantly to lift, carry, push, pull, or otherwise move objects.
• Must have visual acuity to perform activities such as: preparing and analyzing data and figures, viewing a computer screen, and extensive reading.
• This position requires repetitive motion, substantial movements (motions) of the wrist, hands, and/or fingers.
• Must be able to communicate with others to exchange information.

Mental:

Clear and conceptual thinking ability; excellent judgment, troubleshooting, problem‑solving, analysis, and discretion; ability to handle work‑related stress; ability to handle multiple priorities simultaneously; and ability to meet deadlines.

This job operates in a professional workplace or remote office environment and requires standard office equipment and keyboards. Employees who work remotely are expected to maintain their workspace and environment safely and free from safety hazards.

#LI-remote

The annual base salary we reasonably expect to pay is listed. Individual pay decisions depend on various factors, such as primary work location, complexity and responsibility of the role, job duties/requirements, and relevant education, experience and skills.

Pay Transparency

$175,000 - $200,000 USD

The statements contained in this document are intended to describe the general nature and level of work being performed by a colleague assigned to this description. They are not intended to constitute a comprehensive list of functions, duties, or local variances. Management retains the discretion to add or to change the duties of the position at any time.

Iovance is committed to cultivating and offering a diverse and inclusive work environment. As an equal‑opportunity employer, our employees and applicants will be considered without regard to an individual’s race, color, religion, sex, pregnancy, national origin, age, physical and mental disability, marital status, sexual orientation, gender identity, gender expression, genetic information, military and veteran status, and any other characteristic protected by applicable law. If you need assistance or accommodation to apply to one of our opportunities, please contact [e-mail] .