AI Operations Architect & AWS Engineer, Life Sciences

Norstella Group; United States; Remote; USD 180,000 - 230,000; Full time

Job summary

Norstella is seeking an AI Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS infrastructure. You will build agentic AI pipelines powering large-scale clinical data extraction from real-world notes, accelerating drug development and patient access decisions.

The role sits at the intersection of ML infrastructure, cloud engineering, and applied NLP, owning the backbone that turns LLM capabilities into reliable data products while ensuring

Qualifications

• Master's degree in Computer Science, Data Science, or related field.
• 5+ years in production NLP/ML systems on AWS.
• Experience deploying open-weight LLMs (Llama, Gemma, Qwen) at scale.
• Expert Python and LLM orchestration tooling (LangChain/LangGraph).
• Experience with AWS ML services and IaC (Terraform, CloudFormation, CDK).

Responsibilities

• Design and maintain AWS architectures for LLM inference and fine-tuning.
• Build and operate agentic AI pipelines with multi-step LLM orchestration.
• Deploy and manage open-weight LLMs in production with optimized serving.
• Implement evaluation, drift detection, and QC for LLM-based extraction.
• Develop CI/CD pipelines for prompt/versioning and model updates.
• Ensure HIPAA/PHI compliance in LLM deployment and data processing.

Skills

NLP/ML systems

AWS

Python

LLM orchestration

LangChain/LangGraph

Cloud infrastructure

Communication skills

Education

Master's degree in Computer Science/Data Science

Tools

vLLM

TGI

Terraform

CloudFormation

CDK

SageMaker

EC2 GPU

S3

LangChain

Job description

AI Operations Architect & AWS Engineer, Life Sciences

Job Description – AI Operations Architect & AWS Engineer, Life Sciences

About Norstella

Norstella is a premier and critical global life sciences data and AI solutions platform dedicated to improving patient access to life-saving therapies. Norstella supports pharmaceutical and biotech companies across the full drug development lifecycle — from pipeline to patient. Our mission is simple: to help our clients bring therapies to market faster and more efficiently, ultimately impacting patient lives.

Norstella unites five market-leading brands (Citeline, Evaluate, MMIT, Panalgo, and The Dedham Group) that all have a shared vision of improving patient access to life-saving therapy. Norstella delivers must-have answers and insights for critical strategic, clinical, and commercial decision-making. We help our clients:

• Accelerate the drug development cycle
• Make data driven commercial and financial decisions
• Match and recruit patients for clinical trials
• Assess competition and bring the right drugs to market
• Identify and address barriers to patient access
• Leverage the latest AI to turn data into insights for faster and better decision making

Norstella serves almost every pharmaceutical and biotech company in the world, along with regulators like the FDA, and payers. By providing critical proprietary data supporting AI-driven workflows, Norstella helps clients make decisions faster and with greater confidence. Norstella’s investments in AI are transforming how data is consumed and decisions are made, disrupting inefficient legacy workflows and helping the industry become more efficient, innovative, and responsive to patient needs.

About The Position:

We are seeking an LLM Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS infrastructure at Norstella. You will build and maintain the agentic AI pipelines and cloud architectures that power large-scale clinical data extraction from real-world clinical notes, directly accelerating RWD-driven drug development and treatment access for patients with unmet need.

This role sits at the intersection of ML infrastructure, cloud engineering, and applied NLP — you will own the operational backbone that turns LLM capabilities into reliable, scalable data products.

Key Responsibilities:

• Design and maintain AWS cloud architectures optimized for LLM inference and fine-tuning workloads, including GPU instance management, auto-scaling, and cost optimization across spot and on-demand capacity.
• Build and operate agentic AI pipelines — multi-step LLM orchestration workflows with tool use, structured output extraction, retrieval-augmented generation (RAG), and automated quality control loops.
• Deploy and manage open-weight LLMs (Llama, Gemma, Qwen, and successors) in production, including model serving infrastructure, batching strategies, and latency/throughput optimization.
• Implement robust evaluation and monitoring frameworks for LLM-based extraction systems: automated accuracy measurement, drift detection, prompt regression testing, and human-in-the-loop QC integration.
• Develop and maintain CI/CD pipelines for prompt versioning, model updates, and configuration management across development, staging, and production environments.
• Collaborate with NLP scientists, data engineers, and software developers to translate prototype extraction logic into production-hardened, recurrent processing feeds.
• Ensure compliance with data security requirements (HIPAA, PHI handling) in all LLM deployment and data processing workflows, working closely with IT security and compliance teams.
• Analyze and optimize computational resource utilization — GPU hours, storage, network throughput — balancing cost efficiency against processing SLAs.
• Evaluate and integrate emerging LLM serving technologies, orchestration frameworks, and inference optimization techniques (quantization, speculative decoding, structured generation) to maintain operational edge.

Required Qualifications

• Master's in Computer Science, Data Science, or related field.
• 5+ years in production NLP/ML systems development and operations on AWS.
• Hands‑on experience deploying and serving open-weight LLMs (Llama, Gemma, Qwen families) at scale, using frameworks such as vLLM, TGI, or equivalent serving infrastructure.
• Expert Python proficiency and working knowledge of LLM orchestration tooling (e.g., LangChain/LangGraph, custom agent frameworks).
• Experience with AWS ML/compute services (SageMaker, EC2 GPU instances, S3, Step Functions, Lambda) and infrastructure-as-code (Terraform, CloudFormation, or CDK).
• Proven track record deploying large-scale NLP/LLM solutions in healthcare or life sciences.
• Strong analytical and communication skills; comfortable operating across multiple concurrent projects.

Preferred Qualifications

• AWS certifications (Solutions Architect, Machine Learning Specialty, or DevOps Engineer).
• Direct experience with PHI handling and HIPAA-compliant ML infrastructure.
• Experience with AWS Redshift, OpenSearch/Elasticsearch, or similar large-scale data stores.
• Familiarity with ML experiment tracking and lifecycle management (MLflow, Weights & Biases, or equivalent).
• Experience building or operating RAG systems, structured extraction pipelines, or clinical NLP applications.

Our guiding principles for success at Norstella:

• Bold, Passionate, Mission-First We have a lofty mission to Smooth Access to Life Saving Therapies, and we will get there by being bold and passionate about the mission and our clients. Our clients and the mission in what we are trying to accomplish must be in the forefront of our minds in everything we do.
• Integrity, Truth, Reality We make promises that we can keep, and goals that push us to new heights. Our integrity offers us the opportunity to learn and improve by being honest about what works and what doesn’t. By being true to the data and producing realistic metrics, we are able to create plans and resources to achieve our goals.
• Kindness, Empathy, Grace We will empathize with everyone’s situation, provide positive and constructive feedback with kindness, and accept opportunities for improvement with grace and gratitude. We use this principle across the organization to collaborate and build lines of open communication.
• Resilience, Mettle, Perseverance We will persevere — even in difficult and challenging situations. Our ability to recover from missteps and failures in a positive way will help us to be successful in our mission.
• Humility, Gratitude, Learning We will be true learners by showing humility and gratitude in our work. We recognize that the smartest person in the room is the one who is always listening, learning, and willing to shift their thinking.

Norstella is an equal opportunities employer and does not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, color, nationality, ethnic or national origin, religion or belief, disability or age. Our ethos is to respect and value people’s differences, to help everyone achieve more at work as well as in their personal lives so that they feel proud of the part they play in our success. We believe that all decisions about people at work should be based on the individual’s abilities, skills, performance and behavior and our business requirements. Norstella operates a zero-tolerance policy to any form of discrimination, abuse or harassment.