AI Ops Architect & AWS Engineer — Life Sciences

Norstella; United States; On-site; USD 180,000 - 240,000; Full time

Job summary

Norstella is seeking an LLM Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS infrastructure at Norstella.

You will build and maintain the agentic AI pipelines and cloud architectures that power large-scale clinical data extraction from real-world clinical notes, directly accelerating RWD-driven drug development and treatment access for patients with unmet need.

Qualifications

• Master's in Computer Science, Data Science, or related field.
• 5+ years in production NLP/ML systems development and operations on AWS.
• Hands-on experience deploying and serving open-weight LLMs (Llama, Gemma, Qwen) at scale.
• Expert Python proficiency and working knowledge of LLM orchestration tooling (e.g., LangChain/LangGraph, custom agent frameworks).
• Experience with AWS ML/compute services (SageMaker, EC2 GPU instances, S3, Step Functions, Lambda) and infrastructure-as-code (Terraform, CloudFormation, or CDK).
• Proven track record deploying large-scale NLP/LLM solutions in healthcare or life sciences.
• Strong analytical and communication skills; comfortable operating across multiple concurrent projects.

Responsibilities

• Design and maintain AWS cloud architectures optimized for LLM inference and fine-tuning workloads, including GPU instance management, auto-scaling, and cost optimization across spot and on-demand capacity.
• Build and operate agentic AI pipelines — multi-step LLM orchestration workflows with tool use, structured output extraction, retrieval-augmented generation (RAG), and automated quality control loops.
• Deploy and manage open-weight LLMs (Llama, Gemma, Qwen) in production, including model serving infrastructure, batching strategies, and latency/throughput optimization.
• Implement robust evaluation and monitoring frameworks for LLM-based extraction systems: automated accuracy measurement, drift detection, prompt regression testing, and human-in-the-loop QC integration.
• Develop and maintain CI/CD pipelines for prompt versioning, model updates, and configuration management across development, staging, and production environments.
• Collaborate with NLP scientists, data engineers, and software developers to translate prototype extraction logic into production-hardened, recurrent processing feeds.
• Ensure compliance with data security requirements (HIPAA, PHI handling) in all LLM deployment and data processing workflows, working closely with IT security and compliance teams.
• Analyze and optimize computational resource utilization — GPU hours, storage, network throughput — balancing cost efficiency against processing SLAs.
• Evaluate and integrate emerging LLM serving technologies, orchestration frameworks, and inference optimization techniques (quantization, speculative decoding, structured generation) to maintain operational edge.

Skills

NLP/ML on AWS

Python

LLM orchestration tooling

LLM deployment at scale

Terraform / CloudFormation / CDK

AWS ML/compute services

Education

Master's in Computer Science, Data Science, or related field

Tools

vLLM

TGI

LangChain / LangGraph

Job description

Norstella is seeking an LLM Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS infrastructure at Norstella.

You will build and maintain the agentic AI pipelines and cloud architectures that power large-scale clinical data extraction from real-world clinical notes, directly accelerating RWD-driven drug development and treatment access for patients with unmet need.