Senior LLMOps Engineer (AWS & GenAI)

Norstella

India

Remote

INR 2,500,000 - 4,500,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health Insurance
Provident Fund
Life Insurance
Reimbursement of CertificationExpenses
Gratuity
24x7 Health Desk

Job summary

Norstella Group is seeking an LLM Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS. You will build agentic AI pipelines and cloud architectures that power real-world clinical data extraction from notes, accelerating RWD-driven drug development and patient access.

You will own the operational backbone for scalable, compliant data products, collaborating with NLP scientists and software engineers to turn prototypes into robust production feeds.

Qualifications

  • 5+ years in production NLP/ML on AWS.
  • Deploying open-weight LLMs at scale (Llama, Gemma, Qwen) with modern serving infra.
  • Expert Python and LLM orchestration tooling.
  • Experience with AWS ML/compute services and IaC.

Responsibilities

  • Design and maintain AWS cloud architectures for LLM inference and fine-tuning.
  • Build and operate agentic AI pipelines with multi-step LLM orchestration and RAG.
  • Deploy and manage open-weight LLMs in production with scalable serving infra.
  • Implement monitoring, drift detection, and QA for LLM-based extraction.
  • Develop CI/CD pipelines for prompt/version control across environments.
  • Collaborate with NLP scientists and engineers to production-harden extraction feeds.
  • Ensure HIPAA/PHI compliance in LLM deployment and data workflows.
  • Optimize GPU hours, storage, and network throughput to meet SLAs.
  • Evaluate new LLM serving tech and inference optimization techniques.

Education

Master's in Computer Science

Tools

SageMaker
EC2 GPU
S3
Terraform
CDK
vLLM

Job description

Company: Norstella
Location: Remote, India
Date Posted: Nov 4, 2026
Employment Type: Full Time
Job ID: R-2210

About Norstella

Norstella is a premier and critical global life sciences data and AI solutions provider dedicated to improving patient access to life-saving therapies. Norstella supports pharmaceutical and biotech companies across the full drug development lifecycle - from pipeline to patient. Our mission is simple: to help our clients bring therapies to market faster and more efficiently, ultimately impacting patient lives.

Norstella unites market-leading brands - Citeline, Evaluate, MMIT, Panalgo, Skipta and The Dedham Group and delivers must-have answers and insights, leveraging AI, for critical strategic, clinical, and commercial decision-making. We help our clients:

  • Accelerate the drug development cycle
  • Assess competition and bring the right drugs to market
  • Make data driven commercial and financial decisions
  • Match and recruit patients for clinical trials
  • Identify and address barriers to therapies

Norstella serves most pharmaceutical and biotech companies around the world, along with regulators like the FDA, and payers. By providing critical proprietary data supporting AI-driven workflows, Norstella helps clients make decisions faster and with greater confidence. Norstella's investments in AI are transforming how data is consumed and decisions are made, disrupting inefficient legacy workflows and helping the industry become more efficient, innovative, and responsive to patient needs.

Job Description:

We are seeking an LLM Operations Architect & AWS Engineer to design, deploy, and operate production-grade LLM systems on AWS infrastructure at Norstella. You will build and maintain the agentic AI pipelines and cloud architectures that power large-scale clinical data extraction from real‑world clinical notes, directly accelerating RWD‑driven drug development and treatment access for patients with unmet need.

This role sits at the intersection of ML infrastructure, cloud engineering, and applied NLP — you will own the operational backbone that turns LLM capabilities into reliable, scalable data products.

Responsibilities:
  • Design and maintain AWS cloud architectures optimized for LLM inference and fine‑tuning workloads, including GPU instance management, auto‑scaling, and cost optimization across spot and on‑demand capacity.
  • Build and operate agentic AI pipelines — multi‑step LLM orchestration workflows with tool use, structured output extraction, retrieval‑augmented generation (RAG), and automated quality control loops.
  • Deploy and manage open‑weight LLMs (Llama, Gemma, Qwen, and successors) in production, including model serving infrastructure, batching strategies, and latency/throughput optimization.
  • Implement robust evaluation and monitoring frameworks for LLM‑based extraction systems: automated accuracy measurement, drift detection, prompt regression testing, and human‑in‑the‑loop QC integration.
  • Develop and maintain CI/CD pipelines for prompt versioning, model updates, and configuration management across development, staging, and production environments.
  • Collaborate with NLP scientists, data engineers, and software developers to translate prototype extraction logic into production‑hardened, recurrent processing feeds.
  • Ensure compliance with data security requirements (HIPAA, PHI handling) in all LLM deployment and data processing workflows, working closely with IT security and compliance teams.
  • Analyze and optimize computational resource utilization — GPU hours, storage, network throughput — balancing cost efficiency against processing SLAs.
  • Evaluate and integrate emerging LLM serving technologies, orchestration frameworks, and inference optimization techniques (quantization, speculative decoding, structured generation) to maintain operational edge.
Requirements:
  • Master's in Computer Science, Data Science, or related field.
  • 5+ years in production NLP/ML systems development and operations on AWS.
  • Hands‑on experience deploying and serving open‑weight LLMs (Llama, Gemma, Qwen families) at scale, using frameworks such as vLLM, TGI, or equivalent serving infrastructure.
  • Expert Python proficiency and working knowledge of LLM orchestration tooling (e.g., LangChain/LangGraph, custom agent frameworks).
  • Experience with AWS ML/compute services (SageMaker, EC2 GPU instances, S3, Step Functions, Lambda) and infrastructure-as-code (Terraform, CloudFormation, or CDK).
  • Proven track record deploying large‑scale NLP/LLM solutions in healthcare or life sciences.
  • Strong analytical and communication skills; comfortable operating across multiple concurrent projects.
Preferred Qualifications
  • AWS certifications (Solutions Architect, Machine Learning Specialty, or DevOps Engineer).
  • Direct experience with PHI handling and HIPAA‑compliant ML infrastructure.
  • Experience with AWS Redshift, OpenSearch/Elasticsearch, or similar large‑scale data stores.
  • Familiarity with ML experiment tracking and lifecycle management (MLflow, Weights & Biases, or equivalent).
  • Experience building or operating RAG systems, structured extraction pipelines, or clinical NLP applications.
  • Health Insurance
  • Provident Fund
  • Life Insurance
  • Reimbursement of Certification Expenses
  • Gratuity
  • 24x7 Health Desk
Our guiding principles for success at Norstella
01: Bold, Passionate, Mission-First

We have a lofty mission to Smooth Access to Life Saving Therapies and we will get there by being bold and passionate about the mission and our clients. Our clients and the mission in what we are trying to accomplish must be in the forefront of our minds in everything we do.

02: Integrity, Truth, Reality

We make promises that we can keep, and goals that push us to new heights. Our integrity offers us the opportunity to learn and improve by being honest about what works and what doesn’t. By being true to the data and producing realistic metrics, we are able to create plans and resources to achieve our goals.

03: Kindness, Empathy, Grace

We will empathize with everyone's situation, provide positive and constructive feedback with kindness, and accept opportunities for improvement with grace and gratitude. We use this principle across the organization to collaborate and build lines of open communication.

04: Resilience, Mettle, Perseverance

We will persevere - even in difficult and challenging situations. Our ability to recover from missteps and failures in a positive way will help us to be successful in our mission.

05: Humility, Gratitude, Learning

We will be true learners by showing humility and gratitude in our work. We recognize that the smartest person in the room is the one who is always listening, learning, and willing to shift their thinking.

Norstella is an equal opportunities employer and does not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, color, nationality, ethnic or national origin, religion or belief, disability or age. Our ethos is to respect and value people's differences, to help everyone achieve more at work as well as in their personal lives so that they feel proud of the part they play in our success. We believe that all decisions about people at work should be based on the individual's abilities, skills, performance and behavior and our business requirements. Norstella operates a zero tolerance policy to any form of discrimination, abuse or harassment.

Sometimes the best opportunities are hidden by self-doubt. We disqualify ourselves before we have the opportunity to be considered. Regardless of where you came from, how you identify, or the path that led you here- you are welcome. If you read this job description and feel passion and excitement, we’re just as excited about you.

All legitimate roles with Norstella will be posted on Norstella's job board which is located at norstella.com/careers. If a role is not posted on this job board, a candidate should assume the role is not a legitimate role with Norstella. Norstella is not responsible for an application that may be submitted by or through a third-party and candidates should proceed with extreme caution if a third-party approaches them about an open role with Norstella. Norstella will never ask for anything of value or any type of payment during or as part of any recruitment, interview, or pre‑hire onboarding process. If you are aware of or have reason to believe a job posting purportedly for a role with Norstella is fraudulent or otherwise not authorized by Norstella, please contact the Company using the following email address: [emailprotected]

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Technical Operations (RWD Data Delivery)
Technical Operations (RWD Data Delivery)

Norstella • India

Remote
INR 1,200,000 - 1,800,000
Health Insurance
Provident Fund
Gratuity
+1
Delivery Operations
Delivery Operations

Norstella • India

Remote
INR 1,200,000 - 2,000,000
Health Insurance
Provident Fund
Gratuity
+2
CRM Support Specialist (ADM201 certified)
CRM Support Specialist (ADM201 certified)

Norstella • India

Remote
INR 600,000 - 800,000
Health Insurance
Provident Fund
Life Insurance
+3
Multimedia Producer — News & Insights
Multimedia Producer — News & Insights

Norstella • India

On-site
INR 600,000 - 1,200,000
Health Insurance
Provident Fund
Reimbursement of CertificationExpenses
+2
Multimedia Producer — News & Insights
Multimedia Producer — News & Insights

Norstella • Bengaluru

Remote
INR 800,000 - 1,200,000
Health Insurance
Provident Fund
Reimbursement of Certification Costs
+2
Commissions Analyst
Commissions Analyst

Norstella Group • India

On-site
INR 800,000 - 1,200,000
Health Insurance
Provident Fund
Certification Reimbursement
+2
Accounts Payable Specialist
Accounts Payable Specialist

Norstella • Bengaluru

On-site
INR 600,000 - 1,200,000
Senior Fullstack Engineer - Product Engineering
Senior Fullstack Engineer - Product Engineering

Norstella • India

On-site
INR 1,800,000 - 3,200,000
Lead AI/ML Engineer
Lead AI/ML Engineer

UnitedHealth Group • Bengaluru

On-site
INR 3,000,000 - 6,000,000
Data Scientist - II
Data Scientist - II

Netomi • Gurugram District

On-site
INR 1,200,000 - 2,000,000
Flexible working hours
Health insurance
Professional development opportunities