Principal Software Engineer: Real-Time ML & LLM

Hispanic Alliance for Career Enhancement; Trenton (NJ); On-site; USD 144,200 - 288,400; Full time

Job summary

CVS Health is seeking a Principal Software Engineer to lead a real‑time multi‑service clinical platform on Google Cloud, personally building the most critical and performance‑sensitive components and mentoring engineers across UI, backend and ML.

You will design scalable LLM pipelines, implement strict latency targets with sub‑second responsiveness, and deploy models on Vertex AI while driving CI/CD, testing, observability, and robust data contracts to ensure compliant, high‑impact health

Qualifications

• 10+ years of software engineering experience delivering production systems at scale.
• 5+ years as hands-on Staff or Principal-level engineer.
• Proficiency in Python including performance tuning and concurrency.
• 5+ years building real-time distributed systems with latency requirements.
• 5+ years in a major public cloud (GCP preferred).
• Experience with microservices, REST APIs, and resilient systems.
• 3+ years building production ML systems (MLOps).
• 3+ years developing LLM-powered applications in production.
• Experience building modern UI components with React or Angular.
• Strong CS fundamentals in distributed systems and concurrency.

Responsibilities

• Own architecture and execution of a real-time, multi-service clinical platform on GCP.
• Build production-grade LLM systems with prompt engineering, retrieval, and evaluation frameworks.
• Deliver real-time, high-performance systems with sub-second latency and strong fault tolerance.
• Operationalize ML on Vertex AI, including feature pipelines, drift detection, and CI/CD‑integrated quality controls.
• Design scalable APIs, microservices, and data contracts with strong testing and observability.
• Drive high-quality code reviews and establish standards across testing, CI/CD, security, and observability.
• Own outcomes end-to-end and iteratively improve based on real-world data.
• Collaborate with Data Science, Product, Clinical SMEs, AI Governance, and Security to meet enterprise standards.

Skills

Software architecture

Python proficiency

Real-time distributed systems

Public cloud (GCP)

Microservices & REST APIs

ML systems (MLOps)

LLM applications

React/Angular UI

CS fundamentals

Leadership / mentoring

Education

Bachelor's degree in Computer Science or related field

Master's or PhD in CS / ML (a plus)

Tools

Vertex AI

GKE

Cloud Run

Pub/Sub

BigQuery

IAM

React

Job description

CVS Health is seeking a Principal Software Engineer to lead a real‑time multi‑service clinical platform on Google Cloud, personally building the most critical and performance‑sensitive components and mentoring engineers across UI, backend and ML.

You will design scalable LLM pipelines, implement strict latency targets with sub‑second responsiveness, and deploy models on Vertex AI while driving CI/CD, testing, observability, and robust data contracts to ensure compliant, high‑impact health