Senior Machine Learning Engineer

Navanta; Atlanta (GA); On-site; USD 120,000 - 180,000; Full time

Job summary

Navanta leads an AI platform for regulated banking, and seeks a Lead AI/ML Engineer to own retrieval, text-to-metrics, model serving, and tool orchestration for trusted answers.

Working under the SVP of Technology and Commercial AI, you will collaborate with data, platform, and product teams to ensure correctness and verifiability as the default in high-stakes environments.

Qualifications

• 6-10+ years building software, with 2-3+ years shipping production LLM, RAG, or NLP systems used by real people.
• Demonstrated focus on accuracy and evaluation, not just demos.
• Strong Python and solid software-engineering fundamentals.
• Comfort operating self-hosted open-weight models and reasoning about latency, cost, and quality trade-offs.

Responsibilities

• Build Navanta's retrieval and verifications over data systems with citations for every answer.
• Stand up self-hosted open-weight models serving and embeddings inside client environments; advance RAG standards.
• Design MCP tool layer exposing read-only tools (metrics, documents, customer 360) and evolve to read/write tools.
• Build and maintain the evaluation harness with golden-question regression, groundedness, and explicit 'I don't know' behavior.
• Implement LLM guardrails: PII redaction, prompt-injection defenses, and cost controls aligned to regulatory security.
• Partner with data teams so models select governed metrics from semantic layers rather than improvised SQL.
• Document architecture, evaluation methodology, and guardrails for security reviews and audits.
• Track latency, cost, and quality trade-offs across model versions and deployments.

Skills

Python

Software engineering

NLP

Education

Bachelor's degree in computer science, mathematics, or a related technical field

Tools

vLLM

Ollama

NVIDIA Enterprise (NVAIE)

LlamaIndex

Haystack

Qdrant

pgvector

Job description

The Lead AI/ML Engineer owns the brain of the Navanta AI platform - retrieval, text-to-metrics, model serving, tool orchestration, and the evaluation harness that keeps answers honest. Working under the SVP of Technology and Commercial AI and in close collaboration with the data, platform, and product teams, this role makes "correct and verifiable" the product's default - the foundation of trust in a regulated banking environment where a confident wrong number loses the account.

Key Responsibilities

• Build Navanta's retrieval and verifications over data systems, with shown queries and citations for every answer
• Stand up self-hosted open-weight models serving and embeddings inside each bank's environment or shared environments for Navanta; evolve RAG to a dedicated standard
• Design the MCP tool layer that exposes a small, audited set of read-only tools (metrics, documents, customer 360), eventually growing into read/write tools with heavy amounts of regulated, highly sensitive data
• Build and maintain the evaluation harness - golden-question regression, groundedness and retrieval metrics, explicit "I don't know" behavior - and make it a release gate
• Implement LLM guardrails: PII redaction in prompts and context, prompt-injection defenses, and cost and row limits aligned to regulatory security expectations
• Partner with data teams so the model selects governed metrics from the semantic layer rather than improvising SQL
• Document model architecture, evaluation methodology, and guardrail controls to support customer security reviews and audit readiness
• Track latency, cost, and quality trade-offs across model versions and deployment configurations

Core Competencies

• Accuracy and evaluation orientation - a demonstrated focus on verifiability and groundedness, not just compelling demos
• Production LLM/RAG engineering: retrieval pipelines, tool orchestration, prompt engineering, and guardrail implementation
• Security and compliance mindset: PII handling, prompt-injection defense, and least-privilege tool access aligned to NIST CSF 2.0 principles
• Cross-functional collaboration with data and platform engineering to deliver a governed, auditable AI system

Key Performance Indicators (KPIs)

• Golden-question accuracy - maintained or improved release over release against the verified question set
• Groundedness rate: percentage of assistant answers fully supported by retrieved context
• PII redaction coverage and zero prompt-injection incidents in production
• Model serving latency and cost per query within defined targets
• Evaluation harness adoption as a release gate - zero releases without passing the regression suite

Qualifications

To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required.

• 6-10+ years building software, with 2-3+ years shipping production LLM, RAG, or NLP systems used by real people - not prototypes
• A demonstrated focus on accuracy and evaluation, not just demos
• Strong Python and solid software-engineering fundamentals
• Comfort operating self-hosted open-weight models and reasoning about latency, cost, and quality trade-offs

Core Technologies

• Serving & inference: vLLM, Ollama; GPU / CUDA familiarity, NVIDIA Enterprise (NVAIE)
• RAG & retrieval: LlamaIndex or Haystack; Qdrant, pgvector; embeddings
• Orchestration: MCP, tool / function calling
• Structured querying: text-to-SQL; semantic layers (Cube / dbt MetricFlow)
• Evaluation & guardrails: groundedness and eval frameworks, PII redaction, prompt-injection defense

Nice to Have

• Experience in regulated or high-stakes domains where a wrong answer is costly
• Fine-tuning, adapters, and retrieval-quality optimization
• Familiarity with banking and finance terminology

Education and/or Experience

• Bachelor's degree in computer science, mathematics, or a related technical field, or equivalent hands-on experience