ML Engineer: RAG & LLM Validation for Advanced AI

Janus Soft Inc.; McLean (VA); On-site; USD 180,000 - 230,000; Full time

Job summary

Janus Soft Inc. in McLean, Virginia seeks a Machine Learning Subject Matter Expert to lead evaluation, fine-tuning, and validation of state-of-the-art AI capabilities, with emphasis on Retrieval-Augmented Generation (RAG) testing and LLM architectures for national security missions.

You will design rigorous experimentation frameworks and verification benches, leverage GPU-accelerated training, and deploy robust models for semantic search and NLP, ensuring rigor, scalability, and mission-focused

Qualifications

• Active TS/SCI clearance with Full Scope Poly.
• Experience designing and executing LLM validation and RAG benchmarks.
• Experience fine-tuning LLMs on custom datasets for operational use.
• Proficiency with PyTorch, TensorFlow or Keras for model development.
• Hands-on experience with GPUs for training and inference.
• Strong Python or R programming skills.
• Advanced SQL skills including CTEs and subqueries.
• GitHub and CI/CD tooling experience.

Responsibilities

• Lead evaluation, fine-tuning, and validation of LLM-based systems and architectures.
• Design experimentation frameworks and verification benches for LLM testing.
• Benchmark performance against baselines and report results to stakeholders.
• Deploy robust models for semantic search and NLP in mission-focused contexts.

Skills

RAG & LLM Validation

Neural Network Tuning

Deep Learning Frameworks

Python or R

GPU Acceleration

SQL

GitHub & Jenkins

Tools

PyTorch

TensorFlow

Keras

Hugging Face Transformers

Flask

Tableau

AWS

ONNX

Job description

Janus Soft Inc. in McLean, Virginia seeks a Machine Learning Subject Matter Expert to lead evaluation, fine-tuning, and validation of state-of-the-art AI capabilities, with emphasis on Retrieval-Augmented Generation (RAG) testing and LLM architectures for national security missions.

You will design rigorous experimentation frameworks and verification benches, leverage GPU-accelerated training, and deploy robust models for semantic search and NLP, ensuring rigor, scalability, and mission-focused