G2001 - Machine Learning Engineer - Mclean VA (STAR 2219-01)

Janus Soft Inc.; McLean (VA); On-site; USD 180,000 - 230,000; Full time

Job summary

Janus Soft Inc. in McLean, Virginia seeks a Machine Learning Subject Matter Expert to lead evaluation, fine-tuning, and validation of state-of-the-art AI capabilities, with emphasis on Retrieval-Augmented Generation (RAG) testing and LLM architectures for national security missions.

You will design rigorous experimentation frameworks and verification benches, leverage GPU-accelerated training, and deploy robust models for semantic search and NLP, ensuring rigor, scalability, and mission-focused

Qualifications

• Active TS/SCI clearance with Full Scope Poly.
• Experience designing and executing LLM validation and RAG benchmarks.
• Experience fine-tuning LLMs on custom datasets for operational use.
• Proficiency with PyTorch, TensorFlow or Keras for model development.
• Hands-on experience with GPUs for training and inference.
• Strong Python or R programming skills.
• Advanced SQL skills including CTEs and subqueries.
• GitHub and CI/CD tooling experience.

Responsibilities

• Lead evaluation, fine-tuning, and validation of LLM-based systems and architectures.
• Design experimentation frameworks and verification benches for LLM testing.
• Benchmark performance against baselines and report results to stakeholders.
• Deploy robust models for semantic search and NLP in mission-focused contexts.

Skills

RAG & LLM Validation

Neural Network Tuning

Deep Learning Frameworks

Python or R

GPU Acceleration

SQL

GitHub & Jenkins

Tools

PyTorch

TensorFlow

Keras

Hugging Face Transformers

Flask

Tableau

AWS

ONNX

Job description

Job Description Are you an elite Machine Learning Engineer ready to push the boundaries of Large Language Models (LLMs) and advanced AI systems in support of critical national security missions? Janus Soft is seeking a Machine Learning Subject Matter Expert to lead the evaluation, fine-tuning, and validation of state-of-the-art AI capabilities in McLean, VA.

In this pivotal role, you will design rigorous experimentation frameworks and verification test benches to evaluate LLM-based architectures, with an emphasis on Retrieval-Augmented Generation (RAG) testing and validation. Leveraging GPU-accelerated computing and deep learning frameworks (PyTorch, TensorFlow, Keras), you will tune neural networks on custom domain datasets, benchmark performance metrics against baseline standards, and deploy resilient models for semantic search and Natural Language Processing (NLP). If you thrive at the intersection of applied AI research, rigorous testing methodologies, and mission-focused delivery, Janus Soft offers the elite environment and compensation package to support your career growth.

Mandatory Skills

Clearance: Active TS/SCI with Full Scope Polygraph (Strict Requirement).

RAG & LLM Validation: Demonstrated experience in experiment design and execution to test, evaluate, and validate LLM-based systems and architectures, specifically including Retrieval-Augmented Generation (RAG) testing against established benchmarks.

Neural Network Tuning: Proven experience fine-tuning neural networks and Large Language Models (LLMs) on custom datasets and applying outputs to specialized operational use cases.

Deep Learning Frameworks: In-depth professional or academic experience with deep learning frameworks such as PyTorch , TensorFlow , or Keras .

Accelerated Computing: Hands-on experience leveraging GPUs for accelerated computing, model training, and inference.

Programming & Model Development: Strong proficiency in Python or R for developing AI/ML models and ensembles, including feature engineering, data preprocessing, library selection, and model performance analysis.

Relational Databases & SQL: Advanced proficiency with SQL , including common table expressions (CTEs), set operations, aggregate functions, and nested subqueries.

Version Control & CI/CD: Hands-on experience utilizing version control and automation tools such as GitHub and Jenkins .

Desired Skills

Demonstrated experience utilizing the Hugging Face Transformers library and ecosystem.

Experience in the design and development of novel benchmarks to quantify and measure LLM performance.

Experience with semantic search architectures and vector search applications.

Familiarity with Explainable AI ( XAI ) methodologies and Open Neural Network Exchange ( ONNX ).

Experience designing verification and validation (V&V) test benches for AI systems.

Demonstrated capability in text classification and topic modeling utilizing scikit-learn and deep learning techniques.

Experience with cloud computing architectures (e.g., AWS) and lightweight web frameworks such as Flask .

Data visualization and reporting experience with Tableau to communicate methodological approaches and model outcomes to stakeholders.