Senior AI Software Engineer: On-Prem Chatbot & LLM Ops

General Dynamics Information Technology, Inc.; Sterling (VA); On-site; USD 128,000 - 173,000; Full time

Job summary

General Dynamics Information Technology, Inc. seeks a Senior AI Software Engineer to design, develop, and deploy an on-prem AI-powered enterprise chatbot for the U.S. Department of State in Sterling, VA.

You will work across AI stacks—from backend services and model serving to data processing and automated testing—within highly controlled, air-gapped environments. The role emphasizes containerized architectures, OpenWebUI integrations, and secure, scalable AI capabilities with a strong focus on

Qualifications

• Minimum Secret Clearance, no sponsorship
• 5+ years of professional software engineering experience, including backend development
• Strong proficiency in Python and production-quality software
• Experience developing RESTful APIs and integrating distributed application services
• Hands-on experience integrating LLMs or generative AI capabilities into software applications
• Understanding of transformer-based models, inference APIs, prompt engineering, and LLM application development
• Experience with RAG concepts, including document ingestion, embeddings, vector search, and retrieval pipelines
• Experience working with relational databases such as PostgreSQL
• Familiarity with container technologies such as Podman or Docker
• Experience with Linux-based development and application deployment
• Proficiency with Git, automated testing, debugging, and software development best practices
• Experience troubleshooting applications across multiple interconnected services
• Strong analytical, problem-solving, communication, and technical documentation skills
• Ability to work effectively with cross-functional teams and government stakeholders

Responsibilities

• AI Application Development & Integration: Design, develop, test, and maintain Python-based applications, microservices, and APIs for an on-prem chatbot.
• RAG & Document Processing: Implement RAG workflows and integrate document processing tech into ingestion pipelines.
• AI Model Serving & Inference: Integrate and maintain AI inference services and model-serving frameworks.
• Containerization & Platform Modernization: Migrate DS Chat components to containerized services and ensure secure deployments.
• Agentic AI & Future Capabilities: Prototype agentic AI capabilities and secure tool-calling workflows.
• Software Engineering, Testing & DevSecOps: Build automated tests and support CI/CD in disconnected/classified environments.
• Security & Classified Environment Support: Design solutions for secure on-prem environments and RMF/ATO readiness.

Skills

Agile Software Development

Back-End Development

Microservices Architecture

Python

RESTful APIs

LLMs integration

AI model serving

Containerization

PostgreSQL

pgvector

Linux

Git

Education

Bachelor's degree in Computer Science/Software Engineering/AI/Computer Engineering

Tools

OpenWebUI

vLLM

Ollama

PostgreSQL

pgvector

Docker/Podman

Job description

General Dynamics Information Technology, Inc. seeks a Senior AI Software Engineer to design, develop, and deploy an on-prem AI-powered enterprise chatbot for the U.S. Department of State in Sterling, VA.

You will work across AI stacks—from backend services and model serving to data processing and automated testing—within highly controlled, air-gapped environments. The role emphasizes containerized architectures, OpenWebUI integrations, and secure, scalable AI capabilities with a strong focus on