Type of Requisition: Pipeline
Clearance Level Must Currently Possess: Secret
Clearance Level Must Be Able to Obtain: Top Secret
Public Trust/Other Required: None
Job Family: Data Science and Data Engineering
Job Qualifications
- Skills: Agile Software Development, Back-End Development, Microservices Architecture
- Certifications: None
- Experience: 7 + years of related experience
- US Citizenship Required: Yes
Job Description
GDIT is seeking a highly motivated and experienced Senior AI Software Engineer to support the development, modernization, integration, and sustainment of our on-premises, AI-powered enterprise chatbot platform supporting the U.S. Department of State in Sterling, VA. The selected candidate will design, develop, and integrate software solutions leveraging Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), vector databases, document processing technologies, and modern AI inference frameworks. This role requires a strong foundation in software engineering, Python development, API integration, containerization, and AI application architecture. The engineer will help transition existing AI services toward a scalable, containerized architecture while introducing new capabilities that enhance system performance, reliability, security, and user experience. The ideal candidate is a hands‑on software engineer who can work across the AI application stack, from backend development and model serving integration to data processing, automated testing, and deployment. The individual will collaborate with AI/ML engineers, data engineers, infrastructure specialists, cybersecurity teams, and government stakeholders to deliver secure, reliable AI capabilities in highly controlled, classified, and air‑gapped environments.
Key Responsibilities
AI Application Development & Integration
- Design, develop, test, and maintain Python-based applications, microservices, and APIs supporting an on prem chatbot.
- Extend, configure, and integrate OpenWebUI with enterprise AI services and backend systems.
- Develop custom OpenWebUI Functions, Tools, and integrations to support mission‑specific use cases.
- Integrate approved LLMs into chatbot using OpenAI-compatible APIs and inference frameworks such as vLLM and Ollama.
- Develop reusable software components for AI model access, prompt management, response processing, and application workflows.
- Collaborate with AI/ML engineers to improve model inference performance, response quality, and application reliability.
- Support multimodal AI use cases, including document understanding, optical character recognition, and vision‑language model integration.
- Develop backend capabilities to support secure enterprise AI workflows.
Retrieval‑Augmented Generation (RAG) & Document Processing
- Design, implement, and optimize RAG workflows for enterprise document search and question answering.
- Integrate document processing technologies such as Docling, Tesseract, and vision‑language models into automated ingestion pipelines.
- Develop document chunking, metadata extraction, embedding generation, and retrieval workflows.
- Integrate embedding models and reranking models to improve retrieval accuracy and relevance.
- Work with PostgreSQL and pgvector to support vector search, semantic retrieval, and document indexing.
- Develop APIs and background processing services for document ingestion and knowledge base management.
- Collaborate with data engineers to ensure accurate storage, indexing, and retrieval of enterprise information.
- Implement testing and evaluation approaches to measure RAG accuracy, retrieval relevance, and answer quality.
AI Model Serving & Inference Integration
- Integrate and maintain AI inference services using vLLM, Ollama, and compatible model‑serving frameworks.
- Support the deployment and configuration of approved open‑weight LLMs, embedding models, rerankers, and vision‑language models.
- Develop software integrations using OpenAI-compatible REST APIs.
- Assist in optimizing inference configurations, including context length, GPU memory utilization, concurrency, and model serving performance.
- Support the transition from existing inference frameworks to improved architectures as technologies mature.
- Collaborate with AI/ML engineers to troubleshoot inference errors, model loading issues, and integration failures.
- Develop benchmarking and performance‑testing utilities for AI workloads.
Containerization & Platform Modernization
- Support the migration of DS Chat components from traditional deployments into containerized services.
- Develop, configure, and maintain container images using Podman or Docker.
- Create and maintain Podman Compose configurations defining application services, networking, volumes, and environment variables.
- Integrate services such such as OpenWebUI, vLLM, Docling, PostgreSQL, pgvector, PgBouncer, and Redis.
- Develop scripts and automation for repeatable installation, deployment, upgrades, and recovery.
- Implement health checks, readiness checks, logging, a