On-Prem AI Systems Engineer – LLM/Agent Tech

Innovative Defense Technologies (IDT); Mount Laurel Township (NJ); On-site; USD 114,000 - 231,000; Full time

Benefits offered by this job

Generous benefits package

401(k) with matching

9/80 optional schedule

Job summary

Innovative Defense Technologies (IDT) seeks an AI Implementation Engineer to design and deliver on-prem AI solutions for air-gapped environments, with a focus on end-to-end agentic systems and MCP interfaces. The role requires active Secret clearance and on-site work in Mt.

Laurel, NJ or Arlington, VA with 3+ days in the office. The engineer will integrate LLMs and tools, develop RAG pipelines, deploy production-grade AI workflows, and optimize local CPU/GPU workloads while collaborating with

Qualifications

• Bachelor’s degree in IT/CS/CE/EE/Systems Eng/Physics/Math
• 5-10+ years in software/ML/AI engineering or related roles
• Proficiency in Python with NumPy and Pandas
• Hands-on with production local inference engines (vLLM, SGLang, Triton, TensorRT-LLM)
• Experience with PyTorch, TensorFlow, LangChain, LangGraph, Semantic Kernel or AutoGen
• Experience with Linux and hardening (fapolicy/selinux/fips)
• Experience with hosting LLM servers and containerization
• Ability to work independently and in Agile teams
• Strong leadership and documentation skills
• Willingness to travel up to 10%

Responsibilities

• Design and Build AI Solutions for On-Prem systems in Air-Gapped environments with end-to-end agentic capabilities
• Integrate Models and Tools across LLMs, APIs, data sources, MCP interfaces
• Develop Retrieval-Augmented Generation (RAG) pipelines for grounding and relevance
• Engineer Conversational and Agentic Interfaces with memory and multi-turn interactions
• Implement and Evaluate AI Workflows; optimize prompts, orchestration, retrieval, and performance
• Architect local infrastructure to optimize CPU/GPU workloads and quantization for throughput
• Orchestrate disconnected environments and offline model update pipelines
• Define and validate technical requirements with stakeholders
• Collaborate across teams to integrate AI into broader architectures
• Support testing, debugging, and performance optimization of AI-enabled apps
• Stay current with advances in LLMs and agent frameworks
• Document architectures, workflows, and decisions for maintainability
• Travel up to 10% as needed

Skills

Python

NumPy

Pandas

LLM tooling

LLM servers

PyTorch

TensorFlow

LangChain

LangGraph

Semantic Kernel

AutoGen

Linux hardening

Containerization

MCP interfaces

RAG pipelines

Education

Bachelor's degree in IT/CS/CE/EE/Systems Eng/Physics/Math

Tools

vLLM

SGLang

Triton Inference Server

TensorRT-LLM

Docker

Kubernetes

Job description

Innovative Defense Technologies (IDT) seeks an AI Implementation Engineer to design and deliver on-prem AI solutions for air-gapped environments, with a focus on end-to-end agentic systems and MCP interfaces. The role requires active Secret clearance and on-site work in Mt.

Laurel, NJ or Arlington, VA with 3+ days in the office. The engineer will integrate LLMs and tools, develop RAG pipelines, deploy production-grade AI workflows, and optimize local CPU/GPU workloads while collaborating with