Azure OpenAI & RAG Engineer - Gov Cloud Specialist

SkyePoint Decisions

Vansville (MD)

On-site

USD 140,000 - 160,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Certification incentives

PTO

Floating federal holidays

Health insurance

Dental & Vision

401k matched

Job summary

SkyePoint Decisions seeks an experienced AI Engineer with Azure OpenAI to design and implement a retrieval-augmented generation pipeline for the Department of State. The role covers provisioning Azure OpenAI Service on Azure Government, building a RAG workflow grounded in agency playbooks and live feeds, and implementing a confidence scoring mechanism.

The engineer collaborates with a Power Platform Developer on prompt context and with an agency SME on response quality, working on on-site

Qualifications

3+ years of experience with Azure OpenAI Service in enterprise environments.
Experience building RAG pipelines using Azure AI Search or equivalent.
Familiarity with LLM prompt engineering techniques including few-shot prompting, chain-of-thought, and system prompt design.
Experience with FedRAMP-authorized cloud environments, specifically Azure Government.
Experience with REST API integrations for data feeds (JSON APIs, rate limiting, error handling).
Active Top Secret clearance OR ability to obtain TS within program timeline (interim clearance accepted to start).
US Citizenship is required.

Responsibilities

Provision and configure Azure OpenAI Service on Azure Government in compliance with DOS GovCloud FedRAMP High requirements.
Design and implement the RAG pipeline using Azure AI Search: vector index configuration, document chunking strategy for agency playbooks and runbooks, embedding model selection, and semantic search tuning.
Build and maintain automated data ingestion pipelines for CISA KEV (daily), NVD CVE database (daily), Microsoft Security Update Guide (weekly), and agency SharePoint document library.
Implement ServiceNow ticket context injection: when a matching SNOW ticket is retrieved, structure the vulnerability description, CVE identifier, asset details, and current ticket status into the AI prompt context for response generation.
Design and implement the AI confidence scoring mechanism — per-response confidence scores surfaced to agency analysts in the validation queue.
Implement CUI-safe prompt construction: ensure sanitized email content and structured SNOW ticket data are assembled into prompts without including raw CUI elements that have been vaulted.
Configure Azure AI Content Safety filters appropriate to the advisory use case.
Monitor model performance, token consumption, and response latency in the Azure Government environment.
Support the monthly review of AI confidence score distribution, override rate, and coverage metrics.
Produce AI system documentation including prompt templates, RAG configuration, and model monitoring procedures.