AI Infrastructure Engineer — Agents & ML Systems

HavocAI; Town of Providence (NY); On-site; USD 150,000 - 210,000; Full time

Benefits offered by this job

Health, Dental and Vision Insurance (6

Life Insurance

401k (Matching)

Unlimited PTO

Equity Package

Work/Home Office Stipend

Global Entry

Parental Leave (16 weeks)

Health & Wellness Stipend

Job summary

HavocAI in Providence, RI, seeks an AI Infrastructure Engineer – Agents & ML Systems to build production‑grade internal AI infrastructure that safely connects LLMs, agents, data sources, and tooling across engineering teams.

You will develop pipelines, integrations, and observability for AI workflows, ensure secure tool usage, and partner with Software, Data, Simulation, and Product teams to deliver high‑impact internal tools for autonomous systems.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Machine Learning, Data Science, Applied Mathematics, or a related technical field.
• 3+ years of experience in software engineering, infrastructure engineering, ML infrastructure, backend systems, data engineering, developer tools, or related technical roles.
• Strong programming experience in Python, TypeScript, Go, C++, or similar languages.
• Experience building production software systems, APIs, services, data pipelines, or internal platforms.
• Experience with, or strong interest in, LLM applications, AI agents, tool‑using systems, RAG pipelines, or AI developer tools.
• Familiarity with modern AI infrastructure concepts such as embeddings, vector search, prompt management, evaluation, model serving, fine‑tuning, or MLOps.
• Ability to integrate systems across APIs, databases, object stores, documents, logs, internal tools, and structured or unstructured data sources.
• Strong understanding of production engineering fundamentals, including reliability, observability, testing, and maintainability.
• Strong grounding in securing AI and agentic systems, including least‑privilege tool access, prompt‑injection and misuse mitigation, secrets management, and safe handling of sensitive data.
• Ability to work across Software, Data, ML, Infrastructure, and Product teams.
• Strong debugging skills and comfort working with complex distributed systems.
• U.S. citizenship and the ability to obtain and maintain a security clearance.

Responsibilities

• Build internal AI infrastructure that connects LLMs and AI agents with internal tools, APIs, data sources, data lakes, telemetry stores, simulation tools, code repositories, documentation systems, logs, and engineering workflows.
• Develop and maintain agentic AI systems for task automation, data analysis, engineering support, simulation workflows, and internal productivity.
• Build tool integration and connector infrastructure for AI agents, including MCP and other emerging tool‑use standards, spanning servers, tools, resources, prompts, connectors, and secure tool‑use patterns.
• Create pipelines for retrieval, RAG, context management, document processing, embeddings, and internal knowledge search.
• Support ML infrastructure workflows such as data preparation, dataset curation, experiment tracking, model evaluation, fine‑tuning support, and model deployment.
• Build evaluation frameworks for agent performance, tool‑use reliability, task success, model quality, regression testing, and failure analysis.
• Develop observability, logging, tracing, auditability, monitoring, and debugging tools for AI agents, model calls, MCP tools, and ML pipelines.
• Partner with Autonomy, Software, Data, Simulation, Product, and Operations teams to identify high‑value AI use cases and turn them into reliable internal tools.
• Secure agentic AI systems end‑to‑end with least‑privilege tool access, sandboxed tool execution, prompt‑injection and misuse mitigation, secrets management, human‑in‑the‑loop approvals, and safe handling of sensitive and defense data.
• Maintain documentation, reusable examples, templates, and best practices that help internal teams adopt AI tools safely and effectively.

Skills

Python

TypeScript

Go

C++

APIs

ML infra

Education

Bachelor's degree in CS or related

Tools

Kubernetes

Docker

LangChain

Ray

Airflow

MLflow

Job description

HavocAI in Providence, RI, seeks an AI Infrastructure Engineer – Agents & ML Systems to build production‑grade internal AI infrastructure that safely connects LLMs, agents, data sources, and tooling across engineering teams.

You will develop pipelines, integrations, and observability for AI workflows, ensure secure tool usage, and partner with Software, Data, Simulation, and Product teams to deliver high‑impact internal tools for autonomous systems.