Senior AI Infrastructure Engineer—Self-Hosted & Multi-GPU

Carnegie Mellon University

Pittsburgh (Allegheny County)

On-site

USD 140,000 - 190,000

Full time

14 days+

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Benefits offered by this job

Medical, dental, vision insurance

Tuition benefits

Paid time off and holidays

Pittsburgh bus pass

Retirement plan

Job summary

National Robotics Engineering Center (NREC) at Carnegie Mellon University in Pittsburgh seeks a Senior Applied AI Infrastructure Engineer to lead the evaluation, deployment, and integration of secure generative AI tools and LLMs across engineering workflows. You will help design, host, and optimize AI services on secure, multi-GPU infrastructure.

You will collaborate with leadership and multidisciplinary teams to advance AI-assisted workflows, ensure reliability, and push the boundaries of

Qualifications

B.S. in Computer Science, Computer Engineering, Electrical Engineering, or related field or equivalent experience.
5+ years of professional software engineering, machine-learning infrastructure, DevOps, platform engineering, or developer-tools experience.
Strong Python programming skills.
Linux development and system-administration experience.
Familiarity with large language models, retrieval-augmented generation, tool-using agents, or AI-assisted software-development workflows.
Strong technical communication and documentation skills.
3+ of the following: experience deploying and maintaining software services; Docker and container experience; API integrations; modern software engineering practices (Git, code review, CI/CD, testing, logging, debugging); ability to evaluate new technologies and communicate tradeoffs.

Responsibilities

Evaluate generative and agentic AI tools and recommend practical approaches to engineering leadership.
Support cloud-hosted AI tools where appropriate and locally hosted tools where confidentiality requires it.
Design, implement, document, test, and maintain internally hosted AI services and supporting infrastructure.
Deploy and operate large language models on shared GPUs and project-specific platforms.
Integrate AI tools with engineering systems (source control, Jira, Confluence, Jenkins, internal docs, test infra).
Develop secure tool interfaces, Model Context Protocol servers, and sandboxed environments for AI agents.
Prototype and evaluate AI-assisted workflows for software development, testing, docs, and analysis.
Help engineers use supported AI tools across software, embedded, FPGA, mechanical, electrical workflows.
Create internal docs, examples, training materials, and reusable configurations for tools and practices.
Measure reliability and usefulness of AI-assisted workflows (code quality, tests, reviews, failure modes).
Survey emerging tools and implement promising approaches where valuable.
Follow best practices for team software development (peer review, automated testing, version control, issue tracking, security review, docs).