Senior AI Engineer (US)

Penguin Computing, Inc.; United States; Remote; USD 175,000 - 215,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

401k plan

Job summary

Penguin Solutions in the United States is seeking a Senior AI/ML Engineer to design, build, deploy, and troubleshoot enterprise AI solutions across cloud and on-prem environments. You will work on AI models, agentic applications, and MLOps with the NVIDIA AI stack, delivering secure, production-ready systems.

The role requires strong Python, Linux, Docker, Kubernetes skills and the ability to translate customer requirements into scalable architectures.

Qualifications

• 7+ years of experience in software engineering, ML engineering, AI infrastructure, or distributed systems.
• Strong Python programming skills — testing, packaging, async, and API development, not just scripting.
• GPU model serving at scale. NVIDIA AI Enterprise (NIM, NeMo, Triton Inference Server) is commonly deployed; equivalent stacks like vLLM, TGI, Ray Serve, SageMaker are acceptable.
• Production experience with Linux, Docker, Kubernetes, APIs, and distributed systems.
• Hands-on experience building LLM applications that carry production traffic — retrieval-augmented generation, agentic workflows, tool integration, evaluation, and guardrails.
• Experience deploying beyond a single managed cloud: multi-cloud, on-premises, or hybrid environments.
• Strong communication and technical judgment, with experience explaining tradeoffs to non-specialist stakeholders.

Responsibilities

• Architect and build production AI solutions using commercial, open-source, and customer-developed models.
• Create agentic solutions with retrieval, tools, workflow orchestration, memory, evaluation, guardrails, and observability.
• Design enterprise AI environments using the NVIDIA AI Enterprise stack; build portable solutions for AWS, Azure, GCP, on-prem, and hybrid setups.
• Implement secure, scalable LLMOps and MLOps practices for testing, deployment, monitoring, governance, versioning, and rollback.
• Embed security: address prompt injection, insecure tool use, data exposure, and supply-chain risks; enforce access controls and encryption.
• Translate customer requirements into practical architectures; support proofs of concept, production deployments, and ongoing operations.
• Explain technical options and tradeoffs to customers and internal teams clearly.

Skills

Python

LLM applications

Distributed systems

Multi-cloud deployment

Communication

Tools

NVIDIA AI Enterprise

Docker

Kubernetes

LangGraph

LlamaIndex

PyTorch

Hugging Face

MLflow

Kubeflow

OpenTelemetry

Prometheus

Grafana

Terraform

Ansible

Job description

At Penguin Solutions (Nasdaq: PENG) - The AI Factory Platform Company - we're building a team of innovators who thrive on collaboration, creativity, and the opportunity to help shape the future of AI. As part of the AI technology revolution, our teams design, build, deploy, and manage AI factories for enterprises, sovereign AI initiatives, and neocloud providers worldwide.

Headquartered in Silicon Valley, California, Penguin Solutions operates globally through a network of R&D, manufacturing, and sales locations. For nearly three decades, we have operated at the intersection of memory and AI/HPC infrastructure. That engineering expertise positions us to power the next generation of AI workloads, from training to inference and agentic AI at scale.

Penguin Solutions brings together differentiated infrastructure software, advanced memory, compute systems, end-to-end services, and industry-leading partner solutions in a full-stack AI factory platform designed to help customers deploy and scale AI workloads with speed and precision.

At Penguin Solutions, we value ideas over hierarchy and believe in servant leadership, where leaders enable teams to do their best work. We empower employees to take ownership, drive innovation, and grow through challenging work, continuous learning, and exposure to advanced AI tools and technologies. With flexibility where it matters and a strong focus on outcomes, Penguin Solutions is a place to do your best work, grow your career, and make a meaningful impact.

Job Overview

As an Senior AI/ML Engineer, you will be a hands-on technical expert within our post-sales delivery organization, designing, building, deploying, and troubleshooting enterprise AI solutions for our customers. You will work across AI models, agentic applications, security, MLOps, and the NVIDIA AI Enterprise software stack, supporting public-cloud, sovereign, on-premises, and hybrid environments.

This role is ideal for an experienced AI engineer who enjoys solving complex technical challenges, translating customer requirements into practical solutions, and delivering secure, production-ready AI systems.

Responsibilities

Build Production AI Solutions

• Architect and implement applications using commercial, open-source, and customer-developed models.
• Build agentic solutions incorporating retrieval, tool use, workflow orchestration, memory, evaluation, guardrails, and observability.
• Evaluate models and determine when to use prompt engineering, retrieval-augmented generation, fine-tuning, or other adaptation techniques.
• Integrate AI applications with enterprise data, APIs, tools, and business systems.

Design Enterprise AI Environments

• Design and deploy solutions using the NVIDIA AI Enterprise software stack and the broader AI ecosystem.
• Build portable solutions for AWS, Azure, GCP, on-premises data centers, and hybrid environments.
• Work with containers, Kubernetes, GPU-enabled infrastructure, networking, storage, identity, and platform services.
• Implement reliable LLMOps and MLOps practices for testing, deployment, monitoring, governance, versioning, and rollback.

Embed Security

• Address AI-specific risks such as prompt injection, insecure tool use, sensitive-data exposure, excessive permissions, and software supply-chain vulnerabilities.
• Apply enterprise security practices including access controls, encryption, secrets management, network segmentation, and auditability.
• Support deployments with data-privacy, regulatory, sovereignty, or disconnected-environment requirements.

Deliver Customer Solutions

• Translate customer requirements and constraints into practical architectures and implementations.
• Contribute directly to technical discovery, proofs of concept, production deployments, and operational support.
• Troubleshoot complex issues across AI applications, orchestration platforms, integrations, and infrastructure.
• Clearly explain technical options, limitations, and tradeoffs to customers and internal stakeholders.

Qualifications

• 7+ years of experience in software engineering, ML engineering, AI infrastructure, or distributed systems.
• Strong Python programming skills — testing, packaging, async, and API development, not just scripting.
• GPU model serving at scale. NVIDIA AI Enterprise (NIM, NeMo, Triton Inference Server) is what we deploy most often, and prior experience with it is a significant plus - but equivalent experience with vLLM, TGI, Ray Serve, SageMaker, or similar inference stacks is fully acceptable, and we'll support your ramp-up on the NVIDIA stack.
• Production experience with Linux, Docker, Kubernetes, APIs, and distributed systems.
• Hands-on experience building LLM applications that carried real production traffic - retrieval-augmented generation, agentic workflows, tool integration, evaluation, and guardrails.
• Experience deploying beyond a single managed cloud: multi-cloud, on-premises, or hybrid environments.
• Strong communication and technical judgment, with experience explaining tradeoffs to non-specialist stakeholders.

Preferred Qualifications

• Familiarity with AI-specific security risks (for example, the OWASP Top 10 for LLM Applications) and comfort partnering with security teams on enterprise controls.
• Experience with PyTorch and Hugging Face.
• Experience with agentic frameworks such as LangGraph or LlamaIndex.
• Experience with MLOps platforms such as MLflow or Kubeflow.
• Experience with model fine-tuning, data preparation, or other model-adaptation techniques.
• Experience deploying in sovereign, regulated, disconnected, or air-gapped environments.
• Familiarity with observability and infrastructure-automation tooling (OpenTelemetry, Prometheus/Grafana, Terraform, Ansible).
• Knowledge of AI governance, model risk, data privacy, and responsible-AI practices.
• Experience delivering in customer-facing or professional-services environments.

Location

This is a remote position in the United States.

Travel

Flexible (with travel to client sites and Penguin Solutions offices as needed).

Compensation & Benefits

The base pay range that the Company reasonably expects to pay for this position in the United States is $175,000 - 215,000; the pay ultimately offered may vary based on business considerations, including job-related knowledge, skills, experience, and education. The position is bonus-eligible, and there are medical, dental, and vision benefits available. There is a 401k saving plan and other benefits, such as Paid Time Off, Life Insurance, and an Employee Assistance Plan.

Inclusion & Belonging Statement

We are committed to creating an inclusive environment that embraces differences and fosters belonging for all.

Equal Opportunity Statement

We are an Affiliation/Equal Opportunity Employer and strongly committed to all policies which will afford equal opportunity employment to all qualified persons without regard to age, national origin, race, ethnicity, creed, gender, disability, veteran status, or any other characteristic protected by law.