Senior AI Engineer

Penguin Solutions

United States

Remote

USD 175,000 - 215,000

Full time

2 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical, dental, and vision insurance
401(k) plan
Paid time off
Life insurance
Employee Assistance Plan

Job summary

Penguin Solutions, the AI Factory Platform Company, is seeking a Senior AI/ML Engineer to design, build, deploy and troubleshoot enterprise AI solutions across cloud, on-prem, and hybrid environments. You will work with AI models, agentic applications, and MLOps within NVIDIA AI Enterprise stack.

This remote role in the United States offers a salary range of $175,000 to $215,000 plus bonus eligibility and a comprehensive benefits package.

Qualifications

  • 7+ years of experience in software engineering, ML engineering, AI infrastructure, or distributed systems.
  • Strong Python programming and API development experience.
  • Hands-on experience building LLM applications with production traffic.

Responsibilities

  • Build Production AI Solutions using commercial, open-source, and customer models.
  • Design agentic solutions with retrieval, tool use, memory, evaluation, guardrails, and observability.
  • Evaluate models and determine when to use prompt engineering or retrieval-augmented generation.
  • Integrate AI applications with enterprise data, APIs, tools, and business systems.
  • Design Enterprise AI Environments using NVIDIA AI Enterprise stack for cloud and on-prem deployments.
  • Build portable solutions for AWS, Azure, GCP, on-premises data centers, and hybrid environments.
  • Work with containers, Kubernetes, GPU-enabled infrastructure, networking, storage, identity, and platform services.
  • Implement reliable LLMOps and MLOps practices for testing, deployment, monitoring, governance, and rollback.
  • Address AI-specific security risks and partner with security teams on controls.
  • Support deployments with data privacy, regulatory, sovereignty, or disconnected environments.
  • Deliver customer solutions by translating requirements into architectures and implementations.
  • Contribute to technical discovery, proofs of concept, production deployments, and support.
  • Troubleshoot complex issues across AI apps, orchestration, integrations, and infrastructure.
  • Explain technical options, tradeoffs, and limitations to customers and stakeholders.

Skills

Python programming
LLM applications
Distributed systems
Multi-cloud deployment
Strong communication
Model serving at scale

Tools

NVIDIA AI Enterprise (NIM, NeMo, Triton)
Docker
Kubernetes
PyTorch
Hugging Face
OpenTelemetry
Prometheus/Grafana
Terraform
Ansible

Job description

At Penguin Solutions (Nasdaq: PENG) – The AI Factory Platform Company – we’re building a team of innovators who thrive on collaboration, creativity, and the opportunity to help shape the future of AI. As part of the AI technology revolution, our teams design, build, deploy, and manage AI factories for enterprises, sovereign AI initiatives, and AI cloud providers worldwide. Headquartered in Silicon Valley, California, Penguin Solutions operates globally through a network of R&D, manufacturing, and sales locations. For nearly three decades, we have operated at the intersection of memory and AI/HPC infrastructure. That engineering expertise positions us to power the next generation of AI workloads, from training to inference and agentic AI at scale. Penguin Solutions brings together differentiated infrastructure software, advanced memory, compute systems, end‑to‑end services, and industry‑leading partner solutions in a full‑stack AI factory platform designed to help customers deploy and scale AI workloads with speed and precision. At Penguin Solutions, we value ideas over hierarchy and believe in servant leadership, where leaders enable teams to do their best work. We empower employees to take ownership, drive innovation, and grow through challenging work, continuous learning, and exposure to advanced AI tools and technologies. With flexibility where it matters and a strong focus on outcomes, Penguin Solutions is a place to do your best work, grow your career, and make a meaningful impact.

Job Overview

As an Senior AI/ML Engineer, you will be a hands‑on technical expert within our post‑sales delivery organization, designing, building, deploying, and troubleshooting enterprise AI solutions for our customers. You will work across AI models, agentic applications, security, MLOps, and the NVIDIA AI Enterprise software stack, supporting public‑cloud, sovereign, on‑premises, and hybrid environments. This role is ideal for an experienced AI engineer who enjoys solving complex technical challenges, translating customer requirements into practical solutions, and delivering secure, production‑ready AI systems.

Responsibilities
  • Build Production AI Solutions Architect and implement applications using commercial, open‑source, and customer‑developed models.
  • Build agentic solutions incorporating retrieval, tool use, workflow orchestration, memory, evaluation, guardrails, and observability.
  • Evaluate models and determine when to use prompt engineering, retrieval‑augmented generation, fine‑tuning, or other adaptation techniques.
  • Integrate AI applications with enterprise data, APIs, tools, and business systems.
  • Design Enterprise AI Environments Design and deploy solutions using the NVIDIA AI Enterprise software stack and the broader AI ecosystem.
  • Build portable solutions for AWS, Azure, GCP, on‑premises data centers, and hybrid environments.
  • Work with containers, Kubernetes, GPU‑enabled infrastructure, networking, storage, identity, and platform services.
  • Implement reliable LLMOps and MLOps practices for testing, deployment, monitoring, governance, versioning, and rollback.
  • Embed Security Address AI‑specific risks such as prompt injection, insecure tool use, sensitive‑data exposure, excessive permissions, and software supply‑chain vulnerabilities.
  • Support deployments with data‑privacy, regulatory, sovereignty, or disconnected‑environment requirements.
  • Deliver Customer Solutions Translate customer requirements and constraints into practical architectures and implementations.
  • Contribute directly to technical discovery, proofs of concept, production deployments, and operational support.
  • Troubleshoot complex issues across AI applications, orchestration platforms, integrations, and infrastructure.
  • Clearly explain technical options, limitations, and tradeoffs to customers and internal stakeholders.
Qualifications
  • 7+ years of experience in software engineering, ML engineering, AI infrastructure, or distributed systems.
  • Strong Python programming skills — testing, packaging, async, and API development, not just scripting.
  • GPU model serving at scale.
  • NVIDIA AI Enterprise (NIM, NeMo, Triton Inference Server) is what we deploy most often, and prior experience with it is a significant plus — but equivalent experience with vLLM, TGI, Ray Serve, SageMaker, or similar inference stacks is fully acceptable, and we'll support your ramp‑up on the NVIDIA stack.
  • Production experience with Linux, Docker, Kubernetes, APIs, and distributed systems.
  • Hands‑on experience building LLM applications that carried real production traffic — retrieval‑augmented generation, agentic workflows, tool integration, evaluation, and guardrails.
  • Experience deploying beyond a single managed cloud: multi‑cloud, on‑premises, or hybrid environments.
  • Strong communication and technical judgment, with experience explaining tradeoffs to non‑specialist stakeholders.
Preferred Qualifications
  • Familiarity with AI‑specific security risks (for example, the OWASP Top 10 for LLM Applications) and comfort partnering with security teams on enterprise controls.
  • Experience with PyTorch and Hugging Face.
  • Experience with agentic frameworks such as LangGraph or LlamaIndex.
  • Experience with MLOps platforms such as MLflow or Kubeflow.
  • Experience with model fine‑tuning, data preparation, or other model‑adaptation techniques.
  • Experience deploying in sovereign, regulated, disconnected, or air‑gapped environments.
  • Familiarity with observability and infrastructure‑automation tooling (OpenTelemetry, Prometheus/Grafana, Terraform, Ansible).
  • Knowledge of AI governance, model risk, data privacy, and responsible‑AI practices.
  • Experience delivering in customer‑facing or professional‑services environments.
Location

This is a remote position in the United States. Travel Flexible (with travel to client sites and Penguin Solutions offices as needed).

Compensation & Benefits

The base pay range that the Company reasonably expects to pay for this position in the United States is $175,000 – 215,000; the pay ultimately offered may vary based on business considerations, including job‑related knowledge, skills, experience, and education. The position is bonus‑eligible, and there are medical, dental, and vision benefits available. There is a 401k saving plan and other benefits, such as Paid Time Off, Life Insurance, and an Employee Assistance Plan.

Inclusion & Belonging Statement

We are committed to creating an inclusive environment that embraces differences and fosters belonging for all.

Equal Opportunity Statement

We are an Affinit ive Action/Equal Opportunity Employer and strongly committed to all policies which will afford equal opportunity employment to all qualified persons without regard to age, national origin, race, ethnicity, creed, gender, disability, veteran status, or any other characteristic protected by law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Solution Architect
AI Solution Architect

Penguin Computing, Inc. • United States

Remote
USD 130,000 - 175,000
Sr. Solutions Architect
Sr. Solutions Architect

sghcorp.com • Palo Alto (CA)

On-site
USD 185,000 - 230,000
Medical benefits
Dental benefits
Vision benefits
+4
Sr. Solutions Architect
Sr. Solutions Architect

Penguin Computing, Inc. • United States

On-site
USD 185,000 - 230,000
Health benefits
401k plan
Paid time off
Director, Integration Engineering
Director, Integration Engineering

Penguin Computing • Durham (NC)

Hybrid
USD 215,000 - 230,000
Medical benefits
Dental benefits
Vision benefits
+4
AI Solution Architect
AI Solution Architect

sghcorp.com • Idaho

Remote
USD 130,000 - 162,000
Medical, dental, and vision benefits
401k savings plan
Paid Time Off
+2
Vice President, AI Talent Acquisition
Vice President, AI Talent Acquisition

Penguin Computing, Inc. • Fremont (CA)

On-site
USD 240,000 - 270,000
Medical, dental, vision benefits
401k plan
Paid Time Off
+2
Professional Services Engineer
Professional Services Engineer

Penguin Computing, Inc. • Durham (NC)

Hybrid
USD 97,000 - 120,000
Professional Services Engineer
Professional Services Engineer

Penguin Solutions • North Carolina

Hybrid
USD 97,000 - 120,000
Medical benefits
Dental benefits
Vision benefits
+4
Director, Integration Engineering
Director, Integration Engineering

Penguin Solutions • Durham (NC)

Hybrid
USD 215,000 - 230,000
Medical insurance
Dental insurance
Vision insurance
+4
Professional Services Engineer
Professional Services Engineer

Penguin Solutions • North Carolina

Hybrid
USD 97,000 - 120,000
Health benefits
401k plan
Paid time off
+3