On-Prem AI Infra Engineer: Datacenter & AI Workloads

AMAX; Fremont (CA); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k)

Flexible spending account

Commuter benefits

Disability insurance

Job summary

AMAX is seeking a hands-on Infrastructure Engineer to design, build, and operate on-prem infrastructure that mirrors a cloud environment for internal teams, including AI/ML workloads. You’ll own datacenter operations, manage compute/storage/networking, and automate with Terraform and Ansible.

You will deploy HashiCorp Vault/Boundary with Keycloak, implement observability with Prometheus and Grafana, and run container workloads using Docker and Kubernetes.

Qualifications

• Experience with on-prem infrastructure operations.
• Experience with Infrastructure-as-Code and Configuration-as-Code.
• Hands-on with Docker and Kubernetes.
• Experience with Prometheus, Grafana, Alertmanager for observability.
• Familiarity with Terraform and Terragrunt for IaC.
• Experience with automation scripting and internal tooling.

Responsibilities

• Design, build, and operate on-prem infrastructure that behaves like a cloud environment for internal teams, including AI/ML workloads.
• Own datacenter and infrastructure operations: compute, storage, networking, and the automation layered on top.
• Build and maintain Infrastructure-as-Code using Terraform (Terragrunt), Ansible, etc.
• Deploy, secure, and operate our HashiCorp stack (Vault, Boundary) alongside identity/access via Keycloak.
• Build and maintain observability with Prometheus, Grafana, and Alertmanager.
• Deploy and manage containerized workloads via Docker and Kubernetes.
• Design and maintain networking infrastructure: VLANs, routing, firewall rules, and load balancers.
• Write scripts/tools to automate operational toil.
• Provide general datacenter support for customers colocated in the HostMax datacenter—cabling, hardware swaps, rack/stack work, and basic troubleshooting.
• Participate in business-hours on-call rotation for customer and infrastructure support.
• Participate in an SDLC/Scrum-based workflow, using Jira and Confluence for planning and documentation.
• Help define standards, runbooks, and best practices as the team grows.
• Development and deployment of AI infrastructure workloads on-prem (GPU scheduling, model serving infra, self-hosted AI tooling).
• Solid experience with IaC and Configuration-as-Code.
• Real-world on-prem/datacenter operations experience — not cloud-native; environment designed to feel cloud-like.
• Working knowledge of networking fundamentals: VLANs, routing, firewalls, load balancers.
• Hands-on with containerization/orchestration (Docker, Kubernetes).
• Proficiency with Git and standard version control workflows.
• Development/scripting skills for internal tooling and automation.
• Comfort with hands-on datacenter/hardware work (cabling, rack/stack, hardware troubleshooting).
• Medical Insurance
• Dental Insurance
• Vision Insurance
• 401(k)
• Flexible spending account
• Commuter benefits
• Disability insurance

Skills

Infrastructure-as-Code

Docker

Kubernetes

Terraform

Terragrunt

Prometheus

Grafana

Git

Automation

Networking fundamentals

Tools

Terraform (Terragrunt)

Ansible

Keycloak

Job description

AMAX is seeking a hands-on Infrastructure Engineer to design, build, and operate on-prem infrastructure that mirrors a cloud environment for internal teams, including AI/ML workloads. You’ll own datacenter operations, manage compute/storage/networking, and automate with Terraform and Ansible.

You will deploy HashiCorp Vault/Boundary with Keycloak, implement observability with Prometheus and Grafana, and run container workloads using Docker and Kubernetes.