AI Architect

Insight Softmax Consulting, LLC.; Canada; Remote; CAD 150.000 - 210.000; Vollzeit

Zusammenfassung

Insight Softmax Consulting, LLC. is seeking an experienced AI Architect to lead discovery and design of AI platforms, including GPU clusters, inference serving, and AI gateways. You will work closely with customers to translate business goals into scalable architectures and guide engineers through implementation.

The role is remote with work hours aligned to Latin America and Canada time zones. A hands-on, results-driven approach is essential for delivering robust AI infrastructure solutions.

Qualifikationen

• 8+ years in infrastructure, platform or ML engineering.
• 3+ years designing and operating production AI/ML platforms at scale.
• Hands-on experience deploying LLM inference in production (e.g., vLLM, TensorRT-LLM, llm-d, Triton).
• Deep knowledge of GPU infrastructure and Kubernetes scheduling.

Aufgaben

• Lead technical discovery with customers and translate goals into AI platform architectures, phased roadmaps and effort estimates.
• Design inference serving platforms on Kubernetes and bare metal: model serving, GPU scheduling and partitioning, autoscaling, multi-tenancy, and capacity planning.
• Design AI gateway and routing layers: tenant-aware routing, model pools, budgets and quotas, rate limiting, observability, and multi-site failover.
• Define reference architectures for multi-tenant GPU clouds, sovereign and air-gapped deployments, and hybrid cloud and on-prem estates.
• Advise on AI usage governance: cost visibility, token accounting, audit, data handling, and security posture for LLM services.
• Evaluate and recommend build, buy or compose decisions across open source and commercial AI infrastructure components.
• Review implementations delivered by our AI and platform engineers and own the technical quality of the outcome.
• Represent Insight Softmax in customer architecture reviews, partner enablement sessions and executive readouts.
• Reserve time each week for pre-sales support, internal platform work and mentoring.

Kenntnisse

Kubernetes

Linux

AWS

LLM inference

GPU infrastructure

API design

Python

Go

Rust

API gateway

Tools

vLLM

TensorRT-LLM

llm-d

Triton

LitellM

Traefik

Envoy

Tetrate

Kubernetes Gateway API

Jobbeschreibung

Insight Softmax is a leading organization in the fields of machine learning, data science, andhigh-performance computing (HPC). We are dedicated to providing innovative solutions incomputing technology to solve complex problems in various sectors including research,science, manufacturing, finance, and data analytics.

Description:

• We are hiring an AI Architect for our engineering team. You will be the senior technical voicefor customers who are building or operating AI infrastructure: GPU clusters, inferenceplatforms, AI gateways, and the services that sit on top of them.
• Our customers range from chip vendors and neoclouds running thousands of GPUs, toenterprises standing up their first production LLM services, to platform vendors who have usbuild parts of their product. They ask us how the leading AI labs and AI-native companies runtheir platforms, and they expect answers grounded in experience.
• You will lead discovery and architecture phases of engagements, produce referencearchitectures and recommendations, and hand clear designs to our engineers forimplementation. You will also support pre-sales scoping and estimates.
• This is a hands‑on architect role. You are expected to prototype, validate assumptions in a lab,and review the implementation, not only produce slides.
• Our team requires individuals with focus on delivery and results, hunger to learn, ability to adapt, and can thrive in a self‑directed environment where you’ll figure out what’s needed to deliver rather than wait for detailed instructions.

Key Responsibilities:

• Lead technical discovery with customers and translate business goals into AI platformarchitectures, phased roadmaps and effort estimates.
• Design inference serving platforms on Kubernetes and bare metal: model serving, GPUscheduling and partitioning, autoscaling, multi‑tenancy, and capacity planning.
• Design AI gateway and routing layers: tenant‑aware routing, model pools, budgets and quotas,rate limiting, observability, and multi‑site failover.
• Define reference architectures for multi‑tenant GPU clouds, sovereign and air‑gappeddeployments, and hybrid cloud and on‑prem estates.
• Advise on AI usage governance: cost visibility, token accounting, audit, data handling, andsecurity posture for LLM services.
• Evaluate and recommend build, buy or compose decisions across open source and commercialAI infrastructure components.
• Review implementations delivered by our AI and platform engineers and own the technicalquality of the outcome.
• Represent Insight Softmax in customer architecture reviews, partner enablement sessions andexecutive readouts.
• Reserve time each week for pre‑sales support, internal platform work and mentoring.

Qualifications:

• At least 8 years in infrastructure, platform or ML engineering, with at least 3 years designingand operating production AI or ML platforms at scale.
• Hands‑on experience deploying LLM inference in production (for example vLLM,TensorRT-LLM, llm-d, Triton) including tensor and pipeline parallelism, batching, caching andperformance tuning.
• Deep understanding of GPU infrastructure: NVIDIA and AMD accelerators, drivers andruntimes, GPU scheduling on Kubernetes, high‑speed interconnects and storage for AIworkloads.
• Experience designing AI gateway or API management layers (for example LiteLLM, Traefik,Envoy or Tetrate based designs, Kubernetes Gateway API inference extensions).Strong Kubernetes and Linux skills. Able to read and debug the stack from the scheduler downto the node.
• Deep knowledge of at least one of AWS, GCP, or Azure. Preferably AWS.
• Track record of leading architecture engagements with enterprise customers, including writingproposals, estimates and architecture documents that engineers can execute against.
• Clear written and spoken communication with both engineers and executives.

Optional Experience and Skills:

• Experience at an AI lab, AI-native company, or neocloud, or building platform products forthem.
• Multi‑tenant GPU cloud design, including namespace and cluster‑level isolation, quota, billingand chargeback.
• Slurm or other batch schedulers alongside Kubernetes for mixed HPC and AI estates.
• RAG, agentic systems, model evaluation and fine‑tuning pipelines.
• Air‑gapped, FIPS or other regulated deployment experience.
• Spectro Cloud Palette and PaletteAI, Run:ai, or similar GPU platform products.
• Observability for AI services: traces, token and cost metrics, quality monitoring.

Bonus Experience and Skills:

• Python, Go or Rust.
• Software engineering background beyond infrastructure.
• Published talks, blog posts or open source contributions in AI infrastructure.
• Using linux as your primary desktop workstation environment.

Attributes that we value:

• Transparency
• Grit
• Effectiveness
• Willingness to learn

Location and Hours:

• Full time position
• Remote work environment
• 80% of your work schedule will be during the business time zones for Latin America and Canada.
• Some customer meetings will occur each week at non‑standard / nighttime hours, as we currently support a global team across 3-4 continents.
• Occasional travel to customer sites in the US for architecture workshops, load tests and go‑live
  reviews.

References:

• 3 references will be required.

Notes:

• AI systems may be used with the recruitment and screening processes for this job requisition.