Senior AI DevOps / LLMOps

TechBiz Global GmbH; Kraków; On-site; PLN 260,000 - 520,000; Full time

Job summary

TechBiz Global is seeking a Senior AI DevOps / LLMOps specialist to join one of our client teams in Kraków. You will design and implement AI-focused CI/CD, IaC, and MLOps pipelines to enable safe, scalable deployment of models and agents.

Responsibilities include building canary and blue-green strategies, policy-as-code, monitoring of inference endpoints, and ensuring security and compliance across hybrid cloud environments. The role demands 10+ years in DevOps and 2+ years in MLOps/LLMOps.

Qualifications

• 10+ years in DevOps, SRE, or Cloud Engineering.
• 2+ years in MLOps or LLMOps, moving LLMs to production.
• Proven experience managing Hybrid Cloud environments (AWS/Azure + Private Data Center).

Responsibilities

• Automate Build-to-Production for AI, including model weights, dataset versioning, and code.
• Develop PromptOps workflows with version-controlled system prompts.
• Automate deployment of Agentic workflows with stateful AI interactions and multi-agent handoffs.
• Provide AI Infrastructure as Code with GPU clusters and Terraform/Pulumi/Ansible.
• Define Policy-as-Code for endpoints ensuring security, cost usage limits, and data residency.
• Maintain parity between On-Premises and Cloud environments.
• Architect Progressive Delivery strategies (Canary, Blue-Green, Shadowing).
• Build Evaluation-in-the-Loop gates to test bias, hallucination, and performance.
• Implement A/B testing for LLM outputs and agentic behavior.
• Establish deep observability into Inference Endpoints with latency, tokens/second, and drift.

Skills

Kubernetes

CI/CD

IaC

AI Tooling

GPU virtualization

Security & secret management

Hybrid cloud

Tools

Terraform

Pulumi

Ansible

GitHub Actions

GitLab CI

KubeFlow

Ray

NVIDIA Triton

Job description

At TechBiz Global, we are providing recruitment service to our TOP clients from our portfolio. We are currently seeking an Senior AI DevOps / LLMOps specialist to join one of our clients' teams. If you're looking for an exciting opportunity to grow in a innovative environment, this could be the perfect fit for you.

Key Responsibilities

1. Automation of Build-to-Production

- Design and implement robust CI/CD pipelines tailored for AI, covering model weights, dataset versioning, and application code.

- Develop specialized workflows for PromptOps, ensuring that system prompts are version-controlled, tested for regressions, and deployed with the same rigor as traditional code.

- Automate the deployment of Agentic workflows, managing the complexities of stateful AI interactions and multi-agent handoffs.

2. AI Infrastructure as Code (IaC)

- Provision and manage high-performance compute environments (GPU clusters, TPU pods) using Terraform, Pulumi, or Ansible.

- Define and enforce Policy-as-Code for AI endpoints to ensure compliance with security, cost-usage limits, and data residency requirements.

- Maintain a consistent environment across Hybrid Infrastructure, ensuring seamless parity between On-Premises development and Cloud production.

3. Safe Experimentation & Controlled Releases

- Architect Progressive Delivery strategies for AI, including Canary releases, Blue-Green deployments, and Shadowing (where new models run in parallel with production to compare outputs).

- Build Evaluation-in-the-Loop gates within the pipeline to automatically test for bias, hallucination, and performance degradation before a release.

- Implement A/B testing frameworks specifically designed for LLM outputs and agentic behavior.

4. Monitoring & Observability

- Establish deep observability into Inference Endpoints, tracking metrics like tokens-per-second, latency, and drift in model accuracy.

-Integrate feedback loops that capture production edge cases to feed back into the training and fine-tuning pipelines.

Must-Have Technical Skills:

-Orchestration: Advanced Kubernetes (K8s) skills, specifically with KubeFlow, Ray, or NVIDIA Triton.

-CI/CD & IaC: Expertise in GitHub Actions/GitLab CI, and Terraform or Pulumi.

- AI Tooling: Experience with Weights & Biases, MLflow, LangSmith, or Arize Phoenix.

-Hardware: Understanding of GPU virtualization, CUDA drivers, and on-premises hardware management.

-Security: Familiarity with Open Policy Agent (OPA) and secret management (Vault).

Experience:

- 10+ years in DevOps, SRE, or Cloud Engineering.

- 2+ years of hands-on experience in MLOps or LLMOps, specifically moving LLMs from notebook to production.

-Proven experience managing Hybrid Cloud environments (e.g., AWS/Azure + Private Data Center).