AI Ops Engineer (m/f/d)

Insight 42; United States; Remote; USD 106,000 - 168,000; Full time

Benefits offered by this job

Remote work in Germany

Competitive salary

Performance-based incentives

Job summary

Insight42 in Germany is hiring an AI Ops Engineer to design and run GenAI infrastructure across cloud and edge environments. You will architect self-hosted GenAI workflows, optimize GPU/CPU orchestration, and automate model deployment for secure, scalable AI systems.

Applicants should have 4+ years in DevOps/MLOps, hands-on GenAI deployments, strong Docker/Kubernetes skills, and experience with cloud platforms like Azure or GCP. German language is a plus, English required.

Qualifications

• 4+ years of experience in DevOps, MLOps, or AI infrastructure engineering.
• Hands-on experience deploying self-hosted GenAI models (LLMs, multimodal, or diffusion models).
• Strong understanding of containerization, microservices, and orchestration (Docker, Kubernetes).
• Familiarity with agentic ecosystems, AI orchestration frameworks, and vector databases.
• Knowledge of GPU orchestration, distributed inference, or on-prem AI serving.
• Solid programming and automation skills (Python, Bash, or Go).
• Cloud experience with Azure, GCP, or hybrid infrastructure setups.

Responsibilities

• Design and maintain self-hosted GenAI model deployments in secure cloud environments.
• Build and automate workflows for AI model lifecycle management (training, tuning, and versioning).
• Build observability and reliability into AI workloads.
• Collaborate with MLOps, DevOps, and Data teams to align infrastructure with model requirements.
• Integrate models into agentic frameworks and autonomous system ecosystems.
• Optimize infrastructure for performance, cost efficiency, and scalability.
• Establish CI/CD and GitOps practices for AI and ML system delivery.

Skills

DevOps

MLOps

AI infrastructure

GenAI

Docker

Kubernetes

Python

Go

Tools

Docker

Kubernetes

Azure

GCP

Job description

AI Ops Engineer (m/f/d), fully remote in Germany

Job type: Full-time
Experience: 4+ years in DevOps or MLOps

Who you work for

With this AI Ops Engineer job you join Insight42, based in Ingolstadt and led by its founder Martin-Peter Lambert. The company won the German Innovation Award for decentralized, risk-eliminating architectures. Your team turns cloud engineering, MLOps and generative AI into secure, resilient systems for European clients.

What you will work on

You design and run the infrastructure behind Insight42’s GenAI and large language model (LLM) workloads. You architect self-hosted GenAI environments, tune GPU/CPU orchestration for performance and cost, and automate model deployment across edge and cloud systems.

Your tasks

• Design and maintain self-hosted GenAI model deployments in secure cloud environments
• Build and automate workflows for AI model lifecycle management (training, tuning, and versioning)
• Build observability and reliability into AI workloads
• Collaborate with MLOps, DevOps, and Data teams to align infrastructure with model requirements
• Integrate models into agentic frameworks and autonomous system ecosystems
• Optimize infrastructure for performance, cost efficiency, and scalability
• Establish CI/CD and GitOps practices for AI and ML system delivery

What you bring

• 4+ years of experience in DevOps, MLOps, or AI infrastructure engineering
• Hands-on experience deploying self-hosted GenAI models (LLMs, multimodal, or diffusion models)
• Strong understanding of containerization, microservices, and orchestration (Docker, Kubernetes)
• Familiarity with agentic ecosystems, AI orchestration frameworks, and vector databases
• Knowledge of GPU orchestration, distributed inference, or on-prem AI serving
• Solid programming and automation skills (Python, Bash, or Go)
• Cloud experience with Azure, GCP, or hybrid infrastructure setups
• Fluent communication in English; German is a plus

Nice to Have

• Experience with open-source AI frameworks (Ollama, vLLM, FastAPI, LangChain, Haystack)
• Understanding of AI observability and monitoring tools
• Exposure to model compression, quantization, and inference optimization
• Awareness of data privacy, security, and compliance in AI systems

What you get

• No commute or relocation: work from anywhere in Germany
• Competitive salary and performance-based incentives
• Build and run AI infrastructure at scale and grow your expertise
• A collaborative team with its own products, like Secretary42