AI Engineer

Omnitrace; Warszawa; Hybrid; PLN 180,000 - 240,000; Full time

Job summary

Omnitrace is seeking an AI Engineer to take end-to-end ownership of AI systems from prototype to production. You will work across LLM pipelines and agentic workflows, shaping architecture and overseeing reliability and scalability in real-world deployments.

The role demands hands-on development, collaboration with Product/Design, and willingness to explore R&D fronts such as tacit knowledge extraction and personalized agents. Fluency in Polish and English is essential.

Qualifications

• 2-5 years of commercial experience in AI/ML or software engineering with production systems shipped.
• Strong Python skills and backend API development, including async, testing, and CI/CD.
• Ability to rapidly prototype AI solutions and convert to customer value.
• Experience with agent architectures, including context engineering, tool calling, agent loops, skills, outputs, and fallbacks.
• Hands-on LLMOps experience with tracing, evals, and production debugging.

Responsibilities

• Own AI systems end-to-end, prototype to production with reliability and scalability across LLM pipelines and agentive workflows.
• Build AI and data agents, RAG systems, semantic layers, and ontologies for Omniviser’s interface.
• Design production-grade orchestration using tool use, skills, MCP, context engineering, memory/state management, and structured outputs with validation.
• Implement fallback handling, human-in-the-loop flows, guardrails, and tool permission controls.
• Prototype new ML/AI ideas quickly and turn strong concepts into production features.
• Iterate with Product and Design based on user feedback and value.
• Build evaluation/monitoring for agents including tracing, evals, RAG evaluation, and model benchmarks.
• Work in an evaluation-driven development model with LLMOps and MLOps practices.
• Support Ops in AI-native initiatives and customer implementations.
• Explore open R&D problems like tacit knowledge extraction and personalized agents.

Skills

Python

Production backend

LLMOps

LangChain/LangGraph

Agent architecture

PostgreSQL

pgvector

Semantic search

MCP servers/interfaces

Langfuse/MLflow

Azure

Code quality (tests/linters)

English & Polish communication

CI/CD

Tools

Claude Code

Codex

Cursor

GitHub Copilot

Langfuse/MLflow

Azure CLI

Job description

We are looking for an AI Engineer who will take end-to-end ownership of AI systems — from prototype to production, including reliability and scalability across LLM pipelines and agentic workflows. This is a deeply hands-on role with a high level of ownership: one week you may be working in the core product code, another supporting a customer engineering team during an implementation, and another exploring an R&D problem at the edge of what is currently possible. You will define the business problem, design the solution architecture, and take responsibility for what ultimately reaches the user — including R&D topics such as extracting tacit knowledge, automating eval creation, personalizing agents from their traces, and enabling proactive behavior.

Key Responsibilities

• Own AI systems end-to-end, from prototype to production, including reliability and scalability across LLM pipelines and agentic workflows
• Build AI and data agents, RAG systems, semantic layers, and ontologies that let customers work with their business through Omniviser’s conversational interface
• Design production-grade orchestration using tool use, skills and MCP, context engineering, memory and state management, and structured outputs with validation
• Implement fallback handling, human-in-the-loop flows, guardrails, and tool permission controls
• Prototype new ML and AI ideas in days rather than quarters and turn the strongest concepts into production features
• Iterate with Product and Design based on what actually works for users
• Build evaluation and monitoring for agents, including tracing, automated evals, RAG evaluation, LLM-as-a-judge, and model benchmarks
• Work in an evaluation-driven development model, combining LLMOps and MLOps practices to improve quality, cost, and latency
• Support Ops in making the company more AI-native and support Delivery in customer implementations, solution fit, and adoption
• Work on open R&D problems such as tacit knowledge extraction, trace-based agent personalization, intent recognition, and proactive product behavior
• Use agentic coding tools such as Claude Code, Codex, or Cursor, follow the AI ecosystem, and bring useful new practices into the team

Required Qualifications

• 2-5 years of commercial experience in AI/ML or software engineering, with production systems you have actually shipped
• Strong Python skills and experience building and maintaining production backend services and APIs, including async, testing, and CI/CD
• Ability to rapidly prototype AI solutions and quickly turn them into customer value
• Practical knowledge of agent architectures, including context engineering, tool calling, agent loops, skills, structured outputs, and fallback handling
• Experience with orchestration in LangGraph, LangChain, or a custom framework, and with MCP servers or custom tool-calling interfaces
• Experience with data agents and production retrieval over customer data, including semantic layers and ontologies; knowledge of PostgreSQL, pgvector, semantic and hybrid search, and BM25
• Ability to build evaluation and validation systems for AI applications, from golden datasets and scenario tests to regression tests and quality gates
• Hands-on LLMOps experience, including tracing and observability in Langfuse or MLflow, automated evals, prompt optimization, and production debugging for quality, cost, and latency
• Experience deploying and maintaining production applications in the cloud, especially Azure
• Daily use of agentic engineering tools such as Claude Code, Codex, Cursor, or GitHub Copilot, together with strong code quality practices: tests, linters, type checkers, documentation, and code review
• Very good communication skills in Polish and English and the ability to work with Product and Design teams as well as directly with customer engineers

Nice to Have

• Experience building AI products for customers, especially in a 0-1 phase
• Experience deploying AI solutions with measured business value
• Experience in a startup or fast-changing product environment
• Experience with spec-driven development using agentic engineering tools
• Experience securing LLM applications through red teaming, adversarial testing, and anti-jailbreak policy enforcement
• Experience optimizing cost and latency through techniques such as context compression and prompt caching

Who We Are Looking For

• Someone with high agency who can take a feature from idea to deployment without waiting for every task to be defined
• A product-minded engineer who makes decisions based on real user needs and measurable business outcomes
• Someone who combines engineering rigor with pragmatism and can make sensible trade-offs without over-engineering
• A strong communicator who can move between technical conversations with engineers and business conversations with stakeholders, in Polish and English
• An AI-native early adopter who uses generative AI in everyday work, follows the ecosystem, and shares useful new techniques with the team

What We Offer

• End-to-end ownership of production AI systems, from data and agents to what ultimately reaches the user
• A high level of autonomy together with real responsibility for outcomes
• Work across core product development, customer implementations, and R&D
• Direct influence on how we build, evaluate, monitor, and improve AI systems
• Collaboration with Product, Design, Ops, Delivery, and customer engineering teams
• An AI-native, fast-moving environment focused on rapid experimentation and measurable results