LLM AI Engineer - Production Backends, Cloud, Remote

Billennium; Poland; Hybrid; PLN 180,000 - 320,000; Full time

Benefits offered by this job

Udemy for Business

Private medical care

Multisport card

Language lessons

Shopping vouchers

Remote/hybrid work options

Career growth opportunities

Global collaboration

Job summary

Billennium is seeking an experienced AI Engineer with strong Python and cloud skills to design, build, and optimize scalable backend systems supporting AI-powered automation solutions. You will ship LLM-powered features end to end and design agentic workflows, RAG pipelines and tool-calling solutions with LangGraph.

Develop production-ready FastAPI services with streaming, persistence and tests, build evaluation datasets and regression suites, and monitor latency, costs, and observability.

Qualifications

• Strong experience building and operating backend services, APIs and asynchronous applications.
• Hands-on experience taking LLM solutions to production, beyond prototypes.
• Practical experience with RAG and AI/agent orchestration.
• Experience with LangGraph or similar orchestration frameworks.
• Experience with LLM evaluation, tracing, debugging and production monitoring.
• Experience running services in the cloud, preferably Azure.
• Strong Python skills and experience with FastAPI.
• Analytical mindset and ability to make sound technical decisions.
• Fluent English and strong communication skills.
• A proactive, collaborative and curious approach to solving complex problems.

Responsibilities

• Build and ship LLM-powered features end to end
• Design agentic workflows, RAG pipelines and tool-calling solutions, with a strong focus on LangGraph
• Develop production-ready FastAPI services with streaming, persistence and automated tests
• Build evaluation datasets, regression suites and LLM-as-a-judge mechanisms to continuously measure answer quality
• Improve retrieval and context engineering—from query formulation to multi-step research workflows
• Investigate why an answer is wrong and identify whether the root cause is retrieval, query formulation, context or prompting
• Monitor and optimize latency, token usage and infrastructure costs
• Work with different LLM providers and model sizes, including intelligent model routing through an AI gateway
• Collaborate with engineers and stakeholders to make AI solutions reliable, observable and scalable

Skills

Backend services

APIs

Asynchronous applications

LLM production

RAG

Agent orchestration

LangGraph

LLM evaluation

Tracing & debugging

Production monitoring

Azure

Python

FastAPI

English

Tools

LangGraph

Docker

Langfuse

Langfuse

Job description

Billennium is seeking an experienced AI Engineer with strong Python and cloud skills to design, build, and optimize scalable backend systems supporting AI-powered automation solutions. You will ship LLM-powered features end to end and design agentic workflows, RAG pipelines and tool-calling solutions with LangGraph.

Develop production-ready FastAPI services with streaming, persistence and tests, build evaluation datasets and regression suites, and monitor latency, costs, and observability.