AI Engineer

Billennium; Poland; Hybrid; PLN 180,000 - 320,000; Full time

Benefits offered by this job

Udemy for Business

Private medical care

Multisport card

Language lessons

Shopping vouchers

Remote/hybrid work options

Career growth opportunities

Global collaboration

Job summary

Billennium is seeking an experienced AI Engineer with strong Python and cloud skills to design, build, and optimize scalable backend systems supporting AI-powered automation solutions. You will ship LLM-powered features end to end and design agentic workflows, RAG pipelines and tool-calling solutions with LangGraph.

Develop production-ready FastAPI services with streaming, persistence and tests, build evaluation datasets and regression suites, and monitor latency, costs, and observability.

Qualifications

• Strong experience building and operating backend services, APIs and asynchronous applications.
• Hands-on experience taking LLM solutions to production, beyond prototypes.
• Practical experience with RAG and AI/agent orchestration.
• Experience with LangGraph or similar orchestration frameworks.
• Experience with LLM evaluation, tracing, debugging and production monitoring.
• Experience running services in the cloud, preferably Azure.
• Strong Python skills and experience with FastAPI.
• Analytical mindset and ability to make sound technical decisions.
• Fluent English and strong communication skills.
• A proactive, collaborative and curious approach to solving complex problems.

Responsibilities

• Build and ship LLM-powered features end to end
• Design agentic workflows, RAG pipelines and tool-calling solutions, with a strong focus on LangGraph
• Develop production-ready FastAPI services with streaming, persistence and automated tests
• Build evaluation datasets, regression suites and LLM-as-a-judge mechanisms to continuously measure answer quality
• Improve retrieval and context engineering—from query formulation to multi-step research workflows
• Investigate why an answer is wrong and identify whether the root cause is retrieval, query formulation, context or prompting
• Monitor and optimize latency, token usage and infrastructure costs
• Work with different LLM providers and model sizes, including intelligent model routing through an AI gateway
• Collaborate with engineers and stakeholders to make AI solutions reliable, observable and scalable

Skills

Backend services

APIs

Asynchronous applications

LLM production

RAG

Agent orchestration

LangGraph

LLM evaluation

Tracing & debugging

Production monitoring

Azure

Python

FastAPI

English

Tools

LangGraph

Docker

Langfuse

Langfuse

Job description

Billennium is a global technology company delivering innovative software, cloud, data, and AI solutions for enterprise clients worldwide. We combine engineering excellence with a collaborative culture to help organizations accelerate their digital transformation.

We are looking for an experienced AI Engineer with strong Python and cloud engineering skills to design, build, and optimize scalable backend systems supporting AI-powered and intelligent automation solutions.

Key Responsibilities

• Build and ship LLM-powered features end to end
• Design agentic workflows, RAG pipelines and tool-calling solutions, with a strong focus on LangGraph
• Develop production-ready FastAPI services with streaming, persistence and automated tests
• Build evaluation datasets, regression suites and LLM-as-a-judge mechanisms to continuously measure answer quality
• Improve retrieval and context engineering-from query formulation to multi-step research workflows
• Investigate why an answer is wrong and identify whether the root cause is retrieval, query formulation, context or prompting
• Monitor and optimize latency, token usage and infrastructure costs
• Work with different LLM providers and model sizes, including intelligent model routing through an AI gateway
• Collaborate with engineers and stakeholders to make AI solutions reliable, observable and scalable

Requirements

• Strong experience building and operating backend services, APIs and asynchronous applications
• Hands‑on experience taking LLM solutions to production, beyond prototypes and PoCs
• Practical experience with RAG and AI/agent orchestration
• Experience with LangGraph or similar orchestration frameworks
• Experience with LLM evaluation, tracing, debugging and production monitoring
• Experience running services in the cloud, preferably Azure
• Strong Python skills and experience with FastAPI
• Analytical mindset and ability to make sound technical decisions
• Fluent English and strong communication skills
• A proactive, collaborative and curious approach to solving complex problems

Nice to have

Experience with:

• Bicep / Infrastructure as Code
• FAISS and PostgreSQL
• Langfuse and Azure Application Insights
• Docker / Podman
• Mentoring or technical leadership

Perks and benefits:

• Comprehensive benefits-enjoy Udemy for Business, private medical care, Multisport card, veterinary package, language lessons, and shopping vouchers.
• Flexibility-adaptable working hours and remote/hybrid work options to suit your lifestyle & location.
• Career growth-access to professional development and learning opportunities, including perks from our official partnerships with global IT giants: Microsoft, AWS, Snowflake, Salesforce & more.
• Global collaboration-work with a diverse, international team.
• Innovative environment-be part of a forward-thinking and growth-oriented workplace.
• Engaging community-participate in team-building events, hackathons, and CSR initiatives to make an impact beyond work.
• Team-building events, including our company tradition-an annual company event in Mazury.
• A welcome pack to start your journey with us on a positive note.

Recruitment process

• HR call
• Interview with the dedicated Client