AI Product Engineer

Forte Group; Polska; Remote; PLN 180,000 - 280,000; Full time

Benefits offered by this job

Remote-first

International exposure

Competitive compensation

Career growth

Job summary

Forte Group is seeking an AI Product Engineer to design and deliver AI-powered features end-to-end—architecture through production and ongoing optimization. You will work on client engagements across the US and Europe, delivering LLM-powered capabilities, retrieval systems, and AI-native applications.

You will build RAG pipelines, integrate LLM APIs, optimize for latency and cost, and own production operations.

Qualifications

• 5+ years in software engineering.
• 2+ years building AI-powered features or products in production.

Responsibilities

• Design and deliver AI-powered features end-to-end—from architecture to production.
• Build RAG pipelines: document ingestion, chunking strategies, embeddings, vector stores, retrieval evaluation, and re-ranking.
• Integrate LLM APIs into production systems using structured outputs, function calling, and multi-model routing.
• Balance real-world tradeoffs between accuracy, latency, and cost.
• Develop agentic workflows with tool usage, human-in-the-loop checkpoints, and multi-agent coordination.
• Own production operations: latency optimization, cost control, drift monitoring, and incident response.

Skills

Backend development

AI systems

LLM integration

Python

CI/CD

Tools

FastAPI / Django

ASP.NET Core

Node.js / TypeScript

Job description

Forte Group is looking for an AI Product Engineer. We build products where AI is the core capability. Our AI Product Engineers work on client engagements across the US and Europe, delivering LLM-powered features, retrieval systems, agentic workflows, and fully AI-native applications into production.

Recent projects include:

• An AI agent extracting structured settlement data from unstructured financial documents with 95%+ accuracy, including full audit trails and multi-tenant enterprise deployment
• A clinical AI platform that reduced prescription validation time from 20 minutes to 5 minutes across 47 healthcare centers, built and maintained by a team of five engineers
• A revenue optimization platform processing booking data across thousands of facilities using multi-model LLM orchestration, large-scale vector search, and document intelligence pipelines

WHAT YOU WILL DO

You will design and deliver AI-powered features end-to-end — from architecture to production and ongoing optimization.

On a typical engagement, you will:

• Build RAG pipelines: document ingestion, chunking strategies, embeddings, vector stores, retrieval evaluation, and re-ranking
• Integrate LLM APIs into production systems using structured outputs, function calling, and multi-model routing
• Balance real-world tradeoffs between accuracy, latency, and cost
• Develop agentic workflows with tool usage, human-in-the-loop checkpoints, and multi-agent coordination
• Build evaluation systems: automated test sets, field-level accuracy metrics, regression detection in CI/CD, and production monitoring
• Design human-in-the-loop workflows for low-confidence outputs and feedback loops
• Own production operations: latency optimization, cost control, drift monitoring, and incident response

SKILLS & EXPERIENCE

Experience

• 5+ years in software engineering
• 2+ years building AI-powered features or products in production

Languages & Frameworks

• Strong proficiency in at least one modern backend language — e.g., Python, C#/.NET, Java/Kotlin, TypeScript/Node.js, or Go — with the ability to work across stacks when an engagement requires it
• Experience building and shipping production services with a modern web framework in your ecosystem (e.g., FastAPI/Django, ASP.NET Core, Spring Boot, Express/Next.js)
• Solid understanding of async/concurrent patterns, streaming (SSE/WebSockets), and batch processing — regardless of language

LLM & AI Systems

• LLM APIs: OpenAI, Anthropic, open-source models (Ollama, vLLM, or similar)
• Prompt engineering at a systems level: structured output, chain-of-thought, few-shot patterns, function calling, tool use
• Multi-model architectures: routing between models, tiering by cost and complexity, fallback chains
• Fine-tuning pipelines and when fine-tuning is the right approach vs. RAG or prompt engineering

RAG & Retrieval

• Vector databases: Pinecone, Weaviate, pgvector, Chroma, OpenSearch, or equivalent
• Embedding models, chunking strategies (fixed, semantic, sentence-based), and the tradeoffs between them
• Hybrid search (vector + keyword/BM25), re-ranking, and retrieval evaluation methodology
• Document processing: OCR pipelines (Azure Document Intelligence or equivalent), multi-format handling (PDF, Excel, CSV, email, images)

Agent Framework & Orchestration

• MCP (Model Context Protocol) for multi-agent coordination
• Custom orchestration engines: when frameworks add value and when direct API calls are simpler
• Tool use patterns: sandboxed execution, scoped permissions, action approval gates

Evaluation & Observability

• Evaluation frameworks (LangSmith, Braintrust, Ragas, or custom setups)
• Tracing across multi-step pipelines
• Human feedback loops and confidence-based routing

Infrastructure & Production

• Containerization and cloud deployment (AWS, Azure, or GCP)
• CI/CD pipelines for AI systems, including automated evaluation
• Data pipelines (Snowflake, dbt, or similar tools are a plus)
• Latency optimization (streaming, caching, async processing)
• Cost management at scale (model tiering, caching, per-query tracking)

WHAT SETS YOU APART

• You can walk us through a production AI feature you built — including what failed and what you learned
• You speak concretely about RAG (chunking tradeoffs, retrieval accuracy), not just tools
• You’ve handled real-world constraints: latency, cost spikes, unpredictable inputsYou can explain AI system behavior to stakeholders who care about outcomes, not architecture
• You build evaluation into systems from day one — because you’ve seen what happens when you don’t

What’s in it for you

• Work your way — anywhere, anytime. Our remote-first approach lets you choose where and how you work best!
• Experience working with diverse teams and gaining international expertise
• A friendly, supportive team and an enjoyable work environment where your ideas matter
• A chance to work on exciting, challenging projects using cutting-edge technologies that make a real impact
• Individual and group sessions with a psychologist
• Social, corporate, team-building, and knowledge-sharing events
• Partial compensation of external trainings
• Referral bonuses, regular performance reviews, and full support for business trips

About Forte Group

Founded over 25 years ago, Forte Group began with a focus on Quality Assurance and has since evolved into a dynamic force in the tech industry, delivering cutting-edge solutions worldwide. As an American company headquartered in Boca Raton, USA, we've had the privilege of partnering with over 400 clients, including Fortune 500 giants. Our software has made a significant impact, reaching more than 9 million users — comparable to the entire population of New York or Switzerland!

We’re more than just a company — we’re a team of passionate, driven people who love what we do. If you’re looking for a place where your work matters, your ideas are valued, and your growth is supported, you’ve found it!