Senior AI Engineer – Retrieval & Evaluation IRC306816

GlobalLogic Inc.; Kraków; On-site; PLN 240,000 - 420,000; Full time

Benefits offered by this job

Comprehensive benefits

Relocation options

Learning & Development

Diversity & Inclusion

Career development

Job summary

GlobalLogic is partnering with a leading threat intelligence platform to design an enterprise-scale HITL AI agent platform, aimed at accelerating expert analyst decision-making with cutting-edge generative and agentic AI systems.

The role focuses on building hybrid retrieval over enterprise sources, creating a precedent store with human decisions, delivering report generation with source attribution, and developing automated evaluation frameworks.

Qualifications

• 5+ years of engineering experience, including 2+ years on production RAG or search systems.
• Experience with embeddings and vector databases.
• Experience with LLM evaluation methods and tooling.
• Strong Python proficiency from prototype to production.
• Background in information retrieval or NLP research.
• Experience with Vertex AI Search and BigQuery; in-context learning optimization.
• Experience with structured report generation and citation grounding.

Responsibilities

• Build hybrid retrieval over enterprise sources (vector and keyword search), with reranking and permission-aware filtering.
• Build the precedent store: index decision points with human decisions and rewards, and retrieve the right examples at runtime.
• Deliver report generation and explanation layers with source attribution.
• Build automated evaluation: golden datasets, LLM-as-judge, regression suites, and A/B tests of few-shot strategies.
• Apply the dynamic few-shot approach to the LLM stage of alert triage.

Skills

Python

RAG/LLM systems

IR/NLP research

Vector databases

LLM evaluation tooling

Production engineering

Tools

Vertex AI Search

BigQuery

pgvector

DSPy

Job description

GlobalLogic takes an opportunity to partner with a leading global threat intelligence platform to design and deliver an enterprise-scale, Human-in-the-Loop (HITL) AI agent platform. The initiative centers on accelerating expert analyst decision-making by augmenting high-complexity threat intelligence workflows with cutting-edge generative and agentic AI systems.

#LI-OM1

Requirements

• 5+ years of engineering, including 2+ years on production retrieval-augmented generation (RAG) or search systems.
• Embeddings, vector databases (Vertex AI Vector Search, pgvector or similar), hybrid search and rerankers.
• LLM evaluation methods and tooling (Ragas, DeepEval, custom judges, golden-set design).
• Strong Python, from prototype through to production code.
• An information retrieval or NLP research background.
• Vertex AI Search, BigQuery, and in-context learning optimization (for example, DSPy).
• Structured report generation and citation grounding.

Job responsibilities

• Build hybrid retrieval over enterprise sources (vector and keyword search), with reranking and permission-aware filtering.
• Build the precedent store: index decision points with human decisions and rewards, and retrieve the right examples at runtime.
• Deliver report generation and explanation layers with source attribution.
• Build automated evaluation: golden datasets, LLM-as-judge, regression suites, and A/B tests of few-shot strategies.
• Apply the dynamic few-shot approach to the LLM stage of alert triage.

What we offer

Empowering Projects: With 500+ clients spanning diverse industries and domains, we provide an exciting opportunity to contribute to groundbreaking projects that leverage cutting-edge technologies. As a team, we engineer digital products that positively impact people's lives.

Empowering Growth: We foster a culture of continuous learning and professional development. Our dedication is to provide timely and comprehensive assistance for every consultant through our dedicated Learning & Development team, ensuring their continuous growth and success.

DE&I Matters: At GlobalLogic, we deeply value and embrace diversity. We are dedicated to providing equal opportunities for all individuals, fostering an inclusive and empowering work environment.

Career Development: Our corporate culture places a strong emphasis on career development, offering abundant opportunities for growth. Regular interactions with our teams ensure their engagement, motivation, and recognition. We empower our team members to pursue their career goals with confidence and enthusiasm.

Comprehensive Benefits: In addition to equitable compensation, we provide a comprehensive benefits package that prioritizes the overall well-being of our consultants. We genuinely care about their health and strive to create a positive work environment.

Flexible Opportunities: At GlobalLogic, we prioritize work-life balance by offering flexible opportunities tailored to your lifestyle. Explore relocation and rotation options for diverse cultural and professional experiences in different countries with our company.

About GlobalLogic

GlobalLogic, a Hitachi Group Company, is a trusted digital engineering partner to the world's largest and most forward-thinking companies. Since 2000, we’ve been at the forefront of the digital revolution – helping create some of the most innovative and widely used digital products and experiences. Today we continue to collaborate with clients in transforming businesses and redefining industries through intelligent products, platforms, and services.