AI Engineer — Agents & RAG

DevMations; Pakistan; On-site; PKR 1,800,000 - 3,000,000; Full time

Job summary

DevMations in Pakistan is building retrieval pipelines and agent systems for client products, plus the evaluation harnesses that prove they work. You will own AI systems end to end: reading a client’s corpus, designing the retrieval strategy, building the agent loop, and measuring results with real users, not just demos.

The role requires 3+ years shipping production software with LLMs, strong Python or TypeScript, and practical vector search experience.

Qualifications

• 3+ years building production software, with at least one LLM system shipped to real users.
• Strong Python or TypeScript
• Practical experience with vector search and a retrieval framework
• Comfortable measuring things rather than asserting them

Responsibilities

• Design and build retrieval pipelines - chunking, embedding, hybrid search, re-ranking
• Implement agent orchestration with step limits, retries and human handoff
• Build eval suites and wire them into CI alongside the systems they measure
• Track and reduce per-request cost and p95 latency as first-class metrics
• Write up findings for clients in language a non-engineer can act on

Skills

LLM systems experience

Python

TypeScript

Vector search

Job description

Build retrieval pipelines and agent systems for client products, and the evaluation harnesses that prove they work.

full-time Pakistan

You will own AI systems end to end: reading a client's corpus, designing the retrieval strategy, building the agent loop, and measuring the result. The work is judged on whether the system holds up with real users, not on whether the demo is impressive.

What you will do

• Design and build retrieval pipelines - chunking, embedding, hybrid search, re-ranking
• Implement agent orchestration with step limits, retries and human handoff
• Build eval suites and wire them into CI alongside the systems they measure
• Track and reduce per-request cost and p95 latency as first-class metrics
• Write up findings for clients in language a non-engineer can act on

What we are looking for

• 3+ years building production software, with at least one LLM system shipped to real users
• Strong Python or TypeScript
• Practical experience with vector search and a retrieval framework
• Comfortable measuring things rather than asserting them

Nice to have

• Experience with MCP or similar tool-calling protocols
• Published writing or open-source work on evals or retrieval