AI Engineer — Agents & RAG

DevMations; Pakistan; On-site; PKR 1,800,000 - 3,000,000; Full time

Job summary

DevMations is seeking a hands-on engineer to own AI systems end-to-end in a full-time role based in Pakistan. You will read client corpora, design retrieval strategies, build agent loops, and measure outcomes with real users in mind.

Responsibilities include building retrieval pipelines, orchestrating agents with robust controls, creating eval suites, and reducing latency and per-request costs. You will communicate findings clearly to clients who are not engineers.

Qualifications

• 3+ years building production software, with at least one LLM system shipped to real users.
• Strong Python or TypeScript
• Practical experience with vector search and a retrieval framework
• Comfortable measuring things rather than asserting them

Responsibilities

• Design and build retrieval pipelines — chunking, embedding, hybrid search, re-ranking
• Implement agent orchestration with step limits, retries and human handoff
• Build eval suites and wire them into CI alongside the systems they measure
• Track and reduce per-request cost and p95 latency as first-class metrics
• Write up findings for clients in language a non-engineer can act on

Skills

Production software

Python

TypeScript

Vector search

Job description

Build retrieval pipelines and agent systems for client products, and the evaluation harnesses that prove they work.

full-time Pakistan

You will own AI systems end to end: reading a client’s corpus, designing the retrieval strategy, building the agent loop, and measuring the result. The work is judged on whether the system holds up with real users, not on whether the demo is impressive.

What you will do

• Design and build retrieval pipelines — chunking, embedding, hybrid search, re-ranking
• Implement agent orchestration with step limits, retries and human handoff
• Build eval suites and wire them into CI alongside the systems they measure
• Track and reduce per-request cost and p95 latency as first-class metrics
• Write up findings for clients in language a non-engineer can act on

What we are looking for

• 3+ years building production software, with at least one LLM system shipped to real users
• Strong Python or TypeScript
• Practical experience with vector search and a retrieval framework
• Comfortable measuring things rather than asserting them

Nice to have

• Experience with MCP or similar tool-calling protocols
• Published writing or open-source work on evals or retrieval