AI Technical Lead

Benchmarkit; Pune District; Vor Ort; INR 4.000.000 - 7.000.000; Vollzeit

Zusammenfassung

Benchmark IT Solutions India Pvt Ltd in Pune seeks a Technical Lead for AI practice. You will own technical delivery across concurrent client engagements, balancing hands-on engineering with architectural oversight.

The role requires deep Python expertise, multi-LLM production experience, and the ability to mentor engineers, enforce design gates, and communicate risks clearly to clients and leadership.

Qualifikationen

• Two or more LLMs shipped to production with measurable quality gates.
• Deep Python and multi-tenant SaaS system design.
• Evaluation datasets and metrics used to block a release.
• Cloud depth on Azure (Azure AI Foundry, Azure OpenAI, AKS, Key Vault) or equivalents.
• Led a team of three or more engineers through delivery and reviews.
• Judgment on where a model does not belong and the authority to push back with clients.
• Clear written communication for design docs and risk memos.

Aufgaben

• Own technical delivery of AI workstreams across concurrent client engagements: scope, design, sequencing, quality and honest estimates.
• Run the pre-implementation design gate: review engineers' design notes, apply the tiered checklist, maintain the decisions register, and approve or send back before code is written.
• Set and enforce engineering standards for LLM systems — evaluation suites as release gates, prompt and model versioning, cost and latency budgets, guardrails and injection defenses, human approval gates, evidence and audit trails.
• Personally build the hardest components: retrieval and re-ranking architecture, tool-calling and agent orchestration on Temporal, document extraction pipelines, on-premises and offline deployments.
• Review code at scale, including code produced with AI coding tools; catch invalidated model output, leaked secrets, missing tests and silent behavioral change before they merge.
• Mentor early-career engineers with a deliberate plan: pair on design, grade their design notes, expand their ownership as they earn it.
• Represent engineering with clients: explain trade-offs in plain language, push back on unsafe or unbounded asks, and turn vague requirements into a bounded phase-one scope.
• Hold the security posture: secrets management via Vault and Key Vault, data residency, least-privilege access, and client confidentiality on on-premises work.
• Build reusable Centre of Excellence assets — evaluation harness, RAG reference implementation, guardrail library, extraction patterns — so that the method travels across clients even when the code cannot.
• Assess codebases and delivery risk candidly and communicate upward early, including when the news is unwelcome.

Kenntnisse

Two or more LLM systems

Deep Python

System design

Leadership

Client communication

Tools

Azure AI Foundry

AKS

Key Vault

Temporal

Azure OpenAI

Playwright

Jobbeschreibung

Benchmark IT Solutions India Pvt Ltd | Full time

Since our establishment in 2005, we have been focused on helping our clients build their Software Products/Projects.

Our founders had software development engineering background, our focus has always been in having strong delivery capability. This has resulted in extremely happy clients from day one and we continue to be the preferred partner for the clients we have acquired in our first year.

As we continued our journey and growth, we have expanded our team with different technology stacks and right technical solutions. Our 200+ people team continues to pleasantly surprise our clients across different technologies.

Job Description

The role

We need senior hands now. The AI practice runs several concurrent client platforms in regulated industries, and the engineering team supporting them is early-career. The Technical Lead becomes the day-to-day technical owner of that team: the person who turns architecture decisions into implementable designs, runs the design gate, reviews the code, unblocks the hard problems personally, and tells the Chief Architect and delivery leadership the truth about risk.

Expect the split to be roughly half hands-on engineering and half oversight. The hands-on half is the difficult half: retrieval architecture, agent orchestration, extraction accuracy, performance and cost. The oversight half is where you multiply — three to five engineers with one to two years of experience become independent owners of their workstreams within six months because of how you review, mentor and set standards.

What will you do

Own technical delivery of AI workstreams across concurrent client engagements: scope, design, sequencing, quality and honest estimates.

Run the pre-implementation design gate: review engineers' design notes, apply the tiered checklist, maintain the decisions register, and approve or send back before code is written.

Set and enforce engineering standards for LLM systems — evaluation suites as release gates, prompt and model versioning, cost and latency budgets, guardrails and injection defenses, human approval gates, evidence and audit trails.

Personally build the hardest components: retrieval and re-ranking architecture, tool-calling and agent orchestration on Temporal, document extraction pipelines, on-premises and offline deployments.

Review code at scale, including code produced with AI coding tools; catch invalidated model output, leaked secrets, missing tests and silent behavioral change before they merge.

Mentor early-career engineers with a deliberate plan: pair on design, grade their design notes, expand their ownership as they earn it.

Represent engineering with clients: explain trade-offs in plain language, push back on unsafe or unbounded asks, and turn vague requirements into a bounded phase-one scope.

Hold the security posture: secrets management via Vault and Key Vault, data residency, least-privilege access, and client confidentiality on on-premises work.

Build reusable Centre of Excellence assets — evaluation harness, RAG reference implementation, guardrail library, extraction patterns — so that the method travels across clients even when the code cannot.

Assess codebases and delivery risk candidly and communicate upward early, including when the news is unwelcome.

What you must bring

Two or more LLM systems that you shipped to production and operated, each with measurable quality gates. You can describe the incidents, what you measured, what you changed, and what you would do differently.

Deep Python and strong system design: multi-tenant SaaS, asynchronous pipelines, event- or workflow-driven orchestration, data modelling on document and relational stores.

Evaluation rigor as a habit: you have built eval datasets, chosen the metrics, and used them to block a release.

Cloud depth on Azure preferred (Azure AI Foundry, Azure OpenAI, AKS or App Service, Key Vault); AWS or GCP equivalents acceptable if the principles transfer.

Led a team of three or more engineers through delivery, including design reviews and code reviews you were accountable for.

Judgment about where a model does not belong, and the standing to say so to a client who wants one everywhere.

Clear written communication: design documents, decision records, risk memos. Comfortable in client-facing technical conversations.

What would set you apart

Temporal in production; durable, idempotent, resumable workflow design.

Document AI at depth: tiered OCR strategies, layout analysis, extraction accuracy measurement on real-world scanned records.

Computer vision for inspection or verification use cases (DETR-family detectors, DINOv2 backbones, ensembling and box-fusion methods) and release gates expressed as per-class recall.

Compliance frameworks applied to software delivery: GxP and 21 CFR Part 11, SOC 2, ISO 27001, HIPAA, PCI-DSS.

Test automation strategy with AI-assisted authoring (Playwright), and the governance of AI coding tools inside an engineering team.

Experience running a small engineering practice or CoE: standards, reusable assets, training, hiring.