AI Engineer

Onit, Inc.; Pune District; On-site; INR 2,500,000 - 6,000,000; Full time

Job summary

Onit, Inc. is seeking an AI Engineer to design and ship production-grade agentic AI systems that automate complex workflows end-to-end.

This hands-on role carries significant technical ownership and collaborates with the Chief Architect, product, and domain experts to translate ambiguous problems into reliable AI-driven experiences. You will build AI capabilities that improve user outcomes, ensure reliability by design with evals and safety controls, and iterate quickly through a tight loop of

Qualifications

• Strong software engineering background (Python preferred) and backend services experience.
• Hands-on experience building agentic LLM systems from first principles.
• Strong context engineering abilities: retrieval, routing, grounding, long-context tradeoffs.
• Solid evaluation discipline with automated pipelines and online monitoring.
• Experience with LLM APIs (OpenAI, Azure OpenAI, Anthropic, Gemini).
• Excellent debugging and systems thinking.
• Comfortable in fast-paced, ownership-driven environments.

Responsibilities

• Build AI-driven features using LLM APIs and tool-using agents.
• Design multi-agent workflows and reliability controls from day one.
• Own end-to-end context engineering and retrieval strategies.
• Implement retrieval-augmented generation and vector stores.
• Develop backend services with AI capabilities and observability.
• Define evaluation frameworks and CI/CD gates for AI features.
• Collaborate with product, architects, and domain SMEs.

Skills

Software engineering

Agentic LLM systems

Context engineering

Evaluation pipelines

LLM APIs

Debugging skills

Ownership mindset

Tools

Langfuse

OpenTelemetry

Datadog

Airflow

Temporal

AWS

GCP

Job description

We’re seeking a AI Engineer to design and ship production-grade agentic AI systems that automate complex workflows end-to-end. This is a hands‑on role with significant technical ownership. You’ll work closely with the Chief Architect, product, engineering, and domain experts to translate ambiguous, high‑impact problems into reliable AI‑driven user experiences.

What success looks like:

Ship AI capabilities that measurably improve user outcomes (quality, time saved, throughput)

Build systems that are reliable by design: evals, observability, safety, and cost/latency controls from day one

Iterate quickly using a tight loop of instrument → evaluate → improve → deploy

What You’ll Do

Agentic AI Feature & Workflow Development

• Build and integrate AI-driven features using LLM APIs (OpenAI / Azure OpenAI, Anthropic, Gemini on Vertex AI)
• Design and implement tool-using agents (structured function calling, schema validation, retries, fallbacks)
• Build multi-agent workflows when appropriate (e.g., planner/worker, reviewer/critic, specialist routing) and know when a simpler architecture is better
• Create agentic workflows such as document understanding, extraction, reasoning over evidence, task automation, and multi-step decision support
• Own context engineering end‑to‑end:
  • dynamic context assembly (retrieval + state + tool outputs)
  • context budgeting and compression/summarization
  • grounding strategies to reduce hallucinations and improve consistency
• Implement retrieval‑augmented generation (RAG) and search workflows using off‑the‑shelf vector stores and embedding services

Evaluation, Quality & Iteration (Core)

• Establish evaluation frameworks for accuracy, reliability, and output quality
• Build task‑specific eval suites: golden datasets, adversarial cases, regression tests, and rubric‑based scoring
• Set up automated evaluation pipelines and release gates (CI/CD-friendly) tied to prompt/model/version changes
• Define and monitor online metrics (e.g., task success rate, human override rate, safety flags, latency, cost) and run experiments/A‑B tests where appropriate
• Use LLM‑as‑judge responsibly: calibrate, validate, and pair with human labels when needed

Engineering, Integration & Observability

• Develop scalable backend services and APIs that incorporate AI functionality
• Integrate AI pipelines into existing cloud, microservices, and event‑driven architectures
• Implement observability and analytics for all AI features (tracing, evaluations, prompt versioning, cost tracking) Example tooling: Langfuse (and/or OpenTelemetry‑compatible stacks)
• Ensure reliability, uptime, performance, and security of AI services
• Build internal tooling for evaluation, testing, prompt/version management, and safe deployment

Product & Collaboration

• Partner with product managers, designers, the Chief Architect, and domain SMEs to shape AI‑first solutions
• Rapidly prototype concepts and iterate based on user feedback and measurable eval results
• Translate business problems into well‑structured AI workflows without requiring ML model training
• Document system behavior, known failure modes, and operational playbooks

Governance & Safety

• Implement guardrails, checks, and fallback logic for safe and predictable AI behavior
• Help define and follow compliance, privacy, and responsible AI guidelines
• Design for safe tool execution (bounded actions, permissions, escalation paths, human‑in the‑loop review

What You Bring

Core Strengths (Required)

• Strong software engineering background (Python preferred) and experience shipping backend services
• Deep hands‑on experience building agentic LLM systems from first principles: agent loops, tool interfaces, planning/replanning, memory/state, and failure handling
• Strong context engineering ability: retrieval strategies, routing, grounding, context budgeting, and long‑context tradeoffs
• Strong evaluation discipline: golden datasets, regression gating, automated eval pipelines, and online monitoring
• Practical experience with LLM APIs (OpenAI/Azure OpenAI/Anthropic/Gemini) and AI orchestration frameworks
• Excellent debugging, systems thinking, and problem decomposition skills
• Comfortable operating in fast‑paced, ambiguous environments with high ownership

Signals We Value

• You’ve shipped an LLM/agent system in production and can clearly explain:
  • the failure modes you discovered
  • the evals you built to catch regressions
  • how you improved cost/latency while increasing quality
  • how you monitored and iterated safely over time
• You keep up with industry developments (model releases, frameworks, best practices) and can translate them into pragmatic improvement

Nice to Have

• Experience with cloud platforms (AWS and/or GCP), microservices, and event‑driven systems
• Experience with observability stacks (OpenTelemetry, Datadog, Honeycomb) and AI‑specific tooling (e.g., Langfuse, Braintrust, HumanLoop, W&B Weave)
• Experience with workflow orchestration for long‑running jobs (Temporal, Celery, Airflow)
• Experience building enterprise AI features (permissions, auditability, compliance constraints)
• Experience with safety/policy layers (PII handling, prompt injection defenses, sandboxed tool execution)

Why Join Us

• Build core AI capabilities that directly impact users and product strategy
• Work on cutting‑edge, real‑world agentic systems—focused on applied engineering (no model training required)
• High ownership, fast iteration cycles, and strong cross‑functional collaboration
• Competitive compensation and opportunities for rapid advancement

What Your First 90 Days Could Look Like

Ship one production agent workflow end‑to‑end with:

• tracing + observability
• an offline eval suite with regression gates
• cost/latency targets and monitoring
• documented failure modes and fallback path