AI Engineer (Autonomous Agents & Databricks)

Spearsoftech; Hyderabad; On-site; INR 3,000,000 - 5,400,000; Full time

Job summary

Spearsoft is seeking an AI Engineer to design, develop, and deploy autonomous AI agents and production-ready AI applications that elevate Data Science and BI capabilities. You will build, test, and maintain agent workflows, ensuring security, reliability, and responsible AI practices for enterprise-grade quality.

Role involves architecting multi-agent orchestration with LLMs, tool-calling, and RAG; deploying AI workloads on the Databricks Data Intelligence Platform using Mosaic AI, Unity

Qualifications

• 5–8+ years in AI/ML engineering or AI app development.
• Proven Python software engineering practices and testing.
• Experience building AI agents with LangChain, LangGraph, AutoGen, CrewAI, or Semantic Kernel.
• Proficiency with prompt engineering, RAG, vector databases, and LLM API integration.
• Production deployment in cloud environments (Azure preferred) and on Databricks.

Responsibilities

• Design, build, and deploy autonomous AI agents.
• Develop production-ready agent apps with error handling and observability.
• Use Cursor and Copilot to accelerate delivery and ensure code quality.
• Define agent roles, tools, memory strategy, and escalation paths.
• Architect multi-step agent workflows with planning and human-in-the-loop.
• Implement prompt engineering and evaluation frameworks.
• Maintain reusable templates, libraries, and shared prompts.
• Deploy AI agents on Databricks with Mosaic Model Serving / Agent Framework.
• Build CI/CD pipelines for Databricks deployments.
• Implement RAG pipelines with Vector Search and Unity Catalog.
• Apply responsible AI practices and human-in-the-loop gates.

Skills

AI/ML engineering

Python

LangChain

LangGraph

AutoGen

CrewAI

Semantic Kernel

Prompt engineering

RAG architectures

Vector databases

LLM API integration

Azure cloud

Databricks platform

Education

Bachelor's degree in CS/Engineering/Statistics

Master's degree in CS/Engineering/Statistics

Tools

Cursor

GitHub Copilot

Databricks

Delta Lake

Unity Catalog

MLflow

Git

Job description

AI & Intelligent Systems Featured Opportunity Work from office US EST Timezone

Share role:

Role Overview

We are seeking an innovative and highly skilled AI Engineer to design, develop, and deploy autonomous AI agents and production-ready AI applications that elevate Data Science and Business Intelligence capabilities. You will build, test, and maintain agent workflows that automate complex business processes while ensuring security, reliability, responsible AI practices, and enterprise-grade quality.

Key Summary: Architect multi-agent orchestration systems using LLMs, tool-calling, and RAG. Deploy and operationalize AI/ML workloads on the Databricks Data Intelligence Platform using Mosaic AI, Unity Catalog, and Vector Search while maintaining high code velocity using AI-assisted tools like Cursor and GitHub Copilot.

Key Responsibilities

• Design, build, and deploy autonomous AI agents using LLMs, tool-calling, RAG, and multi-agent orchestration patterns (planning agents, execution agents, validation agents).
• Build production-ready agent applications with proper error handling, guardrails, retry logic, and observability (logging, tracing, metrics), including native tracing/observability via MLflow Tracing on Databricks.
• Leverage AI-assisted development tools (Cursor Agent Mode, GitHub Copilot) to accelerate delivery and maintain high code quality through rules, skills, and context management.
• Translate business problems into technical agent designs, defining agent roles, tool inventories, memory strategies, and escalation paths.
• Architect multi-step agent workflows with planning, tool use, memory management, and human-in-the-loop checkpoints.
• Implement prompt engineering best practices, evaluation frameworks, and systematic agent testing strategies leveraging Mosaic AI Agent Evaluation and MLflow Evaluate.
• Build and maintain reusable agent templates, Cursor skills and rules libraries, and shared prompt catalogs for team-wide productivity.
• Deploy, host, and serve AI agents and LLM-backed applications on Databricks using Mosaic AI Model Serving / Agent Framework, Databricks Apps, and Foundation Model APIs with Unity Catalog governance.
• Build and maintain Databricks Asset Bundles (DABs) and CI/CD pipelines (Databricks Workflows, GitHub Actions) to automate repeatable, version-controlled deployment across dev/staging/prod.
• Implement RAG pipelines using Databricks Vector Search and Delta Lake–backed knowledge bases governed through Unity Catalog.
• Apply responsible AI practices: bias assessment, explainability, hallucination prevention, content filtering, and human-in-the-loop approval gates.

Mandatory Skills & Qualifications

• 5–8+ years of hands-on experience in AI/ML engineering, applied machine learning, or AI application development.
• Expert Python and strong software engineering practices (testing, packaging, code review, version control).
• Proven experience building AI agents using frameworks such as LangChain, LangGraph, AutoGen, CrewAI, or Semantic Kernel.
• Proficiency with AI-assisted development tools (Cursor, GitHub Copilot) for rapid, high-quality development including agent mode and context management.
• Experience with prompt engineering, RAG architectures, vector databases, and LLM API integration (OpenAI, Azure OpenAI, Anthropic).
• Experience deploying AI applications to production in cloud environments (Azure preferred) and serving models/agents on Databricks.
• Hands‑on experience with the Databricks platform: notebooks/Databricks Connect, Delta Lake, and Unity Catalog.

Desired Skills & Bonus Attributes

• Experience building multi-agent systems with tool calling, function calling, and MCP (Model Context Protocol) integrations.
• Experience with agent evaluation frameworks (LangSmith, Braintrust, Mosaic AI Agent Evaluation).
• Experience automating deployment using Databricks Asset Bundles (DABs), REST APIs, or Terraform for Databricks.

Python LangChain / AutoGen / CrewAI Databricks Mosaic AI RAG / Vector Search Unity Catalog MLflow Tracing Cursor / Copilot Azure

Education & Work Schedule

Education Requirements

Bachelor’s or Master’s degree in Computer Science, Engineering, Statistics, or a related field.

Must be willing to work from the Hyderabad office on Eastern Standard Time Zone (US EST)

Why Build Your Career at Spearsoft?

At Spearsoft, you work on modern AI solutions, autonomous multi-agent systems, and digital assurance platforms that directly impact enterprise operations globally. We foster a culture of rapid innovation, deep engineering ownership, and continuous skill acceleration.