Senior AI / Python Engineer

Vaisesika Consulting Pvt Ltd; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Vaisesika Consulting Pvt Ltd is hiring a Senior AI / Python Engineer to design and build core platform components for an enterprise MCP platform linking ChatGPT Enterprise to governed data in Databricks Unity Catalog.

You will own the orchestration layer, entity resolution, prompts, observability, and production upgrades, with a focus on accuracy, latency, and cost. The role involves collaboration with AWS/EKS, security, and business users in a fast-moving environment.

Qualifications

• 6+ years of software engineering with Python 3.10+.
• Experience building LLM apps: prompts, tools, orchestration.
• Ownership of production components from design to ops.
• Experience with entity matching, NLP, or retrieval (embeddings, RAG).
• API and security fundamentals: REST, OAuth 2.0 / OIDC.

Responsibilities

• Design and build the multi-agent orchestration layer (intent classification, routing, handoffs).
• Develop account and entity resolution with versioned Python packages.
• Enhance the MCP server (FastMCP) and add features from requirements to release.
• Document trade-offs across accuracy, latency and cost.
• Instrument prompts/tools with Langfuse; track versions and cost.
• Maintain runbooks and triage production issues; ensure quality.

Skills

Python

Async programming

LLM applications

NLP

API security

Prompt engineering

Tools

GitHub Actions

Docker

Kubernetes

AWS

Databricks

Job description

Role Summary

We are hiring a Senior AI / Python Engineer for one of the AI Initiative, which delivers an enterprise MCP (Model Context Protocol) platform connecting ChatGPT Enterprise to governed data in Databricks Unity Catalog. You will design and build core platform components, including the orchestration layer, account and entity resolution, and observability. You will also enhance the MCP server, engineer and evaluate prompts and agent instructions, enable new features, and own upgrades, releases, and production quality.

Key Responsibilities

Platform development

• Design and build the multi-agent orchestration layer: intent classification, semantic routing (embedding similarity with LLM fallback), agent handoffs, and clarification flows.
• Build and evolve account and entity resolution: alias indexes, fuzzy and semantic matching, batch index publishing with versioned swaps, and shadow-to-enforce rollouts, delivered as versioned, tested Python packages.
• Enhance the MCP server (FastMCP): tools, agents, and new features from requirements through release.
• Write design documents before coding and document trade-offs across accuracy, latency, and cost.

Prompt engineering and agent configuration

• Develop and refine prompts, agent instructions, tool descriptions, and semantic examples; reduce hallucination and tool-selection errors.
• Onboard and maintain agents: configuration, vector stores and recipe files, retrieval tuning, and SQL logic against Databricks Unity Catalog.

Observability, evaluation, and quality

• Instrument agents and tools with Langfuse (or equivalent) tracing; track prompt versions, token cost, latency, and quality per agent.
• Maintain evaluation suites (e.g., Promptfoo) with LLM-as-judge grading; add regression tests for every prompt or agent change and gate releases on the results.
• Own the audit and activity pipeline (CloudWatch, S3, Databricks); use logs, traces, and user feedback to improve routing, prompts, and retrieval.

Platform lifecycle and operations

• Plan and run dependency upgrades (OpenAI Agents SDK, openai, pydantic, FastMCP): breaking-change analysis, compatibility testing, phased rollout, and rollback.
• Manage CI/CD (GitHub Actions), nonprod-to-prod promotion, configuration and secrets, and Kubernetes (EKS) rollouts.
• Integrate external services and MCPs (e.g., Salesforce/MuleSoft, APISIX-based MCPs).
• Triage and resolve production issues; write and maintain runbooks.

Scale, security, and governance

• Optimize performance and cost: concurrency, rate limiting, connection pooling, Redis caching, retries and timeouts, model selection, token usage, and multi-pod consistency.
• Implement enterprise controls: OAuth (Azure AD) with user impersonation (OBO), row-level security compatibility, input validation, prompt-injection defenses, PII handling, and audit logging.

Collaboration

• Work with the OpenAI team, AWS/EKS infrastructure, security owners, and business users.
• Use AI-assisted development and review tools while owning design decisions and verifying every change with tests.

Required Qualifications

• 6+ years of professional software engineering experience, with strong Python (3.10+): async programming, typing, packaging, and testing (pytest).
• Hands-on experience building LLM applications: prompt engineering, tool/function calling, and multi-agent or orchestration design.
• Proven ownership of a production system component (e.g., orchestration, routing, matching, or search) from design through operations.
• Experience with entity matching, NLP, or retrieval (fuzzy matching, embeddings, RAG).
• API and security fundamentals: REST, OAuth 2.0 / OIDC.
• Working knowledge of AWS (EKS, S3, CloudWatch, DynamoDB, Redis/ElastiCache), Docker, and Kubernetes.
• Strong SQL and a cloud data platform (Databricks or similar), including notebooks/PySpark.
• Git and CI/CD (GitHub Actions); track record of managing breaking dependency upgrades safely.
• Clear written and verbal communication; able to work independently in an ambiguous, fast-moving environment.

Preferred Qualifications

• Experience with Model Context Protocol (MCP), FastMCP, OpenAI Agents SDK, or comparable agent frameworks (LangChain / LangGraph).
• Experience with LLM observability and evaluation tools (Langfuse, OpenTelemetry, Promptfoo).
• Familiarity with Databricks Unity Catalog governance, row/column-level security, and metric views or semantic layers.
• Background in enterprise security: Azure AD / Entra ID, identity propagation, secrets management.
• Experience optimizing cost and latency of LLM workloads.
• Life sciences, commercial, or enterprise data domain experience.

Technology Stack

Python 3.11 FastMCP OpenAI Agents SDK / ChatGPT Enterprise Databricks Unity Catalog AWS (EKS, CloudWatch, S3, DynamoDB, Redis, ALB, WAF) Azure AD OAuth Langfuse Promptfoo GitHub Actions Docker / Kubernetes