Lead AI Databricks Engineer | Individual Contributor

Enable Data Incorporated; Bengaluru; Hybrid; INR 4,000,000 - 6,000,000; Full time

Job summary

Enable Data Incorporated in Bengaluru (Hybrid/remote) seeks a senior architect and full‑stack engineer to shape data platforms, Databricks pipelines, and AI‑driven apps. You will drive architecture decisions, implement secure coding practices, and deliver end‑to‑end data solutions spanning ingestion, processing, and presentation.

The role covers design reviews, cloud resource planning, and documentation to satisfy security approvals.

Qualifications

• Experience with Azure DevOps, CI/CD pipelines.
• Knowledge of Databricks, PySpark, SQL and Unity Catalog.
• Proficiency in Python, FastAPI and React.
• Familiarity with AI/ML concepts, prompt engineering, and RAG patterns.

Responsibilities

• Adapt architecture and code for security, authentication, and cloud requirements.
• Participate in design reviews and Azure resource planning.
• Create architecture, network, and deployment docs for security approvals.
• Design and develop data ingestion pipelines with PySpark/SQL from SAP and web data.
• Configure Unity Catalog permissions, service principals, and OBO tokens.
• Build and deploy Databricks Apps and integrate AI/ML services.
• Develop lightweight AI apps using FastAPI, React, and LLM APIs.
• Apply prompt engineering, AI guardrails, and caching strategies.
• Troubleshoot frontend, backend, and data-layer components; fix security issues.

Skills

Azure DevOps

CI/CD

Azure Cloud Services

Databricks

PySpark

SQL

Python

FastAPI

React

Prompt Engineering

RAG

LLM integrations

Full Stack Development

Security

Technical Documentation

Education

Bachelor's degree in Computer Science or related field

Tools

Unity Catalog

Azure resource planning

Microsoft Teams integration

Job description

Total Experience: 10+ Years

Notice Period: Immediate Joiner

Location: Bengaluru ( Hybrid/remote)

Solution & Data Architecture

• Adapt application architecture and code to meet security, authentication, and platform requirements.
• Participate in technical design reviews and Azure resource planning.
• Create and maintain architecture, network, and deployment documentation for security and cloud approvals.

Databricks Development

• Design and develop data ingestion pipelines using PySpark and SQL from sources such as SAP and web data.
• Perform testing, validation, and stakeholder reviews to ensure solution quality.
• Configure Unity Catalog permissions, service principals, OBO tokens, and AI/ML-related Databricks services.
• Build and deploy Databricks Apps and integrate custom notifications with Microsoft Teams.

AI & Application Development

• Develop lightweight AI applications using FastAPI, React, and LLM APIs.
• Apply prompt engineering techniques, RAG patterns, AI Search, chunking strategies, caching, and AI guardrails.
• Analyze, re-engineer, and enhance existing codebases to meet business requirements.
• Utilize graph-based approaches to understand dependencies and rule relationships.

Full Stack Engineering

• Troubleshoot and support frontend, backend, middleware, API, and data-layer components.
• Resolve security vulnerabilities and implement secure coding practices.
• Produce technical documentation, solution designs, and development artifacts.

Required Skills

• Azure DevOps, CI/CD, Azure Cloud Services
• Databricks, PySpark, SQL, Unity Catalog
• Python, FastAPI, React
• Prompt Engineering and Generative AI concepts
• RAG, AI Search, LLM integrations, Claude APIs
• Full Stack Application Development
• Security, Architecture Documentation, Technical Design

DevOps & CI/CD

• Build and automate CI/CD pipelines using Azure DevOps for Teams Bots, web applications, and Databricks deployments.
• Implement quality gates such as Snyk security scans, linting, and validation checks.
• Configure and manage deployment environments across development, testing, and production.