Databricks GenAI Solutions Architect

Celebal Technologies

Bengaluru

On-site

INR 4,200,000 - 7,000,000

Full time

12 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Databricks ecosystem exposure

Job summary

Celebal Technologies is seeking a highly skilled Databricks GenAI Solutions Architect to lead design, development, and deployment of enterprise-grade GenAI solutions on the Databricks Lakehouse Platform. You will architect scalable LLM-based applications, implement RAG pipelines, and establish MLOps/LLMOps practices.

Responsibilities include end-to-end GenAI design, selecting suitable LLMs, vector search implementation, prompt engineering, and governance using Unity Catalog.

Qualifications

  • GenAI/LLM expertise.
  • 5+ years of ML/AI with 3+ years focused on GenAI/LLM applications.
  • Hands-on experience with LLM APIs (OpenAI, Anthropic Claude, Azure OpenAI, Google Vertex AI)
  • Strong RAG implementation experience including document processing, chunking strategies, embedding models, and retrieval strategies

Responsibilities

  • GenAI Architecture & Development for end-to-end GenAI solutions on Databricks.
  • Evaluate and select LLMs based on use case and cost.
  • Design scalable vector search solutions using Databricks Vector Search and Mosaic AI.
  • Establish prompt engineering best practices and management strategies.
  • Implement LLMOps pipelines for evaluation, monitoring, and deployment.
  • Lead architectural reviews and mentor engineers on GenAI standards.

Skills

GenAI/LLM Expertise
LLM APIs experience
RAG implementation
Vector databases
Prompt engineering
LLMOps
Databricks Platform
PySpark / Spark SQL
Python programming
API design / microservices

Tools

Databricks Mosaic AI
MLflow
Unity Catalog
Delta Lake
Databricks Workflows
LangChain
Langgraph
Semantic Kernel

Job description

About the Company

Celebal Technologies is a premier software services company in the field of Data Science, Big Data and Enterprise Cloud. Celebal Technologies helps you to discover the competitive advantage by employing intelligent data solutions using cutting-edge technology solutions that can bring massive value to your organization. The core offerings are around "Data to Intelligence", wherein we leverage data to extract intelligence and patterns thereby facilitating smarter and quicker decision making for clients.Celebal Technologies solutions are powered by Robotics, Artificial Intelligence and Machine Learning algorithms which offer improved business efficiency in the interconnected world. Companies clients are Product ISVs and delivery organisations across the globe who use our niche expertise in product development as well as Enterprise project implementations. We have adopted the highest standards of service quality and operational excellence, enabling its clients across a wide range of industries to transform into a data-driven enterprise. Our tailor-made solutions help enterprises maximise productivity, improve speed and accuracy.

About the Role

We are seeking a highly skilled Databricks GenAI Solutions Architect to lead the design, development, and deployment of enterprise-grade Generative AI solutions on the Databricks Lakehouse Platform. This role combines deep expertise in Large Language Models (LLMs) with comprehensive knowledge of the Databricks ecosystem, including Mosaic AI, MLflow, Unity Catalog, and Delta Lake. You will be responsible for architecting scalable LLM-based applications, implementing RAG pipelines, establishing MLOps/LLMOps best practices, and mentoring the team on cutting-edge GenAI technologies within the Databricks environment.

Responsibilities
  • GenAI Architecture & Development
  • Architect end-to-end GenAI solutions including RAG (Retrieval-Augmented Generation), AI agents, and fine-tuned models on Databricks
  • Evaluate and select appropriate LLMs (OpenAI, Anthropic Claude, Azure OpenAI, DBRX, Llama, Mistral) based on use case requirements and cost optimization
  • Design and implement scalable vector search solutions using Databricks Vector Search and Mosaic AI
  • Establish prompt engineering best practices, optimization techniques, and prompt management strategies
  • Implement compound AI systems and multi-agent architectures using Databricks Agent Framework
  • Databricks Platform Expertise
  • Design and implement GenAI solutions leveraging Databricks Mosaic AI capabilities including Model Serving, Foundation Model APIs, Vector Search, AI Gateway, and Feature & Function Serving
  • Architect data pipelines using Delta Lake and Spark for preprocessing, chunking, and embedding generation at scale
  • Implement robust data governance using Unity Catalog for model assets, features, and vector indices
  • Design and manage MLflow experiments for LLM evaluation, tracking, and model registry
  • LLMOps & Production Deployment
  • Design LLMOps pipelines for evaluation, monitoring, A/B testing, and continuous improvement
  • Implement guardrails, safety measures, content filtering, and responsible AI practices
  • Build observability solutions for LLM applications including latency monitoring, cost tracking, and quality metrics
  • Establish CI/CD pipelines for GenAI model deployment using Databricks Workflows and Asset Bundles
  • Leadership & Collaboration
  • Lead technical decisions and conduct architecture reviews for GenAI initiatives
  • Collaborate with business stakeholders to translate requirements into scalable technical solutions
  • Mentor engineers and data scientists on GenAI development standards and Databricks best practices
  • Create technical documentation, reference architectures, and reusable solution accelerators
Qualifications
  • GenAI/LLM Expertise
  • 5+ years of relevant experience in ML/AI with 3+ years focused on GenAI/LLM applications
  • Deep hands-on experience with LLM APIs (OpenAI, Anthropic Claude, Azure OpenAI, Google Vertex AI)
  • Strong RAG implementation experience including document processing, chunking strategies, embedding models, and retrieval strategies (hybrid search, re-ranking, query transformation)
  • Experience with vector databases (Pinecone, Weaviate, ChromaDB, FAISS, Azure AI Search, Databricks Vector Search)
  • Advanced prompt engineering skills including chain-of-thought, few-shot learning, and prompt optimization
  • Proficiency with LLM orchestration frameworks (LangChain, Langgraph, Semantic Kernel)
  • Understanding of fine-tuning techniques (LoRA, QLoRA, PEFT, full fine-tuning)
  • Databricks Platform Skills
  • 3+ years hands-on experience with Databricks platform
  • Strong proficiency in PySpark and Spark SQL for large-scale data processing
  • Experience with Databricks Mosaic AI suite: Model Serving, Vector Search, Foundation Model APIs, AI Gateway
  • Expertise in MLflow for experiment tracking, model registry, and deployment
  • Working knowledge of Unity Catalog for data and AI governance
  • Experience with Delta Lake architecture and optimization
  • Familiarity with Databricks Workflows for orchestration
  • Technical Foundations
  • Strong programming skills in Python; familiarity with SQL
  • Production deployment experience for LLM applications at scale
  • Excellent system design and distributed architecture skills
  • Understanding of API design, microservices, and cloud-native patterns
  • Experience with containerization (Docker, Kubernetes) for model serving
Required Skills
  • GenAI/LLM Expertise
  • Deep hands-on experience with LLM APIs (OpenAI, Anthropic Claude, Azure OpenAI, Google Vertex AI)
  • Strong RAG implementation experience including document processing, chunking strategies, embedding models, and retrieval strategies (hybrid search, re-ranking, query transformation)
  • Experience with vector databases (Pinecone, Weaviate, ChromaDB, FAISS, Azure AI Search, Databricks Vector Search)
  • Advanced prompt engineering skills including chain-of-thought, few-shot learning, and prompt optimization
  • Proficiency with LLM orchestration frameworks (LangChain, Langgraph, Semantic Kernel)
  • Understanding of fine-tuning techniques (LoRA, QLoRA, PEFT, full fine-tuning)
  • Databricks Platform Skills
  • 3+ years hands-on experience with Databricks platform
  • Strong proficiency in PySpark and Spark SQL for large-scale data processing
  • Experience with Databricks Mosaic AI suite: Model Serving, Vector Search, Foundation Model APIs, AI Gateway
  • Expertise in MLflow for experiment tracking, model registry, and deployment
  • Working knowledge of Unity Catalog for data and AI governance
  • Experience with Delta Lake architecture and optimization
  • Familiarity with Databricks Workflows for orchestration
  • Technical Foundations
  • Strong programming skills in Python; familiarity with SQL
  • Production deployment experience for LLM applications at scale
  • Excellent system design and distributed architecture skills
  • Understanding of API design, microservices, and cloud-native patterns
  • Experience with containerization (Docker, Kubernetes) for model serving

Any Location- Jaipur/ Noida/Gurgaon/ Pune/ Bangalore/ Hyderabad

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Science Lead – R01570347
Data Science Lead – R01570347

Brillio • Bengaluru

On-site
INR 3,200,000 - 5,200,000
Databricks AI Engineer
Databricks AI Engineer

Kiya.ai • Hyderabad, Bengaluru

Hybrid
INR 900,000 - 1,300,000
Gen AI Architect
Gen AI Architect

Cybage Software • Pune District

On-site
INR 4,200,000 - 7,000,000
Senior Data Scientist (ML Engineer)
Senior Data Scientist (ML Engineer)

Indsafri • India

On-site
INR 1,800,000 - 3,200,000
Sr Gen AI Engineer
Sr Gen AI Engineer

Safran Engineering Services • Bengaluru

On-site
INR 3,000,000 - 6,000,000
Data Science Lead - R01570347
Data Science Lead - R01570347

Brillio • Bengaluru

On-site
INR 2,500,000 - 4,000,000
Technical Architect- Remote, India
Technical Architect- Remote, India

LeewayHertz Technologies Pvt. Ltd • India

Remote
INR 6,000,000 - 9,000,000
Data Scientist (Gen/Agentic AI solutions)
Data Scientist (Gen/Agentic AI solutions)

Holcim • Navi Mumbai

On-site
INR 2,500,000 - 5,000,000
MLOps Architect With (Databricks)
MLOps Architect With (Databricks)

Anblicks • Hyderabad

On-site
INR 2,000,000 - 3,000,000
Senior AI Engineer - Databricks
Senior AI Engineer - Databricks

Adesso SE • Bengaluru

On-site
INR 1,800,000 - 3,600,000