Azure Databricks consultants

Shree Narayani Networking Solutions Pvt Ltd

Atlanta (GA)

On-site

USD 150,000 - 190,000

Full time

9 days ago
Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Job summary

Shree Narayani Networking Solutions Pvt Ltd in Atlanta, GA is seeking a Senior Databricks & AI Engineer to advance the Ask Anna platform. You will optimize Databricks performance, design robust data pipelines, and develop GenAI-enabled capabilities to drive enterprise AI initiatives.

The role requires strong Azure API integrations, DataOps experience, and hands-on expertise in Python, SQL, and modern data engineering. On-site collaboration with product teams will be essential.

Qualifications

  • 5+ years in Data Engineering or Analytics Engineering.
  • Deep Databricks development, optimization, and administration.
  • Strong DataOps, automation, and CI/CD practices.
  • Experience with Databricks Genie and AI-enabled analytics.
  • Proficiency in SQL, Python, and cloud services (Azure).

Responsibilities

  • Lead Databricks performance tuning and optimization.
  • Design and maintain robust data pipelines in Databricks.
  • Develop AI/GenAI-enabled capabilities and AI workflows.
  • Build Azure-based APIs and integrations with front-end apps.
  • Collaborate on enterprise AI roadmap and strategic planning.

Skills

Databricks development
Performance tuning
DataOps
Azure
API development
GenAI
Python
SQL

Tools

Databricks Genie
REST API design
CI/CD
Cloud integrations

Job description

Databricks & AI Engineer

On-site

Start Date

Oct-04-2026
Atlanta, GA

Description
Position Summary

We are seeking a highly skilled Senior Databricks & AI Engineer to support the continued evolution of the Ask Anna platform. This position is critical to the success of the Ask Anna Performance Enhancement Initiative and will focus on improving platform performance, scalability, and AI capabilities based on production feedback.
The ideal candidate will possess deep expertise in Databricks, Data Engineering, DataOps, Azure, APIs, and AI/GenAI technologies. This role will be instrumental in optimizing response times, enhancing the user experience, expanding data sources, and helping drive the future AI roadmap for the organization.
This is a strategic position with significant visibility and impact, supporting the next generation of enterprise AI solutions.

Key Responsibilities
Databricks Performance & Optimization
  • Lead Databricks performance tuning and optimization initiatives.
  • Analyze and resolve performance bottlenecks impacting Ask Anna response times.
  • Implement scalable data processing and query optimization strategies.
  • Monitor platform performance and recommend continuous improvements.
Data Engineering & DataOps
  • Design, develop, and maintain robust data pipelines within Databricks.
  • Support DataOps best practices, automation, monitoring, and deployment processes.
  • Improve data quality, reliability, and operational efficiency across the platform.
  • Enable onboarding of new datasets and data sources into Ask Anna.
AI & Generative AI Solutions
  • Develop and optimize AI and GenAI-enabled capabilities within the Ask Anna ecosystem.
  • Collaborate with stakeholders to identify opportunities for AI-driven enhancements.
  • Support the development and deployment of AI-powered agents and intelligent workflows.
  • Contribute to enterprise AI platform expansion and future innovation initiatives.
Cloud Architecture & Integrations
  • Support Azure-based architecture, services, and AI solutions.
  • Design and build APIs that enable seamless integration between front-end applications and Databricks services.
  • Partner with application development teams to ensure scalable, secure, and high-performing solutions.
  • Participate in architecture reviews and long-term technology planning.
Strategic Innovation
  • Contribute to the Ask Anna product roadmap and long-term performance strategy.
  • Evaluate emerging Databricks, AI, and cloud technologies for future adoption.
  • Support enterprise scalability initiatives and future AI platform capabilities.
Required Qualifications
  • 5+ years of experience in Data Engineering, Analytics Engineering, or related technical roles.
  • Advanced hands-on experience with Databricks development, optimization, and administration.
  • Strong experience with Data Engineering and modern data pipeline architectures.
  • Proven experience implementing DataOps practices and automation.
  • Experience working with Databricks Genie and AI-enabled analytics capabilities.
  • Strong knowledge of Microsoft Azure services and cloud-native architectures.
  • Experience designing and developing RESTful APIs and data integration services.
  • Hands‑on experience building, deploying, or supporting AI/GenAI solutions.
  • Strong proficiency in SQL, Python, and modern data engineering frameworks.
  • Excellent problem-solving and performance tuning skills.
Preferred Qualifications
  • Experience with Angular front-end development.
  • Experience developing AI Agents and autonomous workflows.
  • Knowledge of Google Cloud Platform (GCP) services and AI offerings.
  • Full-stack application development experience.
  • Experience integrating AI solutions into business applications and enterprise workflows.
  • Familiarity with conversational AI, semantic search, vector databases, or retrieval‑augmented generation (RAG) architectures.
Future Focus Areas
  • Expansion of Ask Anna datasets and knowledge sources.
  • Performance enhancement roadmap execution.
  • Azure-based AI agent development.
  • Databricks AI and machine learning capabilities.
  • Potential Google Cloud AI integrations.
  • Enterprise-scale GenAI platform expansion.
Success Measures
  • Improve Ask Anna performance and response times.
  • Enhance scalability and platform reliability.
  • Accelerate AI and GenAI feature delivery.
  • Enable seamless integration of additional datasets and services.
  • Help establish a long‑term, enterprise-ready AI architecture.
Enterprise Req Skills

Databricks, api, development, CICD

____________________________________________________________________________

Top Skills Details
Databricks Performance Optimization (Must Have)

Deep expertise in Databricks architecture, Spark optimization, query tuning, cluster management, and performance troubleshooting.
The primary business driver is improving Ask Anna performance and response times.

Data Engineering & DataOps

Strong experience building and optimizing data pipelines, ETL/ELT processes, data modeling, orchestration, monitoring, and CI/CD practices.
Needed to support new datasets, scalability, and operational excellence.

AI / GenAI Solutions

Experience building and deploying GenAI applications, RAG solutions, LLM integrations, Databricks Genie, and AI-enabled business solutions.
Critical for future Ask Anna enhancements and AI roadmap initiatives.

Azure Cloud & API Development

Hands‑on experience with Azure services, cloud architecture, API design, and integrations between applications and Databricks.
Required to support AI agents, front-end integrations, and scalable enterprise solutions.

Experience Level

Expert Level

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Databricks AI Engineer
Databricks AI Engineer

IT America Inc • Austin (TX)

Hybrid
USD 150,000 - 190,000
Senior Databricks AI Engineer
Senior Databricks AI Engineer

PowerPlan, Inc. • Smyrna (GA)

On-site
USD 120,000 - 150,000
Work on real AI/ML problems
Influence the evolution of a data + AI platform
Partnership with senior leaders
Databricks AI Engineer
Databricks AI Engineer

Ven Soft LLC • Austin (TX)

On-site
USD 96,000 - 207,000
Azure Databricks Engineer (Dallas, TX)
Azure Databricks Engineer (Dallas, TX)

Cedent • Dallas (TX)

On-site
USD 120,000 - 150,000
Data Engineer/Architect with AI Agent Automation Engineer (Databricks)
Data Engineer/Architect with AI Agent Automation Engineer (Databricks)

Prohires • Austin (TX)

Hybrid
USD 140,000 - 210,000
Sr. Databricks Engineer
Sr. Databricks Engineer

hyrusinc • New York (NY)

On-site
USD 130,000 - 190,000
Lead AI Engineer - Bengaluru, INDIA
Lead AI Engineer - Bengaluru, INDIA

Vytwo • Dallas (TX)

Hybrid
USD 120,000 - 160,000
Sr. Manager, AI Forward Deployed Engineering (FDE)
Sr. Manager, AI Forward Deployed Engineering (FDE)

Vibehackers • California (MO)

Hybrid
USD 233,000 - 320,000
Databricks Architect - R01568474
Databricks Architect - R01568474

Brillio • United States

On-site
USD 130,000 - 190,000
Senior AI Data Engineer
Senior AI Data Engineer

Coalesce Management Consulting • Birmingham (AL)

On-site
USD 90,000 - 120,000