Databricks Forward Deployed Engineer (FDE)

ALIQAN Technologies; India; On-site; INR 3,500,000 - 7,000,000; Full time

Job summary

ALIQAN Technologies is seeking an experienced Databricks Forward Deployed Engineer to design and optimize enterprise data platforms. You will work directly with clients to translate business needs into scalable Lakehouse architectures using Medallion patterns and Delta Lake.

The role emphasizes data engineering, performance tuning, CI/CD deployment, and GenAI-enabled solutions, with strong emphasis on client engagement and end-to-end delivery.

Qualifications

• 5+ years of production data engineering experience in data platforms.
• At least 2 years hands-on with Databricks or Apache Spark.
• Strong programming skills in Python and SQL.
• Experience with Azure, React, and CI/CD pipelines.

Responsibilities

• Lead client discussions and technical discovery to design lakehouse solutions.
• Develop production-grade data pipelines using Python, PySpark, Spark SQL, and Delta Lake.
• Optimize Spark jobs for performance, cost, and scalability.
• Implement batch and streaming ingestion with Auto Loader and Kafka.
• Configure Databricks assets, Unity Catalog, and data governance.
• Deliver end-to-end data engineering services with GenAI and RAG capabilities.

Skills

Python

Databricks

PySpark

Apache Spark

Spark SQL

Azure

GenAI

CI/CD

SQL

React

Tools

Databricks

GitHub Actions

Azure DevOps

Unity Catalog

Auto Loader

Kafka

Delta Lake

DABs

RAG

Vector Databases

LLM Evaluation

Job description

Responsibility

Role Summary

We are looking for an experienced Databricks Forward Deployed Engineer (FDE) to design, develop, and optimize production-grade data platforms for enterprise clients. The ideal candidate should have strong expertise in Python, PySpark, Apache Spark, Databricks, Microsoft Azure, Generative AI, and CI/CD, along with excellent communication and client-facing skills.

The role involves working directly with clients to understand business requirements, design scalable Lakehouse architectures, develop data pipelines, troubleshoot performance issues, and deliver reliable, cost-effective data engineering solutions.

Key Responsibilities

Data Engineering & Lakehouse Architecture

Lead client discussions, technical discovery sessions, and solution design activities.

Design and implement enterprise Lakehouse solutions using Medallion Architecture (Bronze, Silver, and Gold).

Develop and optimize production-grade data pipelines using Python, PySpark, Spark SQL, and Delta Lake.

Implement batch and streaming data ingestion using Auto Loader, Kafka, and related frameworks.

Configure Databricks clusters, Unity Catalog, access controls, and data governance.

Performance Optimization & Deployment

Optimize Spark jobs and Delta Lake tables for performance, scalability, and cost efficiency.

Implement CI/CD pipelines using Databricks Asset Bundles (DABs), GitHub Actions, or Azure DevOps.

Monitor production workloads, troubleshoot technical issues, and implement data quality checks and alerting.

Manage production deployments and ensure reliable, maintainable, and scalable data solutions.

Generative AI & Client Engagement

Support GenAI solutions involving RAG pipelines, vector stores, and LLM evaluation.

Present technical recommendations and architecture decisions to clients and business stakeholders.

Translate business requirements into technical solutions and take ownership of end-to-end delivery.

Explain complex technical concepts clearly to technical and non-technical stakeholders.

Required Skills & Experience

5+ years of experience in data engineering within production environments.

At least 2 years of hands-on experience with Databricks or Apache Spark.

Strong programming skills in Python.

Hands-on experience with Databricks, Apache Spark, PySpark, and Spark SQL.

Experience with Microsoft Azure cloud services.

Experience with React.

Strong SQL and data pipeline development skills.

Experience designing Medallion Architecture and enterprise Lakehouse platforms.

Knowledge of Delta Lake, data ingestion, performance tuning, and data governance.

Hands-on experience with CI/CD pipelines and deployment automation.

Exposure to Generative AI (GenAI) and related solutions.

Excellent communication, client-facing, and stakeholder management skills.

Preferred Skills

Experience with Structured Streaming, Delta Live Tables, or Kafka.

Knowledge of Unity Catalog, Auto Loader, Change Data Capture (CDC), and Slowly Changing Dimensions (SCD).

Exposure to GenAI, Retrieval-Augmented Generation (RAG), vector databases, and LLM evaluation.

Databricks Data Engineer Associate or Professional certification.

Experience with AWS or GCP in addition to Azure.

Previous experience in consulting, Global System Integrators (GSIs), or client-embedded engineering roles.

Communication & Core Competencies

Excellent verbal and written communication skills.

Strong client-facing and stakeholder management capabilities.

Ability to communicate complex technical concepts to diverse audiences.

Strong problem-solving, analytical thinking, and decision-making skills.

Proactive mindset with end-to-end ownership of technical delivery.

Ability to work directly with enterprise clients and deliver high-quality technical solutions.

Key Skills

Databricks | Python | PySpark | Apache Spark | Spark SQL | Microsoft Azure | Delta Lake | Lakehouse Architecture | Medallion Architecture | SQL | React | CI/CD | Databricks Asset Bundles (DABs) | GitHub Actions | Azure DevOps | Unity Catalog | Auto Loader | Kafka | GenAI | RAG | Vector Databases | LLM Evaluation | Client Engagement | Data Engineering