Azure Data Engineer

Ample Success HR; India; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Ample Success HR is seeking a Senior Databricks Developer in India to design, implement, and optimize scalable data platforms on Azure Databricks. You will leverage Spark, Delta Lake, and Azure services to deliver robust data pipelines and governance for financial markets data.

You will collaborate with cross-functional teams to ensure secure, compliant, and highly available cloud data solutions, with a focus on DevOps, infrastructure as code, and scalable architecture.

Qualifications

• Extensive experience designing data platforms on Azure Databricks.

Responsibilities

• Design, develop, and optimize scalable data platforms using Azure Databricks.

Skills

Azure Databricks

Apache Spark

Python

SQL

Delta Lake

Azure Data Factory

Azure Synapse Analytics

Azure Data Lake Storage Gen2

Azure Key Vault

Azure Event Hub

Azure Functions

Terraform

Bicep

Azure DevOps

Git

CI/CD Pipelines

Azure Networking

Virtual Networks

Private Link

Private Endpoints

NSGs

Azure Firewall

DNS Management

ExpressRoute

SQL Server

Azure SQL Database

Tools

Terraform

Bicep

ARM Templates

Job description

Senior Databricks Developer with strong expertise in Azure Cloud Services, Data Engineering, and Financial Markets domain knowledge. The ideal candidate will be responsible for designing, developing, and optimizing scalable data platforms using Azure Databricks, while leveraging Azure networking, security, and DevOps best practices.

Key Responsibilities

• Data Engineering & Databricks
• Design, develop, and maintain scalable data pipelines using Azure Databricks.
• Build ETL/ELT solutions for structured and unstructured data processing.
• Develop Spark-based applications using Py Spark, Scala, or SQL.
• Implement Delta Lake architecture and optimize data storage and processing.
• Create reusable frameworks and components to improve development efficiency.
• Perform performance tuning and optimization of Databricks workloads.
• Implement data quality checks, monitoring, and governance controls.
• Understanding of data masking, security compliance control for financial markets data

Azure Cloud & Architecture

• Design cloud-native data solutions using Azure services.
• Integrate Databricks with:Azure Data Factory (ADF)Azure Synapse AnalyticsAzure Data Lake StorageAzure Event HubAzure Key VaultAzure Functions
• Ensure high availability, scalability, and security of data platforms.

Azure Networking & Security

• Configure and manage:Virtual Networks (V Net)Private EndpointsNetwork Security Groups (NSG)ExpressRoute/VPN connectivityAzure FirewallPrivate Link
• Implement secure connectivity between Databricks and enterprise systems.
• Ensure compliance with enterprise security and regulatory requirements.
• Work closely with infrastructure and security teams to establish cloud networking best practices.

DevOps & Automation

• Build and maintain CI/CD pipelines using Azure DevOps.
• Automate deployments using Infrastructure as Code (Terraform, Bicep, ARM Templates).
• Implement release management and environment promotion strategies.
• Configure automated testing, monitoring, and alerting mechanisms.
• Support version control using Git and branching strategies.

Financial Markets Domain

• Work with business stakeholders in Capital Markets and Financial Services.
• Understand financial instruments
• Translate business requirements into scalable technical solutions.

Required Skills

Technical Skills

• Azure Databricks
• Apache Spark
• Py SparkSQLPython
• Delta Lake
• Azure Data Factory
• Azure Synapse Analytics
• Azure Data Lake Storage Gen2
• Azure Key Vault
• Azure Event Hub
• Azure Functions
• Terraform / Bicep
• Azure DevOps
• Git
• CI/CD Pipelines
• Azure Networking
• Virtual Networks (V Net)
• Private Link
• Private Endpoints
• NSGs
• Azure Firewall
• DNS Management
• ExpressRouteHybrid Networking
• Database Technologies
• SQL Server
• Azure SQL Database

Financial Markets Expertise

• Strong understanding of:Capital Markets ecosystemTrade lifecycle managementMarket and reference data processingRisk and compliance reportingFront, Middle, and Back Office operationsFinancial regulations and reporting standards

Key Competencies

• Strong analytical and problem-solving skills
• Excellent stakeholder communication
• Solution architecture mindset
• Leadership and mentoring capabilities
• Ability to work in Agile/Scrum environments
• Strong ownership and accountability

Target Profile

• Senior Data Engineer / Databricks Engineer who combines deep Azure technical expertise with financial markets knowledge and can independently drive design, development, deployment, and support of enterprise-grade cloud data platforms.