SENIOR SOFTWARE ENGINEER - Azure Databricks

Happiest Minds Technologies; Bengaluru; On-site; INR 1,200,000 - 2,100,000; Full time

Job summary

Happiest Minds Technologies is seeking an Azure Data Engineer in Bengaluru to design, build, and support end-to-end data pipelines on Azure. You will extract data from multiple sources, transform it with PySpark and SQL, and deploy pipelines using ADF and Synapse to support reporting and reconciliation processes.

The ideal candidate will have 5–6 years of hands-on Azure data engineering experience, strong SQL/Python skills, and familiarity with CI/CD practices and data orchestration in the Azure

Qualifications

• Experience with Azure data engineering in building end-to-end pipelines.
• Strong SQL, Python and PySpark for data transformations.
• Familiarity with SFTP data extraction and file-based integrations.

Responsibilities

• Extract data from source systems and files from SFTP locations.
• Build and monitor ADF/Synapse pipelines end-to-end.
• Transform data using SQL, Python and PySpark per mapping documents.
• Generate final reports/files and load into downstream systems.
• Update JIRA and document in Confluence.

Skills

Azure Data Factory

Azure Databricks

Azure Active Directory

Azure Storage

Azure Data Lake Services

Azure Key Vault

Azure Logic Apps

Azure SQL DB

Azure HDInsight

Azure DevOps

Python

PySpark

SQL

Spark SQL

Azure Synapse Analytics

Tools

GitHub

Power BI

REST APIs

Job description

Role Overview

We are looking for an experienced

Job Description: Azure Data Engineer

Experience Required: 5 ? 6 Years

Role Overview

We are looking for an experienced Azure Data Engineer to design, build, and support end-to-end data pipelines on the Azure ecosystem. The role involves extracting and transforming data from multiple source systems, building robust ADF/Synapse/Databricks pipelines, and supporting business-critical reporting and reconciliation processes.

Skills Required

• Azure Data Factory (ADF)
• Azure Databricks (ADB)
• Azure Active Directory
• Azure Storage
• Azure Data Lake Services (ADLS)
• Azure Key Vault
• Azure Logic Apps
• Azure SQL DB
• Azure HDInsight
• Good knowledge of Azure DevOps (ADO) services ? Repos, Boards, Build Pipelines (CI/CD)
• Good knowledge of developing data engineering frameworks and notebooks using Azure Databricks with Spark SQL, PySpark, and Python
• Good knowledge of Azure Synapse Analytics

Good-to-Have Skills

• GitHub
• PowerApps
• Azure Monitor and Log Analytics
• Power BI
• REST APIs

Daily Activities

• Extracting data from source systems and files in any format from SFTP locations.
• Processing the extracted data and storing it into staging tables in Azure Blob Storage.
• Transforming data based on the mapping (STTM) document to meet given requirements in ADB using SQL, Python, and PySpark code.
• Generating a final TLM file or report and loading it into the Smartstream environment.
• Building pipelines end-to-end using ADF and Synapse.
• Monitoring pipelines and alerts in ADF.
• Implementing new enhancements for the LBU.
• Analyzing and fixing issues raised by the business.Updating JIRA and documenting in Confluence.

Ideal Candidate Profile

• 5?6 years of hands-on experience in Azure data engineering, with strong exposure to ADF, Databricks, and Synapse.
• Solid programming skills in SQL, Python, and PySpark for building transformation logic and data engineering frameworks.
• Experience working with SFTP-based data extraction and file-based integrations.
• Familiarity with CI/CD practices using Azure DevOps.
• Strong analytical and issue-resolution skills, with experience supporting business/production issues.
• Good communication skills for documentation (Confluence) and task tracking (JIRA).

Azure Databricks