Databricks - Technical Specialist-Data Engg

Birlasoft; Pune District; On-site; INR 1,800,000 - 2,800,000; Full time

Job summary

Birlasoft is seeking a Databricks - Technical Specialist-Data Engg in Pune to design, develop, and deploy scalable data pipelines using Azure services. You will work with Azure Data Factory, Databricks, PySpark, and SQL to support data integration and analytics initiatives.

The role involves creating technical specifications, handling file formats (XML, CSV, Parquet), and collaborating with BI teams to meet business requirements. Prior experience in on-premises and cloud ETL is beneficial.

Qualifications

• Experience with Azure data services and building scalable data pipelines.
• Strong knowledge of Databricks, PySpark, and SQL for data processing.
• Experience designing technical specifications and data integration solutions.

Responsibilities

• Design, develop and deploy data pipelines using Azure services (ADF, ADLS, Databricks).
• Build and maintain scalable data sources to support increasing data volumes.
• Develop batch and streaming data processing solutions.

Skills

Azure Data Factory

Azure Databricks

PySpark

SQL

ETL development

SSIS

Tools

Informatica

Job description

Select how often (in days) to receive an alert:

Location: INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI

Title: Databricks - Technical Specialist-Data Engg

Description:

Area(s) of responsibility

• Experience in Perform Design, Development & Deployment using Azure Services ( Data Factory, Azure Data Lake Storage, Databricks, PySpark, SQL)
• Develop and maintain scalable data pipelines and build out new Data Source integrations to support continuing increases in data volume and complexity.
• Experience in create the Technical Specification Design, Application Interface Design.
• Files Processing – XML, CSV, Excel, ORC, Parquet file Formats
• Develop batch processing, streaming and integration solutions and process Structured and Non-Structured Data
• Good to have experience with ETL development both on-premises and in the cloud using SSIS, Data Factory, and related Microsoft and other ETL technologies (Informatica preferred)
• Demonstrated in depth skills with Azure Data Factory, Azure Databricks, PySpark, ADLS (must have) with the ability to configure and administrate all aspects of Azure SQL DB.
• Collaborate and engage with BI & analytics and business team
• Deep understanding of the operational dependencies of applications, networks, systems, security and policy (both on premise and in the cloud; VMs, Networking, VPN (Express Route), Active Directory, Storage (Blob, etc.),