Databricks - Technical Specialist-Data Engg

Birlasoft; Pune District; On-site; INR 1,200,000 - 1,600,000; Full time

Job summary

Birlasoft in Pune, India, is seeking a Databricks - Technical Specialist-Data Engg to design, build and deploy scalable data pipelines. The role emphasizes Azure services (Data Factory, ADLS), Databricks and PySpark to process structured and unstructured data.

The candidate will collaborate with BI/analytics teams and business units, handling XML/CSV/Parquet formats and integrating new data sources while ensuring system security and performance.

Qualifications

• Experience in design, development & deployment using Azure services (Data Factory, ADLS, Databricks).
• Develop and maintain scalable data pipelines and build data source integrations.
• Create technical specification design and application interface design.
• Handle files in XML, CSV, Excel, ORC, Parquet formats.
• Develop batch and streaming processing for structured and unstructured data.
• Experience with ETL tools (SSIS, Data Factory, Informatica preferred).
• Deep knowledge of Azure Data Factory, Azure Databricks, PySpark, ADLS and Azure SQL DB.
• Collaborate with BI/analytics and business teams.
• Understand operational dependencies of apps, networks, security and cloud/on-premise balance.

Responsibilities

• Perform design, development and deployment using Azure services and Databricks.
• Build scalable data pipelines and integrate data sources to handle growing data volumes.
• Draft technical specifications and interface designs for data apps.
• Process various file formats such as XML, CSV, Excel, ORC, Parquet.
• Develop batch and streaming ETL solutions across on-premises and cloud.
• Collaborate with BI and analytics teams to satisfy data needs.

Skills

Azure

Databricks

PySpark

SQL

Azure Data Factory

Tools

Azure Data Factory

Azure Data Lake Storage

Databricks

PySpark

SSIS

Informatica

Azure SQL DB

ADLS

Job description

Select how often (in days) to receive an alert:

Location: INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI

Title: Databricks - Technical Specialist-Data Engg

Description:

Area(s) of responsibility

• Experience in Perform Design, Development & Deployment using Azure Services ( Data Factory, Azure Data Lake Storage, Databricks, PySpark, SQL)
• Develop and maintain scalable data pipelines and build out new Data Source integrations to support continuing increases in data volume and complexity.
• Experience in create the Technical Specification Design, Application Interface Design.
• Files Processing – XML, CSV, Excel, ORC, Parquet file Formats
• Develop batch processing, streaming and integration solutions and process Structured and Non-Structured Data
• Good to have experience with ETL development both on-premises and in the cloud using SSIS, Data Factory, and related Microsoft and other ETL technologies (Informatica preferred)
• Demonstrated in depth skills with Azure Data Factory, Azure Databricks, PySpark, ADLS (must have) with the ability to configure and administrate all aspects of Azure SQL DB.
• Collaborate and engage with BI & analytics and business team
• Deep understanding of the operational dependencies of applications, networks, systems, security and policy (both on premise and in the cloud; VMs, Networking, VPN (Express Route), Active Directory, Storage (Blob, etc.),