Senior Data Engineer - MS Fabric

SmartRecruiters, Inc.; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Bosch Global Software Technologies Private Limited in Bengaluru seeks a Senior Data Engineer with Databricks expertise to design, build, and optimize scalable data pipelines for analytics and ML. The role focuses on integrating Power BI, Delta Lake, and cloud services to enable data-driven decisions across the organization.

You will collaborate across data engineering and analytics teams, implement governance, and optimize data workflows for performance and cost.

Qualifications

• 5-8 years of professional work experience in a relevant field.
• Proficient in Microsoft Fabric platform, Azure Databricks, ADF, Delta Lake, SQL Data Warehouse, Unity Catalog.
• Experience with Microsoft Dynamics 365 and semi-structured data; familiarity with streaming and data lake concepts.
• Proficient in creating Azure Data Factory pipelines for ETL/ELT processing; copy activity, custom Azure development, etc.
• Strong SQL and Python for data manipulation, transformation, and analysis.
• Understand business requirements to set functional specifications for reporting applications.

Responsibilities

• Design, develop, and maintain robust, scalable data pipelines using Databricks and Spark.
• Ingest, transform, and integrate data from diverse sources into Databricks for analytics and reporting.
• Develop and optimize data models and datasets in Databricks for Power BI visualizations.
• Optimize data workflows, API calls, and queries on Databricks and Power BI for performance and cost-efficiency.
• Build and maintain data models ensuring quality, consistency, and accessibility for analytics.
• Implement data solutions on cloud platforms (AWS, Azure, GCP) integrated with Databricks and APIs.
• Implement data governance, security, and compliance best practices within Databricks, Power BI, and APIs.

Skills

Python

SQL

Apache Spark

Databricks

Power BI

Azure Databricks

Data Modeling

ETL/ELT

Education

BE/B.Tech or Equivalent Degree

Tools

Delta Lake

Unity Catalog

Azure Data Factory

Azure Synapse

Job description

• Legal Entity: Bosch Global Software Technologies Private Limited

Job Description

We are seeking a skilled Senior Data Engineer with expertise in Databricks to join our dynamic data team. The ideal candidate will design, build, and maintain scalable data pipelines and architectures to support our organization's data-driven initiatives and leverage Databricks to process large-scale datasets, optimize data workflows, enable advanced analytics and machine learning, and integrate Power BI for data visualization and reporting.

Key Responsibilities

5-8 years of professional work experience in a relevant field

Proficient in Microsoft Fabric platform, Azure Databricks, ADF, Delta Lake, SQL Data Warehouse, Unity Catalog.

Good Experience on Microsoft Dynamics 365Experience/ prior knowledge on semi structure data and Structured Streaming, Azure synapse, data lake, data warehouse.

Proficient in creating Azure Data Factory pipelines for ETL/ELT processing; copy activity, custom Azure development etc.

Good knowledge of SQL and Python for data manipulation, transformation, and analysis

Understand business requirements to set functional specifications for reporting applications

Data Pipeline Development

Design, develop, and maintain robust, scalable data pipelines using Databricks, Apache Spark, and other cloud-based technologies.

Data Integration

Ingest, transform, and integrate data from diverse sources, including APIs, databases, streaming platforms, and third-party systems, into Databricks for analytics and reporting.

Power BI Integration

Develop and optimize data models and datasets in Databricks for use in Power BI, ensuring efficient data connections and high-quality visualizations.

Performance Optimization

Optimize data workflows, API calls, and queries on Databricks and Power BI for performance, cost-efficiency, and scalability.

Data Modeling

Build and maintain data models to support business requirements, ensuring data quality, consistency, and accessibility for analytics and reporting.

Cloud Integration

Implement data solutions on cloud platforms (e.g., AWS, Azure, GCP) integrated with Databricks, Power BI, and API ecosystems.

Security & Compliance

Implement data governance, security, and compliance best practices within Databricks, Power BI, and API environments.

Technical Skills

Proficiency in Databricks, including Delta Lake, Spark SQL.

Strong programming skills in Python, Scala, or Java.

Experience with Apache Spark for big data processing.

Knowledge of SQL for querying and transforming data.

Proficiency in Power BI for creating data models, DAX queries, and interactive dashboards.

Preferred Qualifications

Experience with real-time data processing and streaming (e.g., Kafka, Spark Streaming).

Additional Information

BE/B.Tech or Equivalent Degree

Experience

2-5 Years

Mandatory/requires Skills

Strong hands-on expertise in Python (or Java), NLP, RegEx, SpaCy, NLTK and DevOps