Azure Data Engineer(Databricks, Datafactory) Immediate joiners

Tata Consultancy Services; Pune District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

A leading IT services firm in Pune is looking for a Mid-Senior level Talent Acquisition Specialist proficient in SQL and Python, with experience in Apache Spark, Delta files, and data warehousing. The role involves setting up end-to-end analytics and utilizing Microsoft Fabric for data processing solutions. Comprehensive analytics solutions and practical experience are crucial for this full-time position.

Qualifications

• Proficiency in SQL and Python, with SQL being more important.
• Experience with Apache Spark for data processing.
• Understanding of Delta files and Lakehouse architecture.
• Basic data warehouse knowledge or experience.
• Hands-on skills with Microsoft Fabric for analytics.
• Experience setting up end-to-end analytics and data pipelines.

Responsibilities

• Set up end-to-end analytics, managing Lakehouse and Medallion Architecture.
• Utilize Apache Spark and Delta Lake tables for data analytics.
• Ingest data and create pipelines with Data Factory.
• Support Power BI reporting with Direct Lake access.

Skills

Proficiency in SQL

Proficiency in Python

Experience with Apache Spark

Understanding of Delta files

Data warehouse knowledge

Hands-on skills with Microsoft Fabric

Experience with Dataflows Gen2

Experience with Data Factory

Tools

Apache Spark

Delta Lake

Lakehouse architecture

Microsoft Fabric

Data Factory

Dataflows Gen2

Power BI Direct Lake

Job description

Talent Acquisition Specialist | TATA CONSULTANCY SERVICES

• Proficiency in SQL and Python. SQL being far more important.
• Experience with Apache Spark for data processing (Python version)
• Understanding of Delta files and Lakehouse architecture
• Data warehouse basic knowledge or experience
• Hands‑on skills (even gotten only by training) are essential for effectively using Microsoft Fabric.
• Practical experience in setting up end‑to‑end analytics, managing Lakehouse and Medallion Architecture, using Apache Spark and Delta Lake tables, handling data ingestion with Dataflows Gen2, creating pipelines with Data Factory, and setting up data warehouses is crucial.
• Understand the capabilities of Microsoft Fabric for complete analytics solutions, including data ingestion, transformation, storage, and visualization.
• Familiarity with features such as Direct Lake access for Power BI reports, is important.
• Utilize Apache Spark for large‑scale data processing and work with Delta Lake tables for advanced data analytics.
• Ingest data using Dataflows Gen2 and create pipelines with Data Factory capabilities for multi‑step data ingestion and transformation tasks.
• Set up and query data warehouses in Microsoft Fabric, integrating them with other analytics components.
• Learn how to secure a Microsoft Fabric data warehouse and administer the platform effectively.

Seniority level: Mid‑Senior level

Employment type: Full‑time

Job function: Information Technology

Industries: IT Services and IT Consulting