Azure Data Engineer

Tata Consultancy Services; Hyderabad; On-site; INR 1,200,000 - 2,300,000; Full time

Job summary

Tata Consultancy Services is seeking a data engineer with hands-on PySpark experience to design and implement data pipelines for a Fabric-based environment in Hyderabad.

The role requires strong SQL skills, experience with Medallion architecture, and translating SQL procedures to Spark jobs.

Qualifications

• Hands-on experience with PySpark in a data engineering environment.
• Strong SQL skills focusing on aggregation and analytics.
• Experience translating SQL Stored Procedures to PySpark pipelines.

Responsibilities

• Collaborate with technology, architecture, product management, and design teams to improve customer experience.
• Plan, design, and build data solutions based on Medallion architecture principles.
• Load and process data from on-prem DWH or other sources into Fabric environment.
• Demonstrate deep understanding of data warehousing concepts and data aggregation techniques.
• Communicate effectively and demonstrate leadership in software development initiatives.

Skills

Pyspark

SQL Aggregation

Medallion Architecture

Tools

Azure Data Factory

Synapse Analytics

SQL Stored Procedures

Job description

Must-Have

• Hands-on experience in Pyspark, Notebook
• Experience in Azure Data Factory, Synapse Analytics
• Experience in converting SQL Stored Procedure into Pyspark
• Experience in Medallion Architecture
• Experience in SQL Aggregation functions

Good-to-Have

• Certification in DP-600: Implementing Analytics Solutions Using Microsoft Fabric and DP-203T00: Data Engineering on Microsoft Azure
• Power BI and Power platform
• Experience in Agile methodology using JIRA
• Knowledge over Investment functional domain, LOMA certification

Expectations from the Role

1. Collaborate with technology, architecture, product management and design teams to drive outcomes with focus on enriched customer experience
2. Plan, design, architect, and build feasible solutions based on Medallion architecture design principles
3. Data Load and Process from on-prem DWH or other data source to Fabric environment
4. Good understanding of DWH and data aggregation techniques
5. Candidate should possess strong communication skills, exhibit strong emotional intelligent skills, relationship leadership ability and a track record software development.