Azure Data Engineer-20 Aug (Thursday)-6+yrs-Video Interview

Tata Consultancy Services; Hyderabad, Bengaluru, New Delhi; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Tata Consultancy Services in Hyderabad is seeking an Azure Data Engineer to design, build, and maintain data transformation pipelines using Azure services with Python. You will implement pipelines with PySpark and Databricks to support complex calculations and large datasets.

Exposure to Azure Functions and Step Functions, and collaboration with software engineers is required. You will participate in code reviews, design discussions, and contribute to automation, testing, and deployment

Qualifications

• Design and build data pipelines using Azure services (Azure Functions, Step Functions) in Python.
• Implement pipelines with PySpark and Databricks for complex calculations.
• Handle large datasets efficiently and scale data processing.
• Collaborate with Software Engineers and participate in code reviews and design discussions.

Responsibilities

• Design, build, and maintain data transformation pipelines using AZURE services in Python.
• Develop pipelines with PySpark/Databricks for complex calculations.
• Manage large datasets and ensure scalable data processing.
• Collaborate with software engineers, participate in design reviews and code reviews.
• Contribute to automation, testing, and deployment processes.

Skills

Python

PySpark

Databricks

Azure Data Factory

Tools

PostgreSQL

MongoDB

Job description

Azure Data Engineer(Python, PySpark, Databricks, ADF)

• Design, build, and maintain data transformation pipelines using AZURE services (Azure Functions, Step Functions .. ) in Python.
• Design, build, and maintain data transformation pipelines using PySpark and Databricks. Implement and maintain complex calculation engine.
• Manage effeciently large dataset.
• Manage SQL database - Postgresql or MongoDB.
• Implement and maintain complex data calculation formula.
• Open to collaborate with Software Engineers.
• Participate in code reviews, design discussions, and software planning.
• Contribute to automation, testing, and deployment processes.