Azure Data Engineer (Walkin benguluru 12th sep)

Tata Consultancy Services; Bengaluru; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking a data engineer to design, build, and maintain data transformation pipelines using Azure services (Azure Functions, Step Functions), Python, PySpark, and Databricks. You will manage large datasets, oversee SQL databases such as PostgreSQL or MongoDB, and implement complex data calculation formulas to ensure accurate analytics.

Collaborating with cross-functional teams, you will optimize data flows and ensure scalable, high-performance data

Qualifications

• Experience designing and maintaining data pipelines end-to-end.
• Proficient in Python and Azure cloud services.
• Strong knowledge of PySpark and Databricks.
• Experience managing large datasets and relational/non-relational databases.

Responsibilities

• Design, build, and maintain data transformation pipelines using Azure services (Azure Functions, Step Functions) in Python.
• Design, build, and maintain data transformation pipelines using PySpark and Databricks.
• Manage efficiently large dataset.
• Manage SQL databases - PostgreSQL or MongoDB.
• Implement and maintain complex data calculation formulas

Skills

Python

Azure Functions

PySpark

Databricks

SQL

PostgreSQL

MongoDB

Data pipelines

Tools

PostgreSQL

MongoDB

Job description

• - Design, build, and maintain data transformation pipelines using AZURE services (Azure Functions, Step Functions .. ) in Python.
• - Design, build, and maintain data transformation pipelines using PySpark and Databricks. Implement and maintain complex calculation engine.
• - Manage efficiently large dataset.
• - Manage SQL database - Postgresql or MongoDB.
• Implement and maintain complex data calculation formula