Azure Data Factory Developer

Tata Consultancy Services; India; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Tata Consultancy Services is seeking a data engineer to design and optimize data pipelines using Azure Data Factory, store data in Azure Data Lake Gen 2, and build analytics with Azure Databricks and Spark. You will create Power BI reports and collaborate with IT and business teams to deliver scalable data solutions.

The role emphasizes data quality, security, and compliance, requiring hands-on experience with distributed processing and strong visualization skills in Power BI.

Qualifications

• Design and implement end-to-end data pipelines
• Manage and optimize data storage and lake environments
• Perform large-scale data processing and analytics with distributed frameworks
• Create interactive dashboards and reports for business insights

Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory.
• Manage and optimize data storage using Azure Data Lake Gen 2.
• Build and process large-scale data solutions with Azure Databricks and Apache Spark.
• Create interactive reports and dashboards in Power BI for business insights.
• Collaborate with cross-functional IT and business teams to translate requirements into data solutions.
• Ensure data quality, security, and compliance in line with organizational standards.

Skills

Azure Data Factory

Azure Data Lake Gen 2

Azure Databricks / Apache Spark

Power BI

Job description

Role & responsibilities

Azure Databricks, PySpark, Azure Data Factory

Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory.
• Manage and optimize data storage using Azure Data Lake Gen 2.
• Build and process large-scale data solutions with Azure Databricks and Apache Spark.
• Create interactive reports and dashboards in Power BI for business insights.
• Collaborate with cross-functional IT and business teams to translate requirements into data solutions.
• Ensure data quality, security, and compliance in line with organizational standards.

Core Skills Required

• Azure Data Factory: Experience in building and managing data pipelines.
• Azure Data Lake Gen 2: Proficient in data storage and management within Azure's data lake environment.
• Azure Databricks / Apache Spark: Hands-on skill with distributed data processing, transformations, and analytics.
• Power BI: Expertise in data visualization and reporting.