Azure data engineer (ADB / ADF)

Tata Consultancy Services; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Tata Consultancy Services in Bengaluru is seeking a data engineer to design and maintain large-scale data pipelines using Azure Data Factory, ingesting data from multiple sources into Azure storage like Blob Storage and Cosmos DB.

You will optimize SQL queries, collaborate with cross-functional teams, ensure high availability through regional replication, and participate in code reviews to uphold coding standards.

Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Data Factory to move data into Azure storage.
• Collaborate with cross-functional teams to gather requirements and design scalable data architectures.
• Develop complex SQL queries to optimize database performance in Azure Databricks.
• Ensure high availability by implementing replication and redundancy across regions.
• Participate in code reviews to ensure coding standards and best practices.

Skills

Azure Data Factory

Azure Databricks

Cosmos DB

Tools

Blob Storage

Job description

Roles and Responsibilities

• Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure storage solutions such as Blob Storage, Cosmos DB, SQL DBA, etc.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design scalable architectures that meet business needs.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Azure Databricks.
• Ensure high availability of data services by implementing redundancy measures like replication across multiple regions.
• Participate in code reviews to ensure adherence to coding standards and best practices.