Technical Lead -Data Engineer - Azure 1

embIND Embitel Technologies (India) Private Limited; Bengaluru; On-site; INR 800,000 - 1,600,000; Full time

Job summary

embIND Embitel Technologies (India) Private Limited seeks a professional proficient in designing and implementing scalable and secure data pipelines. The ideal candidate will utilize Azure Data Factory and Azure Databricks to build robust data solutions and collaborate with DevOps teams for CI/CD pipeline management.

The successful applicant should demonstrate expertise in Python and cloud services, ensuring high standards of data quality and governance throughout the development lifecycle.

Responsibilities

• Design and implement scalable and secure data pipelines using Azure Data Factory and Azure Databricks (Spark).
• Utilize Azure cloud services such as Blob Storage, Azure Functions, and API Gateway to build and maintain robust data solutions.
• Develop and maintain code using Python or similar programming languages to support ETL workflows and data transformation processes.
• Collaborate with DevOps teams to set up and manage CI/CD pipelines using Azure DevOps or equivalent toolsets.
• Ensure high code and data quality standards through performance tuning, testing, and documentation.
• Work across the full development lifecycle, from design through deployment and maintenance of distributed systems.
• Leverage knowledge of SQL and NoSQL databases to design effective data models and ensure data reliability.
• Apply Object-Oriented Programming and ETL best practices to develop modular and reusable components.
• Troubleshoot and resolve cloud networking and connectivity issues in collaboration with infrastructure teams.
• Translate complex technical problems into simple and scalable solutions, aligning with architectural best practices.
• Advocate for and implement data governance, monitoring, and quality frameworks to ensure the integrity of business-critical data.

Job description

Responsibilities

• Design and implement scalable and secure data pipelines using Azure Data Factory and Azure Databricks (Spark).
• Utilize Azure cloud services such as Blob Storage, Azure Functions, and API Gateway to build and maintain robust data solutions.
• Develop and maintain code using Python or similar programming languages to support ETL workflows and data transformation processes.
• Collaborate with DevOps teams to set up and manage CI/CD pipelines using Azure DevOps or equivalent toolsets.
• Ensure high code and data quality standards through performance tuning, testing, and documentation.
• Work across the full development lifecycle, from design through deployment and maintenance of distributed systems.
• Leverage knowledge of SQL and NoSQL databases to design effective data models and ensure data reliability.
• Apply Object-Oriented Programming and ETL best practices to develop modular and reusable components.
• Troubleshoot and resolve cloud networking and connectivity issues in collaboration with infrastructure teams.
• Translate complex technical problems into simple and scalable solutions, aligning with architectural best practices.
• Advocate for and implement data governance, monitoring, and quality frameworks to ensure the integrity of business-critical data.