Opening For Azure data engineer

Tekskills Inc.; Bengaluru; Vor Ort; INR 1.500.000 - 2.000.000; Vollzeit

Zusammenfassung

Responsibilities include monitoring, cost optimization, security enforcement, and collaboration with data teams to optimize workflows, plus training and documentation to ensure best practices are followed.

Qualifikationen

• Strong analytical and problem-solving skills.
• Strong technical documentation capabilities.
• Agile framework and delivery lifecycle management.
• Team based development capabilities and ability to manage personal deliverables.
• Experience with multi-cloud environments and hybrid cloud strategies.
• Familiarity with DevOps practices and tools such as Azure DevOps, Jenkins, or GitHub Actions and Azure DevOps
• Databricks Certified Associate Developer or equivalent certification.
• Relevant Azure certifications, such as Azure Data Engineer Associate or Azure Solutions Architect
• In-depth knowledge of Databricks architecture, Spark, and Delta Lake.
• Proficiency in managing and optimizing Databricks clusters and jobs.
• Experience with deploying and managing Databricks on Azure, including integration with Azure Data Lake, Azure SQL Database, and other Azure services.
• Strong programming skills in languages such as Python, SQL, and Scala.
• Familiarity with data warehousing concepts and ETL processes.
• Strong understanding of Azure infrastructure and services, such as Azure Virtual Machines, Azure Storage, and Azure Networking.
• Azure Database and data platform
• Database Services, including Azure SQL and Cosmos DB
• Monitoring, Diagnostics, and Analytics with Azure Monitor
• Azure Data Factory and Data Pipeline Orchestration
• Azure Synapse Analytics and Data Warehousing
• Data Modeling and Database Design
• Power BI and Data Visualization
• Mongodb Experience a plus

Aufgaben

• Architect and deploy scalable Databricks solutions on Azure.
• Develop and implement data pipelines, ETL processes, and ML workflows using Databricks and Azure services.
• Monitor Databricks performance and optimize configurations for efficiency and cost-effectiveness.
• Implement best practices for data storage, processing, and analysis.
• Ensure Databricks environments comply with security standards and regulatory requirements.
• Implement and manage data encryption, access controls, and other security measures.
• Perform regular maintenance tasks, including updates, patches, and backups.
• Provide advanced technical support for Databricks-related issues and act as escalation point.
• Collaborate with data engineers, data scientists, and analysts to design and optimize data workflows.
• Work with IT and DevOps teams to integrate Databricks with other Azure services and tools.
• Develop and deliver training sessions for internal teams on Databricks best practices and new features.
• Document architectures, configurations, and procedures for knowledge sharing and future reference.

Kenntnisse

Databricks

Python

SQL

Scala

Azure

ETL

Data warehousing

Documentation

Agile

DevOps

CI/CD

Multi-cloud

Cloud security

Ausbildung

Databricks Certified Associate Developer

Azure Data Engineer Associate

Azure Solutions Architect

Tools

Azure DevOps

Jenkins

GitHub Actions

Jobbeschreibung

Architect and deploy scalable Databricks solutions on Azure.

• Develop and implement data pipelines, ETL processes, and machine learning workflows using Databricks and Azure services.
• Monitor Databricks performance and optimize configurations for efficiency and cost-effectiveness.
• Implement best practices for data storage, processing, and analysis to ensure optimal performance.
• Ensure Databricks environments comply with security standards and regulatory requirements.
• Implement and manage data encryption, access controls, and other security measures.
• Perform regular maintenance tasks, including updates, patches, and backups.
• Provide advanced technical support for Databricks-related issues and act as the escalation point for complex problems.
• Collaborate with data engineers, data scientists, and analysts to design and optimize data workflows.
• Work with IT and DevOps teams to integrate Databricks with other Azure services and tools.
• Develop and deliver training sessions for internal teams on Databricks best practices and new features.
• Document architectures, configurations, and procedures for knowledge sharing and future reference.

To be successful in this role, you should meet the following requirements:

• Strong analytical and problem-solving skills.
• Strong technical documentation capabilities.
• Agile framework and delivery lifecycle management.
• Team based development capabilities and ability to manage personal deliverables.
• Experience with multi-cloud environments and hybrid cloud strategies.
• Familiarity with DevOps practices and tools such as Azure DevOps, Jenkins, or GitHub Actions and Azure DevOps
• Databricks Certified Associate Developer or equivalent certification.
• Relevant Azure certifications, such as Azure Data Engineer Associate or Azure Solutions Architect
• In-depth knowledge of Databricks architecture, Spark, and Delta Lake.
• Proficiency in managing and optimizing Databricks clusters and jobs.
• Experience with deploying and managing Databricks on Azure, including integration with Azure Data Lake, Azure SQL Database, and other Azure services.
• Strong programming skills in languages such as Python, SQL, and Scala.
• Familiarity with data warehousing concepts and ETL processes.
• Strong understanding of Azure infrastructure and services, such as Azure Virtual Machines, Azure Storage, and Azure Networking.
• Azure Database and data platform
• Database Services, including Azure SQL and Cosmos DB
• Monitoring, Diagnostics, and Analytics with Azure Monitor
• Azure Data Factory and Data Pipeline Orchestration
• Azure Synapse Analytics and Data Warehousing
• Data Modeling and Database Design
• Power BI and Data Visualization
• MongoDB Experience a plus