Azure Databricks Architect

Sira Consulting, an Inc 5000 company; Charlotte (NC); On-site; USD 120,000 - 160,000; Full time

Job summary

Sira Consulting, an Inc 5000 company, seeks a Technical Team Lead with extensive Azure Databricks experience to guide a team of developers and engineers in building data-driven solutions.

The role focuses on leading design, development and deployment of scalable data processing pipelines, integrating with Azure Data Lake Storage and Azure SQL DB, and mentoring staff to uphold best practices.

Qualifications

• Proven expertise in Azure Databricks, Spark, data pipelines and data lakes.
• Strong programming skills in Python, Scala or SQL.
• Experience with cloud data storage solutions, especially Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes and data warehousing concepts.
• Demonstrated ability to lead technical teams and manage multiple projects.

Responsibilities

• Lead and mentor a team of technical professionals, fostering a collaborative and high-performance culture.
• Design and implement data processing solutions using Azure Databricks, ensuring scalability and efficiency.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.
• Oversee the development lifecycle from planning to deployment and maintenance.
• Conduct code reviews and provide feedback to ensure code quality and best practices.
• Stay up to date with industry trends in Azure Databricks and data engineering.
• Facilitate communication between technical and non-technical stakeholders on project goals.
• Identify and mitigate risks relating to delivery and team performance.

Skills

Azure Databricks

Spark

Data pipelines

Python

SQL

Leadership

Education

Bachelor's degree in Computer Science / IT / related field

Tools

Azure DevOps

Job description

Location: Charlotte, NC

Years of Experience: 12- 15 Years

Job Summary:

We are seeking a highly skilled and motivated Technical Team Lead with extensive experience in Azure Databricks to join our dynamic team. The ideal candidate will possess a strong technical background, exceptional leadership abilities, and a passion for driving innovative solutions. As a Technical Team Lead, you will be responsible for guiding a team of developers and engineers in the design, development, and implementation of data driven solutions that leverage Azure Databricks.

Responsibilities:

• Lead and mentor a team of technical professionals, fostering a collaborative and high performance culture.
• Design and implement data processing solutions using Azure Databricks, ensuring scalability and efficiency.
• Collaborate with cross functional teams to gather requirements and translate them into technical specifications.
• Oversee the development lifecycle, from planning and design to deployment and maintenance.
• Conduct code reviews and provide constructive feedback to team members to ensure code quality and adherence to best practices.
• Stay up to date with industry trends and emerging technologies related to Azure Databricks and data engineering.
• Facilitate communication between technical and non technical stakeholders to ensure alignment on project goals.
• Identify and mitigate risks associated with project delivery and team performance.

Mandatory Skills:

• Proven expertise in Azure Databricks, including experience with Spark, data pipelines, and data lakes.
• Strong programming skills in languages such as Python, Scala, or SQL.
• Experience with cloud based data storage solutions, particularly Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Demonstrated ability to lead technical teams and manage multiple projects simultaneously.

Preferred Skills:

• Familiarity with Azure DevOps for CI/CD processes.
• Experience with machine learning frameworks and libraries.
• Knowledge of data governance and compliance standards.
• Strong analytical and problem solving skills.
• Excellent communication and interpersonal skills.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 7 10 years of experience in data engineering, software development, or a related technical role.
• Proven track record of leading technical teams and delivering successful projects.
• Relevant certifications in Azure or data engineering are a plus.