Azure Databricks Tech Lead - Remote

Valiant Technologies LLC; Charlotte (NC); Remote; USD 95,000 - 136,000; Full time

Job summary

Valiant Technologies LLC is seeking an experienced Azure Databricks Engineer to lead a technical team in building scalable data pipelines and lakehouse solutions on Azure.

The candidate will mentor developers, design data processing architectures, collaborate with cross-functional teams, and ensure quality through code reviews while staying current with cloud data engineering trends. You will drive delivery from planning to deployment, manage risks, and communicate with stakeholders.

Qualifications

• Proven expertise in Azure Databricks with Spark, data pipelines and data lakes.
• Strong programming skills in Python, Scala or SQL.
• Experience with Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes and data warehousing.
• Demonstrated ability to lead technical teams and manage multiple projects.

Responsibilities

• Lead and mentor a team of technical professionals, fostering a collaborative and high performance culture.
• Design and implement data processing solutions using Azure Databricks, ensuring scalability and efficiency.
• Collaborate with cross functional teams to gather requirements and translate them into technical specifications.
• Oversee the development lifecycle, from planning and design to deployment and maintenance.
• Conduct code reviews and provide constructive feedback to team members to ensure code quality and adherence to best practices.
• Stay up to date with industry trends and emerging technologies related to Azure Databricks and data engineering.
• Facilitate communication between technical and non technical stakeholders to ensure alignment on project goals.
• Identify and mitigate risks associated with project delivery and team performance.

Skills

Azure Databricks

Spark

Python

Scala

SQL

Data Modeling

ETL

Leadership

Team leadership

Tools

Azure Data Lake Storage

Azure SQL Database

Job description

Valiant Technologies LLC is seeking an experienced Azure Databricks Engineer to lead a technical team in building scalable data pipelines and lakehouse solutions on Azure.

The candidate will mentor developers, design data processing architectures, collaborate with cross-functional teams, and ensure quality through code reviews while staying current with cloud data engineering trends. You will drive delivery from planning to deployment, manage risks, and communicate with stakeholders.