Azure Databricks Engineer

PDSSOFT INC.; Charlotte (NC); On-site; USD 120,000 - 190,000; Full time

Job summary

PDSSOFT INC. seeks a Technical Team Lead with deep Azure Databricks expertise to guide a development team in building data-driven solutions. You will oversee design, development, and implementation using Spark, Databricks, and cloud data stores.

The role emphasizes leadership, multi-project management, and delivery of scalable data pipelines and warehousing concepts in a fast-paced environment.

Qualifications

• Proven expertise in Azure Databricks with Spark, data pipelines and data lakes.
• Strong programming skills in Python/Scala/SQL.
• Experience with cloud data storage (Azure Data Lake, Azure SQL).
• Solid data modeling, ETL, and data warehousing fundamentals.
• Proven track record leading technical teams and delivering projects.

Skills

Azure Databricks

Spark

Python

Scala

SQL

Data modeling

ETL

Data warehousing

Leadership

Project management

Education

Bachelor's degree in CS/IT

Tools

Azure Data Lake Storage

Azure SQL Database

Job description

We are seeking a highly skilled and motivated Technical Team Lead with extensive experience in Azure Databricks to join our dynamic team. The ideal candidate will possess a strong technical background, exceptional leadership abilities, and a passion for driving innovative solutions. As a Technical Team Lead, you will be responsible for guiding a team of developers and engineers in the design, development, and implementation of data driven solutions that leverage Azure Databricks.

Mandatory Skills

• Proven expertise in Azure Databricks, including experience with Spark, data pipelines, and data lakes.
• Strong programming skills in languages such as Python, Scala, or SQL.
• Experience with cloud based data storage solutions, particularly Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes, and data warehousing concepts.
• Demonstrated ability to lead technical teams and manage multiple projects simultaneously.

Preferred Skills

• Familiarity with Azure DevOps for CI/CD processes.
• Experience with machine learning frameworks and libraries.
• Knowledge of data governance and compliance standards.
• Strong analytical and problem solving skills.
• Excellent communication and interpersonal skills.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 7 10 years of experience in data engineering, software development, or a related technical role.
• Proven track record of leading technical teams and delivering successful projects.
• Relevant certifications in Azure or data engineering are a plus.