Azure Databricks Engineer

Valiant Technologies LLC; Charlotte (NC); Remote; USD 95,000 - 136,000; Full time

Job summary

Valiant Technologies LLC is seeking an experienced Azure Databricks Engineer to lead a technical team in building scalable data pipelines and lakehouse solutions on Azure.

The candidate will mentor developers, design data processing architectures, collaborate with cross-functional teams, and ensure quality through code reviews while staying current with cloud data engineering trends. You will drive delivery from planning to deployment, manage risks, and communicate with stakeholders.

Qualifications

• Proven expertise in Azure Databricks with Spark, data pipelines and data lakes.
• Strong programming skills in Python, Scala or SQL.
• Experience with Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes and data warehousing.
• Demonstrated ability to lead technical teams and manage multiple projects.

Responsibilities

• Lead and mentor a team of technical professionals, fostering a collaborative and high performance culture.
• Design and implement data processing solutions using Azure Databricks, ensuring scalability and efficiency.
• Collaborate with cross functional teams to gather requirements and translate them into technical specifications.
• Oversee the development lifecycle, from planning and design to deployment and maintenance.
• Conduct code reviews and provide constructive feedback to team members to ensure code quality and adherence to best practices.
• Stay up to date with industry trends and emerging technologies related to Azure Databricks and data engineering.
• Facilitate communication between technical and non technical stakeholders to ensure alignment on project goals.
• Identify and mitigate risks associated with project delivery and team performance.

Skills

Azure Databricks

Spark

Python

Scala

SQL

Data Modeling

ETL

Leadership

Team leadership

Tools

Azure Data Lake Storage

Azure SQL Database

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Job Title: Azure Databricks Engineer
Location: Charlotte, NC(Remote)
Duration: 24 Months + Extension
Bill Rate: $84/hour (C2C)
Job Type: W-2, 1099, C2C
Client: To Be Discussed Later
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Description

We are seeking a highly skilled and motivated Technical Team Lead with extensive experience in Azure Databricks to join our dynamic team. The ideal candidate will possess a strong technical background, exceptional leadership abilities, and a passion for driving innovative solutions. As a Technical Team Lead, you will be responsible for guiding a team of developers and engineers in the design, development, and implementation of data driven solutions that leverage Azure Databricks.

Responsibilities

Lead and mentor a team of technical professionals, fostering a collaborative and high performance culture.
Design and implement data processing solutions using Azure Databricks, ensuring scalability and efficiency.
Collaborate with cross functional teams to gather requirements and translate them into technical specifications.
Oversee the development lifecycle, from planning and design to deployment and maintenance.
Conduct code reviews and provide constructive feedback to team members to ensure code quality and adherence to best practices.
Stay up to date with industry trends and emerging technologies related to Azure Databricks and data engineering.
Facilitate communication between technical and non technical stakeholders to ensure alignment on project goals.
Identify and mitigate risks associated with project delivery and team performance.

Mandatory Skills

Proven expertise in Azure Databricks, including experience with Spark, data pipelines, and data lakes.
Strong programming skills in languages such as Python, Scala, or SQL.
Experience with cloud based data storage solutions, particularly Azure Data Lake Storage and Azure SQL Database.
Solid understanding of data modeling, ETL processes, and data warehousing concepts.
Demonstrated ability to lead technical teams and manage multiple projects simultaneously.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.