Lead Data Engineer - Azure Databricks & ETL

Ethereum Technologies; City of Albany (NY); On-site; USD 84,000 - 120,000; Full time

Job summary

Ethereum Technologies is seeking a Java Full Stack Developer to lead and design scalable data processing solutions, leveraging Azure Databricks and related cloud technologies. The role involves gathering requirements, overseeing the full development lifecycle, and coordinating with stakeholders across the organization.

The ideal candidate brings 8+ years of end-to-end delivery and strong skills in Python, Scala, or SQL, with hands-on Azure Databricks and data warehousing experience.

Qualifications

• Over 8 years of experience in end-to-end project delivery and leading technical teams.
• Strong programming skills in Python, Scala, or SQL.
• Experience with SQL, Informatica, and cloud technologies.
• Hands-on experience with Azure Databricks.
• Experience with Azure Data Lake Storage and Azure SQL Database.
• Solid understanding of data modeling, ETL processes, and data warehousing.
• Ability to lead teams and manage multiple projects simultaneously.

Responsibilities

• Lead and mentor technical professionals, fostering a collaborative and high-performance team.
• Design and implement scalable data processing solutions using Azure Databricks.
• Gather business requirements and translate them into technical specifications.
• Oversee the complete development lifecycle, including planning, design, deployment, and maintenance.
• Conduct code reviews and ensure adherence to coding standards and best practices.
• Coordinate with technical and non-technical stakeholders.
• Identify and mitigate project delivery and team performance risks.
• Stay current with Azure Databricks and data engineering technologies.

Skills

Leadership

Python/Scala/SQL

Cloud technologies

Data modeling/ETL/warehousing

Education

Bachelor's degree in CS/IT or related

Tools

Azure Databricks

Azure Data Lake Storage

Azure SQL Database

CI/CD tooling

Job description

Ethereum Technologies is seeking a Java Full Stack Developer to lead and design scalable data processing solutions, leveraging Azure Databricks and related cloud technologies. The role involves gathering requirements, overseeing the full development lifecycle, and coordinating with stakeholders across the organization.

The ideal candidate brings 8+ years of end-to-end delivery and strong skills in Python, Scala, or SQL, with hands-on Azure Databricks and data warehousing experience.