Azure Databricks Data Engineer for Enterprise Pipelines

HIGH BRIDGE CONSULTING LLC; New York (NY); Hybrid; USD 120,000 - 170,000; Full time

Job summary

High Bridge Consulting LLC is seeking a Data Engineer with strong Azure Databricks and Azure data engineering experience to build and scale enterprise data pipelines across Finance, Compliance, Facilities, RESO, and HCG in the NYC area. This role focuses on modern cloud data engineering, ETL/ELT pipelines, and enterprise system integration within a large-scale Azure data ecosystem.

Hybrid work model: 3 days onsite per week in NYC, collaboration with cross-functional teams and emphasis on data

Qualifications

• 5+ years of Data Engineering or Cloud Data Engineering experience.
• Hands-on Azure Databricks, Spark, and SQL expertise.
• Experience with Azure Data Factory and API integration.
• Strong Python or Scala programming skills and data governance awareness.

Responsibilities

• Design, build, and maintain Azure Databricks + Data Factory pipelines.
• Develop ETL/ELT workflows for structured and unstructured data.
• Build scalable data models and lakehouse architectures.
• Integrate enterprise systems (ServiceNow, Workday, Dataverse, APIs).
• Optimize SQL queries and Spark jobs for performance.
• Ensure data governance, security, and data quality standards.
• Support monitoring, troubleshooting, and pipeline reliability.
• Collaborate across business and technical teams.

Skills

Data Engineering

Azure Databricks

SQL

Python

APIs

Spark

Azure Data Factory

Azure SQL

PowerShell

Tools

Azure Databricks

Data Factory

Purview

Job description

High Bridge Consulting LLC is seeking a Data Engineer with strong Azure Databricks and Azure data engineering experience to build and scale enterprise data pipelines across Finance, Compliance, Facilities, RESO, and HCG in the NYC area. This role focuses on modern cloud data engineering, ETL/ELT pipelines, and enterprise system integration within a large-scale Azure data ecosystem.

Hybrid work model: 3 days onsite per week in NYC, collaboration with cross-functional teams and emphasis on data