Senior Data Engineer — Remote Databricks on Azure

Hylant; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Hylant is seeking a Data Engineer to build and operationalize the company’s enterprise data architecture on Databricks and Azure. You will translate designs into reliable pipelines and datasets, working with the Lead Data Architect to align with governance standards.

You will design ETL/ELT pipelines, create analytics-ready data layers, implement Unity Catalog security, and own CI/CD for data platforms while collaborating with source system owners and downstream consumers across the organization.

Qualifications

• Bachelor’s degree or equivalent practical experience in CS/engineering/data science or related field
• 5–7+ years of experience as a Data Engineer designing, implementing, and validating production-grade data pipelines
• Hands-on experience with Azure Databricks, Azure Data Factory, Delta Lake, Unity Catalog and core Azure cloud services
• Strong proficiency in SQL and PySpark for data transformation and pipeline development
• Experience implementing data quality checks, reconciliations, and validation processes in complex data environments
• Experience owning or contributing to CI/CD pipelines for data platforms (Azure DevOps Pipelines preferred)
• Ability to troubleshoot and resolve complex data, pipeline, and performance issues
• Insurance industry experience is a plus but not required; exposure to Power BI is a plus
• Must be legally authorized to work in the United States

Responsibilities

• Design, build, and maintain data transformation logic, business rules, and analytics-ready datasets using Databricks notebooks, PySpark, and SQL
• Develop, orchestrate, and monitor ETL/ELT pipelines using Azure Databricks and Azure Data Factory
• Create target schemas, curated tables, and analytics-ready data layers aligned to architectural designs
• Enforce data governance, security, and access controls within Unity Catalog
• Conduct data profiling, validation, reconciliation, and testing to ensure data quality
• Serve curated datasets through Databricks SQL and Power BI datasets for reporting
• Own the CI/CD process for Databricks and Azure Data Factory using Azure DevOps Pipelines
• Monitor and optimize the data platform for performance and cost
• Participate in agile delivery as part of the Data & Digital Solutions team

Skills

Communication

Collaboration

Problem-solving

Attention to detail

Education

Bachelor’s degree in CS/Engineering/Data Science

Tools

Azure Databricks

Azure Data Factory

Delta Lake

Unity Catalog

SQL

PySpark

Power BI

Azure DevOps Pipelines

Job description

Hylant is seeking a Data Engineer to build and operationalize the company’s enterprise data architecture on Databricks and Azure. You will translate designs into reliable pipelines and datasets, working with the Lead Data Architect to align with governance standards.

You will design ETL/ELT pipelines, create analytics-ready data layers, implement Unity Catalog security, and own CI/CD for data platforms while collaborating with source system owners and downstream consumers across the organization.