Senior Data Engineer - Azure Databricks & Lakehouse

Brillio; United States; On-site; USD 150,000 - 190,000; Full time

Job summary

Brillio is seeking a Lead Data Engineer in the United States to join the Data Pipeline team, building modern data pipelines with Databricks, Azure Synapse Analytics, and Azure Data Explorer, while creating scalable ETLs and real-time and batch processing.

You will collaborate with business, design, test and development teams to shape data capture, health monitoring, and data-driven decision making at scale, delivering high quality data architectures.

Qualifications

• 5+ years' experience with SQL required.
• 5+ years' experience designing and implementing scalable ETL processes including data movement and quality tools.
• 3+ years' experience with modern Big Data Analytics using Data Lake, Spark and formats like Parquet.
• 2+ years' experience building cloud hosted data systems. Azure highly preferred.

Responsibilities

• Build scalable ETLs and processes in Databricks and Data Factory to support real-time and batch analytics.
• Contribute to the design and maintenance of state-of-the-art data architectures at scale.
• Collaborate with business, design, test and development teams to shape how data is captured to improve data-driven development.

Skills

AKS

Event Hub

Azure DevOps

Cosmos DB

Azure Functions

Databricks

Azure Synapse Analytics

Azure Data Explorer

ETL design

Data modeling

Big Data

Tools

Databricks

Job description

Brillio is seeking a Lead Data Engineer in the United States to join the Data Pipeline team, building modern data pipelines with Databricks, Azure Synapse Analytics, and Azure Data Explorer, while creating scalable ETLs and real-time and batch processing.

You will collaborate with business, design, test and development teams to shape data capture, health monitoring, and data-driven decision making at scale, delivering high quality data architectures.