Azure Data Engineer

Aon plc

United States

Remote

USD 120,000 - 160,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Aon plc in Krakow is seeking an Azure Data Engineer to design and deliver scalable data pipelines and Lakehouse solutions on Azure, leveraging PySpark, SQL, Delta Lake, Unity Catalog and Power BI enablement to empower enterprise analytics.

The role combines hands-on engineering with technical leadership across governance, performance, and CI/CD for Databricks, with 50/50 hybrid or fully remote options, and a focus on secure, reliable data platforms.

Qualifications

  • Experience designing scalable ETL/ELT pipelines on Azure Databricks with PySpark, SQL and Delta Lake.
  • Proficiency in building Lakehouse architectures and governance using Unity Catalog.
  • Ability to develop performance-optimized datasets for BI tools like Power BI.
  • Experience with CI/CD practices using Git, Azure DevOps or GitHub.
  • Knowledge of finance/insurance domains is a plus.

Responsibilities

  • Facilitate workshops to discuss business reporting requirements.
  • Establish best practices across data architecture, governance, security, performance, and reliability.
  • Develop scalable data integration pipelines on Azure Databricks using PySpark, SQL, Notebooks, Jobs, and Workflows.
  • Build and maintain Lakehouse architectures with Delta Lake across bronze/silver/gold layers.
  • Implement Unity Catalog for centralized data governance and access control.
  • Define data contracts and collaborate with BI developers on semantic models and refresh strategies.
  • Optimize queries and troubleshoot Power BI–Databricks integrations (Direct Lake/Direct Query).
  • Configure Databricks clusters, policies, and job schedules for cost and reliability.
  • Contribute to CI/CD pipelines for Databricks code and explore ML pipelines.

Skills

Azure Databricks
PySpark
SQL
Delta Lake
Unity Catalog
Power BI
ETL/ELT pipelines
CI/CD
Notebooks/Jobs/Workflows
Data governance

Tools

Git
GitHub
Azure DevOps
Databricks

Job description

Posting Description:

Azure Data Engineer As an Azure Databricks Data Engineer, you will work with the team to design and deliver scalable data pipelines, Lakehouse solutions, and governed data platforms on Azure. You will apply your expertise in PySpark, SQL, Delta Lake, Unity Catalog, and Power BI enablement to create secure, reliable, and high-performing data solutions.

Our office is located in Krakow and this role is available both in 50/50 hybrid and fully remote working model.

Aon is in the business of better decisions At Aon, we shape decisions for the better to protect and enrich the lives of people around the world. As an organization, we are united through trust as one inclusive team and we are passionate about helping our colleagues and clients succeed.

What the day will look like
Your responsibilities will include:
  • Facilitate workshops to discuss, review and document business reporting requirements.
  • Establish best practices across data architecture, governance, security, performance, and platform reliability
  • Developing and implementing scalable data integration pipelines on Azure Databricks using PySpark, SQL, Notebooks, Jobs, and Workflows.
  • Building and maintaining Lakehouse architectures using Delta Lake and bronze, silver, and gold data layers to support analytics and reporting.
  • Crafting robust, reusable systems and collections for data transformation.
  • Implementing and managing Unity Catalog to provide centralized governance of data, tables, and permissions.
  • Defining and enforcing role-based access control, data access controls, and auditing in line with security and compliance requirements.
  • Standardizing catalog, schema, and table naming conventions across environments.
  • Delivering well-modelled, performance-optimized datasets for Power BI and other business analytics tools.
  • Collaborating with BI developers and analysts to define semantic models, data contracts, and refresh strategies.
  • Supporting design and development of new reports
  • Troubleshooting and optimizing query performance between Power BI and Databricks, including Direct Lake and Direct Query where applicable.
  • Configuring and improving Databricks clusters, cluster policies, and job schedules for cost, performance, and reliability.
  • Implementing monitoring, logging, and alerting for data pipelines and jobs.
  • Contributing to and improving CI/CD pipelines for Databricks code using Git and Azure DevOps or GitHub.
  • Explore AI/ML technologies and opportunities to deploy machine learning pipelines that generate measurable business value.
How this opportunity is different

This role provides the opportunity to develop scalable and governed data solutions that directly support enterprise analytics and business decision-making. You will combine hands‑on engineering with technical leadership, helping to establish best practices across data architecture, governance, security, performance, and platform reliability.

Skills and experience that will lead to success
  • Strong experience crafting and implementing scalable ETL and ELT pipelines on Azure Databricks using PySpark, SQL, Notebooks, Jobs, and Workflows.
  • Expertise in Lakehouse architecture, Delta Lake, and bronze, silver, and gold data‑layer patterns, with the ability to develop reusable data transformation frameworks and libraries.
  • Experience implementing Unity Catalog, role‑based access control, data access controls, auditing, and consistent catalog, schema, and table standards.
  • Ability to develop performance‑optimized datasets for Power BI and collaborate on semantic models, data contracts, refresh strategies, Direct Lake, and Direct Query solutions where applicable.
  • Experience optimizing Databricks clusters and job scheduling, implementing monitoring and alerting, and contributing to CI/CD practices using Git and Azure DevOps or GitHub.
  • Knowledge of Finance and Insurance business processes and terminology.
  • Work independently, Self‑motivated and organized, Manages own time to ensure task are completed with the ability to work to tight deadlines and under high pressure situations
  • Can communicate openly, honestly and independently with stakeholders at all levels of the organisation to understand their business goals and processes.

#technologyandsecurity #li-mk1 #LI-remote

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer, Azure Databricks
Senior Data Engineer, Azure Databricks

Nextech Solutions • United States

Remote
USD 130,000 - 170,000
Data Engineer – Azure Databricks
Data Engineer – Azure Databricks

HIGH BRIDGE CONSULTING LLC • New York (NY)

Hybrid
USD 120,000 - 170,000
Azure Data Engineer - Remote Databricks & Lakehouse Lead
Azure Data Engineer - Remote Databricks & Lakehouse Lead

Aon plc • United States

Remote
USD 120,000 - 160,000
Azure Data Engineer
Azure Data Engineer

Aroha Technologies, Inc • Greenfield (IN)

On-site
USD 120,000 - 180,000
Azure data engineer
Azure data engineer

Valce Talent Solutions • United States

Remote
USD 49,000 - 81,000
Azure Data Engineer
Azure Data Engineer

Signature IT World • New Jersey

Hybrid
USD 110,000 - 150,000
Senior Data Architect
Senior Data Architect

Sowelo Consulting sp. z o.o. sp. k. • United States

Remote
USD 45,000 - 87,000
Azure Data Engineer
Azure Data Engineer

Signature IT World • Pennsylvania

Hybrid
USD 120,000 - 150,000
Azure Databricks Data Engineer - Real-Time Analytics
Azure Databricks Data Engineer - Real-Time Analytics

Luxoft • United States

Remote
USD 120,000 - 180,000
Senior Databricks & Azure Data Platform Engineer
Senior Databricks & Azure Data Platform Engineer

Kestra Holdings • Tempe (AZ)

On-site
USD 120,000 - 150,000
Competitive pay and benefits
401(k) and health insurance
Opportunities for training and development