Remote Azure Databricks Data Engineer - Scalable Pipelines

Fairweather, LLC; United States; Remote; USD 120,000 - 150,000; Full time

Benefits offered by this job

Remote work

Flexible hours

Diversity & inclusion program

Welcome back program

Job summary

Exavalu Solutions India Pvt Ltd. is seeking a remote Azure Databricks Data Engineer to design and implement end-to-end data pipelines using Azure-native services and Databricks.

The role focuses on building scalable data platforms, defining robust data frameworks, and translating business requirements into reliable data solutions. You will also implement Unity Catalog governance, data ingestion patterns, and data transformation practices.

Qualifications

• Hands-on experience with PySpark for large-scale data processing.
• Experience defining and enforcing best practices for Databricks workloads and Spark-based pipelines.
• Strong analytical and problem-solving skills.
• Excellent communication skills to effectively work with cross-functional stakeholders.
• Prior experience in working in Insurance P&C cloud-based data & analytics projects.

Responsibilities

• Design and implement end-to-end data pipeline using Azure-native services and Databricks.
• Build scalable, high-performance data platforms and robust data frameworks.
• Partner with stakeholders to translate business requirements into reliable data solutions.
• Create PySpark notebooks and data transformation procedures.
• Handle performance tuning in Databricks and Azure SQL DBs.

Skills

Azure Databricks

PySpark

Data ingestion & orchestration

Data governance (Unity Catalog)

Distributed data processing

Tools

Azure SQL DB

Job description

Exavalu Solutions India Pvt Ltd. is seeking a remote Azure Databricks Data Engineer to design and implement end-to-end data pipelines using Azure-native services and Databricks.

The role focuses on building scalable data platforms, defining robust data frameworks, and translating business requirements into reliable data solutions. You will also implement Unity Catalog governance, data ingestion patterns, and data transformation practices.