Senior Azure Databricks Lakehouse Data Engineer

EXL; Warren (MI); In loco; USD 120.000 - 190.000; Tempo pieno

Descrizione del lavoro

EXL is seeking a skilled Databricks Engineer with 5+ years of hands-on Databricks experience to design, develop, and optimize scalable data platforms for enterprise analytics and AI initiatives. The role focuses on data engineering, ETL/ELT development, cloud technologies (Azure), and a Lakehouse architecture to unify data across sources.

The candidate will collaborate with analytics and AI teams, implement governance, and optimize Spark/Databricks workloads for performance and cost efficiency.

Competenze

• Bachelor's or Master's degree in CS/IT/Engineering or related field.
• 9-15 years of experience in Data Engineering and Data Warehousing.
• Minimum 5+ years hands-on with Azure Data Engineering technologies.
• Minimum 4+ years hands-on with Azure Databricks and Spark ecosystem.
• Strong understanding of data lake, lakehouse, and cloud-native architecture patterns.
• Experience handling large-scale structured and unstructured datasets.
• Strong analytical, problem-solving, and troubleshooting skills.

Mansioni

• Design, develop, and maintain data pipelines using Databricks, Spark, and cloud-native services.
• Build and optimize ETL/ELT workflows for large-scale data.
• Develop data models and implement data quality, validation, and governance frameworks.
• Integrate data from multiple sources into a unified Lakehouse architecture.
• Optimize Spark jobs and Databricks workloads for performance and cost efficiency.
• Implement security controls and data governance using Unity Catalog.
• Collaborate with business, analytics, and AI/ML teams to deliver data products.
• Monitor, troubleshoot, and resolve data pipeline issues.
• Support CI/CD, DevOps, and infrastructure automation practices.
• Maintain technical documentation and best practices.

Conoscenze

Databricks Lakehouse Platform

SQL

Python

ETL/ELT Development

Data Modeling

Data Warehousing

Data Quality & Validation

Streaming & Real-Time Processing

Azure Data Engineering

Spark

Formazione

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field

Strumenti

Terraform

GitHub Actions

Azure DevOps

Azure Databricks

Power BI

Snowflake

Descrizione del lavoro

EXL is seeking a skilled Databricks Engineer with 5+ years of hands-on Databricks experience to design, develop, and optimize scalable data platforms for enterprise analytics and AI initiatives. The role focuses on data engineering, ETL/ELT development, cloud technologies (Azure), and a Lakehouse architecture to unify data across sources.

The candidate will collaborate with analytics and AI teams, implement governance, and optimize Spark/Databricks workloads for performance and cost efficiency.