Azure Databricks Data Engineer

Riana Technologies; Warren (MI); Presencial; USD 120.000 - 180.000; Jornada completa

Descripción de la vacante

Riana Technologies is seeking a skilled Databricks Engineer with 5+ years of hands-on experience to design, develop, and optimize scalable data platforms using the Databricks Lakehouse Platform.

You will build data pipelines, implement data governance, collaborate with analytics and AI teams, and ensure secure, cost-efficient Spark workloads across Azure cloud services.

Formación

• Bachelor's or Master's in CS/IT/Engineering or related field.
• 5+ years in Data Engineering and Data Warehousing.
• 5+ years in Azure Data Engineering technologies.
• 4+ years in Azure Databricks and Spark ecosystem.
• Strong understanding of lakehouse/cloud-native architecture patterns.
• Experience with large-scale structured and unstructured data.
• Strong analytical and troubleshooting skills.

Responsabilidades

• Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.
• Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.
• Develop data models and implement data quality, validation, and governance frameworks.
• Integrate data from multiple sources into a unified Lakehouse architecture.
• Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
• Implement security controls, access management, and data governance using Unity Catalog.
• Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.
• Monitor, troubleshoot, and resolve data pipeline issues.
• Support CI/CD, DevOps, and infrastructure automation practices.
• Maintain technical documentation and best practices.

Conocimientos

Databricks

Apache Spark

Azure Data Engineering

ETL/ELT

Data modeling

Data governance

CI/CD

DevOps

Spark tuning

Azure

Educación

Bachelor's or Master's in CS/IT/Engineering

Herramientas

Azure Databricks

Unity Catalog

Snowflake

Power BI

Microsoft Fabric

Kafka

Event Hubs

Descripción del empleo

We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.

Key Responsibilities

• Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.
• Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.
• Develop data models and implement data quality, validation, and governance frameworks.
• Integrate data from multiple sources into a unified Lakehouse architecture.
• Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
• Implement security controls, access management, and data governance using Unity Catalog.
• Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.
• Monitor, troubleshoot, and resolve data pipeline issues.
• Support CI/CD, DevOps, and infrastructure automation practices.
• Maintain technical documentation and best practices.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
• 5+ years of experience in Data Engineering and Data Warehousing.
• Minimum 5+ years of hands-on experience with Azure Data Engineering technologies.
• Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem.
• Strong understanding of data lake, lakehouse, and cloud-native architecture patterns.
• Experience in handling large-scale structured and unstructured datasets.
• Strong analytical, problem-solving, and troubleshooting skills.

Preferred Qualifications

• Experience with Snowflake, Power BI, or Microsoft Fabric.
• Experience in real-time streaming solutions using Kafka/Event Hubs.
• Exposure to Data Governance and Master Data Management initiatives.

Soft Skills

• Strong stakeholder management and communication skills.
• Ability to lead technical initiatives and drive architecture discussions.
• Experience working in Agile/Scrum environments.
• Excellent documentation and presentation skills.
• Strong mentoring and team leadership abilities.