Databricks Data Engineer

GlobalSource IT; Teletrabalho; BRL 120.000 - 160.000; Tempo integral

Resumo da oferta

A leading IT services company in Brazil seeks a fully remote Databricks Data Engineer. The ideal candidate will have hands-on experience with Databricks, specializing in Spark, PySpark, and Delta Lake. Responsibilities include building and optimizing ETL/ELT pipelines, ingesting and transforming data from various sources, and delivering insights through Power BI dashboards. This contract role offers a dynamic environment for experienced professionals looking to enhance their data engineering skills.

Qualificações

• Strong hands-on experience with Databricks, specifically Spark, PySpark, and Delta Lake.
• Advanced SQL skills for processing large datasets.
• Ability to integrate data from multiple sources.

Responsabilidades

• Build and optimize ETL/ELT pipelines in Databricks using PySpark and Spark SQL.
• Ingest, clean, and transform data from various sources.
• Design datasets and deliver insights through Power BI dashboards.
• Implement best practices in data development and orchestration.

Conhecimentos

Databricks experience

SQL for large-scale data processing

Data integration skills

Distributed computing knowledge

Power BI skills

CI/CD pipelines in Azure

Ferramentas

Databricks

Spark

PySpark

Delta Lake

Power BI

Descrição da oferta de emprego

Databricks Data Engineer Fully Remote Contract

We’re looking for a hands‑on Databricks Data Engineer with strong experience building scalable data pipelines using Spark, PySpark, SQL, and Delta Lake. This role focuses on ingesting data from multiple sources, transforming it for analytics, and publishing high-quality datasets and visualizations.

Responsibilities

• Build and optimize ETL / ELT pipelines in Databricks using PySpark, Spark SQL, and Delta Lake.
• Ingest, clean, and transform data from diverse sources (APIs, SQL databases, cloud storage, SAP / legacy systems, streaming).
• Develop reusable pipeline frameworks, data validation logic, and performance‑tuned transformations.
• Design curated datasets and deliver insights through Power BI dashboards.
• Implement best practices for lakehouse development, orchestration, and version control.
• Troubleshoot pipeline performance issues and ensure data accuracy, reliability, and quality.

Required Skills

• Strong hands‑on Databricks experience (Spark, PySpark, Delta Lake).
• Advanced SQL for large‑scale data processing.
• Experience integrating data from multiple structured and unstructured sources.
• Solid understanding of distributed computing, performance tuning, and debugging Spark jobs.
• Power BI (reports, models, DAX preferred).
• Experience with CI / CD pipelines in an Azure environment

Nice to Have

• Experience with data quality frameworks, Lakehouse monitoring, or DQX
• Knowledge of Airflow, ADF, IoT, Kafka or other tools is a plus
• Experience with SAP data is a plus.