Data Engineer (Azure/Databricks) | SR

Jobgether; Brasil; Presencial; BRL 240 000 - 360 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Access to modern Azure/Databricks tech

Exposure to AI and RAG use cases

Cloud-native Data Lake/Data Lakehouse

Professional development in tech

Resumo da oferta

Jobgether seeks a Senior Data Engineer (Azure/Databricks) in Brazil to build scalable data pipelines and data lakehouse architectures. You will design models, implement ETL/ELT processes, and integrate data from APIs and databases using Databricks and Azure tools.

The role emphasizes data quality, reliability, and performance, with exposure to AI-enabled use cases and RAG workflows in a cloud-native data platform. Collaboration with stakeholders drives innovative data solutions.

Qualificações

• Advanced professional experience with Databricks and Spark
• Strong experience with Azure Data Factory and ADLS
• Proven track record in Data Lake/Data Lakehouse architectures
• Proficient in Python and distributed data processing
• Experience with Airflow for orchestration and SQL/NoSQL databases
• Knowledge of APIs, data integration, and data quality best practices

Responsabilidades

• Build and maintain scalable data pipelines using modern distributed processing technologies
• Design data models and pipelines across Databricks, SAP Datasphere, ADF, and ADLS
• Create and maintain Data Lakes and Data Lakehouses and analytical tables
• Structure and manage data tables within Databricks for performance and accessibility
• Develop ETL/ELT pipelines from APIs and data sources; validate data quality
• Structure RAG solutions using contracts and business documents

Conhecimentos

Databricks

Azure Data Factory

Python

SQL/NoSQL databases

Apache Airflow

Azure Data Lake Storage

Data modeling

Apache Spark

AI/RAG experience

API integration

Ferramentas

Databricks

Azure Data Factory

Azure Data Lake Storage

SAP Datasphere

PostgreSQL

MongoDB

Cassandra

dbt

Kafka

Descrição da oferta de emprego

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer (Azure/Databricks) | SR based in Brazil.This is a senior-level data engineering role focused on building scalable, reliable, and high-performance data solutions.
You will design and maintain modern data pipelines across Azure and Databricks environments.
The role combines structured and unstructured data processing, data modeling, and cloud-based data architecture.
You will work with large volumes of data from APIs, databases, documents, and other enterprise sources.
You will also contribute to Data Lake and Data Lakehouse initiatives, including advanced use cases involving RAG and AI.
The position requires strong technical ownership, attention to data quality, and the ability to transform complex data into usable assets.
You will collaborate in a technology-driven environment where modern data platforms and emerging AI capabilities are central to innovation.

Accountabilities

• Build and maintain scalable, reliable data pipelines using modern distributed processing technologies, ensuring efficient data ingestion, transformation, and delivery.
• Design and implement data models using Databricks, SAP Datasphere, Azure Data Factory, Azure Data Lake Storage (ADLS), Python, and relational databases.
• Extract, transform, and organize data to create and maintain Data Lakes, Data Lakehouses, and analytical tables.
• Structure and manage data tables within Databricks, ensuring performance, consistency, and accessibility.
• Develop ETL pipelines for both structured and unstructured data, using APIs and specialized tools for file analysis and data validation.
• Integrate data from APIs, including CXL sources, and transform and process the resulting datasets within Databricks.
• Structure RAG (Retrieval-Augmented Generation) solutions using contracts and other business documents as data sources.
• Apply data engineering best practices to ensure data quality, scalability, reliability, and performance.
• Work with multiple data sources and technologies to support analytical, operational, and AI-driven use cases.
• Contribute to the evolution of cloud-based data architecture and help identify opportunities to improve data processing and integration workflows.
  

Requirements

• Advanced professional experience and strong technical knowledge of Databricks.
• Solid experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).
• Proven experience implementing Data Lake and Data Lakehouse architectures.
• Strong programming skills in Python and experience with Apache Spark and distributed data processing.
• Professional experience with Apache Airflow for data pipeline orchestration.
• Experience working with SQL and NoSQL databases, including technologies such as PostgreSQL, MongoDB, and Cassandra.
• Strong understanding of data modeling, ETL/ELT processes, data integration, and large-scale data processing.
• Ability to work with both structured and unstructured datasets and transform complex sources into reliable analytical assets.
• Analytical mindset and strong problem-solving skills, with attention to data quality and technical details.
• Ability to work independently on complex data engineering challenges while collaborating effectively with technical and business stakeholders.
• Desirable: Knowledge of Kafka and event-driven data architectures.
• Desirable: Experience with dbt and modern data transformation practices.
• Desirable: Knowledge of AWS Glue and BigQuery.
• Desirable: Experience with AI, RAG, or document-based data solutions.
• Desirable: Familiarity with SAP Datasphere.
  

Benefits

• Opportunity to work with modern Azure and Databricks data technologies.
• Exposure to advanced AI, Generative AI, and RAG use cases.
• Opportunity to work with cloud-native Data Lake and Data Lakehouse architectures.
• Professional development in a technology-focused and innovation-driven environment.
• Access to modern AI-powered tools and technologies.
• Opportunities to work with distributed and borderless teams.
• Exposure to emerging technologies and current industry trends.
• Career development opportunities and continuous learning.
• Opportunity to contribute to projects focused on digital transformation and AI-driven innovation.