Data Engineer

Jobtailor

Osasco

Presencial

BRL 180 000 - 320 000

Tempo integral

Há 4 dias

Torna-te num dos primeiros candidatos

Recebe mais respostas dos empregadores

Envia um currículo específico para a oferta em poucos minutos.

Resumo da oferta

Jobtailor is seeking a Senior Data Architect/Engineer to lead data engineering architectures and provide technical leadership for ingestion, processing, storage, and delivery using Azure and Databricks.

You will design Data Lake/Lakehouse solutions, set standards, drive observability and governance, ensure LGPD compliance, and mentor mid‑ and senior engineers across multidisciplinary teams.

Qualificações

Advanced experience in Data Engineering and Data Architecture.
Proficiency in Python.
Advanced SQL and query optimization.
Spark/PySpark in large-scale environments.
Advanced Databricks expertise.
Azure Data Lake Storage.
Azure Data Factory.
Delta Lake.
Lakehouse Architecture.
Data Warehouses and Data Lakes.
Data modeling.
Batch and streaming processing.
Kafka/Event Hubs or equivalent technologies.
APIs and integration patterns.
CI/CD and automation.
Infrastructure as Code (Terraform or equivalent).
Pipeline observability and monitoring.
Data security, governance, cataloging, and lineage.
Knowledge of Brazil’s LGPD and data protection.
Experience designing resilient and highly available solutions.
Preferred qualifications: Unity Catalog, Databricks Workflows, Azure Synapse, Microsoft Fabric, Kubernetes, etc.

Responsabilidades

Define and evolve Data Engineering architectures.
Provide technical leadership for data ingestion, processing, storage, and delivery solutions.
Define standards, frameworks, reusable components, and best practices.
Architect solutions using Azure and Databricks.
Design Data Lake/Lakehouse solutions for processing large volumes of data.
Evaluate technologies, frameworks, and modernization strategies.
Define strategies for performance, scalability, resilience, and high availability.
Address highly complex technical challenges and incidents.
Define observability, data quality, lineage, and governance.
Ensure compliance with security, privacy, and data protection requirements.
Support Architects and leadership in making technical decisions.
Conduct solution assessments and architectural reviews.
Serve as a technical reference and mentor for Mid-Level and Senior Data Engineers.
Collaborate with multidisciplinary teams, vendors, and stakeholders.
Translate business needs into scalable and sustainable technical solutions.

Conhecimentos

Python Programming

SQL Query Optimization

Spark/PySpark

Databricks

Azure Data Lake Storage

Azure Data Factory

Delta Lake

Lakehouse Architecture

Data Modeling

Batch & Streaming Processing

Kafka / Event Hubs

APIs & Integration

CI/CD & Automation

Terraform / IaC

Observability & Governance

LGPD Knowledge

Kubernetes

Security & Data Protection

Ferramentas

Azure Data Lake Storage

Azure Data Factory

Databricks Workflows

Microsoft Fabric

Kubernetes

Descrição da oferta de emprego

Define and evolve Data Engineering architectures
Provide technical leadership for complex data ingestion, processing, storage, and delivery solutions
Define standards, frameworks, reusable components, and best practices
Architect solutions using Azure and Databricks
Design Data Lake/Lakehouse solutions for processing large volumes of data
Evaluate technologies, frameworks, and modernization strategies
Define strategies for performance, scalability, resilience, and high availability
Address highly complex technical challenges and critical incidents
Define mechanisms for observability, data quality, lineage, and governance
Ensure compliance with security, privacy, and data protection requirements
Support Architects and leadership in making technical decisions
Conduct solution assessments and architectural reviews
Serve as a technical reference and mentor for Mid-Level and Senior Data Engineers
Collaborate with multidisciplinary teams, vendors, and the bank’s stakeholders
Translate business needs into scalable and sustainable technical solutions

Requirements

Advanced experience in Data Engineering and Data Architecture
Proficiency in Python
Advanced SQL and query optimization
Spark/PySpark in large-scale environments
Advanced Databricks expertise
Azure Data Lake Storage
Azure Data Factory
Delta Lake
Lakehouse Architecture
Data Warehouses and Data Lakes
Data modeling
Batch and streaming processing
Kafka/Event Hubs or equivalent technologies
APIs and integration patterns
CI/CD and automation
Infrastructure as Code (Terraform or equivalent)
Pipeline observability and monitoring
Data security, governance, cataloging, and lineage
Knowledge of Brazil’s General Data Protection Law (LGPD) and data protection
Experience designing resilient and highly available solutions
Preferred qualifications: Unity Catalog, Databricks Workflows, Azure Synapse, Microsoft Fabric, Kubernetes, event-driven architectures, FinOps applied to data platforms, Microsoft Azure and/or Databricks certifications, and experience in large-scale, mission-critical banking environments
Technical leadership and influence
Ability to make architectural decisions
Systems thinking and strategic mindset
Excellent analytical skills
Executive and technical communication
Ability to operate across business, architecture, and engineering
Team mentoring and technical development
Ownership, collaboration, and results orientation

Core Competencies

Demonstrates advanced expertise in Data Engineering and Data Architecture, with a strong focus on designing scalable solutions using Azure and Databricks. Proficient in data governance, security, and compliance, particularly with Brazil’s LGPD, while providing technical leadership and mentoring to engineering teams.

Highest-signal resume keywords