Senior Data Engineer (Databricks & AI-Enabled Data Platforms)

Mercado de Ti; São Paulo; Hybrid; BRL 240,000 - 360,000; Full time

Benefits offered by this job

Hybrid work arrangement

On-site in Pinheiros, São Paulo

Job summary

EVOX is seeking a Senior Data Engineer to design, build, and optimize modern data platforms that enable advanced analytics, AI/ML, and BI solutions in our São Paulo team. You will lead Databricks-based data pipelines, Delta Lake architectures, and cloud-native data services across AWS and Azure.

You'll mentor engineers, establish engineering standards, and drive data governance, observability, and CI/CD for scalable deployments. Hybrid work with three on-site days in Pinheiros, São Paulo.

Qualifications

• Bachelor’s degree in CS/IT/Engineering or related field.
• Strong experience with Databricks, Spark, and cloud data platforms.
• Proven ability to lead technical initiatives and mentor teams.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks and Spark.
• Build batch and real-time ingestion frameworks and data products.
• Implement data quality, observability, lineage, and governance.
• Design Delta Lake architectures and Medallion data models.
• Establish CI/CD and IaC for Databricks deployments.
• Design cloud-native architectures across Azure and AWS.
• Collaborate with data scientists to operationalize ML solutions.
• Provide technical leadership and mentoring to engineers.

Skills

Databricks

Apache Spark

Python

SQL

Delta Lake

Data Governance

MLOps

Azure

AWS

ETL/ELT

Education

Bachelor’s degree in CS/IT/Engineering

Master’s degree preferred

Tools

Databricks Notebooks

Delta Live Tables

Unity Catalog

Terraform

Azure Data Factory

GitHub

Job description

EVOX is looking for a Senior Data Engineer (Databricks & AI-Enabled Data Platforms) to work with one of our clients.We are seeking a highly skilled Senior Data Engineer to design, build, and optimize modern data platforms that enable advanced analytics, AI/ML, and business intelligence solutions. The ideal candidate will possess deep expertise in Databricks, cloud-based data engineering, data architecture, and modern software engineering practices. This role will provide technical leadership, establish engineering standards, and drive the development of scalable, secure, and reliable data solutions supporting enterprise-wide initiatives.**Key Responsibilities:****Data Platform Engineering:*** Design, develop, and maintain scalable data pipelines using Databricks, Spark, and cloud-native technologies;* Build and optimize batch and real-time data ingestion frameworks;* Develop reusable engineering patterns, frameworks, and accelerators for enterprise data solutions;* Implement data quality, observability, lineage, and governance capabilities across the data ecosystem;* Support large-scale data modernization and cloud migration initiatives.**Databricks Engineering:*** Design and implement Delta Lake architectures and Medallion data models;* Build optimized ETL/ELT pipelines using Databricks notebooks, workflows, and Delta Live Tables;* Establish performance tuning, cost optimization, and workload management best practices;* Leverage Unity Catalog for data governance, security, and metadata management;* Implement CI/CD pipelines and Infrastructure-as-Code for Databricks deployments.**Cloud Data Engineering:*** Design cloud-native data architectures across Azure and AWS environments;* Integrate enterprise data sources including SQL Server, Oracle, APIs, SaaS platforms, and streaming platforms;* Build resilient and secure data services supporting high availability and disaster recovery requirements;* Partner with infrastructure and security teams to implement cloud security controls and compliance standards.**AI and Advanced Analytics Enablement:*** Build and optimize data products supporting AI, machine learning, and generative AI workloads;* Develop feature engineering pipelines and curated datasets for model training and inference;* Support vector databases, semantic search, RAG (Retrieval-Augmented Generation), and LLM powered applications;* Collaborate with Data Scientists and AI Engineers to operationalize machine learning solutions;* Implement MLOps and DataOps practices to support scalable AI delivery.**Technical Leadership:*** Serve as a technical lead for complex data engineering initiatives;* Define architecture standards, engineering best practices, and development guidelines;* Mentor junior and mid-level engineers;* Conduct architecture reviews and provide technical recommendations;* Drive adoption of automation, observability, and platform engineering principles.**Required Qualifications / Skills:*** Bachelor’s degree in Computer Science, Information Technology, Engineering, or related discipline;* Strong experience in data engineering, data warehousing, or data platform development;* Years of hands-on experience with Databricks and Apache Spark;* Experience designing and implementing enterprise-scale cloud data platforms;* Experience leading technical initiatives and mentoring engineering teams;* Data Engineering: Databricks, Apache Spark (PySpark, Spark SQL), Delta Lake, Delta Live Tables, Unity Catalog, Data Warehousing, and Data Modeling;* Programming: Python, SQL, Scala, PowerShell, and scripting automation;* Cloud: Microsoft Azure, AWS, and cloud-native storage and compute services;* Databases: SQL Server, Oracle, PostgreSQL, and Snowflake;* Data Integration: Azure Data Factory, Databricks Workflows, Kafka/Event Streaming, and REST APIs;* DevOps & Automation: GitHub, Azure DevOps, CI/CD pipelines, Terraform, and Infrastructure as Code (IaC);* Observability: Monitoring, alerting, data quality, performance optimization, logging, and troubleshooting;* AI & Emerging Technologies: Machine Learning pipelines, Feature Stores, MLOps, Generative AI, Vector Databases, RAG, and AI Governance;* Fluent English.**Preferred:*** Master’s degree preferred.* Databricks Certified Data Engineer Professional;* Azure Data Engineer Associate certification;* AWS Data Analytics certification;* Experience supporting regulated industries such as Insurance, Financial Services, or Healthcare;* Experience implementing enterprise data governance programs;* Experience integrating AI solutions into production environments.**Additional Information:*** Hybrid position, with 3 days per week working on-site in Pinheiros, São Paulo.