Data Engineer

Casado; São Paulo; Hybrid; BRL 180,000 - 300,000; Full time

Job summary

Casado is seeking a hands-on Databricks Engineer to build and support data pipelines on the raw layer, ensuring production-grade reliability and cost efficiency. The role emphasizes end-to-end ownership from requirements to deployment and ongoing support.

You will transform raw data into curated layers, implement data quality checks with Unity Catalog, and collaborate with business users and the Platform team across time zones in a hybrid Brazil-based setup.

Qualifications

• 8+ years of experience in data engineering.
• 5+ years hands-on Databricks experience in production.
• Proficiency in PySpark, Python and advanced SQL.
• Solid understanding of Delta Lake, data modeling and ETL/ELT patterns.
• Experience supporting production workloads, with strong debugging and data validation skills.
• Working knowledge of Git and CI/CD, ideally Azure DevOps (Repos, Pipelines, Boards).
• Bachelor's degree in Computer Science, Engineering or related field, or equivalent experience.
• Excellent written and verbal English.
• Self-starter who takes initiative, works independently and proactively solves problems.
• Ability to own work end to end, from requirements to production support, and meet commitments.
• Comfortable working with business users and explaining technical topics simply.
• Collaborative communicator who shares proactive updates, raises risks early and works well across time zones

Responsibilities

• Build and optimize Databricks pipelines using PySpark, SQL and Delta Lake.
• Transform Raw data into curated and business-ready layers.
• Own production deployments through Azure DevOps (ADO), including code reviews, testing and releases
• Track work in ADO Boards
• Monitor, troubleshoot and tune jobs for performance, reliability and cost
• Implement data quality checks and governance using Unity Catalog
• Partner with business users and the Platform team to turn requirements into technical solutions
• Maintain clear documentation and support runbooks

Skills

Databricks

PySpark

Python

SQL

Delta Lake

ETL/ELT

Data modeling

CI/CD

Azure DevOps

Git

Education

Bachelor's degree in Computer Science, Engineering or related field

Tools

Azure DevOps

Git

Job description

Mission Context

As part of a data engineering project, our client is looking for a hands-on Databricks Engineer to build and support data pipelines on top of the raw layer owned by the Platform team.

Missions

• Build and optimize Databricks pipelines using PySpark, SQL and Delta Lake
• Transform Raw data into curated and business-ready layers
• Own production deployments through Azure DevOps (ADO), including code reviews, testing and releases
• Track work in ADO Boards
• Monitor, troubleshoot and tune jobs for performance, reliability and cost
• Implement data quality checks and governance using Unity Catalog
• Partner with business users and the Platform team to turn requirements into technical solutions
• Maintain clear documentation and support runbooks

Required Profile

• 8+ years of experience in data engineering
• 5+ years of hands-on Databricks experience in production
• Strong hands-on experience with Databricks, including notebooks, jobs, workflows and clusters
• Proficiency in PySpark, Python and advanced SQL
• Solid understanding of Delta Lake, data modeling and ETL/ELT patterns
• Experience supporting production workloads, with strong debugging and data validation skills
• Working knowledge of Git and CI/CD, ideally Azure DevOps (Repos, Pipelines, Boards)
• Bachelor's degree in Computer Science, Engineering or related field, or equivalent experience
• Excellent written and verbal English
• Self-starter who takes initiative, works independently and proactively solves problems
• Ability to own work end to end, from requirements to production support, and meet commitments
• Comfortable working with business users and explaining technical topics simply
• Collaborative communicator who shares proactive updates, raises risks early and works well across time zones

Additional Information

• Location: Brazil (Hybrid 2x days in office per week)
• Working Hours: Must be willing to work US CST hours
• Work type: Contract/freelance