Tech Lead Data - Databricks

Ciandt; Brazil; Remote; BRL 300,000 - 540,000; Full time

Benefits offered by this job

Health and dental insurance

Meal and food allowance

Childcare assistance

Extended paternity leave

Wellbeing programs

Job summary

CI&T Brazil is seeking a senior, hands-on Tech Lead Data to drive technical leadership of data engineering initiatives on Databricks in Azure. You will partner with the Data Architect to define standards, remove roadblocks, and enable the team to deliver scalable pipelines within the Medallion architecture.

You will oversee pipelines across Bronze/Silver/Gold, implement DataOps, and ensure security, performance, and governance.

Qualifications

• Strong experience in Data Engineering and projects, with experience as a Tech Lead/Senior leading teams technically and operationally.
• Hands-on Databricks expertise: PySpark and Spark SQL; batch pipelines with exposure to streaming.
• Mastery of Delta Lake and Medallion architecture (Bronze/Silver/Gold).
• Azure data stack experience: Azure Databricks, ADLS, and secure identity integration.
• Strong SQL and Python skills; data modeling and contract/schema practices.
• DataOps experience: Git, CI/CD for data pipelines, testing, and documentation.
• Proven ability to review designs, resolve performance issues, and mentor teams.
• Advanced English to collaborate with stakeholders and produce technical docs.

Responsibilities

• Provide technical leadership to Data Engineering squads and unblock delivery.
• Co-define with the Data Architect the technical vision, standards, and best practices for the Lakehouse.
• Design and oversee pipelines in Databricks across Bronze/Silver/Gold with governance.
• Implement DataOps practices: version control, CI/CD, testing, and documentation.
• Optimize jobs/pipelines: partitioning, Spark tuning, cost management, observability.
• Ensure security, access control, and data policies in Azure/Databricks.
• Collaborate with stakeholders to translate requirements into scalable solutions and delivery plans.
• Mentor engineers and create templates and playbooks to accelerate the team.

Skills

Data Engineering

Tech Lead

Databricks PySpark

SQL

Python

DataOps

English communication

Tools

Databricks

Azure Databricks

Delta Lake

ADLS

Git

CI/CD for data pipelines

Job description

Tech Lead Data - Databricks

Brazil

About the Role

We are looking for a senior, hands-on Data Tech Lead to drive the technical leadership of our data engineering initiatives on Databricks in Azure.

You will partner directly with the Data Architect to define standards and references, remove roadblocks, and enable the Data Engineering team to deliver scalable, reliable pipelines aligned with the business within the Medallion architecture.

Key Responsibilities

• Provide technical leadership to Data Engineering squads, supporting design, reviews, and decision-making to unblock delivery.

• Co-define with the Data Architect the technical vision, coding standards, modeling conventions, and best practices for the Lakehouse (Delta Lake + Medallion).

• Design and oversee pipelines in Databricks (PySpark/Spark SQL) across Bronze/Silver/Gold layers, ensuring security, performance, and governance.

• Implement and reinforce DataOps practices: version control (Git), CI/CD, testing (unit/integration/data quality), and documentation.

• Optimize jobs/pipelines: partitioning, Spark tuning, cost management, and reliability (baseline observability and alerting).

• Ensure non-functional requirements: security, access control, and data policies in Azure/Databricks aligned with defined governance.

• Collaborate with stakeholders (business/analytics) to translate requirements into scalable solutions with realistic delivery plans.

• Mentor engineers, promote standards, and create references (templates, playbooks) to accelerate the team.

Required Skills and Qualifications

Must-have Skills

• Strong experience in Data Engineering and projects, with experience as a Tech Lead/Senior leading teams technically and operationally.

• Strong hands-on experience in Databricks: PySpark and Spark SQL; batch pipelines (and working knowledge of streaming when needed).

• Mastery of Delta Lake and practical application of the Medallion architecture (Bronze/Silver/Gold).

• Solid experience with Azure for data, including at least: Azure Databricks, Azure Data Lake Storage (ADLS), and integration with security/identity services (e.g., Azure Active Directory).

• Strong SQL and Python skills; fundamentals in modeling (dimensional/consumption) and good practices for contracts/schemas.

• Experience with DataOps: Git, CI/CD for data pipelines, automated testing, and clear documentation.

• Proven ability to review design/code, resolve performance/scale issues, and guide teams.

• Advanced English to work with stakeholders and produce technical documentation.

Nice-to-have Skills

• Unity Catalog (governance/lineage/access control) and Databricks Workflows/Repos; Delta Live Tables.

• Data quality frameworks (Great Expectations/Soda) and/or dbt on the Lakehouse.

• Orchestration (Airflow/Databricks Workflows) and observability (metrics, logs, alerts).

• Experience with CDC/streaming ingestion (Event Hubs/Kafka) when applicable.

• Consulting experience and international client exposure; Databricks/Azure certifications.

Our benefits:

• Health and dental insurance
• Meal and food allowance
• Childcare assistance
• Extended paternity leave
• Partnership with gyms and health and wellness professionals via Wellhub (Gympass) TotalPass;
• Profit Sharing and Results Participation (PLR);
• Life insurance
• Continuous learning platform (CI&T University);
• Discount club
• Free online platform dedicated to physical, mental, and overall well-being
• Pregnancy and responsible parenting course
• Partnerships with online learning platforms
• Language learning platform

And many more! More details about our benefits here: https://ciandt.com/br/pt-br/carreiras

Inclusion

At CI&T, inclusion starts at the first contact. If you are a person with a disability, it is important to present your assessment during the selection process.

See which data needs to be included in the report by clicking here.

This way, we can ensure the support and accommodations that you deserve.

If you do not yet have the assessment, don't worry: we can support you in obtaining it.

We have a dedicated Health and Well-being team, inclusion specialists, and affinity groups who will be with you at every stage. Count on us to make this journey side by side.