Data Engineering Consultant

Jobtailor; São Paulo; Presencial; BRL 180 000 - 320 000; Tempo integral

Resumo da oferta

Jobtailor in São Paulo, Brazil, is seeking a senior Data Engineer to develop, optimize, and maintain data pipelines using Databricks. You will work with Lakehouse architectures in Azure cloud, implement CI/CD, and ensure data governance and security. Collaboration with international teams in English is a key part of the role.

The candidate should have strong SQL, Python/PySpark, and data orchestration experience, with a focus on performance, reliability, and scalable data solutions.

Qualificações

• Experience with Databricks and data engineering in large-scale environments.
• Advanced SQL knowledge and proficiency in Python/PySpark.
• Cloud experience, preferably Microsoft Azure.
• Knowledge of data modeling and Lakehouse/Data Lake architectures.
• Experience with data orchestration and integration tools.
• Familiarity with Git and DevOps/DataOps practices.
• Able to resolve complex data issues and optimize performance.
• Experience collaborating with international teams.
• Strong communication skills in English.

Responsabilidades

• Develop, maintain, and optimize data pipelines using Databricks.
• Build ingestion, transformation, and processing solutions for large data volumes.
• Work with Lakehouse architectures and cloud environments.
• Implement data engineering best practices, version control, and CI/CD.
• Monitor pipeline performance, quality, and reliability.
• Analyze and troubleshoot data- and performance-related issues.
• Participate in architectural decisions and evolve the data platform.
• Collaborate with global teams in English.

Conhecimentos

Databricks

SQL

Python

PySpark

Cloud Azure

Data Modeling

Lakehouse

Data Governance

Performance Monitoring

Troubleshooting

International Collaboration

CI/CD

Git

Data Orchestration

Data Integration

Data Processing

Data Pipelines

Data Security

English Communication

Ferramentas

Databricks

Git

DevOps

DataOps

Azure

CI/CD

Descrição da oferta de emprego

• Develop, maintain, and optimize data pipelines using Databricks
• Build ingestion, transformation, and processing solutions for large volumes of data
• Work with Lakehouse architectures and cloud environments
• Implement data engineering best practices, version control, and CI/CD
• Monitor pipeline performance, quality, and reliability
• Analyze and troubleshoot data- and performance-related issues
• Participate in architectural decisions and the evolution of the data platform
• Collaborate with global teams in technical and functional meetings conducted in English
• Ensure data governance, security, and standardization
• Support business teams with analytics solutions and data integration

Requirements

• Strong experience with Databricks
• Experience in data engineering and distributed processing
• Advanced knowledge of SQL
• Experience with Python and/or PySpark
• Experience working in cloud environments, preferably Microsoft Azure
• Knowledge of data modeling and Lakehouse/Data Lake architecture
• Experience with data orchestration and integration tools
• Knowledge of Git and DevOps/DataOps practices
• Experience helping resolve complex data issues and optimize performance
• Experience working with international teams
• Collaborative and analytical profile with strong communication skills
• Advanced/fluent English required for verbal communication, global meetings, and technical documentation

Core Competencies

Demonstrates expertise in developing and optimizing data pipelines using Databricks, with a strong focus on data engineering best practices, cloud environments, and data governance. Proficient in SQL, Python, and data orchestration tools, while effectively collaborating with global teams in English.

Highest-signal resume keywords

• Databricks
• Data Engineering
• SQL
• Python
• Microsoft Azure

ATS Optimization Keywords

Hard Skills

• Data Pipeline Development
• Data Transformation
• Data Processing Solutions
• Data Modeling
• Data Orchestration
• Version Control
• CI/CD
• Data Governance
• Performance Monitoring
• Troubleshooting

Soft Skills

• Collaborative
• Analytical
• Strong Communication Skills

Industry Keywords

• Lakehouse Architecture
• Cloud Environments
• Data Integration
• Data Security
• International Collaboration

Tools & Technologies

• PySpark
• Git
• DevOps
• DataOps