Data Developer (AI / Databricks) Sênior

Jobgether; Brasil; Híbrido; BRL 250 000 - 360 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Remote work in Brazil

Health insurance

Dental insurance

Meal allowances

Childcare assistance

Parental leave

Wellbeing platform

Profit-sharing

Life insurance

Continuous learning

Online courses

Language learning

Discount program

Employee wellness

Pregnancy support

Inclusion support

Resumo da oferta

Jobgether is seeking a Senior Data Developer (AI / Databricks) based in Brazil to design, build, and maintain scalable data pipelines on Microsoft Azure using Databricks, PySpark, Python, and SQL. You will ensure data quality, governance, security, and reliability while supporting cloud migration and AI model deployment in production.

The role is remote within Brazil with attendance in the Campinas region as required by policy, collaborating with data scientists, engineers, and business teams to

Qualificações

• Senior data engineer with production-grade pipeline experience.
• Strong Azure data services and Databricks expertise.
• Proficiency in PySpark, Python, and SQL for transformation and automation.
• Knowledge of ETL/ELT, ingestion, orchestration, and delivery.

Responsabilidades

• Design, build, and optimize scalable data pipelines on Azure Databricks.
• Develop automated data ingestion, transformation, and delivery workflows.
• Maintain CI/CD pipelines and GitHub/Azure DevOps workflows.
• Model data views and ensure governance, quality, and security.
• Collaborate with data scientists and business teams to deploy AI models.

Conhecimentos

Azure Databricks

PySpark

Python

SQL

Data Modeling

CI/CD

GitHub

Azure DevOps

Ferramentas

Azure Data Factory

Azure Synapse

Apache Spark

Git

Databricks MLflow

Descrição da oferta de emprego

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Data Developer (AI / Databricks) Sênior based in Brazil.

The Senior Data Developer will design, build, and maintain scalable data engineering solutions that support AI initiatives and business transformation. This role focuses on developing robust data pipelines within Microsoft Azure, leveraging Databricks, PySpark, Python, and SQL to process and integrate large volumes of data. You will help ensure data quality, governance, security, and reliability throughout the data lifecycle while optimizing performance and automating deployment workflows. Working closely with business teams, data scientists, and technical specialists, you will contribute to cloud migration projects and the deployment of AI models in production environments. This opportunity is ideal for an experienced data professional who enjoys solving complex engineering challenges, improving data architecture, and delivering dependable, cloud-native solutions. The position is remote within Brazil, with office attendance required for professionals residing in the Campinas metropolitan region, in accordance with the applicable attendance policy.

Accountabilities

• Design, develop, maintain, and optimize ETL/ELT data pipelines using Microsoft Azure, Databricks, and PySpark.
• Implement automated workflows for data ingestion, transformation, integration, processing, and delivery to downstream systems and applications.
• Process and manage large volumes of customer registration and master data, ensuring data accuracy, consistency, integrity, and availability.
• Develop and maintain CI/CD pipelines to automate deployments, streamline development workflows, and support reliable releases.
• Manage source code, version control, and collaborative development workflows using GitHub and Azure DevOps.
• Design, implement, and optimize conceptual, logical, and physical data models to support scalability, performance, and effective data governance.
• Work with Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Lake Storage, and Spark to build and evolve cloud-based data solutions.
• Contribute to cloud migration initiatives, ensuring secure data transfer, integrity, quality, and continuity throughout the migration process.
• Collaborate with business stakeholders, data scientists, and engineering teams to integrate data solutions and deploy AI models in cloud environments.
• Establish and maintain data quality controls, including data profiling, validation, consistency checks, and ongoing monitoring.
• Monitor pipeline performance and delivery indicators, identify anomalies and bottlenecks, and implement improvements to data processing workflows.
• Automate recurring processes and queries to improve operational efficiency, reliability, and maintainability.
• Investigate data-related issues, analyze root causes, and recommend effective technical solutions.
• Document data architectures, pipeline designs, technical processes, and engineering best practices to support knowledge sharing and maintainability.
• Contribute to the evolution of data architecture, governance practices, and AI-enabled workflows, applying sound engineering principles across the data lifecycle.

Requirements

• Substantial professional experience in senior-level data engineering or data development, including the design and maintenance of production-grade data pipelines.
• Strong experience with Microsoft Azure data services and Databricks, including the development and optimization of distributed data processing solutions.
• Proficiency in PySpark, Python, and SQL for data transformation, analysis, integration, and automation.
• Strong knowledge of ETL/ELT processes and data pipeline architecture, including ingestion, transformation, orchestration, and delivery.
• Experience with Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, and Apache Spark.
• Practical knowledge of CI/CD practices and automated deployment workflows.
• Experience with Git and GitHub for source control, as well as familiarity with Azure DevOps for versioning and deployment automation.
• Demonstrated experience in conceptual, logical, and physical data modeling, with an understanding of performance optimization, scalability, and data governance.
• Experience integrating and processing large datasets, maintaining data consistency, and ensuring data quality across complex workflows.
• Familiarity with cloud migration projects and the technical considerations involved in secure and reliable data transfers.
• Ability to collaborate with business and data science teams to support AI model deployment and cloud-based data solutions.
• Strong analytical and problem-solving skills, with the ability to investigate data issues, identify root causes, and implement effective improvements.
• Excellent technical documentation and communication skills, with the ability to work collaboratively across multidisciplinary teams.
• Preferred qualifications: Experience with AI projects and automated data pipelines; knowledge of data-oriented architectures and governance best practices; involvement in Master Data Management (MDM) initiatives; familiarity with Databricks MLflow for model lifecycle management; and experience with Data Lake and Data Warehouse architectures.
• Experience with Azure Machine Learning or similar tools for deploying and managing AI models is an advantage.

Benefits

• Remote work: Home-office arrangement for professionals based in Brazil, subject to the specific office attendance policy applicable to residents of the Campinas metropolitan region.
• Health insurance: Medical coverage.
• Dental insurance: Dental coverage.
• Food and meal allowances: Benefits to support food and meal expenses.
• Childcare assistance: Financial support for eligible childcare expenses.
• Extended parental leave: Enhanced parental leave provisions to support employees and their families.
• Fitness and wellbeing: Partnerships with fitness, health, and wellness platforms, including Wellhub (Gympass) and TotalPass.
• Profit-sharing: Participation in a profit-sharing program (PLR), subject to applicable program terms.
• Life insurance: Life insurance coverage.
• Continuous learning: Access to a dedicated learning platform, including internal professional development resources.
• Online education: Partnerships with online course providers to support technical and professional development.
• Language learning: Access to a language-learning platform.
• Discount program: Access to a club offering discounts and special offers.
• Physical and mental wellbeing: Access to a free online platform dedicated to health, mental wellbeing, and overall wellness.
• Parenting support: Courses focused on pregnancy and responsible parenting.
• Inclusive workplace support: Dedicated health and wellbeing professionals, inclusion specialists, and employee affinity groups. Candidates with disabilities can request appropriate support and accommodations during the recruitment process.
• Compensation details: The original job description does not specify a salary range, bonus amount, or equity compensation. Confirm compensation, benefit eligibility, and applicable terms with the hiring team.

#LI-CL1