Data & AI Engineer

Jobgether; Brasil; Remote; BRL 180,000 - 320,000; Full time

Benefits offered by this job

Remote-First Work

Collaborative Environment

Technical Growth

Meaningful Client Impact

Inclusive Workplace

Job summary

Jobgether is seeking a Data & AI Engineer based in Brazil to design and deliver modern data platforms and production-ready AI solutions for enterprise clients. You will build scalable Lakehouse architectures, optimize pipelines, and transform complex datasets into reliable foundations for analytics and ML deployments.

You'll work with Databricks, Apache Spark, Delta Lake, and MLflow across cloud environments, collaborating with data architects and client teams to translate requirements into

Qualifications

• At least three years of cloud-native data engineering experience, including hands-on Databricks, Spark, Delta Lake, and MLflow.
• Strong proficiency in Python and SQL.
• Experience with CI/CD and Infrastructure-as-Code (Terraform or similar).
• Databricks Certification strongly preferred.
• Experience in agile delivery and client-facing consulting.
• Excellent communication and collaboration skills.
• Ownership mindset with focus on measurable business outcomes.

Responsibilities

• Lakehouse Architecture and Data Pipelines: Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, Apache Spark, and Delta Lake across cloud environments.
• Data Ingestion and Transformation: Build robust workflows for processing structured and unstructured data for analytics and real-time AI workloads.
• Performance and Cost Optimization: Tune Spark jobs and workflows to improve latency, throughput, scalability, and cloud cost efficiency.
• MLOps and AI Deployment: Collaborate to operationalize AI models using MLflow for experiment tracking and deployment.
• CI/CD and Infrastructure Automation: Implement automated deployment pipelines and IaC practices using Terraform and Databricks CLI.
• Data Governance and Security: Establish data access controls and governance standards, supporting GDPR/HIPAA/SOC 2.
• Client Collaboration: Work with client leaders to translate data/AI requirements into technical solutions and measurable outcomes.
• Documentation and Knowledge Transfer: Produce technical docs and coordinate handoffs to client teams.

Skills

Python

SQL

Spark

Databricks

Delta Lake

MLflow

Terraform

CI/CD

Databricks CLI

Job description

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Data & AI Engineer based in Brazil.

This role offers the opportunity to design and deliver modern data platforms and production-ready AI solutions for enterprise clients. You will build scalable Lakehouse architectures, optimize data pipelines, and transform complex datasets into reliable foundations for analytics and machine learning. Working with technologies such as Databricks, Apache Spark, Delta Lake, and MLflow, you will help bring AI initiatives from concept to real-world deployment. You will collaborate closely with data architects, data scientists, engineering teams, and client leaders in a fast-paced, delivery-focused environment. The position combines hands‑on engineering, cloud optimization, MLOps, and data governance to create secure, efficient, and scalable solutions. It is ideal for an engineer who values technical excellence, ownership, continuous improvement, and measurable business impact.

Accountabilities:

• Lakehouse Architecture and Data Pipelines: Design, develop, and optimize scalable ETL/ELT pipelines using Databricks, Apache Spark, and Delta Lake across cloud environments, including AWS, Azure, and Google Cloud Platform.

• Data Ingestion and Transformation: Build robust workflows for processing structured and unstructured data, automating ingestion, storage, and feature engineering to support analytics and real-time AI workloads.

• Performance and Cost Optimization: Tune Spark jobs and data processing workflows to improve latency, throughput, scalability, and cloud cost efficiency across client environments.

• MLOps and AI Deployment: Collaborate with machine learning engineers and data scientists to operationalize AI models within Databricks, using MLflow for experiment tracking, model versioning, and deployment.

• CI/CD and Infrastructure Automation: Implement automated deployment pipelines and Infrastructure-as-Code practices using tools such as Terraform and Databricks CLI to ensure reliable, repeatable production releases.

• Data Governance and Security: Establish and maintain data access controls, security policies, and governance standards using Unity Catalog and cloud-native capabilities, supporting requirements such as GDPR, HIPAA, and SOC 2.

• Client Collaboration: Work directly with client technology leaders, product managers, architects, and engineering teams to translate complex data and AI requirements into practical technical solutions and measurable business outcomes.

• Documentation and Knowledge Transfer: Produce clear technical documentation and coordinate smooth solution handoffs to client engineering teams, ensuring maintainability and long‑term operational success.

Requirements:

• Professional Experience: At least three years of experience in cloud-native data engineering, including a minimum of two years of hands‑on experience with Databricks, Apache Spark, Delta Lake, and MLflow.

• Programming and Data Engineering: Strong proficiency in Python, SQL, and Spark-based frameworks, with experience designing and maintaining scalable data processing solutions.

• Cloud and Lakehouse Technologies: Practical experience building data architectures across cloud environments and implementing modern Lakehouse patterns for enterprise data workloads.

• MLOps and Model Lifecycle Management: Familiarity with MLflow and the processes required to track, version, deploy, and operationalize machine learning models.

• DevOps and Automation: Experience implementing CI/CD pipelines and Infrastructure-as-Code solutions using Terraform, Databricks CLI, or comparable tools.

• Databricks Certification: An active Databricks Certified Data Engineer Associate or Professional certification, or Databricks Machine Learning Professional certification, is strongly preferred.

• Consulting and Agile Delivery: Proven experience working in fast-paced Agile or DevOps consulting environments, delivering cloud-agnostic solutions and adapting to evolving client requirements.

• Communication and Collaboration: Strong communication skills, with the ability to explain technical concepts clearly, work effectively with multidisciplinary teams, and maintain productive client relationships.

• Ownership and Continuous Improvement: A proactive, accountable approach to problem‑solving, with a commitment to engineering quality, continuous learning, and delivering measurable outcomes.

Benefits:

• Remote-First Work: The flexibility to work remotely while collaborating with a globally connected team.

• Collaborative Environment: The opportunity to work alongside skilled professionals in a supportive, innovative, and fast‑paced setting.

• Technical Growth: Exposure to modern Lakehouse architectures, cloud platforms, data engineering, MLOps, and production AI solutions.

• Meaningful Client Impact: The opportunity to build scalable, production-ready solutions that address real business challenges.

• Inclusive Workplace: A culture that values diversity, equal opportunity, collaboration, curiosity, and individual initiative.

We appreciate your interest and wish you the best!

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1