Data Engineer (Azure Databricks + Python) - Remote Work | REF#294142

BairesDev; Rio de Janeiro; Remote; BRL 607,000 - 910,000; Full time

Benefits offered by this job

Remote work

Competitive pay

Home setup provided

Flexible hours

Parental leave

Global team

Job summary

BairesDev is hiring a Data Engineer to design and develop scalable data pipelines and Lakehouse architectures using Azure Databricks and Python. You will play a key role in integrating AI capabilities into data workflows and automating classification tasks.

You will collaborate with AI and product teams to implement data-driven solutions, monitor pipelines, and ensure security and governance across distributed cloud environments. 4+ years of relevant experience is expected.

Qualifications

• 4+ years of experience in Data Engineering, ML Engineering, or Software Engineering.
• Proven expertise with Azure Databricks and Python.
• Experience integrating ML/AI solutions and LLM APIs.
• Strong data manipulation and automation skills.

Responsibilities

• Design, build, and optimize end-to-end data pipelines and Lakehouse architectures on Azure Databricks.
• Integrate LLM APIs and ML models into data processing to enable automated classification and insights.
• Develop and maintain high-quality Python code and SQL procedures for data transformation and ingestion.
• Collaborate with AI and product teams to deliver data-driven solutions with business impact.
• Monitor pipeline performance and ensure reliability and scalability in cloud environments.
• Implement data governance and security best practices for data platform integrity.

Tools

Azure Databricks

Python

SQL

ML/AI integration

Cloud architectures

Job description

At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting‑edge solutions to giants like Google and the most innovative startups in Silicon Valley.

Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide.

When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success.

As a Data Engineer, you will design and develop scalable data pipelines leveraging Azure Databricks and Python. You will play a key role in integrating AI capabilities into data workflows, enabling the organization to process complex datasets and automate intelligent classification tasks.

What You’ll Do

• Design, build, and optimize end-to-end data pipelines and Lakehouse architectures using Azure Databricks.
• Integrate LLM APIs and machine learning models into data processing streams to support automated classification and insights.
• Develop and maintain high-quality Python code and SQL procedures to ensure efficient data transformation and ingestion.
• Collaborate with AI and product teams to implement data-driven solutions that drive significant business value.
• Monitor and troubleshoot pipeline performance, ensuring reliability and scalability across distributed cloud environments.
• Implement data governance and security best practices to maintain the integrity and compliance of the data platform.

What We Are Looking For

• 4+ years of experience in Data Engineering, Machine Learning Engineering, or Software Engineering.
• Proven expertise in developing and optimizing data pipelines using Azure Databricks and Python.
• Proficiency in integrating ML/AI solutions and LLM APIs into automated data workflows.
• Strong background in data manipulation, automation, and supporting classification tasks.
• Deep understanding of cloud-native data architectures and AI integration patterns.
• Advanced proficiency in English.

How we do make your work (and your life) easier:

• 100% remote work (from anywhere).
• Excellent compensation in USD or your local currency if preferred
• Hardware and software setup for you to work from home.
• Flexible hours: create your own schedule.
• Paid parental leaves, vacations, and national holidays.
• Innovative and multicultural work environment: collaborate and learn from the global Top 1% of talent.
• Supportive environment with mentorship, promotions, skill development, and diverse growth opportunities.

Join a global team where your unique talents can truly thrive and make a significant impact!