Databricks Engineer - Remote Work | REF#297090

BairesDev; Brasil; Remoto; BRL 657.000 - 986.000; Tempo pieno

Vantaggi offerti da questo lavoro

100% remote work

Compensation in USD or local currency

Home hardware and software setup

Flexible hours

Parental leave and holidays

Mentorship and growth opportunities

Descrizione del lavoro

BairesDev, a leading tech company with 15+ years of global delivery, is seeking a Databricks Engineer to design and maintain data pipelines on the Databricks Lakehouse platform.

You will collaborate with data scientists and analysts to deliver high-performance analytics, governance using Unity Catalog, and ML lifecycle management with MLflow, all in a 100% remote environment. This role emphasizes scalability, cost efficiency, and secure data access across the data lifecycle.

Competenze

• 4+ years of experience in Data Engineering, Software Engineering, or Database Administration.
• Proven expertise in building data pipelines and analytics solutions on the Databricks Lakehouse platform.
• Proficiency in managing notebooks, jobs, and Unity Catalog governance.
• Hands-on experience with Delta Lake and MLflow for data processing and lifecycle management.
• Advanced proficiency in English.

Mansioni

• Design and implement scalable data pipelines using Databricks notebooks and automated jobs.
• Build and optimize data lakehouse architectures leveraging Delta Lake for ACID transactions and reliability.
• Manage data governance and security frameworks using Unity Catalog to ensure compliant data access.
• Develop and maintain machine learning lifecycles and experiment tracking using MLflow.
• Collaborate with data scientists and analysts to provide a high-performance platform for advanced analytics.
• Optimize cluster configurations and job execution to ensure cost-effectiveness and system efficiency.

Conoscenze

Databricks Lakehouse

Delta Lake

MLflow

Unity Catalog governance

Data pipelines

Strumenti

Databricks notebooks

Descrizione del lavoro

At BairesDev, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley.

Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide.

When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success.

As a Databricks Engineer, you will build and maintain data pipelines and analytics solutions on the Databricks Lakehouse platform. You will serve as the connection between raw data and actionable insights, ensuring high performance, scalability, and robust governance across the data lifecycle.

What You'll Do

• Design and implement scalable data pipelines using Databricks notebooks and automated jobs.
• Build and optimize data lakehouse architectures leveraging Delta Lake for ACID transactions and reliability.
• Manage data governance and security frameworks using Unity Catalog to ensure compliant data access.
• Develop and maintain machine learning lifecycles and experiment tracking using MLflow.
• Collaborate with data scientists and analysts to provide a high-performance platform for advanced analytics.
• Optimize cluster configurations and job execution to ensure cost-effectiveness and system efficiency.

What We Are Looking For

• 4+ years of experience in Data Engineering, Software Engineering, or Database Administration.
• Proven expertise in building data pipelines and analytics solutions on the Databricks Lakehouse platform.
• Proficiency in managing notebooks, jobs, and Unity Catalog governance.
• Hands-on experience with Delta Lake and MLflow for data processing and lifecycle management.
• Advanced proficiency in English.

How we do make your work (and your life) easier:

• 100% remote work (from anywhere).
• Excellent compensation in USD or your local currency if preferred
• Hardware and software setup for you to work from home.
• Flexible hours: create your own schedule.
• Paid parental leaves, vacations, and national holidays.
• Innovative and multicultural work environment: collaborate and learn from the global Top 1% of talent.
• Supportive environment with mentorship, promotions, skill development, and diverse growth opportunities.

Join a global team where your unique talents can truly thrive and make a significant impact!