Senior Data Engineer / Architect, Databricks, Latin America, Mexic, Canada

CloudGeometry; Belo Horizonte; Presencial; BRL 120.000 - 150.000; Tempo integral

Resumo da oferta

A leading company in the life sciences industry is seeking a Senior Data Engineer/Architect to join their flagship project. This role involves designing and optimizing data pipelines using Databricks, leading architecture discussions, and ensuring data quality. Ideal candidates will have extensive experience in data engineering and strong communication skills, contributing to innovative RNA-based solutions for industry leaders.

Qualificações

• 8+ years in software development, preferably in data engineering.
• 5+ years with the DataBricks ecosystem.

Responsabilidades

• Design and optimize data pipelines within the Databricks platform.
• Lead architecture discussions with product managers and data scientists.
• Ensure data quality and integrity across processing stages.

Conhecimentos

Python

TypeScript

Data Engineering

Data Analytics

Formação académica

Bachelor’s degree in Computer Science

Master’s degree

Ferramentas

Databricks

AWS

Spark

Glue

Delta tables

Iceberg

Git

Descrição da oferta de emprego

Move From Belgium To Canada - We Help Belgium Residents Get A Work Offer In Canada. Don't Miss The Chance: Sign Up Now! Track Record Of Thousands Of Clients & 98.7% Success: Sign Up & Start Working In Canada!

Find your next engineering job at DEME. Apply today for exciting career opportunities. Explore engineering jobs at DEME. Apply now and start your engineering career today!

Shape the future of sustainable energy with Luminus—flexible roles, meaningful projects. Achieve true work-life balance at Luminus, where your impact in renewable energy matters. Zero carbon footprint. Jobs with impact. Improve your skills. Types: Data & Analytics, IT Architecture & Cloud.

CloudGeometry

15/05/2025

We are looking for a Senior Data Engineer/Architect with extensive hands-on experience in the Databricks ecosystem and exceptional communication skills to join our flagship project: a cutting-edge Data Platform for the life sciences industry. This platform supports industry leaders such as Pfizer, Moderna, and Novartis in developing innovative RNA-based solutions, leveraging data-driven research, cloud computing, and advanced AI capabilities. This role offers a unique opportunity for an experienced data engineer to take the next step in their career, playing a key technical leadership role in the rapidly evolving world of AI-driven solutions.

Responsibilities

1. Design, develop, and optimize data pipelines and workflows within the Databricks platform.
2. Lead architecture discussions with engineering, product managers, and data scientists to implement advanced analytics solutions that drive business insights.
3. Build, optimize, and fine-tune Databricks workflows to improve performance and reliability.
4. Work closely with data scientists and analysts to ensure data quality.
5. Ensure the integrity, accuracy, and security of data across all processing stages.
6. Implement data ingestion from various sources into Databricks, ensuring data quality and reliability.
7. Participate in daily Scrum ceremonies and collaborate with team members in the US and Europe, with required online presence from 9 AM to 5 PM EST (for candidates in Europe until 1 PM EST).

Qualifications

1. Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred.
2. 8+ years of experience in software development, preferably in data engineering, data warehousing, or data analytics teams.
3. 5+ years of experience with the DataBricks ecosystem.
4. Expertise in Python and TypeScript.
5. Hands-on experience with Lake House architecture.
6. Experience with Spark/Glue and Delta tables/Iceberg.
7. Proven ability to design and implement complex, scalable data pipelines/ETL processes using Databricks.
8. Skilled in cloud-based data storage and processing technologies, especially AWS services such as S3, Step Functions, Lambda, and Airflow.
9. Familiar with CI/CD practices, version control (Git), automated testing, and Agile environments.
10. Experience working in US-led high-tech companies and startups.

Nice to Have

1. DataBricks certifications.
2. AWS or Azure DevOps or SA certifications.
3. Knowledge of basic DevOps and MLOps principles.
4. Experience working with Data Scientists and ML Developers.
5. Management and lead developer roles in technology services companies.