Data Engineer Databricks (Performance & Security)

Syffer; Lisboa; Presencial; EUR 45 000 - 65 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health insurance

Delivery of work equipment

Benefits Plan

Resumo da oferta

A consulting company based in Lisbon seeks a Data Engineer specialized in Databricks. The role involves optimizing performance, maintaining governance, and contributing to feature enablement. Candidates should have deep knowledge of Databricks, experience in SQL and PySpark, and fluency in Portuguese and English. Benefits include a competitive wage, health insurance, and a hybrid work model.

Qualificações

• Deep knowledge of Databricks configuration, optimization, and administration.
• Experience with PySpark, SQL, and data performance tuning.
• Strong understanding of data governance, cost optimization, and cluster policies.

Responsabilidades

• Optimize Databricks performance across jobs, clusters, and data workloads.
• Implement and maintain governance, security, and permissions via Unity Catalog and Terraform.
• Monitor cluster and job performance using Grafana and internal observability tools.

Conhecimentos

Databricks configuration

PySpark

SQL

Data performance tuning

Data governance

Cost optimization

Kafka

Fluent in Portuguese

Fluent in English

Ferramentas

Databricks

Azure ADLS Gen2

Terraform

Airflow

Grafana

Talend

Descrição da oferta de emprego

Data Engineer Databricks (Performance & Security)

Syffer is an all-inclusive consulting company focused on talent, tech and innovation. We exist to elevate companies and humans all around the world, making change, from the inside to the outside.

We believe that technology + human kindness positively impacts every community around the world. Our approach is simple, we see a world without borders, and believe in equal opportunities. We are guided by our core principles of spreading positivity, good energy and promote equality and care for others.

Our hiring process is unique! People are selected by their value, education, talent and personality. We dont present ethnicity, religion, national origin, age, gender, sexual orientation or identity.

Its time to burst the bubble, and we will do it together!

What you’ll do:

• Optimize Databricks performance across jobs, clusters, and data workloads;
• Implement and maintain governance, security, and permissions via Unity Catalog and Terraform;
• Contribute to Databricks feature enablement Delta Live Tables, Databricks Asset Bundles (DABs), and cost efficiency improvements;
• Monitor cluster and job performance using Grafana and internal observability tools;
• Support data engineers in troubleshooting and fine-tuning jobs and SQL queries;
• Explore and progressively implement streaming features (Structured Streaming, Autoloader, Kafka integration);
• Manage Databricks environments integrated with Airflow on AKS, Azure ADLS Gen2, and Terraform;
• Collaborate with DataOps to ensure high availability, logging, and monitoring coverage;
• Implement performance and security benchmarks for production workloads;
• Utilize stack components including Talend, Azure ADLS Gen2, Azure Databricks, Airflow (AKS), Terraform, SQL, Python, PySpark, Grafana, Kafka (Aiven), GitLab, Databricks Asset Bundles (DABs), and DQX (Data Quality Framework);
• Hybrid Work Model.

Who you are:

• Deep knowledge of Databricks configuration, optimization, and administration;
• Experience with PySpark, SQL, and data performance tuning;
• Familiarity with Unity Catalog, DABs, and Terraform integration;
• Strong understanding of data governance, cost optimization, and cluster policies;
• Basic to intermediate experience with Kafka or interest in learning streaming processing;
• Fluent in Portuguese and English;

What you’ll get:

• Wage according to candidate’s professional experience;
• Allocation of health insurance from the beginning of the employment;
• Delivery of work equipment adjusted to the performance of functions;
• Implementation of a hybrid work regime whenever possible;
• Benefits Plan;
• And others.

Work together with expert teams on projects of large magnitude and intensity, long term together with our clients, all leaders in their industries.

Are you ready to step into a diverse and inclusive world with us?

Send your apply for Data Engineer Databricks (Performance & Security)