Data Engineer - Databricks (Performance & Security)

syffer; Leiria; Híbrido; EUR 40 000 - 70 000; Tempo integral

Resumo da oferta

A consulting company in Leiria is looking for a skilled Databricks Specialist to optimize data workloads and maintain governance. The ideal candidate has deep knowledge of Databricks and proficiency in PySpark and SQL. This role offers competitive pay based on experience, health insurance, and a flexible hybrid work model.

Serviços

Wage according to experience

Remote Work options

Health insurance allocation

Work equipment provided

Qualificações

• Deep knowledge of Databricks configuration, optimization, and administration.
• Experience with PySpark, SQL, and data performance tuning.
• Familiarity with Unity Catalog, DABs, and Terraform integration.

Responsabilidades

• Optimize Databricks performance across jobs, clusters, and data workloads.
• Implement and maintain governance, security, and permissions via Unity Catalog and Terraform.
• Monitor cluster and job performance using Grafana and internal observability tools.

Conhecimentos

Databricks configuration and optimization

PySpark

SQL

Data performance tuning

Understanding of data governance

Fluent in Portuguese

Fluent in English

Ferramentas

Databricks

Terraform

Airflow

Grafana

Kafka

Descrição da oferta de emprego

Syffer is an all-inclusive consulting company focused on talent, tech and innovation. We exist to elevate companies and humans all around the world, making change, from the inside to the outside.

We believe that technology + human kindness positively impacts every community around the world. Our approach is simple, we see a world without borders, and believe in equal opportunities. We are guided by our core principles of spreading positivity, good energy and promote equality and care for others.

Our hiring process is unique! People are selected by their value, education, talent and personality. We dont present ethnicity, religion, national origin, age, gender, sexual orientation or identity.

What You'll do:

• Optimize Databricks performance across jobs, clusters, and data workloads;
• Implement and maintain governance, security, and permissions via Unity Catalog and Terraform;
• Contribute to Databricks feature enablement Delta Live Tables, Databricks Asset Bundles (DABs), and cost efficiency improvements;
• Monitor cluster and job performance using Grafana and internal observability tools;
• Support data engineers in troubleshooting and fine-tuning jobs and SQL queries;
• Explore and progressively implement streaming features (Structured Streaming, Autoloader, Kafka integration);
• Manage Databricks environments integrated with Airflow on AKS, Azure ADLS Gen2, and Terraform;
• Collaborate with DataOps to ensure high availability, logging, and monitoring coverage;
• Implement performance and security benchmarks for production workloads;
• Utilize stack components including Talend, Azure ADLS Gen2, Azure Databricks, Airflow (AKS), Terraform, SQL, Python, PySpark, Grafana, Kafka (Aiven), GitLab, Databricks Asset Bundles (DABs), and DQX (Data Quality Framework);
• Hybrid Work Model;

What You Are:

• Deep knowledge of Databricks configuration, optimization, and administration;
• Experience with PySpark, SQL, and data performance tuning;
• Familiarity with Unity Catalog, DABs, and Terraform integration;
• Strong understanding of data governance, cost optimization, and cluster policies;
• Basic to intermediate experience with Kafka or interest in learning streaming processing;
• Fluent in Portuguese and English;

What you'll get:

• Wage according to candidate's professional experience;
• Remote Work whenever possible;
• Allocation of health insurance from the beginning of the employment;
• Delivery of work equipment adjusted to the performance of functions;
• And others.

Work together with expert teams on projects of large magnitude and intensity, long term together with our clients, all leaders in their industries.

Are you ready to step into a diverse and inclusive world with us?

Together we will promote uniquess!