Senior Data Engineer

Dsv Inc; Lisboa; Híbrido; EUR 70 000 - 110 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Permanent contract

Private medical care

English courses

Food allowance

Hybrid work

Buddy

Good public transport access

Resumo da oferta

DSV - Global Transport and Logistics is seeking a senior data engineering lead to design Lakehouse architectures with Databricks and Delta Lake. You will define reference patterns for batch and streaming workloads, implement governance, and collaborate on cost optimization across cloud environments.

The role requires expertise in the Microsoft BI stack, Python, and Power BI, with strong English communication and leadership abilities in a fast-paced, multinational setting.

Qualificações

• Expert knowledge of Microsoft’s BI stack incl. Azure SQL, Synapse, Analysis Services, ADF, Databricks, DevOps.
• Strong analytical skills with root-cause analysis ability.
• Experience with Agile methodologies and project planning.

Responsabilidades

• Design end-to-end Lakehouse architectures using Databricks and Delta Lake.
• Define reference architectures for batch, streaming, and near-real-time workloads.
• Lead data engineering teams and review designs and code.

Conhecimentos

Databricks

Spark

Delta Lake

Power BI

Python

DAX Language

Data Modeling

Azure Synapse

DevOps

English Proficiency

Formação académica

Bachelor’s or Master’s degree in Computer Sciences

Ferramentas

Terraform

ARM

CloudFormation

CI/CD tooling

Descrição da oferta de emprego

Key Responsibilities

• Design end-to-end Lakehouse architectures (Bronze/Silver/Gold) using Databricks and Delta Lake.
• Define reference architectures for batch, streaming, and near-real-time workloads.
• Select and standardize patterns for ingestion, transformation, orchestration, and serving.
• Ensure scalability, security, cost optimization, and high availability across environments.
• Build and optimize ETL/ELT pipelines using Spark (PySpark / SQL).
• Implement ingestion from APIs, event streams, files, databases, and message brokers.
• Implement incremental processing, CDC patterns, and schema evolution.
• Governance, Quality & Security
  
  
  
  • Implement Unity Catalog for data governance, lineage, and access control.
  
  
  • Define data quality rules, validation frameworks, and monitoring.
  
  
  • Enforce data contracts, naming conventions, and lifecycle management.
  
  
  
  
• Integrate Databricks with cloud services (Azure, AWS and GCP).
• Implement CI/CD for data pipelines and notebooks.
• Manage environments (dev/test/prod) and infrastructure as code.
• Collaborate on cost governance and FinOps optimization.
• Analytics & Consumption Enablement
  
  
  
  • Model curated datasets for BI, analytics, and ML use cases.
  
  
  • Support ML workflows and feature engineering where relevant.
  
  
  
  
• Act as technical lead for data engineering teams.
• Review designs and code, define best practices, and raise engineering standards.
• Mentor junior engineers and contribute to technical roadmaps.
• Influence data strategy and platform evolution.

Qualifications

• Expert knowledge of Microsoft’s BI stack, namely Azure SQL Database/Azure Synapse, Azure Analysis Services, ADF, Databricks, DevOps. Expertise with Delta Lakes, PowerBI, and Python preferred.
• Capable of systematically and rationally analyzing the root cause of problems
• Experienced with Agile methodology, and the promotion & facilitation of respective technical deliveries. Includes estimation and project plan creation for the timely delivery of milestones, analysis, and presentation of written and/or oral information, prioritization, and planning.
• Self-starter, takes responsibility for driving his/her portfolio of tasks and projects
• Willingness to challenge the status quo and to drive innovation and change, being authentic, approachable, open and honest, having a high commitment and positive attitude in the face of setbacks and obstacles, and being capable of generating creative ideas/solutions
• Comfortable in international business communication, able to communicate openly and confidently, able to listen effectively, and able to influence and convince others in a way that results in acceptance and agreement
• Ability to work with different cultures and to manage cross-functional interfaces, high priority for teamwork, capable of building strong and long-term team relationships within and across functions and geographical boundaries, and based on ethics and trust, ready to take personal accountability for achieving individual and shared goals
• Bachelor’s or Master’s degree in Computer Sciences
• 6-10 years of experience in BI projects, preferably in the Logistics industry
• Proficient Level of English (spoken and written)
• Expert knowledge in Tabular Models development and maintenance, and DAX Language.
• Good knowledge of PowerShell Scripting. (Nice to Have)
• Excellent communication and collaboration skills in English (written and oral).
• Ability to work independently and as part of a team in a fast-paced, dynamic environment.
• Strong problem-solving and analytical skills.

Technical skills

• Core
  
  
  
  • Databricks (Workflows, Delta Lake, Unity Catalog)
  
  
  • Apache Spark (PySpark / Spark SQL)
  
  
  • Lakehouse architecture patterns
  
  
  • Data modeling (dimensional, analytical, medallion)
  
  
  
  
• Azure (ADF, ADLS, Synapse) or AWS (S3, Glue, Redshift) or GCP
• Infrastructure as Code (Terraform, ARM, CloudFormation) (Nice-to-Have)
• Batch & streaming pipelines
• CDC and incremental loading
• Performance tuning & optimization
• Data quality frameworks.
• Governance & Security
  
  
  
  • Role-based access control
  
  
  • Lineage and metadata management
  
  
  • Data privacy and compliance principles
  
  
  
  
• Nice to have skills
  
  
  
  • Streaming (Kafka, Event Hubs, Kinesis)
  
  
  • ML integration (MLflow, Feature Stores)
  
  
  • Power BI / Tableau / Looker integration
  
  
  • Cost optimization & FinOps practices
  
  
  • Domain experience (Logistics, Finance, Supply Chain, Sustainability) (Preferred)
  
  
  
  

Our offer

• Permanent contract (with basic salary in 14 installments per year)
• Private medical care
• English courses - financial or co-financed by company
• Food allowance (EUR 10.20, per working day)
• Hybrid and flexible work (3 days in a week at the office)
• Buddy
• Good access to public transports

DSV - Global Transport and Logistics

Global transport and logistics - road, air, sea, rail freight and warehousing. Contact us today.