Senior Databricks Engineer - AI & Data Platform

L584 VTV PORTUGAL, UNIPESSOAL LDA; Lisboa; Presencial; EUR 40 000 - 70 000; Tempo integral

Resumo da oferta

L584 VTV PORTUGAL, UNIPESSOAL LDA in Lisboa is looking for a skilled data engineer to support AI application development using Azure Databricks. With a commitment to innovation and collaboration, you will design and maintain data platforms and produce scalable technical solutions.

The ideal candidate has over 5 years of experience in data engineering, strong Python and SQL skills, and familiarity with AI workflows, particularly in data pipeline optimization and security management.

Qualificações

• 5+ years in data engineering or platform engineering roles.
• 3+ years hands-on experience with Azure Databricks.
• Strong proficiency in Python and SQL.
• Experience in managing workspace-level security and access.
• Experience building production data pipelines.
• Familiarity with GenAI/LLM application patterns is desirable.
• Strong communication skills for technical presentations.

Responsabilidades

• Develop and deploy AI-powered applications and workflows.
• Design and maintain data platform foundations.
• Optimize production data pipelines for analytics.
• Evaluate requirements for data ingestion and storage.
• Establish coding standards and best practices.
• Collaborate with cross-functional teams for solutions.

Conhecimentos

Data engineering

Python

SQL

Azure Databricks

Data pipeline development

MosaicAI

Communication skills

Ferramentas

Databricks

Unity Catalog

DeltaLake

SAP

Descrição da oferta de emprego

Overview

Vantiva (Euronext Paris: VANTI) is a global technology leader in connectivity technologies. With over 130 years of experience, we deliver solutions that connect what matters most, redefining connectivity with industry‑leading broadband, video, and IoT‑driven smart systems. We combine a customer‑focused approach with decades of software development, electronics hardware design, and supply chain expertise to deliver high‑quality solutions at scale. Our commitment to sustainability and responsible business practices earned us multiple Gold and Platinum Medals from EcoVadis, placing us among the top 2% of organizations in our category worldwide. Headquartered in Paris with major offices in Australia, Brazil, China, India, South Korea, the United Kingdom, and the United States, we serve a diverse global customer base. The acquisition of CommScope’s Home Networks business in January2024 further bolstered our commitment to innovation.

Key Responsibilities

• AI Application Enablement: Support the development and deployment of AI‑powered applications, including LLM/GenAI workflows, RAG pipelines, and integration with Mosaic AI tooling.


• Data Platform Foundation: Design, build, and maintain the core Databricks Lakehouse platform, including workspace configuration, Unity Catalog setup, cluster policies, security, and governance.


• Data Engineering: Develop and optimize production‑grade data pipelines, delivering clean, reliable datasets for analytics and AI workloads.


• Solution Architecture: Evaluate requirements, produce reference architectures, and recommend patterns for data ingestion, transformation, storage, and consumption that align with AI use cases.


• Standards & Best Practices: Establish engineering standards for code quality, CI/CD (Databricks Asset Bundles, Repos), testing, monitoring, and cost optimization.


• Collaboration: Work cross‑functionally with data analysts, cloud/infra teams, and business units to translate requirements into scalable technical solutions.

Required Qualifications

• 5+ years in data engineering or platform engineering roles.


• 3+ years hands‑on experience with Azure Databricks (workspace administration, DeltaLake, Spark, Unity Catalog).


• Strong proficiency in Python and SQL.


• Experience implementing and managing workspace‑level security, including network security (PrivateLink, VNet injection, IP access lists), identity and access management, cluster policies, secrets management, and RBAC at workspace, catalog, and object levels.


• Experience building and operating production data pipelines at scale.


• Experience with MosaicAI (AI Playground, AI Functions, Agent Framework).


• Familiarity with GenAI/LLM application patterns (RAG, prompt engineering, vector databases) is highly desirable.


• Strong communication skills, ability to produce architecture diagrams, design documents, and present trade‑offs to technical and non‑technical audiences.

Nice to Have

• Databricks certifications (Data Engineer Associate/Professional, GenAI Engineer Associate).


• Experience with Unity Catalog governance at enterprise scale (metastore design, data access policies, lineage, audit logging).


• Hands‑on experience with SAP integration, extracting data from S/4HANA or BW/4HANA into a Lakehouse platform.


• Experience leading or participating in data migration from SAP BW/4HANA to Databricks, including InfoProvider mapping, transformation logic re‑implementation, data validation, and cutover planning.


• Exposure to AI/ML workflows: model training infrastructure, feature stores, MLflow, or equivalent MLOps tooling, and ability to prepare SAP‑sourced datasets for downstream ML consumption.

What You’ll Be Part Of

You will join a growing portfolio of AI initiatives ranging from predictive analytics to generative AI applications. You will help migrate legacy systems to the Databricks Lakehouse platform that will serve as the organization’s single source of truth. You will work in a collaborative, engineering‑driven culture that values automation, quality, and continuous improvement.

Commitment to Diversity and Inclusion

We are proactive in supporting equality and maintaining an inclusive work environment, developing, and enhancing career opportunities for all. Vantiva is committed to ensuring that all its employees receive a fair wage that is sufficient to live with dignity. If you require a reasonable accommodation at any step of the application process, please let us know.