Senior Databricks Data Engineer – Lisbon (Hybrid)

Sysmatch; Lisboa; Híbrido; EUR 35 000 - 45 000; Tempo integral

Resumo da oferta

Sysmatch in Lisbon is seeking a Senior Databricks Data Engineer to design, build, and maintain scalable data pipelines and data products in a hybrid work setup. You will collaborate with Data Architects, Product Owners, and governance teams to deliver enterprise-grade solutions.

You will use Databricks, Python, PySpark, and SQL to implement Delta Lake architectures, Medallion patterns, and Unity Catalog with strong emphasis on data quality, governance, and cost efficiency.

Qualificações

• 5+ years of professional Data Engineering experience.
• 3+ years of hands-on Databricks experience.
• Strong Python and PySpark development skills.
• Advanced SQL, including query and performance optimization.
• Strong hands-on experience with Delta Lake.
• Experience implementing Medallion Architecture.
• Hands-on experience with Databricks Unity Catalog.
• Strong experience designing and building scalable ETL/ELT pipelines.
• Solid understanding of data modelling and data warehousing concepts.
• Experience working with Git and CI/CD/DevOps practices.
• Experience working with cloud-based data platforms, preferably AWS.
• Experience working within Agile delivery teams.
• Ability to independently design and implement Data Engineering solutions.

Responsabilidades

• Design, build, and maintain scalable data pipelines using Databricks.
• Develop and optimize ETL/ELT processes for large-scale enterprise data workloads.
• Design and implement data models and data products following modern architectural principles.
• Apply Medallion Architecture patterns across data solutions.
• Work with structured, semi-structured, and streaming data sources.
• Develop data processing solutions using Python, PySpark, and SQL.
• Design and optimize solutions using Delta Lake.
• Implement data governance and access management using Databricks Unity Catalog.
• Support Data Quality, metadata management, lineage, and governance initiatives.
• Optimize data processing for performance, scalability, reliability, and cost efficiency.
• Contribute to automated testing, CI/CD, and deployment processes.
• Collaborate closely with Data Architects, Product Owners, Data Governance teams, and business stakeholders.
• Contribute to the design and evolution of enterprise data products.

Conhecimentos

Databricks

Python

PySpark

SQL

Delta Lake

Medallion Architecture

Unity Catalog

ETL/ELT pipelines

Data modelling

Data warehousing

Git & CI/CD

AWS

Agile

Descrição da oferta de emprego

Sysmatch is an IT Consultancy and Outsourcing Company with a wide range of technological skills, resulting from the accumulation of know-how and the experience of our consultants, allowing the implementation of solutions adapted to the needs of our clients, privileging their robustness and scalability.
More than two decades in the IT market, we present ourselves as a flexible, adaptable, innovative and forward-looking Company.
At the moment we are looking for the following candidate:

Senior Databricks Data Engineer – Lisbon (Hybrid) (M/F)

Key Responsibilities

• Design, build, and maintain scalable data pipelines using **Databricks**.
• Develop and optimize ETL/ELT processes for large-scale enterprise data workloads.
• Design and implement data models and data products following modern architectural principles.
• Apply Medallion Architecture patterns across data solutions.
• Work with structured, semi-structured, and streaming data sources.
• Develop data processing solutions using Python, PySpark, and SQL.
• Design and optimize solutions using Delta Lake.
• Implement data governance and access management using Databricks Unity Catalog.
• Support Data Quality, metadata management, lineage, and governance initiatives.
• Optimize data processing for performance, scalability, reliability, and cost efficiency.
• Contribute to automated testing, CI/CD, and deployment processes.
• Collaborate closely with Data Architects, Product Owners, Data Governance teams, and business stakeholders.
• Contribute to the design and evolution of enterprise data products.

Required Skills & Experience

• **5+ years of professional Data Engineering experience.**
• **3+ years of hands-on Databricks experience.**
• Strong **Python and PySpark** development skills.
• Advanced **SQL**, including query and performance optimization.
• Strong hands-on experience with **Delta Lake**.
• Experience implementing **Medallion Architecture**.
• Hands-on experience with **Databricks Unity Catalog**.
• Strong experience designing and building scalable **ETL/ELT pipelines**.
• Solid understanding of **data modelling and data warehousing concepts**.
• Experience working with large-scale enterprise datasets.
• Experience with **Git and CI/CD/DevOps practices**.
• Experience working with cloud-based data platforms, preferably **AWS**.
• Experience working within **Agile delivery teams**.
• Ability to independently design and implement Data Engineering solutions.

Strong Advantage

• Experience with **Databricks Workflows**.
• Experience with **Databricks Asset Bundles**.
• Practical experience with **Data Products and Data Mesh** architectures.
• Experience implementing enterprise **Data Governance** solutions.
• Data Quality and observability experience.

Nice to Have

• Data Governance platforms and metadata management solutions.
• Data Quality frameworks and monitoring tools.
• Data lineage solutions.
• AI, Machine Learning, or Generative AI exposure.
• REST API integration experience.
• Experience working in regulated or complex enterprise environments.

Salary Range: €35k - €45k gross/year + Other Components/Benefits.

Recruitment, selection and integration process

The first contacts will be made by phone call or video call. We are prepared to ensure the monitoring and integration of new employees and support them in this new phase of their professional career.