Senior Databricks Data Engineer – Lisbon (Hybrid) (M/F)

SysMatch; Lisboa; Híbrido; EUR 35 000 - 45 000; Tempo integral

Resumo da oferta

SysMatch is hiring a Senior Databricks Data Engineer for Lisbon in a hybrid role. You will design, build and maintain scalable data pipelines with Databricks, and optimize ETL/ELT processes for large enterprise workloads.

Design data models, apply Medallion Architecture, and implement Delta Lake and Unity Catalog. You will collaborate with Data Architects, Product Owners and stakeholders in an Agile setting, using Python, PySpark and SQL daily.

Qualificações

• 5+ years of professional Data Engineering experience.
• 3+ years of hands-on Databricks experience.
• Strong Python and PySpark development skills.
• Advanced SQL, including query and performance optimization.
• Strong hands-on experience with Delta Lake.
• Experience implementing Medallion Architecture.
• Hands-on experience with Databricks Unity Catalog.
• Strong experience designing and building scalable ETL/ELT pipelines.
• Solid understanding of data modelling and data warehousing concepts.
• Experience working with large-scale enterprise datasets.
• Experience with Git and CI/CD/DevOps practices.
• Experience working with cloud-based data platforms, preferably AWS.
• Experience working within Agile delivery teams.

Responsabilidades

• Design, build, and maintain scalable data pipelines using Databricks.
• Develop and optimize ETL/ELT processes for large-scale enterprise data workloads.
• Design and implement data models and data products following modern architectural principles.
• Apply Medallion Architecture patterns across data solutions.
• Work with structured, semi-structured, and streaming data sources.
• Develop data processing solutions using Python, PySpark, and SQL.
• Design and optimize solutions using Delta Lake.
• Implement data governance and access management using Databricks Unity Catalog.
• Support Data Quality, metadata management, lineage, and governance initiatives.
• Collaborate closely with Data Architects, Product Owners, Data Governance teams, and business stakeholders.
• Contribute to automated testing, CI/CD, and deployment processes.

Conhecimentos

Databricks

Python

PySpark

SQL

Delta Lake

Medallion Architecture

Data Modeling

CI/CD

Git

AWS

Agile

Ferramentas

Databricks Unity Catalog

Databricks Workflows

Descrição da oferta de emprego

### Senior Databricks Data Engineer – Lisbon (Hybrid) (M/F)Sysmatch is an IT Consultancy and Outsourcing Company with a wide range of technological skills, resulting from the accumulation of know-how and the experience of our consultants, allowing the implementation of solutions adapted to the needs of our clients, privileging their robustness and scalability.More than two decades in the IT market, we present ourselves as a flexible, adaptable, innovative and forward-looking Company.At the moment we are looking for the following candidate:Senior Databricks Data Engineer – Lisbon (Hybrid) (M/F)Key Responsibilities\\* Design, build, and maintain scalable data pipelines using \\*\\*Databricks\\*\\*.\\* Develop and optimize \\*\\*ETL/ELT processes\\*\\* for large-scale enterprise data workloads.\\* Design and implement data models and data products following modern architectural principles.\\* Apply \\*\\*Medallion Architecture\\*\\* patterns across data solutions.\\* Work with structured, semi-structured, and streaming data sources.\\* Develop data processing solutions using \\*\\*Python, PySpark, and SQL\\*\\*.\\* Design and optimize solutions using \\*\\*Delta Lake\\*\\*.\\* Implement data governance and access management using \\*\\*Databricks Unity Catalog\\*\\*.\\* Support Data Quality, metadata management, lineage, and governance initiatives.\\* Optimize data processing for performance, scalability, reliability, and cost efficiency.\\* Contribute to automated testing, CI/CD, and deployment processes.\\* Collaborate closely with Data Architects, Product Owners, Data Governance teams, and business stakeholders.\\* Contribute to the design and evolution of enterprise data products.Required Skills & Experience\\* \\*\\*5+ years of professional Data Engineering experience.\\*\\*\\* \\*\\*3+ years of hands-on Databricks experience.\\*\\*\\* Strong \\*\\*Python and PySpark\\*\\* development skills.\\* Advanced \\*\\*SQL\\*\\*, including query and performance optimization.\\* Strong hands-on experience with \\*\\*Delta Lake\\*\\*.\\* Experience implementing \\*\\*Medallion Architecture\\*\\*.\\* Hands-on experience with \\*\\*Databricks Unity Catalog\\*\\*.\\* Strong experience designing and building scalable \\*\\*ETL/ELT pipelines\\*\\*.\\* Solid understanding of \\*\\*data modelling and data warehousing concepts\\*\\*.\\* Experience working with large-scale enterprise datasets.\\* Experience with \\*\\*Git and CI/CD/DevOps practices\\*\\*.\\* Experience working with cloud-based data platforms, preferably \\*\\*AWS\\*\\*.\\* Experience working within \\*\\*Agile delivery teams\\*\\*.\\* Ability to independently design and implement Data Engineering solutions.## Strong Advantage\\* Experience with \\*\\*Databricks Workflows\\*\\*.\\* Experience with \\*\\*Databricks Asset Bundles\\*\\*.\\* Practical experience with \\*\\*Data Products and Data Mesh\\*\\* architectures.\\* Experience implementing enterprise Data Governance solutions.\\* Data Quality and observability experience.## Nice to Have\\* Data Governance platforms and metadata management solutions.\\* Data Quality frameworks and monitoring tools.\\* Data lineage solutions.\\* AI, Machine Learning, or Generative AI exposure.\\* REST API integration experience.\\* Experience working in regulated or complex enterprise environments.Salary Range: €35k - €45k gross/year + Other Components/Benefits.Recruitment, selection and integration process:The first contacts will be made by phone call or video call.We are prepared to ensure the monitoring and integration of new employees and support them in this new phase of their professional career.To respond to this opportunity, please send us your CV, indicating the reference RS/SDE/C to the following email address [e-mail]