Nearshore Sector | Databricks Data Engineer

Devoteam; Lisboa; Presencial; EUR 45 000 - 65 000; Tempo integral

Resumo da oferta

A technology consulting firm in Lisbon is seeking a Databricks Data Engineer to design and maintain data solutions on the Databricks platform. The ideal candidate has over 3 years of experience in data engineering, strong programming skills in Python or Scala, and proficiency with cloud platforms like Azure and AWS. This role emphasizes collaboration with different stakeholders and offers opportunities for professional growth.

Qualificações

• 3+ years of experience as a Data Engineer with expertise in Databricks.
• Hands-on experience with a major cloud platform (Azure, AWS, or GCP).
• Strong analytical and problem-solving abilities.

Responsabilidades

• Design, develop, and deploy scalable data pipelines using Databricks.
• Utilize the full capabilities of the Databricks platform.
• Implement data quality checks and validation procedures.

Conhecimentos

Data Pipeline Engineering

Databricks Ecosystem Expertise

Cloud Integration

Data Quality Assurance

Strong programming skills in Python or Scala

Formação académica

University Degree in Computer Science or specialized master's in Machine Learning or AI

Ferramentas

Databricks

Azure

AWS

GCP

Descrição da oferta de emprego

Job Description

We are seeking a talented and passionate Databricks Data Engineer to join our expanding Data & AI team Databricks Partner in our Nearshore Team, at Devoteam Portugal. In this role, you will be a crucial player in designing, building, and maintaining cutting-edge data solutions on the Databricks platform. You will have the opportunity to work with a variety of clients across different industries, collaborating with both internal teams and client stakeholders to deliver high-quality data pipelines and enable data-driven decision-making:

• Data Pipeline Engineering: Design, develop, and deploy robust and scalable data pipelines using Databricks, incorporating data extraction from diverse sources (databases, APIs, streaming platforms), transformation and cleansing using Spark, and loading into target systems (data lakes, data warehouses, etc.).
• Databricks Ecosystem Expertise: Utilize the full capabilities of the Databricks platform, including Databricks SQL, Delta Lake, Databricks Runtime, and Databricks Workflows, to orchestrate complex data workflows and ensure data quality and pipeline reliability.
• Cloud Integration: Seamlessly integrate Databricks with cloud services (Azure, AWS, or GCP) for storage, compute, and security, leveraging services like Azure Data Lake Storage, AWS S3, Google Cloud Storage, and more.
• Data Architecture and Modeling: Contribute to the design and implementation of robust data models and architectures for data lakes and data warehouses, ensuring data consistency, integrity, and accessibility for various use cases.
• Data Quality Assurance: Implement rigorous data quality checks and validation procedures throughout the data pipeline to maintain high accuracy and reliability. Adhere to data governance policies and best practices.

Qualifications

• Hold a University Degree in Computer Science, (Business) Engineering, or a specialized master's in Machine Learning or AI.
• At least 3+ years of experience as a Data Engineer with demonstrated expertise in Databricks.
• Hands-on experience with at least one major cloud platform (Azure, AWS, or GCP), with a good understanding of cloud data services and infrastructure.
• Strong programming skills in Python or Scala, with experience in data manipulation libraries (e.g., PySpark, Spark SQL).
• Solid understanding of data warehousing principles, ETL processes, data modeling techniques, and database systems.
• Advanced SQL skills for data querying, transformation, and analysis.
• Excellent communication and interpersonal skills, with the ability to effectively collaborate with technical and non-technical stakeholders.
• Strong analytical and problem-solving abilities, with a proactive approach to identifying and resolving data-related challenges.
• Fluent in English.

Additional Information

The Devoteam Group works for equal opportunities, promoting its employees based on merit and actively fights against all forms of discrimination. We are convinced that diversity contributes to the creativity, dynamism and excellence of our organization. All of our vacancies are open to people with disabilities.