Lead Databricks Lakehouse Engineer (Remote)

Cognizant; Comunidad de Madrid; Presencial; EUR 60.000 - 90.000; Jornada completa

Descripción de la vacante

Cognizant seeks a highly skilled Databricks Developer to design and build enterprise data pipelines powering analytics, reporting, and AI initiatives for a leading energy-sector company. The role is fully remote, offering hands-on work with Lakehouse technology and high-impact data projects.

You will own data pipelines, optimize Spark workloads, and collaborate with data scientists to enable BI dashboards and GenAI applications in a scalable, cloud-based environment.

Formación

• 5+ years of dedicated data engineering experience in an enterprise environment.
• Expert-level proficiency in Python and SQL.
• Extensive hands-on experience with Databricks, Apache Spark and Delta Lake.
• Strong understanding of distributed systems, big data architecture and data modeling techniques (e.g., Kimball, Data Vault).
• Deep familiarity with cloud-native data services (AWS, Azure or GCP), specifically cloud storage (S3/ADLS) and compute provisioning.
• Proven experience with version control (Git), CI/CD methodologies and agile software development life cycles.

Responsabilidades

• Design, build and maintain scalable ETL/ELT pipelines using Python, SQL and Apache Spark within the Databricks environment.
• Implement and manage a robust Medallion architecture (Bronze, Silver, Gold) to process data from diverse sources.
• Develop and maintain the Gold semantic layer for high-performance BI consumption (e.g., Power BI).
• Optimize Databricks workloads, cluster configurations and Spark queries for performance and cost efficiency.
• Work with Delta Lake and Apache Iceberg to ensure ACID compliance and time travel capabilities.
• Migrate legacy workloads from traditional data warehouses (e.g., AWS Redshift) into the Lakehouse.

Conocimientos

Python

SQL

Databricks

Apache Spark

Delta Lake

Cloud platforms

Git

CI/CD

Data modeling

Educación

Bachelor's degree or higher in CS/IT/DS

Herramientas

Terraform

GitHub Actions

Azure DevOps

Streamlit

Descripción del empleo

Cognizant seeks a highly skilled Databricks Developer to design and build enterprise data pipelines powering analytics, reporting, and AI initiatives for a leading energy-sector company. The role is fully remote, offering hands-on work with Lakehouse technology and high-impact data projects.

You will own data pipelines, optimize Spark workloads, and collaborate with data scientists to enable BI dashboards and GenAI applications in a scalable, cloud-based environment.