Senior Databricks Data Engineer: PySpark Medallion

Talent To Hire Inc.; Madrid; Hybrid; EUR 70,000 - 110,000; Full time

Job summary

Talent To Hire Inc. in Madrid is seeking a hands-on Senior Databricks Data Engineer for a hybrid, six‑month contract with extensions. You will design, build and optimize scalable data pipelines using Databricks, PySpark, Python and SQL.

The role requires working with large datasets, integrating APIs, databases and files, and implementing Bronze, Silver and Gold medallion layers. You will stay hands-on with Databricks/Spark and collaborate with Data Architects and business stakeholders.

Qualifications

• Hands-on production experience with Databricks, PySpark and SQL.
• Strong data pipeline design and implementation.
• Experience with Bronze, Silver and Gold medallion architecture.
• Ingestion from APIs, databases, files and multiple sources.
• Ability to optimize large-scale distributed data processing.
• Proficiency in data quality, monitoring and governance.

Responsibilities

• Design, develop, and maintain scalable Databricks-based data pipelines.
• Build end-to-end data solutions from source ingestion through Gold datasets.
• Develop high-performance workflows using PySpark, Python, Spark and SQL.
• Integrate data from APIs, databases, files, cloud storage and external platforms.
• Design and optimize Databricks architectures for ingestion, processing and storage.
• Work with large-scale datasets and distributed data processing environments.
• Perform Spark/Databricks performance tuning to improve speed and cost efficiency.
• Build ETL/ELT workflows with data quality, monitoring and governance controls.
• Collaborate with Data Architects, Engineers and stakeholders to productionize solutions.
• Contribute to engineering standards and Databricks best practices.

Skills

Databricks

Python

Advanced SQL

Medallion Architecture

Data ingestion

Large-scale data processing

Data transformation

Performance tuning

Data quality monitoring

Cloud data workloads

Tools

AWS S3

Glue ETL

Lambda

Step Functions

ECS

CloudWatch

Snowflake

DBT

Terraform

BigQuery

Kafka

Git

CI/CD

Databricks Certified Data Engineer Associate

Job description

Talent To Hire Inc. in Madrid is seeking a hands-on Senior Databricks Data Engineer for a hybrid, six‑month contract with extensions. You will design, build and optimize scalable data pipelines using Databricks, PySpark, Python and SQL.

The role requires working with large datasets, integrating APIs, databases and files, and implementing Bronze, Silver and Gold medallion layers. You will stay hands-on with Databricks/Spark and collaborate with Data Architects and business stakeholders.