Big Data Architect

Solera Holdings, LLC.; Madrid; Presencial; EUR 65.000 - 90.000; Jornada completa

Descripción de la vacante

An international IT services company is seeking a skilled Big Data Architect in Madrid. The role involves designing scalable data architectures, leading DaaS API solutions, and optimizing Spark job performance. Candidates should have at least 7 years of experience in big data architecture with strong knowledge of Cloudera and Spark. This full-time position emphasizes sustainability and innovation in data practices.

Formación

• At least 7 years of experience in big data architecture, preferably Cloudera and Spark.
• Excellent problem-solving, analytical, and communication skills in English.
• Understanding of data governance, privacy, and security practices.

Responsabilidades

• Design and implement scalable data architectures for processing pipelines.
• Lead development of API solutions for Data as a Service (DaaS).
• Establish best practices for data processes to ensure quality and availability.
• Collaborate with teams to gather requirements and translate them into specifications.
• Optimize Spark job performance for large datasets.

Conocimientos

Cloudera

Spark

API architectures

DaaS solutions

Scala

Python

Java

Hadoop

Kafka

SQL

Educación

Bachelor’s or Master’s in Computer Science, Data Science

Herramientas

HDFS

Hive

Impala

HBase

AWS

Azure

Docker

Kubernetes

Descripción del empleo

Join to apply for the

Big Data Architect

role at

Solera Holdings, LLC.

2 weeks ago Be among the first 25 applicants

Position Purpose :

• We are seeking a highly skilled and experienced Big Data Architect to join our international team. You will play a pivotal role in shaping our Big Data environments and projects, including the Global Data Lake, while enhancing our Sustainable Estimatics offerings. Sustainable Estimatics is a leading suite within the company, recognized for its substantial impact on the industry. With our innovative and certified algorithms, we provide our customers with significant cost savings by minimizing waste and optimizing resource usage. By embedding sustainability principles into our Estimatics practices, we actively contribute to the industry's collective effort to reduce environmental impact. Our commitment to sustainability goes beyond individual projects;

we aim to drive industry-wide innovation through the continuous development of new technologies and practices that create a positive ripple effect for both the environment and society.

Responsibilities :

Design and implement scalable and efficient data architectures supporting data processing pipelines using Cloudera, Spark, and other relevant technologies.

Lead the development of scalable API solutions to facilitate Data as a Service (DaaS), providing seamless access to data for both external and internal customers.

Establish best practices for data ingestion, transformation, and storage processes to ensure data quality, integrity, and availability across international locations.

Collaborate with cross-functional teams to gather business requirements and translate them into architectural specifications.

Optimize data workflows and Spark job performance to meet latency and throughput requirements for large datasets.

Troubleshoot and tune performance of Cloud or On-premises infrastructure to identify bottlenecks and improve resource utilization.

Utilize tools like New Relic for performance monitoring and Graylog for log analysis.

Work with data scientists and analysts to ensure reliable data sets for analytics and machine learning.

Implement data governance and ensure compliance with data privacy and security regulations globally.

Stay updated on emerging technologies and propose innovative solutions to enhance data processing capabilities.

Provide technical leadership, mentorship, and guidance to engineering teams, fostering collaboration and innovation.

Participate in agile practices, including sprint planning, architecture reviews, and CI / CD processes.

Requirements :

Bachelor’s or Master’s in Computer Science, Data Science, or related fields.

At least 7 years of experience in big data architecture, with preference for Cloudera and Spark expertise.

Strong understanding of API architectures and experience in DaaS solutions.

Proficiency in Scala, Python, or Java for complex data solutions.

Deep knowledge of distributed computing frameworks like Hadoop and Spark.

Experience with Cloudera tools such as HDFS, Hive, Impala, HBase.

Knowledge of data modeling, schema design, partitioning, and indexing.

Experience with SQL, NoSQL, ETL, and data warehousing.

Proven ability to design, implement, and optimize Spark data pipelines.

Familiarity with data ingestion tools like Kafka, Flume, Sqoop, Nifi.

Cloud platform experience (AWS, Azure) and containerization (Docker, Kubernetes) is a plus.

Understanding of data governance, privacy, and security practices.

Excellent problem-solving, analytical, and communication skills in English.

Additional Details : Seniority level :

Mid-Senior level

Employment type : Full-time

Job function : Engineering and IT

Industries :

IT Services and Consulting

Referrals increase your chances of interviewing at Solera Holdings, LLC. by 2x.

Get notified about new Data Architect jobs in

Madrid, Community of Madrid, Spain .

J-18808-Ljbffr