Senior Data Engineer

Frontiers Software SL; Madrid; A distancia; EUR 60.000 - 100.000; Jornada completa

Descripción de la vacante

An innovative organization is seeking a Senior Data Engineer to lead the design and development of data pipelines and systems that drive data-driven decision-making. In this pivotal role, you will ensure the integrity and accessibility of data for AI models, collaborating with engineering teams to gather requirements and deliver impactful solutions. With a focus on continuous learning and a commitment to diversity, this role offers a unique opportunity to contribute to a mission-driven company that values your expertise and fosters a supportive work environment. If you are passionate about data engineering and eager to make a difference, this role is for you.

Servicios

Remote-first organization

4 additional wellbeing days

Access to learning platforms

Free online yoga classes

Employee assistance plan

3 days dedicated to volunteering

Formación

• Lead design and development of scalable data pipelines and systems.
• Oversee data quality monitoring and troubleshooting workflows.
• Collaborate with teams to gather requirements and deliver solutions.

Responsabilidades

• Design, develop, and maintain robust data pipelines and systems.
• Establish best practices and technical standards for data engineering.
• Mentor junior team members and provide guidance.

Conocimientos

Data Engineering Practices

Data Governance

SQL

Python

Cloud Platforms

CI/CD Pipelines

Airflow

Terraform

Big Data

Machine Learning

Herramientas

Apache Hive

Hadoop

Redshift

Spark

AWS

Apache Pig

NoSQL

Kafka

Scala

Descripción del empleo

At Frontiers, we are on a mission to make all science open, enabling the research community to develop solutions we need to live healthy lives on a healthy planet.

Join the publishing revolution.

Follow the links below to learn more about our work.

We can offer you

We provide a range of benefits across our locations, including but not limited to:

• We are a remote-first organization, and most roles are work-from-home.
• 4 additional wellbeing days beyond the existing annual leave allowance.
• Access to learning platforms and dedicated learning & development time.
• A range of wellbeing initiatives, including free online yoga classes and an employee assistance plan.
• Employees can dedicate three days each year to volunteer.

The role:

As a Senior Data Engineer, you will lead the design, development, and maintenance of data pipelines, platforms, and tools to drive data-driven decision-making across the organization. The role is based in the Artificial Intelligence team, playing a key role in ensuring the integrity and accessibility of data for AI models, both in experiments and production.

Responsibilities:

• Lead the design, development, and maintenance of robust and scalable complex data pipelines and systems.
• Oversee the monitoring of data quality and the implementation and troubleshooting of data processing workflows.
• Establish and maintain best practices and technical standards.
• Provide significant input on the use and optimization of platform team tools and infrastructure.
• Mentor junior team members and provide guidance.
• Collaborate with engineering teams to gather requirements and deliver solutions.

Skills:

• Deep understanding of data engineering practices, data governance, and data quality standards.
• Proficiency in data engineering processes and understanding of business requirements.
• Knowledge of industry trends and best practices, with the ability to propose technical solutions to business problems.
• Experience with source control systems, CI/CD pipelines, and DevOps tools.
• Expertise in relational/non-relational databases and programming languages (SQL and Python).
• Mastery of at least one cloud platform, data system, and other modern data engineering tools.
• Proficiency with orchestration tools (e.g., Airflow) and data modeling techniques.
• Experience with real-time streaming processing systems.
• Experience with cloud infrastructure provisioning and Infrastructure as Code (e.g., Terraform).
• Strong problem-solving abilities and analytical mindset.
• Leadership and mentoring skills, including negotiating with other teams.
• Ability to evaluate solutions considering dependencies, code complexity, costs, security, and coordination overhead.
• Strong documentation skills for complex systems.
• Knowledge of vector/graph databases and Machine Learning is a plus.
• Excellent communication skills to manage multiple priorities and a commitment to continuous learning.

Apply now

Please click the apply button on this page and submit your application in English. Inform us if you require any special accommodations to participate fully in our recruitment process. You can contact us at [contact info] for additional support.

Equal opportunity statement

Frontiers actively embraces diversity and is a safe, welcoming workplace. Recruitment is free from discrimination based on race, nationality, ethnicity, age, religion, disability, sex, gender identity, or sexual orientation. With employees from over 50 nations, our diversity creates vibrant teams and challenges us to appreciate multiple perspectives.

Required Experience

Senior IC

Key Skills

Apache Hive, S3, Hadoop, Redshift, Spark, AWS, Apache Pig, NoSQL, Big Data, Data Warehouse, Kafka, Scala