Data Engineer

Decskill; Viseu; Presencial; EUR 42 000 - 56 000; Tempo integral

Resumo da oferta

Decskill is actively seeking a Data Engineer to design and optimize robust data pipelines, data models, and analytics-ready datasets. The role supports modern cloud-first projects across our data ecosystem, involving real-time streaming and large-scale processing in public cloud environments.

Ideal candidates bring 3+ years in data engineering, hands-on experience with AWS, Terraform, Python, SQL/NoSQL, and Spark-based tools, joining a collaborative team that values continuous learning and

Qualificações

• Minimum 3 years in data engineering.
• Experience with AWS cloud platforms.
• Strong IaC knowledge, especially Terraform.
• Proficient in Python or Java/JS/Scala.
• Solid SQL and NoSQL data stores.
• Experience with Spark/Hadoop ecosystems.

Responsabilidades

• Develop and maintain data pipelines.
• Model and optimize data structures.
• Ensure data quality and testing.
• Tackle big data processing projects.
• Implement real-time data streaming.
• Collaborate with cross-functional teams.

Conhecimentos

Data engineering

AWS

Terraform

Python

SQL

Big Data

Data modelling

NoSQL

Airflow

Data architecture

Ferramentas

Apache Spark

Hadoop

Cassandra

Apache Hive

Terraform

Descrição da oferta de emprego

Decskill, founded in 2014 as an IT Consulting Company, places paramount importance on its greatest asset: its people. Our main mission is to deliver value through knowledge and talent, and we achieve this by fostering a culture of excellence and investing in the development and well-being of our people. With over 600 dedicated professionals and offices in Lisbon, Porto, Madrid, and Luxembourg, Decskill operates across three core areas: Decskill Talent: We believe that our people are key to our success.

Through Decskill Talent, we empower our team to embrace the digital transformation challenges of our clients. We collaborate with clients to drive innovation, ensuring project success and business growth. Decskill Boost: Equipping our team with the latest tools and methodologies, we optimize Time-to-Market and deliver innovative solutions exceeding client expectations. Decskill Connect: Our team collaborates closely with clients to implement and manage IT infrastructures that generate long-term value.

At Decskill, we believe that by nurturing and empowering our people to confront the challenges of digital transformation, we create value not only for our clients but also for our entire ecosystem, fostering a digital community dedicated to growth and progress.

Responsabilities:

• Data Pipeline Development: Design and maintain robust data pipelines, ensuring seamless data flow from various sources to storage and analytical platforms;
• Data Modeling: Craft and optimize data models for efficient data retrieval and analysis, balancing performance and scalability;
• Data Quality Assurance: Implement rigorous testing and validation protocols to guarantee the accuracy and integrity of data;
• Troubleshooting and System Optimization: Proactively identify and solve data processing issues, enhancing the performance and reliability of data systems;
• Big Data Processing: Engage in large-scale data projects, utilizing frameworks like Apache Spark for efficient data handling and processing;
• Real-Time Data Streaming: Work on cutting-edge projects that process and analyze data in real-time, ensuring swift data delivery and insights.

Requirements:

• At least 3 years of experience in data engineering;
• Experience working withpublic cloud platforms, especially AWS;
• Experience with Ia C tools, such as Terraform;
• Ability to code in at least one programming language (Python, Java, Java Script, Scala) is a must;
• Experience working withand creating data-driven architectures, preferably in public cloud platforms;
• Experience using a querying language such as SQL;
• Experience with Big Data and data modelling;
• Advanced knowledge of multiple SQL/no SQL data environments and a strong. understanding of why and when to use a particular solution;
• Familiarity with tools in the Big Data ecosystem (like Hadoop, Cassandra, Apache Hive, Apache Spark, and others alike);
• Experience with orchestration tools like Airflow is a big plus.

Decskill is committed to equality and non-discrimination with all our talents. We recruit and promote talent, based on diversity and inclusion, regardless of age, gender, ethnicity, race, nationality or any other form of discrimination incompatible with the dignity of the human being.

Data Engineer Portalegre, Portuguese Republic, PT