Big data Engineer

KCS iT; Lisboa; Híbrido; EUR 60 000 - 90 000; Tempo integral

Resumo da oferta

KCS IT, based in Lisbon, is seeking an Amazing: Big Data Engineer with 4+ years of experience to design, develop and maintain scalable data platforms for processing large volumes of data. The role involves collaborating with data scientists, analysts and engineers across diverse projects.

You will work with PySpark, Apache Spark, Hadoop ecosystems and Hive, building robust data pipelines, models and ETL/ELT processes to support analytics and reporting needs. Hybrid/remote options are available.

Qualificações

Responsabilidades

• Design and maintain scalable Big Data solutions.
• Develop and optimize batch data pipelines with PySpark.
• Implement data solutions in Hadoop ecosystems for performance and reliability.
• Create data models and ETL/ELT processes for analytics.
• Use Hive for warehousing and large-scale queries.
• Collaborate with data scientists and engineers to deliver end-to-end solutions.
• Ensure data quality, governance, security and compliance.
• Document technical solutions and support operations.

Conhecimentos

PySpark

Apache Spark

Hadoop ecosystems

Hive

ETL/ELT

Data modeling

Data pipelines

Data governance

Data quality

Cloud data platforms

Documentation

Descrição da oferta de emprego

We’re looking for the special, unique and amazing YOU!

@ KCS IT, we look for the ones that stands out, for those that always wants to be better and fight for it, and for those who has the same values that we do: dedication, energy, integrity, transparency, flexibility, trust, honesty, hard work, proactivity, team work.

At KCS we stand for equality and value diversity. We create a safe, diverse environment where opportunities are equal for everyone! We do not discriminate based on age, ethnicity, sexual orientation, gender, disability or any factor other than merit. All applications with skills for the position are welcome!

We’re looking for the special, unique and amazing YOU!

We are looking for an Amazing: Big Data Engineer

Seniority: +4 years

Location : Lisbon

Skills Needed?

• Design, develop, and maintain scalable Big Data solutions for processing large volumes of structured and unstructured data.
• Develop and optimize data pipelines using PySpark and Apache Spark for batch and large-scale data processing.
• Implement and manage data solutions within Hadoop ecosystems, ensuring performance, scalability, and reliability.
• Create and maintain data models, datasets, and ETL/ELT processes to support analytics and reporting needs.
• Utilize Hive for data warehousing, querying, and large-scale data analysis.
• Collaborate with Data Scientists, Analysts, Architects, and Software Engineers to deliver end-to-end data solutions.
• Troubleshoot and resolve data platform issues, optimizing performance and resource utilization.
• Ensure data quality, governance, security, and compliance standards are met.
• Contribute to the design and implementation of cloud-based data platforms and modern data architectures.
• Produce technical documentation and support operational activities related to data platforms.

Why should you become part of our family?

• You can develop a career that fits you: your career development is personalized, taking in consideration your needs and goals from a short to long term
• Interesting Challenges Ahead: you can work for several clients from different sectors of activity
• Free training programs: Our training and certification programs in languages, tech, behavior and business will help you to reach your full potential faster
• International projects in Benelux: you can gain international experience in Benelux and balance a new way of living with work
• Type of projects (depending on the project you might find one of this types of projects):
• In hybrid Systems: Is important to balance work with socialization, that´s why a hybrid system works for us and for you
• Full Remote projects: If you want to work while you enjoy the comfort of your home
• Full Onsite projects: if you prefere the company of your colleagues!
• Take care of your well-being: Enjoy our free nutrition, psicologist, general medicine appointments and our yoga and personal training days… all remote!

Who are we?

Founded in 2008 and based in Lisbon, KCS IT is a consulting company in the field of Information Technology and Services, focused on creating value for our clients through three main areas: Consulting, Outsourcing, Inovation and Training. Our commitment to talent development is unmistakable in the recent opening of the Porto, Leiria and Azores hubs, which aims to develop technology for the national and international market. Since 2018 we have been elected one of the 10 major companies in the “Index of Excellence”, an initiative that aims to reward organizations that invest most in the development and satisfaction of their employees.We believe transparency, trust and integrity are the pillars of any business and relationship and that is what we promise our clients and employees who join our company KCS IT.