Desenvolvedor full stack

TMC; Portugal; Presencial; EUR 45 000 - 65 000; Tempo integral

Resumo da oferta

TMC is looking for a skilled Data Engineer in Portugal. The role involves designing and optimizing data processing systems, building scalable pipelines, and ensuring data delivery across the organization.

You'll collaborate with software engineering, data science, and business intelligence teams, automating workflows and maintaining data integrity. Candidates should be proficient in Python, SQL, and familiar with big data frameworks like Apache Spark and cloud platforms.

Qualificações

• Advanced proficiency in Python for data engineering tasks.
• Expert-level mastery of SQL and NoSQL database systems.
• Hands-on experience with Big Data technologies like Apache Spark and Hadoop.

Responsabilidades

• Develop and maintain efficient ETL/ELT pipelines for data processing.
• Design and optimize data lakes and warehouse schemas.
• Automate manual workflows to enhance system delivery.

Conhecimentos

Python

SQL

Apache Spark

Hadoop

Kafka

AWS

Azure

GCP

Airflow

Descrição da oferta de emprego

We are seeking a skilled and detail-oriented Data Engineer to design, construct, and optimize our large-scale data processing systems. In this role, you will build robust data pipelines, maintain data warehouses, and ensure stable data delivery across the entire organization. You will collaborate closely with software engineering, data science, and business intelligence teams to support data infrastructure needs and enable data-driven decision-making

• Pipeline Development: Build, test, and maintain scalable Extract, Transform, Load (ETL/ELT) architectures.
• Infrastructure Management: Design and optimize data lakes, databases, and warehouse schemas.
• Automation: Identify and automate manual workflows to optimize system delivery.
• Data Integration: Connect disparate data sources and third-party APIs into central systems.
• Quality Assurance: Implement validation methods to monitor data integrity and reliability.
• Security Compliance: Enforce corporate data governance policies and security protocols.

Cross-Team Collaboration: Assist data analysts and product teams with technical data challenges.

• Programming: Advanced proficiency in languages like Python
• Database Querying: Expert-level mastery of SQL and deep familiarity with NoSQL architectures.
• Big Data Frameworks: Hands-on experience with platforms like Apache Spark, Hadoop, or Kafka.
• Cloud Environments: Strong familiarity with cloud ecosystems such as AWS (Redshift, S3), Azure, or GCP or Snowflake
• Data Modeling: Understanding of dimensional data modeling, schema design, and structuring metadata.
• Orchestration Tools: Experience with workflow managers like Airflow, Prefect, or Dagster.