GenAI & Data Pipelines Engineer

Jobtailor; Lisboa; Presencial; EUR 55 000 - 75 000; Tempo integral

Resumo da oferta

Jobtailor in Lisbon, Portugal seeks a data engineer to design, build, and maintain scalable data pipelines and ETL/ELT workflows leveraging Python and AWS services. You will collaborate with data scientists and product teams to deliver robust data solutions that drive key business metrics.

The role requires strong Python skills, ML/GenAI project experience, and hands-on AWS infrastructure expertise, including CDK, Spark, and CI/CD practices.

Qualificações

• Bachelor's degree in computer science, software engineering, or comparable work experience.
• Proficiency in Python and data engineering frameworks.
• Experience in ML/GenAI projects.
• Strong AWS experience with S3, Lambda, Step Functions, SQS/SNS, EventBridge, CloudWatch, IAM, DynamoDB, batch processing (Glue Jobs/Workflows, Batch).
• Hands-on Infrastructure as Code experience (AWS CDK).
• Strong experience with Spark/PySpark or similar frameworks.
• Proficiency with Git, repository structure management, Docker, testing (unit tests, mocking, isolation).
• Experience with CI/CD pipelines (GitLab/GitHub Actions/CircleCI).
• Experience building scalable, production-grade data pipelines.
• Excellent English skills (written and spoken).

Responsabilidades

• Designing, building, and maintaining scalable data pipelines and ETL/ELT workflows supporting ML and GenAI use cases using Python, and AWS services.
• Building analytics tools that use the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Managing data ingestion, orchestration, transformation, quality, and curation at scale.
• Implementing and maintaining systems via event-driven architectures and APIs.
• Collaborating closely with Data Scientists, product teams, and business stakeholders to translate data needs into robust solutions.
• Ensuring high engineering standards: unit testing, mocking, CI/CD, code quality, repository structure, environment setup.
• Providing technical guidance, code reviews, and mentoring to team members.

Conhecimentos

Python proficiency

ML/GenAI projects

AWS experience

Spark/PySpark

Git

Docker

Testing

CI/CD

Production-grade data pipelines

English language

Formação académica

Bachelor's degree in computer science / software engineering

Ferramentas

AWS CDK

Descrição da oferta de emprego

Jobtailor in Lisbon, Portugal seeks a data engineer to design, build, and maintain scalable data pipelines and ETL/ELT workflows leveraging Python and AWS services. You will collaborate with data scientists and product teams to deliver robust data solutions that drive key business metrics.

The role requires strong Python skills, ML/GenAI project experience, and hands-on AWS infrastructure expertise, including CDK, Spark, and CI/CD practices.