Remote Senior Data Engineer - Web Scraping & Pipelines

Jobgether; Portugal; Presencial; EUR 55 000 - 85 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Fully remote position

Full‑time opportunity

Autonomy and ownership

Resumo da oferta

Jobgether invites a data engineering professional to work remotely on advanced web scraping, data processing, and automation. You’ll design scrapers that convert diverse web sources into reliable datasets and build scalable data pipelines for downstream use.

The role combines Python development, Pandas data transformation, SQL, and Airflow-based orchestration with Docker/Kubernetes and cloud services. You’ll collaborate with analysts and engineers to deliver production‑ready data products and

Qualificações

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical discipline.
• 4–6 years of professional experience in data engineering or a closely related field.
• Strong programming skills in Python and strong knowledge of SQL and database technologies.
• Advanced hands-on expertise with the Python Pandas library for data cleaning, manipulation, exploration, and transformation.
• Strong web-scraping experience with Selenium, Scrapy, Fiddler, Postman, and XPath.
• Strong experience with Apache Airflow for workflow orchestration and data pipeline management.

Responsabilidades

• Design, develop, deploy, and maintain web scrapers using a range of scraping techniques and tools to collect datasets from diverse sources.
• Use Python and Pandas to clean, explore, transform, manipulate, and prepare large datasets for downstream consumption.
• Build and maintain efficient data pipelines that ingest scraped data into databases and data warehouses.
• Develop and manage scheduled workflows using Apache Airflow and other orchestration tools to ensure reliable and timely data delivery.
• Collaborate with analysts and cross‑functional stakeholders to translate requirements into technical solutions.
• Develop quality‑control checks to validate data availability, accuracy, consistency, and integrity.
• Maintain alerting systems, investigate data incidents, and resolve operational issues for reliable delivery.
• Design and implement tools and automation to improve the scraping platform’s capabilities and efficiency.
• Contribute to infrastructure and data‑product design to support data scientists and technology teams.

Conhecimentos

Python programming

SQL

Pandas

Web scraping

Airflow

Formação académica

Bachelor's or Master's in CS/Engineering

Ferramentas

Selenium

Scrapy

Fiddler

Postman

XPath

Apache Airflow

Docker

Kubernetes

Jenkins

GitHub Actions

AWS (S3, RDS, SNS, SQS, Lambda)

Descrição da oferta de emprego

Jobgether invites a data engineering professional to work remotely on advanced web scraping, data processing, and automation. You’ll design scrapers that convert diverse web sources into reliable datasets and build scalable data pipelines for downstream use.

The role combines Python development, Pandas data transformation, SQL, and Airflow-based orchestration with Docker/Kubernetes and cloud services. You’ll collaborate with analysts and engineers to deliver production‑ready data products and