Senior Web Scraping Data Engineer - Remote & Automation

Jobgether; Ireland; On-site; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Fully remote position

Autonomy and ownership

Exposure to complex datasets and data‑

Job summary

Jobgether is seeking a Senior Data Engineer - Web Scraping, fully remote in Ireland. You will design and maintain advanced web scrapers, process diverse sources into reliable datasets, and build scalable data pipelines.

This role emphasizes hands-on Python development, data transformation, and orchestration with Airflow. You will collaborate with analysts, engineers, and cross-functional teams to deliver timely data products and production-ready datasets, with ownership and autonomy to optimize

Qualifications

• Bachelor’s or Master’s degree in CS, Engineering, or related technical discipline.
• 4–6 years of professional data engineering experience.
• Strong Python, SQL, and database skills.
• Advanced Pandas experience for cleaning and transforming data.
• Proficient web scraping with Selenium, Scrapy, Fiddler, Postman, XPath.
• Experience with Apache Airflow for orchestration.
• Knowledge of HTML, JavaScript, APIs.
• Experience with Docker; Kubernetes is a plus.
• CI/CD tools: Jenkins, GitHub Actions.
• AWS services: S3, RDS, SNS, SQS, Lambda.

Responsibilities

• Design, develop, deploy, and maintain web scrapers using diverse techniques to collect datasets.
• Use Python and Pandas to clean, explore, transform, and prepare large datasets.
• Build and maintain data pipelines into databases and data warehouses.
• Develop and manage scheduled workflows with Apache Airflow and orchestration tools.
• Collaborate with analysts and cross-functional stakeholders to translate data requirements into solutions.
• Develop quality-control checks to validate data availability, accuracy, and integrity.
• Maintain alerting systems and resolve operational data issues.
• Design tools and automation to improve the web-scraping platform.
• Contribute to infrastructure and data-product design for data teams.
• Work independently while collaborating with teams.

Skills

Python

SQL

Pandas

Web scraping

Airflow

Docker

Kubernetes

Jenkins

GitHub Actions

AWS

Education

Bachelor’s or Master’s in CS/Engineering

Tools

Selenium

Scrapy

Fiddler

Postman

XPath

Job description

Jobgether is seeking a Senior Data Engineer - Web Scraping, fully remote in Ireland. You will design and maintain advanced web scrapers, process diverse sources into reliable datasets, and build scalable data pipelines.

This role emphasizes hands-on Python development, data transformation, and orchestration with Airflow. You will collaborate with analysts, engineers, and cross-functional teams to deliver timely data products and production-ready datasets, with ownership and autonomy to optimize