Senior ETL & Web Data Extraction Engineer

WebDataGuru; Indiana; On-site; USD 120,000 - 170,000; Full time

Job summary

WebDataGuru is seeking an exceptionally skilled Senior Data Engineer to design, build, and optimize ETL pipelines, and to drive bulk data extraction via web scraping and crawling techniques. You will manage large-scale data projects with emphasis on security, scalability, and performance across cloud environments.

You will collaborate with cross-functional teams, implement efficient data solutions, and ensure data quality while leveraging Python libraries and frameworks like Scrapy and Selenium

Qualifications

• ETL pipeline development and optimization for large datasets.

Responsibilities

• Develop, optimize, and troubleshoot ETL pipelines across environments.

Skills

ETL Development

Bulk Data Extraction

Parallel Processing

Python Data Stack

Database Management

Cloud Technologies

REST API

SQL Proficiency

Version Control

SDLC & CI/CD

Problem Solving

Tools

Selenium

Scrapy

Requests

XPath

urllib

lxml

Job description

WebDataGuru is seeking an exceptionally skilled Senior Data Engineer to design, build, and optimize ETL pipelines, and to drive bulk data extraction via web scraping and crawling techniques. You will manage large-scale data projects with emphasis on security, scalability, and performance across cloud environments.

You will collaborate with cross-functional teams, implement efficient data solutions, and ensure data quality while leveraging Python libraries and frameworks like Scrapy and Selenium