Data Scraping Engineer: Public Data Pipelines

Trabajosihay; Colombia; Hybride; COP 66 960 000 - 133 920 000; Plein temps

Résumé du poste

Trabajosihay is seeking a Data Scraper to collect, organize, and normalize data from public and government sources into a consistent, structured format. You will work on data acquisition challenges, investigate unfamiliar sources, and transform outputs into predefined formats for downstream systems.

The role emphasizes handling messy datasets, researching source structures, and creating reusable automation workflows with strong documentation and independent work styles.

Qualifications

• Strong experience with web scraping and data extraction.
• Practical programming experience using Python.
• Experience with HTML parsing, APIs, HTTP requests, FTP sources, and structured or unstructured data.
• Ability to evaluate, debug, and improve scraping solutions.
• Strong analytical and problem-solving skills.
• Experience building reusable automation workflows rather than one-off scripts.
• Familiarity with relational databases (PostgreSQL preferred) and a normal Git workflow.
• Strong documentation and communication skills.
• Ability to work independently and take ownership of technical challenges.
• High attention to detail and commitment to data accuracy.

Responsabilités

• Research and identify public and government data sources.
• Extract and normalize data from websites, APIs, feeds, and online repositories.
• Build reusable, maintainable, and re-runnable scripts and scraping workflows.
• Deliver structured outputs in predefined formats.
• Provide sample outputs for review before processing larger datasets.
• Document data sources, extraction methodologies, challenges encountered, and re-run procedures.
• Capture and report any relevant information discovered during extraction, including inconsistencies, amendments, effective dates, repeal notes, or related metadata.
• Troubleshoot data acquisition issues and propose alternative approaches when needed.
• Collaborate with stakeholders through regular check-ins and written communication.
• Maintain version-controlled code repositories and follow standard development practices.

Connaissances

Web Scraping

Data Extraction

Python

HTML Parsing

APIs

Problem Solving

Data Normalization

Data Transformation

Git

Structured Data

Unstructured Data

Data Validation

Public Data Sources

Government Data

Technical Documentation

Playwright

Selenium

Puppeteer

Scrapy

PostgreSQL

Outils

PostgreSQL

Git

Playwright

Selenium

Puppeteer

Scrapy

Description du poste

Trabajosihay is seeking a Data Scraper to collect, organize, and normalize data from public and government sources into a consistent, structured format. You will work on data acquisition challenges, investigate unfamiliar sources, and transform outputs into predefined formats for downstream systems.

The role emphasizes handling messy datasets, researching source structures, and creating reusable automation workflows with strong documentation and independent work styles.