Data Collection Engineer (Spain)

CENTRIC SOFTWARE INC; España; A distancia; EUR 40.000 - 60.000; Jornada completa

Descripción de la vacante

A leading software firm based in Spain seeks a Data Collection Engineer to develop scalable data collection systems and maintain high data quality. The ideal candidate will possess strong technical skills in web crawling, monitoring tools, and cloud infrastructure. Excellent English communication and collaboration abilities are essential, alongside a proactive approach to innovation. This role offers opportunities for mentoring and knowledge sharing within a dynamic team.

Formación

• Experience with CI/CD pipelines and code reviews.
• Excellent English communication skills, both written and spoken.
• Strong analytical thinking and problem-solving abilities.

Responsabilidades

• Design and maintain web crawlers for data extraction.
• Build scalable CI/CD pipelines for automation.
• Ensure data integrity and accuracy through validation mechanisms.
• Monitor and maintain data collection systems for performance.

Conocimientos

Git workflows

Cloud infrastructure (AWS)

Monitoring systems (Grafana, Sentry)

Web environment standards

TCP/IP stacks

Low-level web networking

Descripción del empleo

LOCATION Candidates must be legally based in Spain due to employment and compliance regulations.

About Us

In today's complex retail landscape, characterized by economic fluctuations and supply chain challenges, consumers are more discerning, often comparing prices and seeking compelling products.

Centric Pricing™ addresses this by enabling retailers and brands to deeply understand the competitive landscape post-product launch.

By leveraging AI-driven insights, businesses can make informed decisions quickly, aligning product development, sourcing, costing, and pricing strategies with real-time market demands. The integration of Centric Pricing™ into Centric Software's platform provides an end-to-end solution that combines intelligence and execution capabilities.

This empowers brands and retailers to optimize product availability, reduce time to market, and enhance product quality, ultimately improving the consumer experience and driving profitability. We are a key innovation partner for iconic and emerging brands across the world.

Our Platform can analyze the info of more than 1.000 retailers, processing data from more than 600.000 brands, tracking millions of products.

Platform Overview

As a Data Collection Engineer, you will be instrumental in building scalable and high-quality data collection systems, collaborating across teams to drive innovation and maintain the robustness of our data pipeline.

Responsibilities

• Design and Build Robust Web Crawlers: Develop and maintain spiders for high-scale data extraction using Scrapy, ensuring modular, reusable components and employing anti-bot bypass techniques such as rotating proxies, captcha solving, and fingerprinting.
• Enhance and Maintain Infrastructure: Build scalable CI/CD pipelines for automated testing, deployment, and monitoring of spiders; utilize Scrapyd for centralized scheduling and cloud deployment for high-throughput crawling.
• Code Quality and Consistency: Uphold coding standards, conduct thorough reviews, mentor junior engineers, and maintain detailed version control documentation.
• Monitoring, Maintenance & Reliability: Integrate performance monitoring (Grafana, Sentry), schedule periodic spider audits, troubleshoot failures, and optimize resource usage.
• Data Integrity and Accuracy: Implement data validation mechanisms, collaborate with internal consumers, and automate recovery strategies for anomalies.
• Collaboration and Knowledge Sharing: Work cross-functionally with product, engineering, and other data teams, promote documentation culture, and contribute to internal knowledge bases.
• Training Initiatives: Lead training sessions to keep the team current on scraping techniques and emerging technologies.

Desired Technical Skills and Experience

• Core requirements: Comfort with Git workflows, code reviews, and CI/CD pipelines; experience with cloud infrastructure like AWS; expertise in monitoring/observability systems (Grafana, Sentry); deep knowledge of web environment standards, TLS/SSL, TCP/IP stacks, and low-level web networking.
• Bonus / Senior-Level expectations: Proficient in designing fault-tolerant systems and deploying them at scale; familiarity with containerized deployments; prior experience mentoring or leading junior developers.

Soft Skills and Work Ethic

Excellent communication skills in English, both written and spoken. A collaborative mindset with a proactive approach to knowledge sharing. Strong analytical thinking and problem-solving abilities. Commitment to continuous improvement, mentoring, and agile team dynamics. Remain up-to-date with technology trends to keep our software as innovative as possible.

Equal Employment Opportunity

Centric Software provides equal employment opportunities to all qualified applicants without regard to race, sex, sexual orientation, gender identity, national origin, color, age, religion, protected veteran or disability status or genetic information.

Powered by JazzHR