Data Solutions & Automation Engineer

Bright Data LTD.; Bogotá ciudad; Híbrido; COP 90.000.000 - 150.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Lunch bonus COP 500k

Uber office-home

Training stipend USD 2000

Continuous learning

Google Cloud certs paid

Corporate travel opportunities

Laptop provided

Flexible schedule

Descripción de la vacante

Bright Data LTD. is seeking a skilled data-extraction engineer to design, develop, and maintain scrapers and REST APIs. You will work on cloud architectures, data pipelines, and observability while ensuring security and compliance across English/Spanish-speaking stakeholders.

The role emphasizes robust testing, data quality, and collaboration within a growing team, with a strong focus on automation and scalable solutions.

Formación

• Experience building robust scrapers (proxy rotation, CAPTCHA handling, authentication)
• API development with authentication, pagination, and security best practices
• Strong emphasis on testing, data quality, and observability
• Proactive collaboration with stakeholders (ES/EN) and clear communication

Responsabilidades

• Translate business requirements into maintainable technical solutions
• Design resilient cloud architectures
• Develop robust scrapers with proxy rotation and CAPTCHA handling
• Implement REST APIs with auth, pagination, and security best practices
• Orchestrate and monitor data pipelines (scheduling, retries, alerts, logging, metrics)
• Normalize and transform data (ETL/ELT) and design storage schemas
• Improve uptime and performance; ensure data quality and testing
• Promote security, CI/CD, documentation, and code reviews
• Collaborate with stakeholders and clients (Spanish/English)

Conocimientos

Web scraping

Python

FastAPI

CI/CD

Cloud platforms

SQL

NoSQL

Docker

Kubernetes

Observability

Security best practices

English (B2+)

Educación

Bachelor's degree in CS or related

Herramientas

Playwright

Selenium

BeautifulSoup/lxml

Requests/HTTPX

Git

CI/CD tools

PostgreSQL/MySQL

MongoDB/Redis

Docker

Kubernetes

Descripción del empleo

Design, develop, and maintain data extraction and processing solutions, and API-based backend services with security, scalability, and observability standards; while stabilizing and improving existing scraping products to ensure operational continuity and accelerate deliveries in an expanding team.

Key Responsibilities

• Translate business requirements into maintainable technical solutions
• Design resilient and cost-efficient architectures on cloud platforms
• Develop robust scrapers (proxy rotation, CAPTCHA handling, authentication, DOM change tolerance)
• Design and implement REST APIs with authentication, pagination, and security best practices
• Orchestrate and monitor data pipelines (scheduling, retries, alerts, logging, metrics)
• Normalize and transform data (ETL/ELT) and design storage schemas
• Reduce downtime when target sources change (self-healing capabilities)
• Elevate quality and performance standards (profiling, optimization, automation)
• Ensure compliance with Terms of Service, copyright, robots.txt, and data protection regulations
• Promote best practices (security, versioning, CI/CD, documentation)
• Conduct unit and integration testing; validate data quality
• Implement observability: structured logging, metrics, and traces
• Create technical documentation and participate in code reviews
• Collaborate with stakeholders and clients (Spanish/English)

Requirements

Must Have:

• Web scraping tools: Requests/HTTPX, Playwright/Selenium, BeautifulSoup/lxml, CSS/XPath selectors
• API development with FastAPI (or equivalent), authentication (tokens, OAuth), pagination
• Git (PRs, code review) and CI/CD (GitHub Actions/GitLab CI)
• SQL (PostgreSQL/MySQL) and NoSQL (MongoDB/Redis); modeling and performance optimization
• Docker; orchestration and deployment concepts (K8s desirable)
• Observability (structured logging, metrics, traces, and alerts)
• Cloud fundamentals (AWS/Azure/GCP)
• Advanced English (B2+/C1, oral and written)

Nice to Have:

• Messaging/queues (Kafka/RabbitMQ), scheduled tasks (Celery/Arq)
• Data lakes/warehouses (S3/BigQuery/Snowflake) and dbt/Airbyte/Prefect
• Vector DBs/embeddings (FAISS/Pinecone) and LLM ops
• Go/Rust/Node.js for high-performance components
• Security (secrets management, rate limiting, OWASP)
• Lunch bonus: $500,000 COP
• Uber between office and home (specific hours)
• $2,000 USD for training after the first year
• Continuous learning opportunities
• Google Cloud certifications paid by the company
• Corporate travel opportunities
• Laptop and tools provided
• Flexible schedule with team coordination

What We're Looking For

Proactivity and sense of urgency; extreme ownership

Analytical thinking and creative problem-solving

Adaptability to frequent changes and effective prioritization

Clear communication with technical and non-technical audiences

Collaborative work and willingness to share knowledge

Customer orientation and quality focus

Continuous learning and curiosity about new tools/technologies

Success Metrics (First 90 Days)

Success rate / availability (%)

Lead time of changes / throughput

−30% in 90 days

API reliability

SLA/SLO (e.g., 99.5%)

Monthly compliance

30-60-90 Day Plan

30 hours (1 week)

Technical onboarding (repos, pipelines, standards). Take ownership of 1 scraper and 1 API. Map failure points and quick wins.

Success: 1 stable release; 5+ PRs; basic documentation

60 hours (2 weeks)

Fix active scrapers. Performance improvements (profiling/optimization). Automate key tests and observability.

90 hours (3 weeks)

Deliver architecture/orchestration improvements. Reduce change lead time. Propose quarterly roadmap.

Success: −30% lead time; SLOs met; roadmap approved

In Scope

• Development of robust scrapers (proxy rotation, CAPTCHA handling, authentication, DOM change tolerance)
• Design/implementation of REST APIs with authentication, pagination, and security best practices
• Pipeline orchestration and monitoring (scheduling, retries, alerts, logging, metrics)
• Data normalization and transformation (ETL/ELT) and storage schema design
• Unit and integration testing; data quality validation
• Observability: structured logging, metrics, and traces
• Technical documentation and code reviews; collaboration with stakeholders (ES/EN)

Out of Scope

• UI/UX/front-end design beyond API endpoints and contracts
• Direct sales/commercial negotiation (technical support when required)
• On-premise physical infrastructure operation (cloud-based work)

Data Ethics & Compliance

Responsibility to comply with Terms of Service, copyright, robots.txt policies, and applicable data protection regulations (e.g., Law 1581 of 2012 in Colombia, GDPR if applicable). Escalate legal doubts before proceeding with new data sources.