Automation Engineer

Invisible Technologies; United States; A distancia; USD 90.000 - 140.000; Jornada completa

Descripción de la vacante

Invisible Technologies in the United States is seeking a skilled Python crawler engineer to build and maintain advanced web scrapers.

You will leverage Scrapy, Selenium, BeautifulSoup and SQLAlchemy to collect data, manage pipelines, and overcome blocks, IP bans, and captchas.

The role involves deploying scripts to Linux-based AWS servers and collaborating with cross-functional teams in a fast-growing, automation-driven environment.

Formación

• Strong hands-on Python programming experience.
• Proficient with scraping libraries: Scrapy, Selenium, BeautifulSoup, Requests.
• 3+ years with web frameworks: Flask, Django, FastAPI, Tornado.
• Strong SQL knowledge with relational databases.
• Git version control and REST API basics.
• Familiarity with JavaScript, HTML5 and CSS3.

Responsabilidades

• Create crawlers for source URLs using Python modules including Scrapy, Selenium, Requests, BeautifulSoup, and Splash
• Maintain Scrapy pipelines and middlewares to manage crawler output
• Develop crawlers capable of overcoming technical roadblocks like IP bans, geolocation bans, and captchas
• Write SQL queries to manage database operations using SQLAlchemy
• Deploy Python scripts and crawlers to Linux-based AWS servers

Conocimientos

Python

REST APIs

JavaScript

HTML5

CSS3

OOP

Git

Herramientas

Scrapy

Selenium

BeautifulSoup

Requests

SQLAlchemy

SQL

Descripción del empleo

Responsibilities

• Create crawlers for source URLs using Python modules including Scrapy, Selenium, Requests, BeautifulSoup, and Splash
• Maintain Scrapy pipelines and middlewares to manage crawler output
• Develop crawlers capable of overcoming technical roadblocks like IP bans, geolocation bans, and captchas
• Write SQL queries to manage database operations using SQLAlchemy
• Deploy Python scripts and crawlers to Linux-based AWS servers

Requirements

• Strong hands-on experience in Python programming
• Proficiency with scraping libraries: Requests, BeautifulSoup, Selenium, and Scrapy
• 3+ years of experience with web frameworks such as Flask, Django, FastAPI, Tornado, or Pandas
• Strong SQL knowledge and experience with RDBMS
• Proficiency with Git version control
• Understanding of REST APIs, JavaScript, HTML5, and CSS3
• Solid grasp of Object-Oriented Programming (OOP) concepts

Preferred Qualifications

• Experience with UI frameworks like Angular or ReactJS
• Knowledge of data engineering platforms such as Airflow
• Experience with Google Cloud, Docker, Celery, or ETL processes
• Familiarity with Agile/Scrum methodologies and Jira/Confluence

About the Company

Invisible Technologies offers unusual services, combining outsourcing and automation, to fast-scaling, innovative companies.