Data Engineer (Web Scraping)

ACROMETA LIFESTYLE PTE. LTD.; Greater London; On-site; GBP 35,000 - 50,000; Full time

Job summary

A leading data solutions provider in Greater London is looking for a skilled Data Specialist with expertise in data scraping and processing. The ideal candidate will have a Bachelor's degree in Computer Science and 1-3 years of relevant experience. Responsibilities include gathering procurement data, cleaning raw data, and ensuring data quality. Proficiency in Python is essential, along with familiarity in multilingual processing. This role offers the opportunity to work in a dynamic environment with competitive compensation.

Qualifications

• 1-3 years of experience in data scraping, processing, or analysis.
• Proficiency in Python and familiarity with JavaScript.
• Experience in web scraping development is preferred.

Responsibilities

• Gather procurement requirements through scripting.
• Clean and format scraped data for analysis.
• Verify authenticity and validity of data.

Skills

Data scraping

Data analysis

Multilingual processing

Python

JavaScript

Git

Problem-solving

Education

Bachelor’s degree in Computer Science or related fields

Tools

Scrapy

BeautifulSoup

Selenium

Pandas

MySQL

Job description

Key Responsibilities

1. Data Scraping and Collection

• Gather procurement requirements from target markets by writing scripts (Python, JavaScript, etc.) or using tools
• Collect enterprise information, product details, and trade data from multiple sources (websites, APIs, databases, etc.)
• Maintain and optimize data scraping scripts to ensure efficiency and stability
• Handle anti-scraping mechanisms to ensure data collection compliance

2. Data Cleaning and Processing

• Clean and format the raw scraped data
• Identify and remove duplicate and invalid data
• Standardize data formats (country, city, category, HS code, etc.)
• Process multilingual data (Chinese, English, Japanese, Thai, etc.)

3. Data Review and Screening

• Verify the authenticity and validity of procurement requirements according to platform standards
• Verify the accuracy and completeness of enterprise information
• Screen high-quality data that meets platform requirements
• Identify and flag suspicious or low-quality data

4. Data Entry and Management

• Store the approved data in the platform database
• Maintain the data classification and labeling system
• Manage data versions and update records
• Ensure the accuracy and completeness of data entry

5. Data Quality Monitoring

• Regularly check data quality metrics (completeness, accuracy, timeliness, etc.)
• Monitor the execution status of data scraping tasks
• Identify and resolve data quality issues
• Generate data quality reports

6. Tool Development and Optimization

• Develop or optimize data scraping tools and scripts
• Establish data review workflows and standards
• Enhance the automation level of data processing
• Optimize data processing efficiency

7. Target Market Research

• Research data sources and acquisition channels in target markets
• Understand the data characteristics and formats of different markets
• Identify new data collection opportunities
• Monitor market changes and adjust data collection strategies

II. Job Requirements

Educational Background

• Bachelor’s degree or above in Computer Science, Data Science, Information Technology, or related fields

Work Experience

• 1-3 years of experience in data scraping, data processing, or data analysis
• Experience in web scraping development is preferred
• Experience in data cleaning and auditing is preferred
• Experience in B2B platforms or trade data is preferred
• Candidates with cross-border trade industry background are preferred

Technical Skills

• Programming Languages: Proficient in Python (required); familiarity with JavaScript, Java, etc. is preferred
• Web Scraping Frameworks: Familiar with Scrapy, BeautifulSoup, Selenium, Playwright, and other scraping tools
• Data Processing: Familiar with Pandas, NumPy, and other data processing libraries
• Database: Familiar with MySQL, PostgreSQL, MongoDB, and other database operations
• API: Understanding of RESTful API and JSON data processing
• Tools: Familiar with Git version control, Jupyter Notebook, and other development tools
• Regular Expressions: Able to use regular expressions for text matching and processing
• Multilingual Processing: Experience in multilingual text processing (Chinese, English, Japanese, Thai, etc.) is preferred

Business Skills

• Understanding of B2B cross-border trade business processes
• Familiar with the data structure of Request for Quotation (RFQ)
• Understanding of trade-related concepts such as company information, product categories, and HS codes
• Capable of assessing the authenticity and validity of data

Soft Skills

• Fluent in English and Mandarin (spoken and written)
• Detail-oriented and meticulous, capable of handling large volumes of repetitive tasks
• Excellent problem-solving skills and logical thinking
• Strong learning ability and adaptability
• Capable of working under pressure and completing tasks on time
• Strong teamwork spirit

Nice to Have

• Project experience with web scraping frameworks such as Scrapy and Selenium
• Experience in data cleaning and ETL
• Understanding of machine learning or natural language processing
• Proficient in Docker and Linux systems
• Experience in data visualization (Tableau, Power BI, etc.)

Please send your resume to [e-mail]