Senior Data Engineer

Recorded Future; United Kingdom; Remote; GBP 50,000 - 80,000; Full time

Job summary

A global intelligence company is seeking a professional to build and maintain data collection from dark web sources. The ideal candidate has over 5 years of experience in Python and data engineering, is skilled in troubleshooting data issues, and has a solid understanding of cybersecurity. This role emphasizes innovation and reliability in data processing, contributing to intelligence that protects customers.

Qualifications

• 5+ years experience with Python in a professional software engineering environment.
• Strong problem solving capabilities and experience troubleshooting data issues.
• 5+ years experience maintaining production-ready data engineering pipelines.

Responsibilities

• Build, maintain, monitor and collect data from dark web sites.
• Own the reliability of automatic data processing.
• Maintain and improve data ingestion pipeline.

Skills

Python

Problem solving

Data engineering pipelines

NLP

MongoDB

Elasticsearch

AWS cloud services

Job description

Recorded Future is a global intelligence company with 1,000 intelligence professionals, over $300M in sales, and serving over 1,900 clients worldwide. The Ransomware squad on the Dark Web team builds data collection and analytics that transform raw data from ransomware blogs into structured intelligence used to defend customers.

What You'll Do:

• Build, maintain, monitor and collect data from hard-to-reach dark web sites and ransomware blogs.
• Own the reliability of the automatic processing of this structured raw data. Investigate and resolve data processing problems by improving our data processing code so it will automatically handle similar scenarios in the future.
• Maintain and improve our data ingestion pipeline that processes and indexes metadata from large files. This includes analytic-generation services that use NLP to extract content with the highest security value.
• Lead by example when architecting, collaborating and delivering new and improved solutions.

What You'll Bring:

• 5+ years experience with Python in a professional software engineering environment with thorough code testing and review processes.
• Strong problem solving capabilities and experience troubleshooting data issues within your own processes and those of your peers.
• 5+ years experience maintaining production-ready, efficient data engineering pipelines and systems with extensive monitoring to handle complex data ingestion problems.
• Very flexible and outcome-oriented; comfortable in a work environment where urgent issues arise often and quick resolution has immediate security value for customers.
• A solid understanding of cybersecurity and an interest in analyzing dark web activities related to computer and network security.
• Experience with NLP, MongoDB, Elasticsearch, and AWS cloud services is highly desirable.

Why join Recorded Future?

Recorded Future employees (or “Futurists”) represent over 40 nationalities and embody core values of high standards, inclusion, and ethics. The company is trusted by Gartner with a 4.8-star user rating and serves more than 45 of the Fortune 100 companies.