Remote Big Data Engineer for AI Threat Detection

Lumen; Columbus (OH); Remote; USD 132,000 - 194,000; Full time

Job summary

Lumen is seeking a Big Data Developer to build and optimize large-scale data solutions that identify and disrupt malicious threats across the Internet. You’ll leverage Spark, Kafka, Hadoop, Hive, and related technologies within a collaborative cybersecurity team.

As part of a mission-driven group, you’ll lead projects, influence technical direction, and grow your expertise while protecting critical digital infrastructure for US-based customers in a remote capacity.

Qualifications

• Master’s or Bachelor’s degree in Computer Science, Engineering, Mathematics, or related field, or equivalent work experience.
• Proven track record in building data ingestion, processing and analytics pipelines with big data technologies such as Spark, Kafka, Hadoop, Hive, etc.
• Proficient in Python, Java, Scala, or SQL.
• Experience with DevOps practices and tools.
• Strong analytical and problem-solving skills, and attention to detail.
• Strong communication, leadership, and teamwork skills.
• Eager to learn new skills and technologies, and willing to take on challenges.

Responsibilities

• Lead the design, development, and deployment of big data solutions to identify and disrupt malicious threats on the Internet.
• Architect and build scalable data pipelines using Spark, Kafka, Hadoop, Hive, and related big data tech.
• Collaborate with cross-functional teams to understand data requirements and deliver solutions meeting business objectives.
• Extract, transform, and load data from diverse sources such as relational DBs, APIs, JSON, XML, and CSV.
• Ensure data quality, integrity, and security across all data processes.
• Develop scalable systems enabling real-time validation of identified resources.
• Mentor and guide junior developers, fostering continuous learning and improvement.
• Evaluate new technologies and run PoCs to improve threat detection and data processing efficiency.

Skills

Spark

Kafka

Hadoop

Hive

Python

Java

Scala

SQL

DevOps

Leadership

Education

Master's or Bachelor's in CS/Engineering/Math

Job description

Lumen is seeking a Big Data Developer to build and optimize large-scale data solutions that identify and disrupt malicious threats across the Internet. You’ll leverage Spark, Kafka, Hadoop, Hive, and related technologies within a collaborative cybersecurity team.

As part of a mission-driven group, you’ll lead projects, influence technical direction, and grow your expertise while protecting critical digital infrastructure for US-based customers in a remote capacity.