Remote Big Data Engineer - Real-Time Threat Detection

Lumen; Phoenix (AZ); Remote; USD 132,000 - 176,000; Full time

Job summary

Lumen is seeking a Big Data Developer for remote US-based work to build and optimize large-scale data solutions that identify and disrupt threats online. You will lead projects, collaborate with engineers and security professionals, and help protect critical digital infrastructure.

You will work on scalable pipelines with Spark, Kafka, and Hadoop, mentor juniors, and explore PoCs to adopt innovative data technologies.

Qualifications

• Master’s or Bachelor’s degree in Computer Science, Engineering, Mathematics, or related field, or equivalent work experience.
• Experience developing data ingestion, data processing and analytical pipelines with big data technologies such as Spark, Kafka, Hadoop, Hive.
• Proficient in programming languages like Python, Java, Scala, or SQL.
• Experience with DevOps practices and tools.
• Strong analytical and problem-solving skills and attention to detail.
• Strong communication, leadership, and teamwork skills.
• Eager to learn new skills and technologies and take on challenges.

Responsibilities

• Lead the design, development, and implementation of big data solutions to identify and disrupt threats on the Internet.
• Architect and build scalable data pipelines using Spark, Kafka, Hadoop, and similar technologies.
• Collaborate with cross-functional teams to understand data requirements and achieve business objectives.
• Extract, transform, and load data from relational databases, APIs, JSON, XML, CSV, and more.
• Ensure data quality, integrity, and security across data processes.
• Develop scalable systems enabling real-time validation of resources.
• Mentor and guide junior developers to foster continuous learning and improvement.
• Explore PoCs to determine new tech solutions to detect threats or improve data processing efficiency.

Skills

Python

Java

Scala

SQL

Education

Master's or Bachelor's in CS/Engineering/Math

Tools

Spark

Kafka

Hadoop

Hive

Job description

Lumen is seeking a Big Data Developer for remote US-based work to build and optimize large-scale data solutions that identify and disrupt threats online. You will lead projects, collaborate with engineers and security professionals, and help protect critical digital infrastructure.

You will work on scalable pipelines with Spark, Kafka, and Hadoop, mentor juniors, and explore PoCs to adopt innovative data technologies.