Hybrid Big Data ML Engineer: Real-Time Pipelines & Cloud

Spectraforce Technologies; Plano (TX); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Spectraforce Technologies in Plano, TX is seeking a Big Data ML Engineer to design, build, and operate large-scale batch and real-time data pipelines with high reliability and low latency. You will collaborate with Agile teams to architect end-to-end data solutions—from ingestion and orchestration through event streaming and downstream delivery.

The role demands 4+ years in data pipelines and engineering with Java, Python, and SQL, plus experience with distributed tools (Kafka, Spark, EMR,

Qualifications

• 4+ years of experience building or operating data pipeline and orchestration systems.
• 4+ years of data or application engineering experience, including Java, Python and SQL.
• 4+ years of experience building and operating real-time or event-driven data systems.
• 4+ years of experience with distributed data and computing tools (Kafka, Spark, EMR, Hadoop, or equivalent).
• 4+ years of experience with cloud data warehousing platforms at scale.
• 4+ years of experience with cloud platforms (AWS, Azure, or GCP).
• 3+ years of experience with pipeline scheduling and workflow orchestration tools.
• 2+ years of experience with secure secrets management and credential handling in production systems.
• 2+ years of experience working in Agile engineering environments.
• Familiarity with data observability practices - monitoring, alerting, SLA tracking, and data quality frameworks.

Responsibilities

• Design, build, and operate large-scale batch and real-time data pipelines with high reliability and low latency.
• Collaborate with Agile teams to architect end-to-end data solutions from ingestion to downstream delivery.

Skills

Big Data

ML Engineer

Java

Python

SQL

Cloud platforms

Data pipelines

Agile

Data observability

Tools

Kafka

Spark

EMR

Hadoop

AWS

Azure

GCP

Job description

Spectraforce Technologies in Plano, TX is seeking a Big Data ML Engineer to design, build, and operate large-scale batch and real-time data pipelines with high reliability and low latency. You will collaborate with Agile teams to architect end-to-end data solutions—from ingestion and orchestration through event streaming and downstream delivery.

The role demands 4+ years in data pipelines and engineering with Java, Python, and SQL, plus experience with distributed tools (Kafka, Spark, EMR,