Senior Lead Data Engineer: Real-Time Streaming Architect

Information Technology Senior Management Forum; Plano (TX); On-site; USD 209,000 - 239,000; Full time

Job summary

Information Technology Senior Management Forum in Plano, TX is seeking a Senior Lead Data Engineer to build enterprise-grade streaming platforms onsite. You will lead design, implementation, and collaboration across engineering and cross-functional partners, delivering scalable, low-latency analytics pipelines.

You will work with Java, Scala, Python, and Spark/Flink, while mentoring teams, ensuring CI/CD practices, and aligning with security standards.

Qualifications

• Bachelor’s Degree.
• At least 6 years of application development experience (internship excluded).
• At least 2 years of big data technologies experience.
• At least 1 year of cloud computing experience (AWS, Azure, or GCP).
• Master’s Degree.
• 9+ years of application development experience including Python, SQL, Scala, or Java.
• 4+ years of experience with a public cloud (AWS, Azure, or GCP).
• 5+ years of experience with distributed data or computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, etc.).
• 4+ years of experience with real-time data and streaming applications.
• 4+ years of NoSQL implementation (Mongo, Cassandra).
• 4+ years of data warehousing experience (Redshift or Snowflake).
• 4+ years UNIX/Linux and shell scripting.
• 2+ years of Agile engineering practices.
• Experience leveraging interactive AI tooling to accelerate productivity.

Responsibilities

• Lead the design and build of an enterprise-level streaming data platform that is scalable, low-latency, and fault-tolerant to deliver timely insights.
• Develop next-generation distributed streaming data pipelines and analytics data stores using streaming frameworks such as Flink and Spark Streaming, with languages including Java, Scala, and Python.
• Lead engineering efforts building data pipelines on medium to large scale datasets using big data technologies such as Spark, Flink, Kafka, Snowflake, AWS Big Data Services, and Redshift.
• Set and influence best practices for data pipeline design, data architecture, and processing for both structured and unstructured data.
• Work in a creative, collaborative environment using Agile methodologies with a focus on CI/CD and application resiliency standards, partnering with Cyber & Security teams.
• Collaborate with Agile teams to design, develop, test, implement, and support full-stack technical solutions across the toolchain.
• Lead developers with experience spanning machine learning, distributed microservices, and full stack systems.
• Use programming languages such as Java, Scala, and Python, along with open source relational and NoSQL systems and cloud data warehousing services including Redshift and Snowflake.
• Stay current with technology trends by experimenting, learning new tools, contributing to internal and external communities, and mentoring engineering peers.
• Partner with digital product managers to deliver robust cloud-based solutions that support experiences for millions of Americans working toward financial empowerment.
• Perform unit tests and conduct code reviews to ensure code is rigorously designed, elegantly written, and tuned for performance.

Skills

Python

Java

Scala

SQL

Big data

Distributed systems

Education

Bachelor’s Degree

Master’s Degree

Tools

AWS

Azure

GCP

Snowflake

Redshift

Kafka

Spark

Flink

Hadoop

EMR

Mongo

Cassandra

Job description

Information Technology Senior Management Forum in Plano, TX is seeking a Senior Lead Data Engineer to build enterprise-grade streaming platforms onsite. You will lead design, implementation, and collaboration across engineering and cross-functional partners, delivering scalable, low-latency analytics pipelines.

You will work with Java, Scala, Python, and Spark/Flink, while mentoring teams, ensuring CI/CD practices, and aligning with security standards.