Senior Lead Data Engineer

Information Technology Senior Management Forum; Plano (TX); On-site; USD 209,000 - 239,000; Full time

Job summary

Information Technology Senior Management Forum in Plano, TX is seeking a Senior Lead Data Engineer to build enterprise-grade streaming platforms onsite. You will lead design, implementation, and collaboration across engineering and cross-functional partners, delivering scalable, low-latency analytics pipelines.

You will work with Java, Scala, Python, and Spark/Flink, while mentoring teams, ensuring CI/CD practices, and aligning with security standards.

Qualifications

• Bachelor’s Degree.
• At least 6 years of application development experience (internship excluded).
• At least 2 years of big data technologies experience.
• At least 1 year of cloud computing experience (AWS, Azure, or GCP).
• Master’s Degree.
• 9+ years of application development experience including Python, SQL, Scala, or Java.
• 4+ years of experience with a public cloud (AWS, Azure, or GCP).
• 5+ years of experience with distributed data or computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, etc.).
• 4+ years of experience with real-time data and streaming applications.
• 4+ years of NoSQL implementation (Mongo, Cassandra).
• 4+ years of data warehousing experience (Redshift or Snowflake).
• 4+ years UNIX/Linux and shell scripting.
• 2+ years of Agile engineering practices.
• Experience leveraging interactive AI tooling to accelerate productivity.

Responsibilities

• Lead the design and build of an enterprise-level streaming data platform that is scalable, low-latency, and fault-tolerant to deliver timely insights.
• Develop next-generation distributed streaming data pipelines and analytics data stores using streaming frameworks such as Flink and Spark Streaming, with languages including Java, Scala, and Python.
• Lead engineering efforts building data pipelines on medium to large scale datasets using big data technologies such as Spark, Flink, Kafka, Snowflake, AWS Big Data Services, and Redshift.
• Set and influence best practices for data pipeline design, data architecture, and processing for both structured and unstructured data.
• Work in a creative, collaborative environment using Agile methodologies with a focus on CI/CD and application resiliency standards, partnering with Cyber & Security teams.
• Collaborate with Agile teams to design, develop, test, implement, and support full-stack technical solutions across the toolchain.
• Lead developers with experience spanning machine learning, distributed microservices, and full stack systems.
• Use programming languages such as Java, Scala, and Python, along with open source relational and NoSQL systems and cloud data warehousing services including Redshift and Snowflake.
• Stay current with technology trends by experimenting, learning new tools, contributing to internal and external communities, and mentoring engineering peers.
• Partner with digital product managers to deliver robust cloud-based solutions that support experiences for millions of Americans working toward financial empowerment.
• Perform unit tests and conduct code reviews to ensure code is rigorously designed, elegantly written, and tuned for performance.

Skills

Python

Java

Scala

SQL

Big data

Distributed systems

Education

Bachelor’s Degree

Master’s Degree

Tools

AWS

Azure

GCP

Snowflake

Redshift

Kafka

Spark

Flink

Hadoop

EMR

Mongo

Cassandra

Job description

Build enterprise-grade streaming solutions in a role centered on scalable, low-latency platforms and reliable analytics pipelines. This onsite Senior Lead Data Engineering position in Plano, TX pairs hands-on platform work with team leadership, best-practice ownership, and close collaboration with engineering and cross-functional partners including Cyber & Security.

Compensation for this role is $209,000 - $238,500 per year, with performance-based incentive compensation that may include cash bonus(es) and/or long-term incentives (LTI). You’ll also receive a comprehensive, competitive, and inclusive benefits package designed to support your total well-being.

Responsibilities

• Lead the design and build of an enterprise-level streaming data platform that is scalable, low-latency, and fault-tolerant to deliver timely insights
• Develop next-generation distributed streaming data pipelines and analytics data stores using streaming frameworks such as Flink and Spark Streaming, with languages including Java, Scala, and Python
• Lead engineering efforts building data pipelines on medium to large scale datasets using big data technologies such as Spark, Flink, Kafka, Snowflake, AWS Big Data Services, and Redshift
• Set and influence best practices for data pipeline design, data architecture, and processing for both structured and unstructured data
• Work in a creative, collaborative environment using Agile methodologies with a focus on CI/CD and application resiliency standards, partnering with Cyber & Security teams
• Collaborate with Agile teams to design, develop, test, implement, and support full-stack technical solutions across the toolchain
• Lead developers with experience spanning machine learning, distributed microservices, and full stack systems
• Use programming languages such as Java, Scala, and Python, along with open source relational and NoSQL systems and cloud data warehousing services including Redshift and Snowflake
• Stay current with technology trends by experimenting, learning new tools, contributing to internal and external communities, and mentoring engineering peers
• Partner with digital product managers to deliver robust cloud-based solutions that support experiences for millions of Americans working toward financial empowerment
• Perform unit tests and conduct code reviews to ensure code is rigorously designed, elegantly written, and tuned for performance

Requirements

• Bachelor’s Degree
• At least 6 years of experience in application development (internship experience does not apply)
• At least 2 years of experience in big data technologies
• At least 1 year of experience with cloud computing (AWS, Microsoft Azure, or Google Cloud)
• Master’s Degree
• 9+ years of application development experience including Python, SQL, Scala, or Java
• 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
• 5+ years experience with distributed data or computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
• 4+ years of experience working on real-time data and streaming applications
• 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
• 4+ years of data warehousing experience (Redshift or Snowflake)
• 4+ years of experience with UNIX/Linux including basic commands and shell scripting
• 2+ years of experience with Agile engineering practices
• Experience leveraging interactive AI tooling to accelerate productivity, using capabilities beyond basic code completion

Technologies

• Flink, Spark Streaming
• Java, Scala, Python
• Spark, Kafka
• Snowflake, AWS Big Data Services, Redshift
• Open Source RDBMS, NoSQL databases, Mongo, Cassandra
• MapReduce, Hadoop, Hive, EMR, Gurobi, MySQL
• UNIX/Linux, shell scripting
• Agile methodologies, CI/CD
• SQL, AWS, Microsoft Azure, Google Cloud
• Linux