Senior Real-Time Big Data Engineer: Kafka, Redis & PySpark

Centraprise, Corp.; Plano (TX); Sur place; USD 120 000 - 180 000; Plein temps

Résumé du poste

Centraprise, Corp. is seeking an experienced Big Data Engineer with a strong background in real-time data processing to join our Plano, TX team.

The candidate will work on complex streaming pipelines using Kafka, Hadoop, and PySpark while collaborating with data scientists and analytics professionals. Ideal applicants have 9+ years of experience, proficiency in Python, and hands-on exposure to NoSQL databases like Redis, MongoDB, or HBase.

Qualifications

• Bachelor's or Master's degree in Science or Engineering, or a related field.
• Minimum of 5 years of industry experience, with at least 3 years focused on hands-on work in Big Data and real-time data processing technologies.
• Proficiency in real-time streaming technologies like Kafka (KStreams, KTable).
• Proficiency in Hadoop and PySpark.
• Strong knowledge of NoSQL databases like Redis, MongoDB, or HBase.
• Proficiency in programming languages such as Python, Scala, Java, and Bash/Shell Scripting.
• Excellent problem-solving abilities and the capability to deliver effective solutions for business-critical applications.
• Strong understanding of distributed systems, data partitioning, and fault-tolerant architectures.
• Experience with additional real-time streaming technologies like Flink or Storm.
• Familiarity with Cloud Technologies such as Azure, AWS, or GCP.
• Working knowledge of machine learning algorithms, statistical analysis, and programming languages (Python or R) to conduct data analysis and develop predictive models.
• Proficiency in Data Integration and Data Security within real-time and Big Data ecosystems, including knowledge of Kerberos.
• Strong command of Visual Analytics Tools, with a focus on Tableau.

Connaissances

Kafka

Redis

Hadoop

PySpark

Python

Scala

Java

Bash/Shell scripting

Formation

Bachelor's or Master's degree in Science or Engineering

Outils

KStreams

KTable

MongoDB

HBase

Tableau

Kerberos

Azure

AWS

GCP

Flink

Storm

Description du poste

Centraprise, Corp. is seeking an experienced Big Data Engineer with a strong background in real-time data processing to join our Plano, TX team.

The candidate will work on complex streaming pipelines using Kafka, Hadoop, and PySpark while collaborating with data scientists and analytics professionals. Ideal applicants have 9+ years of experience, proficiency in Python, and hands-on exposure to NoSQL databases like Redis, MongoDB, or HBase.