Senior Big Data Engineer - Kafka, Redis & PySpark

Tata Consultancy Services Limited; Pennington (NJ); On-site; USD 95,000 - 115,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Medical Coverage

Family Leaves

Commuter Benefits

Certification Reimbursement

Vacation & Holidays

401K Plan

Job summary

Tata Consultancy Services Limited in Pennington, NJ is seeking an Engineer in Technology to deliver real-time data processing solutions. You will work with Kafka, Redis, Hadoop and PySpark, applying distributed systems concepts to mission-critical applications.

A Bachelor's in Computer Science or related field is required, with 5+ years of hands-on Big Data experience and proficiency in Python, Java or Scala. The role offers competitive salary in the stated range and full benefits.

Qualifications

• Bachelor's or Master's degree in Science or Engineering, or a related field.
• Minimum of 5 years industry experience, with at least 3 years hands-on Big Data and real-time data processing.
• Proficiency in real-time streaming technologies like Kafka (KStreams, KTable).
• Proficiency in Hadoop and PySpark.
• Strong knowledge of NoSQL databases like Redis, MongoDB, or HBase.
• Proficiency in programming languages such as Python, Scala, Java, and Bash/Shell Scripting.
• Experience with additional real-time streaming technologies like Flink or Storm.
• Familiarity with Cloud Technologies such as Azure, AWS, or GCP.
• Working knowledge of machine learning algorithms, statistical analysis, and programming languages (Python or R) to conduct data analysis and develop predictive models.
• Proficiency in Data Integration and Data Security within real-time and Big Data ecosystems, including knowledge of Kerberos.
• Strong command of Visual Analytics Tools, with a focus on Tableau.

Responsibilities

• Bachelor's or Master's degree in Science or Engineering, or a related field.
• Minimum of 5 years industry experience, with at least 3 years focused on hands-on work in Big Data and real-time data processing technologies.
• Proficiency in real-time streaming technologies like Kafka (KStreams, KTable)
• Proficiency in Hadoop and PySpark
• Strong knowledge of NoSQL databases like Redis, MongoDB, or HBase.
• Proficiency in programming languages such as Python, Scala, Java, and Bash/Shell Scripting.
• Excellent problem-solving abilities and the capability to deliver effective solutions for business-critical applications.
• Strong understanding of distributed systems, data partitioning, and fault-tolerant architectures.
• Experience with additional real-time streaming technologies like Flink or Storm.
• Familiarity with Cloud Technologies such as Azure, AWS, or GCP.
• Working knowledge of machine learning algorithms, statistical analysis, and programming languages (Python or R) to conduct data analysis and develop predictive models.
• Proficiency in Data Integration and Data Security within real-time and Big Data ecosystems, including knowledge of Kerberos.
• Strong command of Visual Analytics Tools, with a focus on Tableau.

Skills

Kafka

Redis

Hadoop

PySpark

Python

Education

Bachelor's in Computer Science

Tools

KStreams

KTable

MongoDB

HBase

Job description

Tata Consultancy Services Limited in Pennington, NJ is seeking an Engineer in Technology to deliver real-time data processing solutions. You will work with Kafka, Redis, Hadoop and PySpark, applying distributed systems concepts to mission-critical applications.

A Bachelor's in Computer Science or related field is required, with 5+ years of hands-on Big Data experience and proficiency in Python, Java or Scala. The role offers competitive salary in the stated range and full benefits.