Technical Lead

Cognizant; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Cognizant Hyderabad is seeking an experienced Big Data Engineer to independently design and build big data pipelines, troubleshoot complex data problems, and collaborate with senior developers through the full lifecycle from design to production. You will apply Spark, PySpark, Java/Scala, and a broad Hadoop ecosystem while working in an agile, cloud-enabled environment.

Prior hands-on experience with AWS Glue and related tools is valued, along with effective communication and stakeholder

Qualifications

• 8-15 yrs. of IT experience with more than 4 yrs. of big data implementation experience.
• Hands-on experience with big data pipelines and production support.

Responsibilities

• Independently Design and build/configure Big data pipelines as per the design
• Able to identify and solve big data problems independently.
• Work closely with Sr. Big data developer to report any issues in build
• Work on defect fixes during QA & UAT.
• Work through the life cycle of the code from low-level design to production and Postproductions support.

Skills

Spark stack

PySpark / Python / Scala / Java

Hadoop ecosystem (HDFS, Hive, Impala,

Kafka / NoSQL databases

Cloud experience (AWS, Azure, GCP)

CI/CD & DevOps

Tools

AWS Glue

Cloudera Manager

Cloudera Navigator

NiFi

Oozie

Ambari

Job description

• Independently Design and build/configure Big data pipelines as per the design


• Able to identify and solve big data problems independently.


• Work closely with Sr. Big data developer to report any issues in build


• Work through the life cycle of the code from low-level design to production and Postproductions support.

Purpose

• Independently Design and build/configure Big data pipelines as per the design


• Able to identify and solve big data problems independently.


• Work closely with Sr. Big data developer to report any issues in build


• Work on defect fixes during QA & UAT.


• Work through the life cycle of the code from low-level design to production and Postproductions support.

Experience

• 8-15 yrs. of IT experience with more than 4 yrs. of big data implementation experience.

Capabilities

• Strong technical and analytic skills to solve complex big data problems.


• Experienced with Spark software stack, Prior experience in working on Spark using Python or Scala or Java and PySpark based big data implementations.


• Have strong knowledge and hands-on few of HDFS, Map reduce Hive, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, Mongo DB, Kafka, Spark, Storm, Knox, Ranger, Flume, NiFi, Falcon, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator and Ambari.

Enterprise Competencies

• Independent High level design and developer


• Strong communicator


• Ability to manage stakeholders' expectations.

Technical Competencies

• Big Data Stack (Spark, Hive, Hbase, Apache Pheonix, Pig), Python , Scala, Kafka, Spark Streaming, Atlas, Pig, Java ,SQL, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, Mongo DB,


• Kafka, Spark, Storm, Knox, Ranger, Flume, Nifi, Falcon, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator and Ambari.


• Good Handson Experience in AWS Glue


• Good Experience in CICD & DevOps concepts


• Good experience working in any cloud environment (Azure / AWS Glue/ GCP) using Agile methodology