ADAPT Engineer

Cognizant; Hyderabad; On-site; INR 1,000,000 - 2,000,000; Full time

Job summary

Cognizant Hyderabad is seeking an experienced Big Data Engineer to independently design and build large data pipelines, solve complex data problems, and work with senior developers across the project lifecycle.

The role requires 8–15 years IT experience with 4+ years in big data, strong Spark, Python/Scala/Java, and hands-on knowledge of HDFS, Hive, Kafka, and related tools.

Qualifications

• 8-15 yrs of IT experience with more than 4 yrs of big data implementation experience.
• Experience with Spark stack and PySpark/Scala/Java based big data implementations.
• Hands-on knowledge of HDFS, Hive, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, MongoDB, Kafka, Spark, Storm, Knox, Ranger, Flume, NiFi, Falcon, Kerberos, Sentry, Cloudera Manager, Navigator and Ambari.

Responsibilities

• Independently design and build/configure big data pipelines.
• Identify and solve big data problems independently.
• Work closely with a senior big data developer to report build issues.
• Fix defects during QA and UAT.
• Handle code lifecycle from low-level design to production and postproduction support.

Skills

Spark

Python

Scala

Java

HDFS

Hive

Kafka

PySpark

Tools

Cloudera Manager

Ambari

Oozie

Sqoop

Flume

NiFi

Falcon

Kerberos

Sentry

Job description

Purpose

Independently Design and build/configure Big data pipelines as per the design

Able to identify and solve big data problems independently.

Work closely with Sr. Big data developer to report any issues in build

Work on defect fixes during QA & UAT.

Work through the life cycle of the code from low-level design to production and Postproductions support.

Experience

8-15 yrs. of IT experience with more than 4 yrs. of big data implementation experience.

Capabilities

• Strong technical and analytic skills to solve complex big data problems.
• Experienced with Spark software stack, Prior experience in working on Spark using Python or Scala or Java and PySpark based big data implementations.
• Have strong knowledge and hands-on few of HDFS, Map reduce Hive, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, Mongo DB, Kafka, Spark, Storm, Knox, Ranger, Flume, NiFi, Falcon, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator and Ambari.

Enterprise Competencies

• Independent High level design and developer
• Strong communicator
• Ability to manage stakeholders' expectations.

Technical Competencies

• Big Data Stack (Spark, Hive, Hbase, Apache Pheonix, Pig), Python , Scala, Kafka, Spark Streaming, Atlas, Pig, Java ,SQL, Impala, Tej, Sqoop, Pig, Oozie, HBase, Cassandra, Mongo DB,
• Kafka, Spark, Storm, Knox, Ranger, Flume, Nifi, Falcon, Kerberos, Sentry, Cloudera Manager, Cloudera Navigator and Ambari.

Good Handson Experience in AWS Glue

Good Experience in CICD & DevOps concepts

Good experience working in any cloud environment (Azure / AWS Glue/ GCP) using Agile methodology