Data Engineer

The Hiring Club; Chennai District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Publicis Sapient in Chennai is seeking a Senior Associate L1 Data Engineer to design, develop and deliver data ingestion, integration and transformation solutions. You will work on end-to-end data pipelines using Hadoop ecosystem tools and cloud platforms.

You should have hands-on experience with Java, Scala or Python, and strong knowledge of data ingestion, processing and analytics. This role requires collaboration in agile teams and delivering scalable data workloads.

Qualifications

• 3.5+ years IT experience with 1.5+ years in Data technologies.
• 1.5+ years experience in Big Data technologies.
• Hands-on experience with Hadoop stack including HDFS, Sqoop, Kafka, Pulsar, NiFi, Spark and related components for end-to-end data pipelines.
• Strong programming experience in Java, Scala, or Python.
• Hands-on knowledge of NoSQL and MPP platforms like HBase and MongoDB.

Responsibilities

• Design, development and delivery of data ingestion, integration and transformation solutions.
• Data storage, computation frameworks and performance optimizations for analytics pipelines.
• Build functionality for data ingestion from multiple heterogeneous sources in batch and real-time.
• Data analytics, search and aggregation capabilities.
• Infrastructure and cloud computing components of data platforms.
• Build and maintain data pipelines end-to-end and ensure data quality.

Skills

Java

Scala

Python

Hadoop

Spark

Kafka

Pulsar

NiFi

Airflow

HBase

MongoDB

SQL

Tools

HDFS

Sqoop

Kafka

NiFi

Flink

Oozie

Airflow

Hive

Job description

We at Publicis Sapient, enable our clients to thrive in Next and to create

business value through expert strategies, customer-centric experience design,

The future of business is disruptive, transformative and becoming digital to the

core.

In our 20 + years in IT, never before have we seen such a dire need for

transformation in every major industry - from financial services to automotive,

consumer products, retail, energy, and travel.

To make this transformative journey a reality in these exciting times, we seek

thought leaders and rock stars who will:

• brave it out to go do the next; what will be from what is
• exhibit the optimism that says there is no limit to what we can achieve
• Reimagine the way the world works to help businesses improve the
  daily lives of people and the world.

Our people thrive because of the belief that it is both our privilege and

responsibility to usher our clients and the world into Next.

Our work is fueled by

• multidisciplinary collaboration,
• highly agile teams, and
• the power of the newest technologies and platforms.

If thats you, come talk to us!

This is the world-class engineering team where you should build your career.

Job Summary:

As Senior Associate L1 in Data Engineering, you will do technical design, and

understanding of data integration and big data design principles in creating

custom solutions or implementing package solutions. You will independently

drive design discussions to insure the necessary health of the overall solution

The role requires a hands‑on technologist who has strong programming

background like Java / Scala / Python, should have experience in Data

Ingestion, Integration and data Wrangling, Computation, Analytics pipelines

and exposure to Hadoop ecosystem components. Having hands‑on knowledge

on at least one of AWS, GCP, Azure cloud platforms will be preferable.

Role & Responsibilities:

Job Title: Senior Associate L1 Data Engineering

Your role is focused on Design, Development and delivery of solutions

involving:

• Data Ingestion, Integration and Transformation
• Data Storage and Computation Frameworks, Performance
• Optimizations
• Analytics & Visualizations
• Infrastructure & Cloud Computing
• Data Management Platforms
• Build functionality for data ingestion from multiple heterogeneous
  sources in batch & real-time
• Build functionality for data analytics, search and aggregation

Experience Guidelines:

Mandatory Experience and Competencies:

# Competency

1. 1. Overall 3.5+ years of IT experience with 1.5+ years in Data related technologies
2. 2. Minimum 1.5 years of experience in Big Data technologies
3. 3. Hands‑on experience with the Hadoop stack HDFS, sqoop, kafka, Pulsar, NiFi, Spark,
   Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in
   building end to end data pipeline. Working knowledge on real-time data pipelines is added
   advantage.
4. 4. Strong experience in at least of the programming language Java, Scala, Python. Java
5. 5. Hands‑on working knowledge of NoSQL and MPP data platforms like Hbase, MongoDb,

Preferred Experience and Knowledge (Good to Have):

# Competency

1. 1. Good knowledge of traditional ETL tools (Informatica, Talend, etc) and database
   technologies (Oracle, MySQL, SQL Server, Postgres) with hands on experience
2. 2. Knowledge on data governance processes (security, lineage, catalog) and tools like
3. 3. Knowledge on distributed messaging frameworks like ActiveMQ / RabbiMQ / Solace,
   search & indexing and Micro services architectures
4. 4. Performance tuning and optimization of data pipelines
5. 5. CI/CD Infra provisioning on cloud, auto build & deployment pipelines, code quality
6. 6. Working knowledge with data platform related services on at least 1 cloud platform, IAM
   and data security
7. 7. Cloud data specialty and other related Big data technology certifications

Job Title: Senior Associate L1 Data Engineering

Personal Attributes:

• Strong written and verbal communication skills
• Good team player
• Self-starter who requires minimal oversight
• Ability to prioritize and manage multiple tasks