Data Engineer (pl/sql, Python, Spark, Hadoop & Etl)

Cognizant; Chennai District; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Cognizant in Chennai is seeking a Data Engineer with hands-on expertise in PL/SQL, Python, Apache Spark, Hadoop and ETL to design and maintain scalable data pipelines for enterprise data. The role involves optimizing large-scale processing, ensuring data quality, and integrating data across systems.

The ideal candidate will work with Oracle/PostgreSQL, Hive, HDFS and related technologies, collaborating with architects and analysts. Familiarity with BI data warehousing concepts is preferred.

Qualifications

• Experience with PL/SQL and SQL performance tuning.
• Hands-on Python for data engineering and automation.
• Strong knowledge of Apache Spark (PySpark).
• Experience with Hadoop ecosystem (HDFS, Hive, MapReduce, or YARN).
• Experience with enterprise ETL tools (Informatica, Talend, DataStage, SSIS, NiFi, Glue or Matillion).
• Understanding of data warehousing concepts and dimensional modeling.
• Experience with relational databases (Oracle, PostgreSQL, SQL Server, MySQL).

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for large data volumes.
• Develop and optimize PL/SQL procedures, functions, packages, and SQL queries.
• Build data processing applications using Python and Spark (PySpark).
• Develop and maintain big data solutions using Hadoop ecosystem.
• Design, implement, and optimize ETL workflows using enterprise ETL tools.
• Perform data extraction, transformation, validation, and loading from multiple sources.
• Analyze and resolve data quality issues, bottlenecks, and gaps.
• Optimize batch processing jobs and overall performance.
• Collaborate with data architects, analysts, and application teams.
• Ensure data accuracy, security, and compliance with standards.
• Prepare technical documentation and support deployments.

Skills

PL/SQL

Python

Apache Spark

Hadoop

ETL

SQL performance tuning

Relational databases

Tools

Informatica

Talend

DataStage

SSIS

Apache NiFi

AWS Glue

Matillion

Job description

Job Title: Data Engineer (PL/SQL, Python, Spark, Hadoop & ETL)

Experience: 6 and above
Location: Chennai

Job Summary :-We are looking for a skilled Data Engineer with strong expertise in PL/SQL, Python, Apache Spark, Hadoop, and ETL tools to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, optimizing large-scale data processing, and ensuring high-quality data integration across enterprise systems. The role requires hands-on experience with big data technologies, ETL development, and database programming.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for processing large volumes of structured and unstructured data.
• Develop and optimize complex PL/SQL procedures, functions, packages, and SQL queries.
• Build data processing applications using Python and Apache Spark (PySpark).
• Develop and maintain big data solutions using the Hadoop ecosystem.
• Design, implement, and optimize ETL workflows using enterprise ETL tools.
• Perform data extraction, transformation, validation, and loading from multiple source systems.
• Analyze and resolve data quality issues, performance bottlenecks, and data gaps.
• Optimize batch processing jobs and improve overall system performance.
• Collaborate with data architects, business analysts, and application teams to understand business requirements.
• Ensure data accuracy, consistency, security, and compliance with organizational standards.
• Prepare technical documentation and support production deployments.

Required Technical Skills

• Strong expertise in PL/SQL and SQL performance tuning.
• Hands-on experience with Python for data engineering and automation.
• Strong knowledge of Apache Spark (PySpark).
• Experience working with the Hadoop ecosystem (HDFS, Hive, MapReduce, or YARN).
• Experience with ETL tools such as Informatica, Talend, DataStage, SSIS, Apache NiFi, AWS Glue, or Matillion.
• Good understanding of data warehousing concepts and dimensional modeling.
• Experience working with relational databases such as Oracle, PostgreSQL, SQL Server, or MySQL.
• Strong analytical and problem-solving skills.