Big Data Engineer (8-16 Years)

Infosys; Dadri, Hyderabad, Bengaluru; Hybride; INR 1 800 000 - 3 000 000; Plein temps

Résumé du poste

Infosys is seeking experienced Big Data Developers to design, build, and optimize large-scale data processing systems and analytics platforms across PAN India. The role demands strong expertise in Hadoop, Spark, and streaming technologies to support data-driven initiatives.

The candidate will design scalable data solutions, build ETL/ELT pipelines, and collaborate with architects and analysts to ensure data quality and security across ecosystems. Immediate to 30 days notice preferred.

Qualifications

• Strong hands-on experience with Hadoop ecosystem and Spark-based processing.
• Expertise in ETL/ELT development and data warehousing concepts.
• Proficiency in Python, Scala, Java, and SQL for data engineering tasks.
• Experience with distributed data processing and real-time streaming architectures.

Responsabilités

• Design, develop, and maintain scalable Big Data solutions for large data volumes.
• Build and optimize data pipelines, ETL/ELT processes, and real-time streaming apps.
• Develop data ingestion frameworks from APIs, databases, files, and streaming sources.
• Implement batch and real-time data processing using Big Data tech.
• Collaborate with architects, scientists, analysts, and stakeholders.
• Ensure data quality, governance, and security across data ecosystems.
• Optimize performance and troubleshoot production issues.
• Participate in architecture and code reviews.
• Support cloud migration and modernization initiatives.
• Develop and maintain technical documentation and procedures.

Connaissances

Python

Scala

Java

SQL

Hadoop Ecosystem

Apache Spark (Core, SQL, Streaming)

Hive

HDFS

YARN

Apache Kafka

Spark Streaming

Apache Flink

Apache NiFi

Outils

Databricks

Snowflake

Airflow

AWS

Azure

GCP

Description du poste

Job description

Position: Big Data Developer
Experience: 5 to 15 Years
Location: PAN India
Employment Type: Full-Time

About the Role

We are looking for experienced Big Data Developers to design, build, and optimize large-scale data processing systems and analytics platforms. The ideal candidate will possess strong expertise in Big Data technologies, distributed computing, cloud platforms, and data engineering practices to support enterprise data-driven initiatives.

Key Responsibilities

• Design, develop, and maintain scalable Big Data solutions for processing and analyzing large volumes of structured and unstructured data.
• Build and optimize data pipelines, ETL/ELT processes, and real-time streaming applications.
• Develop data ingestion frameworks from multiple sources including APIs, databases, files, and streaming platforms.
• Implement batch and real-time data processing solutions using Big Data technologies.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and stakeholders to understand business requirements.
• Ensure data quality, integrity, governance, and security across the data ecosystem.
• Optimize data processing performance and troubleshoot production issues.
• Participate in architecture reviews, code reviews, and technical discussions.
• Support cloud migration and modernization initiatives.
• Develop and maintain technical documentation and operational procedures.

Required Skills

Big Data Technologies

• Strong hands-on experience with:
• Hadoop Ecosystem
• Apache Spark (Core, SQL, Streaming)
• Hive
• HDFS
• YARN
• Experience developing distributed data processing applications.

Data Engineering

• Expertise in ETL/ELT development.
• Strong knowledge of data modeling and data warehousing concepts.
• Experience with large-scale data transformation and migration projects.
• Understanding of data governance and data quality frameworks.

Programming Skills

• Strong proficiency in:
• Python
• Scala
• Java
• SQL
• Experience with scripting and automation.

Streaming Technologies

• Experience with:
• Apache Kafka
• Spark Streaming
• Apache Flink (Preferred)
• Apache NiFi

Cloud Technologies

• Experience with one or more cloud platforms:
• AWS
• Azure
• Google Cloud Platform (GCP)

Database Technologies

• Strong experience with:
• SQL Server
• Oracle
• PostgreSQL
• MySQL
• MongoDB
• Cassandra (Preferred)

Tools & Technologies

• Apache Spark
• Hadoop
• Hive
• Kafka
• Scala
• Databricks
• Snowflake
• AWS
• Azure

Notice Period: Immediate to 30 Days Preferred

Keywords: Big Data, Spark, Hadoop, Kafka, Hive, Databricks, Data Engineering, Python, Scala, AWS, Azure, GCP, Snowflake, Airflow, ETL, Data Lake, Data Warehouse, Real-Time Streaming.