Big Data Engineer (5-15 Years)

Infosys; Dadri, Hyderabad, Bengaluru; Hybrid; INR 1,200,000 - 2,200,000; Full time

Job summary

Infosys is seeking an experienced Big Data Developer to design and optimize large-scale data processing systems and analytics platforms. The role focuses on distributed computing, cloud platforms, and data engineering practices to support data-driven initiatives.

You will build scalable pipelines, implement ETL/ELT processes, and enable real-time streaming applications while ensuring data quality and governance across the ecosystem.

Qualifications

• 5+ years of experience in Big Data development and architecture design.
• Proven expertise in designing, building, and optimizing large-scale data processing systems.
• Strong command of distributed data processing, data modeling, and data warehousing concepts.
• Hands-on experience with cloud platforms (AWS/Azure/GCP) and data governance practices.
• Ability to collaborate with data architects, scientists, analysts and stakeholders.

Responsibilities

• Design, develop, and maintain scalable Big Data solutions for processing and analyzing large volumes of data.
• Build and optimize data pipelines, ETL/ELT processes, and real-time streaming applications.
• Develop data ingestion frameworks from APIs, databases, files, and streaming sources.
• Implement batch and real-time data processing leveraging Big Data tech stacks.
• Collaborate with stakeholders to translate business requirements into data solutions.
• Ensure data quality, governance, and security across the data ecosystem.
• Optimize processing performance and troubleshoot production issues.
• Participate in architecture reviews, code reviews, and technical discussions.
• Support cloud migration and modernization initiatives.
• Develop and maintain technical documentation and operational procedures.

Skills

Hadoop Ecosystem

Apache Spark

Hive

HDFS

YARN

Python

Scala

Java

SQL

Apache Kafka

Spark Streaming

Apache Flink

Apache NiFi

Databricks

Snowflake

AWS

Azure

GCP

SQL Server

Oracle

PostgreSQL

MySQL

MongoDB

Cassandra

Tools

Databricks

Snowflake

Kafka

Job description

Position: Big Data Developer
Experience: 5 to 15 Years
Location: PAN India
Employment Type: Full-Time

About the Role

We are looking for experienced Big Data Developers to design, build, and optimize large-scale data processing systems and analytics platforms. The ideal candidate will possess strong expertise in Big Data technologies, distributed computing, cloud platforms, and data engineering practices to support enterprise data-driven initiatives.

Key Responsibilities

• Design, develop, and maintain scalable Big Data solutions for processing and analyzing large volumes of structured and unstructured data.
• Build and optimize data pipelines, ETL/ELT processes, and real-time streaming applications.
• Develop data ingestion frameworks from multiple sources including APIs, databases, files, and streaming platforms.
• Implement batch and real-time data processing solutions using Big Data technologies.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and stakeholders to understand business requirements.
• Ensure data quality, integrity, governance, and security across the data ecosystem.
• Optimize data processing performance and troubleshoot production issues.
• Participate in architecture reviews, code reviews, and technical discussions.
• Support cloud migration and modernization initiatives.
• Develop and maintain technical documentation and operational procedures.

Required Skills

Big Data Technologies

• Strong hands-on experience with:
• Hadoop Ecosystem
• Apache Spark (Core, SQL, Streaming)
• Hive
• HDFS
• YARN
• Experience developing distributed data processing applications.

Data Engineering

• Expertise in ETL/ELT development.
• Strong knowledge of data modeling and data warehousing concepts.
• Experience with large-scale data transformation and migration projects.
• Understanding of data governance and data quality frameworks.

Programming Skills

• Strong proficiency in:
• Python
• Scala
• Java
• SQL
• Experience with scripting and automation.

Streaming Technologies

• Experience with:
• Apache Kafka
• Spark Streaming
• Apache Flink (Preferred)
• Apache NiFi

Cloud Technologies

• Experience with one or more cloud platforms:
• AWS
• Azure
• Google Cloud Platform (GCP)

Database Technologies

• Strong experience with:
• SQL Server
• Oracle
• PostgreSQL
• MySQL
• MongoDB
• Cassandra (Preferred)

Tools & Technologies

• Apache Spark
• Hadoop
• Hive
• Kafka
• Scala
• Databricks
• Snowflake
• AWS
• Azure

Notice Period

Notice Period: Immediate to 30 Days Preferred

Keywords

Keywords: Big Data, Spark, Hadoop, Kafka, Hive, Databricks, Data Engineering, Python, Scala, AWS, Azure, GCP, Snowflake, Airflow, ETL, Data Lake, Data Warehouse, Real-Time Streaming.