Data Engineer

IGS Global LTD; Bengaluru; On-site; INR 1,400,000 - 1,700,000; Full time

Job summary

IGS GLOBAL is seeking a PySpark Data Engineer to design, build, and optimize scalable data pipelines. You will work on high-performance ETL workflows with Spark and cloud-native technologies to ensure reliable data delivery across platforms.

This is a hands-on, individual contributor role in a fast-paced engineering environment. The ideal candidate will work with large datasets, nested JSON, and complex SQL queries while collaborating with data scientists and engineers to deliver robust data

Qualifications

• Proficient in PySpark for designing and building data pipelines.
• Strong Python experience with NumPy and Pandas for data transformation.
• Expertise in writing complex SQL queries and optimizing data processing.
• Hands-on experience building ETL workflows and data pipelines.
• Able to handle nested JSON and complex data structures.

Responsibilities

• Design, build, and optimize scalable ETL pipelines using PySpark.
• Develop high-performance batch and real-time data processing systems.
• Extract/transform/load data from databases, APIs, and files.
• Work with large datasets including nested JSON structures.
• Implement robust data quality and performance improvements across pipelines.
• Collaborate with data scientists and engineers to enable data-driven solutions.

Skills

PySpark

Python NumPy Pandas

SQL

ETL

JSON handling

Tools

Airflow

AWS

GCP

Iceberg Delta Lake

Job description

Job Description

We are looking for a

Company: IGS GLOBAL

Website: Visit Website

LinkedIn: Visit LinkedIn

Business Type: Small/Medium Business

Company Type: Product & Service

Business Model: B2B

Funding Stage: Series C

Industry: Information Technology

Salary Range: ₹ 14-17 Lacs PA

Job Description

We are looking for a PySpark Data Engineer to design, build, and optimize scalable data pipelines and distributed data processing systems. You will work on high-performance ETL workflows using Spark and cloud-native technologies, ensuring reliable and efficient data delivery across platforms. This is a hands-on individual contributor role focused on building robust data products in a fast-paced, engineering-driven environment.

Key Responsibilities

• Design, build, and maintain scalable ETL pipelines using PySpark
• Develop high-performance batch and real-time data processing systems
• Extract, transform, and load data from multiple sources (databases, APIs, files)
• Work extensively with large and complex datasets, including nested JSON structures
• Build efficient data transformation logic using Python, Spark, NumPy, and Pandas
• Write and optimize complex SQL queries for data processing and analytics
• Collaborate with data scientists, analysts, and engineers to support data-driven applications
• Ensure data quality, consistency, and performance across pipelines
• Optimize Spark jobs for scalability and performance

Must-Have Skills

• Strong hands-on experience with PySpark (mandatory)
• Advanced Python skills with NumPy and Pandas
• Strong SQL skills (complex queries, joins, transformations)
• Experience building data pipelines and ETL workflows
• Ability to handle complex data transformations and nested JSON structures
• Experience integrating data from APIs, databases, and flat files
• Strong problem-solving skills in data processing and manipulation

Good-to-Have Skills

• Experience with Apache Airflow or similar workflow orchestration tools
• Familiarity with AWS or GCP cloud platforms
• Knowledge of modern data lake technologies (e.g., Iceberg, Delta Lake)
• Experience working in startup or fast-paced product environments
• Exposure to distributed systems and large-scale data platforms

Ideal Candidate Profile

We Are Looking For Engineers Who Are

• Strong in hands-on coding (not just theoretical knowledge)
• Passionate about data engineering and scalable systems
• Comfortable working in fast-paced, high-ownership environments
• Highly analytical, detail-oriented, and problem-solving driven
• Proactive, energetic, and committed to delivering high-quality data solutions