Pyspark + SQL (Data Engineer)

Sonyo Management Consultants; Chennai District, Bengaluru, Pune District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Sonyo Management Consultants seeks a Pyspark + SQL expert for 6+ to 12 years of experience. The role requires strong SQL, Spark, Hadoop, Hive, and data processing skills. Location is Pune, Bangalore, Chennai, or Hyderabad.

Notice period 0-30 days; joiners only. JD emphasizes CASCADE-like joins, HDFS, Hive bucketing, and SparkSession usage for Hive tables. Only candidates with strong SQL and big data tech stack should apply.

Qualifications

• Strong SQL with complex queries and optimization.
• Experience with SparkSession, RDD, DataFrames, transformations.
• Familiarity with Hive partitions and bucketing.
• Knowledge of Hadoop and HDFS commands for file operations.

Skills

SQL

PySpark

Spark

Hadoop

Hive

Tools

HDFS

SparkSession

RDD

DataFrames

Job description

Pyspark + SQL

Job Description

Experience : 6+ Yrs till 12 Yrs

Grade : C1 /C2

Location : Pune/Bangalore/Chennai/Hyderabad Only

Notice period : 0-30 days Jul/Aug Joiner Only

NOTE:- Very Strong SQL Experience Required,

JD Preferred :-

• SQL: Joins, aggregate functions, CASCADE
• Hadoop: Basic HDFS commands for file transfer, read, search
• HIVE: Partitioning & Bucketing,
• Apache Spark: knowledge of SparkSession, RDD, Dataframes, Transformation, Shared variables, Data processing for Hive tables using spark.