pyspark

Tekskills Inc.; Hyderabad; Hybrid; INR 1,200,000 - 2,100,000; Full time

Job summary

Tekskills Inc. is seeking a Data Engineer with 4+ years of PySpark and GCP experience to design and implement scalable data pipelines. You will work with Airflow, Dataproc, and BigQuery to transform data, writing advanced SQL and Python, and collaborating with cross-functional teams.

The role requires strong analytical skills, ability to operate under deadlines, and hands-on experience with ETL tools. Hyderabad/Pune on-site position offering growth opportunities.

Qualifications

• At least 3-5 years of experience in PySpark with GCP (Airflow, Dataproc, BigQuery) capable of configuring data pipelines.
• Strong Experience in writing complex SQL queries to perform data analysis on Databases SQL Server, Oracle, Hive.
• Possess the following technical skills SQL, Python, PySpark, Hive, ETL, Unix, Control-M (or similar scheduling tools)
• Ability to work independently on specialized assignments within the context of project deliverables

Responsibilities

• Take ownership of providing solutions and tools that iteratively increase engineering efficiencies.
• Design should help embed standard processes, systems and operational models into the BAU approach for end-to-end execution of Data Pipelines
• Proven problem solving and analytical abilities including the ability to critically evaluate information gathered from multiple sources
• Communicate openly and honestly with advanced oral/written/visual communication and presentation skills

Skills

PySpark

GCP

SQL

Python

ETL

Unix

Tools

Airflow

Dataproc

BigQuery

Hive

Control-M

Datastage

Prophecy

Pentaho

Job description

JOB DISCRIPTION

Experience: 4+Years

Location: Hyderabad/pune

Must have skill - Pyspark + GCP - The must-have skills are deep ETL tool experience, PySpark with GCP pipeline expertise, advanced SQL, Python.

Description:

• At least 3-5 years of experience in Pyspark with GCP (Airflow, Dataproc, Big query) capable of configuring data pipelines.
• Strong Experience in writing complex SQL queries to perform data analysis on Databases SQL server, Oracle, HIVE etc.
• Possess the following technical skills SQL, Python, Pyspark, Hive, ETL, Unix, Control-M (or similar scheduling tools)
• Ability to work independently on specialized assignments within the context of project deliverables
• Take ownership of providing solutions and tools that iteratively increase engineering efficiencies.
• Design should help embed standard processes, systems and operational models into the BAU approach for end-to-end execution of Data Pipelines
• Proven problem solving and analytical abilities including the ability to critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details and apply sound business and technical domain knowledge
• Communicate openly and honestly. Advanced oral, written and visual communication and presentation skills - the ability to communicate efficiently at a global level is paramount.
• Ability to deliver materials of the highest quality to management against tight deadlines.
• Ability to work effectively under pressure with competing and rapidly changing priorities
• Good to have: experience in any of the ETL tools Prophecy, Datastage 11.5/11.7, Pentaho.. etc.