BI & Data Management – ETL / Data Pipeline, Pyspark and AWS GLUE

Glan Management Consultancy; Gurugram District; On-site; INR 800,000 - 1,400,000; Full time

Job summary

A consulting firm located in Gurugram is seeking a BI & Data Management professional with 3-8 years of experience to develop and manage ETL/Data Pipelines. The ideal candidate should have strong skills in Pyspark and AWS Glue, along with the ability to handle various data formats and optimize queries. This role involves building data platforms on AWS and requires immediate joining. Interested candidates can send their updated resume to the provided email.

Qualifications

• 3-8 years of experience in building and managing ETL/Data Pipelines.
• Deep technical skills in AWS Glue including Crawler and Data Catalog.
• Strong knowledge in query writing and optimization.
• Experience to build data platforms on AWS using core AWS services Step functions, EMR, Lambda, Athena
• Good experience in processing variety of data format (ORC, Parquet, json, xml) using pyspark
• Good experience in modern data platform components such as the Hadoop, Hive, Pig, kafka

Responsibilities

• Build and manage ETL/Data Pipelines for handling large amounts of data.
• Data extraction, transformation, and processing using Pyspark.
• Build data platforms on AWS using core AWS services.

Skills

ETL/Data Pipeline management

Pyspark

AWS Glue

Data extraction and transformation

SQL query optimization

AWS EMR

Hive

Kafka

Education

Any graduate

Tools

AWS EMR

Hadoop

Kafka

Job description

Share Job :# BI & Data Management – ETL / Data Pipeline, Pyspark and AWS GLUE **Company:** Glan Management Consultancy **Location:** Gurgaon **Experience:** 3-10 year **Salary:** 0 **Employment Type:****Job Description:**Position – BI & Data Management – ETL / Data Pipeline, Pyspark and AWS GLUE Experience- 3-8 yearSalary – negotiableLocation – GurgaonIndustry - ITQualification - Any graduateImmediate joiner prefers.Job description: · Experience in building and managing ETL/Data Pipelines handling large amounts of data. · Data extraction, transformation, designing, coding, and tuning big data processes usingSpark,Pyspark · Deep technical skills in AWS Glue (Crawler, Data Catalog, DataBrew) · Experience to build data platforms on AWS using core AWS services Step function, EMR,Lambda,Athena · Good experience in processing variety of data format (ORC ,Parquet,json,xml)using pyspark · good experience in modern data platform components such as the Hadoop, Hive, Pig,kafka · Strong knowledge in query writing and query optimization mail me updated resume with below details-current ctc-expected ctc-notice period-current [e-mail]: 8802749743website: www.glansolutions.com### Apply for BI & Data Management – ETL / Data Pipeline, Pyspark and AWS GLUE