Data Engineer - Only Immediate Joiner

Tekskills Inc.; Chennai; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Tekskills Inc. seeks a hands-on Data Engineer with strong Python and PySpark skills to build PySpark applications using Spark DataFrames in Python, and optimize Spark jobs that process large volumes of data.

The role involves working with AWS data services such as EMR, Glue and Athena, utilizing S3 for storage, and collaborating with teams using Git for version control.

Qualifications

• Hands-on experience in Python and PySpark.
• Build PySpark applications using Spark DataFrames in Python.
• Experience optimizing Spark jobs processing large data volumes.
• Knowledge of AWS data services and storage (EMR, Glue, Athena, S3).
• Version control with Git is required.

Skills

Python

PySpark

Spark DataFrames

Git

AWS

Amazon EMR

Amazon Athena

AWS Glue

AWS Lambda

Amazon EC2

S3

Parquet

ORC

DynamoDB

Redshift

RDS

Aurora

Job description

• Mandatory - Hands on experience in Python and PySpark.
• Build pySpark applications using Spark Dataframes in Python.
• Worked on optimizing spark jobs that processes huge volumes of data.
• Hands on experience in version control tools like Git.
• Worked on Amazons Analytics services like Amazon EMR, Amazon Athena, AWS Glue.
• Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS.
• Good to have knowledge of datawarehousing concepts – dimensions, facts, schemas- snowflake, star etc.
• Have worked with columnar storage formats- Parquet,Avro,ORC etc. Well versed with compression techniques – Snappy, Gzip.
• Good to have knowledge of AWS databases (atleast one) Aurora, RDS, Redshift, ElastiCache, DynamoDB.

Bigdata, AWS, python, Pyspark, AWS services - IAM, lambda, EMR, glue