EY - GDS Consulting - AI and DATA - AWS - Senior

EY; Ernakulam; On-site; INR 10,00,000 - 15,00,000; Full time

Job summary

A leading professional services firm in Ernakulam seeks a Senior AWS Databricks professional with 3-7 years of experience in data warehousing and ETL design. The ideal candidate will have hands-on experience with programming languages like Python and expertise in AWS services. The role requires excellent analytical and communication skills, as well as deep knowledge of data quality management and SQL query writing.

Qualifications

• 3-7 years of experience in Data warehousing and ETL.
• Hands-on experience with Python, PySpark/Scala.
• Proficiency in SQL and database components.

Responsibilities

• Implement data warehousing and ETL solutions.
• Manage and utilize AWS services effectively.
• Create and perform unit testing for ETL jobs.

Skills

Data warehousing and ETL Design

Programming languages like Python

Spark architecture and internals

Databricks on AWS

AWS services (Glue, Lambda, S3, etc.)

Complex SQL query writing

Data Quality Management

Analytical skills

Excellent Communication Skills

Job description

Overview

AWS Databricks - Senior - Job Description

Responsibilities

• Expertise in Data warehousing and ETL Design and implementation
• Hands on experience with Programming language like Python, PySpark/Scala
• Good understanding of Spark architecture along with internals
• Expert in working with Databricks on AWS
• Hands on experience using AWS services like Glue(Pyspark), Lambda, S3, Athena, RDS, IAM, Lake formation
• Hands on experience on implementing different loading strategies like SCD1 and SCD2, Table/ partition refresh, insert update, Swap Partitions
• Experience in consuming and writing data from and to Flat files, RDMBS systems, MPPS, JSON and XML, Service, Streams, queues, CDC etc. Awareness of scheduling and orchestration tools
• Awareness of OS Compute, Networking, internal working and architecture of DB and ETL Server and its impact on ETL
• Experience on RDBMS systems and concepts
• Expertise in writing and complex SQL queries and developing Database components including creating views, stored procedures, triggers etc.
• Create test cases and perform unit testing of ETL Jobs
• Analytical mind with a problem-solving and debugging skills
• Excellent Communication Skills
• Awareness of Replication, Synchronization and Disaster Management techniques
• Hand on experience in Data Quality Management
• Awareness of data governance concepts and implementation

Experience

Experience 3-7 Years of experience.