AWS Glue Pyspark Developer - 8th April - Virtual Interview - Hyderabad

V2 Solutions; Hyderabad; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

V2 Solutions in Hyderabad is seeking a PySpark & AWS Glue Developer with 5-10 years of experience to design, develop, and maintain scalable ETL pipelines on AWS. The role focuses on PySpark 기반 AWS Glue jobs for robust data ingestion and transformation in analytics use cases.

You'll work with SQL in data validation, leverage UNIX/Linux scripting, and contribute to data warehouse or data lake architectures. S3/IAM exposure is expected; CI/CD and Airflow familiarity are a plus.

Qualifications

• 4–5 years of hands-on experience in ETL / Data Engineering.
• Strong PySpark experience for large-scale data processing.
• Hands-on AWS Glue experience (Glue Jobs, Crawlers, Data Catalog).

Responsibilities

• Design, develop, and maintain ETL pipelines using PySpark and AWS Glue.
• Build and optimize AWS Glue jobs (PySpark-based) for batch processing.
• Ingest and transform data into data lakes or data warehouses.
• Implement data quality checks, validations, and reconciliation.
• Optimize performance via partitioning, caching, and Spark transforms.
• Write and optimize SQL for data validation and analytics consumption.
• Prepare and maintain technical documentation including ETL mappings and LLDs.
• Support SIT/UAT and troubleshoot production issues as required.

Skills

PySpark

AWS Glue

SQL

UNIX/Linux

Shell scripting

Data warehousing

Job description

Location: PySpark & AWS Glue Developer Experience Range: 5-10 Years Location: Hyderabad Virtual Interview : 8th April (Wednesday) Timing : 10 am - 5pm

Job Description

We are looking for a PySpark & AWS Glue Developer with 45 years of experience to design, develop, and maintain scalable data ingestion and transformation pipelines on AWS. The role focuses on building robust ETL solutions using PySpark and AWS Glue to support analytics and reporting use cases.

Key Responsibilities

• Design, develop, and maintain ETL pipelines using PySpark and AWS Glue.
• Build and optimize AWS Glue jobs (PySpark-based) for batch data processing.
• Ingest and transform structured and semi-structured data into data lakes or data warehouses.
• Implement data quality checks, validations, and reconciliation mechanisms.
• Optimize performance through partitioning, caching, and efficient Spark transformations.
• Write and optimize SQL for data validation and downstream analytical consumption.
• Prepare and maintain technical documentation including ETL mappings and LLDs.
• Support SIT/UAT activities and troubleshoot production issues as required.

Required Skills & Experience

• 4–5 years of hands-on experience in ETL / Data Engineering roles.
• Strong experience with PySpark for large-scale data processing.
• Hands-on experience with AWS Glue (Glue Jobs, Crawlers, Data Catalog).
• Strong SQL skills for querying, transformations, and validation.
• Good understanding of data warehousing and data lake concepts.
• Exposure to AWS services such as S3 and IAM.
• Experience with UNIX/Linux and basic shell scripting.

Good-to-Have Skills

• Exposure to AWS orchestration services such as Step Functions or Airflow.
• Experience in performance tuning and cost optimization on AWS.
• Exposure to CI/CD practices and version control tools like Git.
• Exposure to DataStage or migration projects is a plus.