AWS Data Engineer - AWS Glue

Advance Career Solutions; Pune District; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Advance Career Solutions in Pune, Maharashtra, is seeking an experienced Data Engineer with hands-on expertise in AWS Glue and Databricks to design, develop, and maintain scalable data pipelines and data processing solutions.

You will build ETL/ELT pipelines using PySpark, Python, and SQL, work with AWS S3, Delta Lake, and Spark, and optimize pipelines for performance, reliability, and cost.

Strong knowledge of distributed data processing and CI/CD practices is a plus.

Qualifications

• Hands-on experience with AWS Glue and Databricks.
• Strong PySpark, Python, and SQL skills.
• Knowledge of Delta Lake and data warehousing concepts.

Responsibilities

• Develop and maintain scalable ETL /ELT pipelines using AWS Glue, Databricks, and PySpark.
• Build data ingestion and transformation workflows using Python and SQL.
• Work with AWS S3, Data Lake, Spark, and Delta Lake for large-scale data processing.
• Optimize data pipelines for performance, scalability, reliability, and cost.
• Perform data validation, troubleshooting, and production support.
• Collaborate with data architects, analysts, and other engineering teams to deliver data solutions.
• Follow best practices for data quality, security, monitoring, and deployment.

Skills

AWS Glue

Databricks

PySpark

Python

SQL

Apache Spark

AWS S3

Delta Lake

Git

CI/CD

Agile methodologies

Job description

AWS Glue / Databricks Data Engineer

We are looking for an experienced Data Engineer with strong hands-on expertise in AWS Glue and Databricks to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Develop and maintain scalable ETL /ELT pipelines using AWS Glue, Databricks, and PySpark.
• Build data ingestion and transformation workflows using Python and SQL.
• Work with AWS S3, Data Lake, Spark, and Delta Lake for large-scale data processing.
• Optimize data pipelines for performance, scalability, reliability, and cost.
• Perform data validation, troubleshooting, and production support.
• Collaborate with data architects, analysts, and other engineering teams to deliver data solutions.
• Follow best practices for data quality, security, monitoring, and deployment.

Required Skills

• Strong hands-on experience with AWS Glue and Databricks.
• Good experience with PySpark, Python, and SQL.
• Strong understanding of Apache Spark and distributed data processing.
• Experience with AWS S3 and cloud-based data lakes.
• Experience developing and supporting production-grade data pipelines.
• Knowledge of Delta Lake and data warehousing concepts is preferred.
• Familiarity with Git, CI/CD, and Agile methodologies is a plus.

Primary Skills

AWS Glue, Databricks, PySpark, Python, SQL, AWS S3, Apache Spark