VS01472 - Data Migration Engineer

E4 Software Services Pvt Ltd.; Mumbai, Pune District, Bengaluru; On-site; INR 800,000 - 1,300,000; Full time

Job summary

A tech company specializing in data solutions is seeking a Data Engineer to design, develop, and deploy data solutions using Python, SQL, and Databricks. The ideal candidate will have 4 to 8 years of experience with AWS services and big data frameworks, along with strong problem-solving skills. Join a collaborative environment where you can work on cutting-edge data solutions.

Qualifications

• 4 to 8 years of experience in Databricks and big data frameworks.
• Certifications: AWS Certified Solutions Architect, Databricks Certified Professional, or similar are a plus.
• Familiarity with Batch and real-time processing.

Responsibilities

• Design, develop, test, and deploy high-performance data solutions using Python, SQL.
• Collaborate with cross-functional teams to translate business requirements.
• Implement efficient and maintainable code using best practices.
• Develop and maintain ETL processes using Databricks notebooks.
• Optimize data pipelines for data transformation and integration.

Skills

Data engineering with strong skills in Python

Proficient in AWS services

Experience in Databricks and big data frameworks

Strong problem-solving and analytical skills

Excellent communication and collaboration abilities

Education

Bachelor’s degree in Computer Science, Engineering, Data Science, or related field

Tools

Python

PySpark

SQL

AWS (S3, Glue, Redshift, Lambda)

Databricks

Job description

Overview

Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL

Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications.

Implement efficient and maintainable code using best practices and coding standards.

AWS & Databricks Implementation

• Work with Databricks platform for big data processing and analytics.
• Develop and maintain ETL processes using Databricks notebooks.
• Implement and optimize data pipelines for data transformation and integration.
• Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines.
• Leverage PySpark for large-scale data processing and transformation tasks.
• Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks.
• Share knowledge with the team and contribute to a culture of continuous improvement.

SQL Database Management

• Utilize expertise in SQL to design, optimize, and maintain relational databases.
• Write complex SQL queries for data retrieval, manipulation, and analysis

Qualifications & Skills

• Education: Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees are a plus.
• 4 to 8 Years of experience in Databricks and big data frameworks
• Proficient in AWS services and data migration
• Experience in Unity Catalogue
• Familiarity with Batch and real time processing
• Data engineering with strong skills in Python, PySpark, SQL
• Certifications: AWS Certified Solutions Architect, Databricks Certified Professional, or similar are a plus.

Soft Skills

• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.
• Ability to work in a fast-paced, agile environment