VS01472 - Data Migration Engineer

E4 Software Services Pvt Ltd.; Mumbai, Pune District, Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A software development firm in Mumbai is seeking a Data Engineer to design and deploy high-performance data solutions. The ideal candidate will have 4-8 years of experience in Databricks and strong skills in Python, PySpark, and SQL. Proficiency in AWS services is essential. The role requires collaboration with cross-functional teams and an agile work environment. This position offers competitive compensation and opportunities for growth.

Qualifications

• 4 to 8 years of experience in Databricks and big data frameworks.
• Proficient in AWS services and data migration.
• Experience in Unity Catalog.
• Familiarity with Batch and real-time processing.

Responsibilities

• Design, develop, test, and deploy high-performance data solutions using Python and SQL.
• Collaborate with teams to translate business requirements into technical specifications.
• Implement and optimize data pipelines for data integration.
• Utilize Databricks for big data processing and analytics.
• Maintain and optimize relational databases with SQL.

Skills

Data engineering

Python

PySpark

SQL

AWS

Collaboration

Problem-solving

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

AWS services (S3, Glue, Redshift, Lambda)

Job description

Overview

Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL

Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications.

Implement efficient and maintainable code using best practices and coding standards.

AWS & Databricks Implementation

• Work with Databricks platform for big data processing and analytics.
• Develop and maintain ETL processes using Databricks notebooks.
• Implement and optimize data pipelines for data transformation and integration.
• Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines.
• Leverage PySpark for large-scale data processing and transformation tasks.
• Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks.
• Share knowledge with the team and contribute to a culture of continuous improvement.

SQL Database Management

• Utilize expertise in SQL to design, optimize, and maintain relational databases.
• Write complex SQL queries for data retrieval, manipulation, and analysis

Qualifications & Skills

• Education: Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees are a plus.
• 4 to 8 Years of experience in Databricks and big data frameworks
• Proficient in AWS services and data migration
• Experience in Unity Catalogue
• Familiarity with Batch and real time processing
• Data engineering with strong skills in Python, PySpark, SQL
• Certifications: AWS Certified Solutions Architect, Databricks Certified Professional, or similar are a plus.

Soft Skills

• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.
• Ability to work in a fast-paced, agile environment