Senior Databricks Engineer with Python

Luxoft; Pune District; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Luxoft India LLP seeks a Senior Databricks Engineer with strong Python skills to design and deploy scalable data pipelines using Databricks in Pune. You will work on large datasets and collaborate with cross-functional teams to derive insights.

The role emphasizes SQL proficiency, data quality checks, performance optimization, and guiding stakeholders on technical solutions.

Qualifications

• Proficient in Python programming.
• Experience with big data technologies such as Hadoop or Spark.
• Knowledge of cloud-based data platforms and databases.
• Strong problem-solving and attention to detail.
• Ability to collaborate in a team environment.
• Solid understanding of data modeling and database design.

Responsibilities

• Design, develop, and deploy scalable data pipelines using Databricks.
• Collaborate with cross-functional teams to identify business requirements and develop solutions.
• Develop and maintain complex SQL queries to extract insights from large datasets.
• Implement data quality checks and validation processes to ensure accuracy and integrity.
• Optimize database performance and troubleshoot issues as needed.
• Work closely with stakeholders to understand their needs and provide technical guidance.

Skills

Python

Databricks

SQL

Big Data

Tools

Databricks

Job description

We are looking for a skilled Senior Databricks Engineer with expertise in Python to join our team at Luxoft India LLP. The ideal candidate will have experience working with large datasets and developing scalable data pipelines.

Roles and Responsibility

• Design, develop, and deploy scalable data pipelines using Databricks.
• Collaborate with cross-functional teams to identify business requirements and develop solutions.
• Develop and maintain complex SQL queries to extract insights from large datasets.
• Implement data quality checks and validation processes to ensure accuracy and integrity.
• Optimize database performance and troubleshoot issues as needed.
• Work closely with stakeholders to understand their needs and provide technical guidance.

Job Requirements

• Strong proficiency in Python programming language.
• Experience working with big data technologies such as Hadoop or Spark.
• Knowledge of cloud-based databases and data platforms.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a team environment.
• Strong understanding of data modeling and database design principles.