Databricks Developers

Cynosure Corporate Solutions; Chennai District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Cynosure Corporate Solutions is seeking a Data Engineer to develop and optimize data pipelines using Databricks, Spark SQL, and PySpark. The role includes integrating data sources, transforming data, and enabling analytics with dashboards. You will collaborate with front-end teams and use GitLab for CI/CD and Jira for task management.

Acquire experience with Unity Catalog, ETL workflows, and cloud platforms (AWS/Azure) while working on healthcare or clinical trial data.

Qualifications

• Master’s or Bachelor’s degree in Data Engineering or related field.
• 3+ years experience with Databricks and advanced SQL.
• Experience with ETL processes, views, and procedures.
• Strong experience with Databricks, Spark SQL, PySpark, and SQL.
• Expertise in creating and optimizing views and stored procedures in Databricks.
• Experience building ETL workflows and data models.
• Knowledge of cloud platforms (AWS, Azure) and version control tools.
• Experience with healthcare or clinical trial data.

Responsibilities

• Develop and optimize data pipelines in Databricks for transforming and processing data from various sources.
• Integrate data using Unity Catalog and external data sources (data lakes, APIs, etc.).
• Write Spark SQL and PySpark scripts for data transformations, optimizations, and creating views/procedures.
• Perform data analysis to identify quality issues, optimize pipelines, and enhance data processing for analytics.
• Collaborate on report generation and dashboard creation with front-end teams.
• Use GitLab for version control, CI/CD automation, and task management (Jira).

Skills

Databricks

Spark SQL

PySpark

SQL

ETL

Data pipelines

Data modeling

GitLab

CI/CD

Jira

Education

Master’s or Bachelor’s degree in Data Engineering or related field

Tools

GitLab

Jira

Job description

Cynosure Corporate Solutions | Full time

Cynosure Corporate Solutions is a human capital services company. Our focus is to provide Executive Search, Recruitment, Training, Temporary Staffing services, Statutory Compliance's, and other HR aligned services. We understand the business goals of our clients and their need to align human resources to these goals. We are committed to provide high quality manpower in accordance to global standards and their requirements. Our candidates stand testimony to our professional competency.

We work with clients in IT/ITES, Manufacturing, Automobile, Health, Telecom, Media.

Job Description

• Develop and optimize data pipelines in Databricks for transforming and processing data from various sources.
  
• Integrate data using Unity Catalog and external data sources (data lakes, APIs, etc.).
  
• Write Spark SQL and PySpark scripts for data transformations, optimizations, and creating views/procedures.
  
• Perform data analysis to identify quality issues, optimize pipelines, and enhance data processing for analytics.
  
• Collaborate on report generation and dashboard creation with front-end teams.
  
• Use GitLab for version control, CI/CD automation, and task management (Jira).
  

Requirements

• Master’s or Bachelor’s degree in Data Engineering or related field.
  
• 3+ years experience with Databricks and advanced SQL.
  
• Experience with ETL processes, views, and procedures.
  
• Strong experience with Databricks, Spark SQL, PySpark, and SQL.
  
• Expertise in creating and optimizing views and stored procedures in Databricks.
  
• Experience building ETL workflows and data models.
  
• Knowledge of cloud platforms (AWS, Azure) and version control tools (Git, GitLab).
  
• Experience with healthcare or clinical trial data.