Databricks Data Engineer - Contract-to-Hire

Customized Energy Solutions; Pune District; On-site; INR 1,600,000 - 2,800,000; Full time

Benefits offered by this job

Medical insurance

Paid time off

Professional development

Job summary

Customized Energy Solutions India Pvt. Ltd. (CES India) is seeking a Databricks Data Engineer in Pune, India. This role will drive a large modernization effort to migrate an enterprise SQL Server warehouse to the Databricks Lakehouse, emphasizing scalable pipelines, Delta Lake, and governance.

The ideal candidate has 4–7 years of data engineering experience with Spark/SQL and Databricks on AWS (or Azure), working with analytics, BI, and AI/ML teams to enable advanced analytics.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 4-7 years of overall data engineering experience.
• Hands-on Databricks on AWS (preferred); Azure acceptable.
• Strong proficiency in Spark and SQL.
• Experience migrating on-prem SQL Server data warehouses to cloud platforms.
• Experience converting SSIS-based ETL to Spark-based solutions.
• Understanding of data warehousing, dimensional modeling, and analytics workloads.
• Familiarity with Delta Lake, data versioning, and Databricks tools.

Responsibilities

• Design, build, and optimize scalable data pipelines with Databricks.
• Migrate ~20TB on-prem SQL Server data warehouse to Databricks.
• Convert SSIS ETL processes to Databricks notebooks and workflows.
• Support migration of cube-based analytics into Databricks SQL and Delta tables.
• Implement data quality, validation, reconciliation, and audit controls.
• Optimize performance and cost via Spark tuning and partitioning.
• Collaborate with analytics, BI, and AI teams for reporting and analytics.
• Apply data governance and access-control standards using Unity Catalog.

Skills

Spark (PySpark/Scala)

SQL

Databricks on AWS/Azure

Data warehousing concepts

Delta Lake

Databricks Workflows/Jobs

Education

Bachelors in CS/Engineering

Tools

Databricks

Azure

AWS

Job description

• Full-time

Company Description

Customized Energy Solutions India Pvt. Ltd. (CES India) is part of Customized Energy Solutions (CES), a globally recognized, privately held energy market intelligence, consulting, and operational services firm supporting retail and wholesale electricity and natural gas markets worldwide.

Headquartered in Philadelphia, CES operates across the United States, Canada, Japan, India, and Vietnam, with approximately 400 associates globally. The company has earned multiple distinctions from INC. 500 | 5000 and the Philadelphia Business Journal, including Hall of Fame status and repeated recognition as a Best Place to Work.

Established in 2010, CES India plays a critical role in supporting both domestic and international energy initiatives. The India team brings expertise in emerging technologies transforming the energy landscape, including energy storage, electric mobility, wholesale electricity markets, microgrids, and renewable energy systems. Associates in India collaborate closely with global teams while contributing deep technical and analytical capabilities to complex energy market challenges.

CES India actively engages with industry bodies and ecosystem partners that are shaping India’s energy and technology landscape, including organizations such as the India Electronics and Semiconductor Association (IESA). Through industry collaboration, technology innovation, and engagement with public and private stakeholders, CES India supports the broader goal of advancing sustainable, high-growth energy and manufacturing sectors in India.

At CES India, you will work in a fast-evolving industry where innovation, accountability, and professional growth are valued. Your work will contribute to meaningful outcomes across both regional and global energy markets.

Job Description

We are seeking a Databricks Data Engineer to join our growing data engineering team in Pune, India. This role will play a key part in a large-scale modernization initiative to migrate a complex, enterprise-grade Microsoft SQL Server data warehouse ecosystem to the Databricks Lakehouse Platform. The ideal candidate has strong hands-on experience across Databricks data engineering capabilities, with exposure to AI/ML features being a plus, while maintaining a core focus on scalable, reliable data pipelines and analytics workloads.

Key Responsibilities

• Design, build, and optimize scalable data pipelines using Databricks (Apache Spark, Delta Lake, Unity Catalog).
• Participate in the migration of a ~20TB compressed on-prem Microsoft SQL Server data warehouse to Databricks.
• Convert and modernize hundreds of SQL Server tables, thousands of SSIS jobs, and downstream SSRS/SSAS workloads.
• Re-engineer SSIS ETL processes into Databricks notebooks, workflows, and orchestration frameworks.
• Support migration or redesign of cube-based analytics (SSAS) into Databricks SQL, Delta tables, and modern semantic models.
• Implement data quality, validation, reconciliation, and audit controls during migration.
• Optimize performance and cost through efficient Spark usage, partitioning, and query tuning.
• Collaborate with analytics, BI, and AI/ML teams to enable downstream reporting and advanced analytics.
• Apply data governance, security, and access-control standards using Unity Catalog.
• Contribute to reusable frameworks, documentation, and platform best practices.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 4-7 years of overall data engineering experience.
• Hands-on experience with Databricks on AWS (preferred); Azure experience acceptable.
• Strong proficiency in Spark (PySpark and/or Scala) and SQL.
• Proven experience migrating on-prem SQL Server data warehouses to cloud-based data platforms.
• Experience converting SSIS-based ETL pipelines into Spark-based data engineering solutions.
• Solid understanding of data warehousing concepts, dimensional modeling, and analytical workloads.
• Experience with Delta Lake, incremental processing patterns, and data versioning.
• Familiarity with Databricks Workflows, Jobs, and production-grade deployments.
• Practical experience with performance tuning and large-volume data processing.

Preferred Skills

• Experience modernizing SSAS cube-based reporting solutions.
• Exposure to Databricks SQL Warehouses and BI integrations (Power BI preferred).
• Working knowledge of cloud-native data engineering practices aligned with Databricks best practices.
• Familiarity with MLflow, feature engineering, or AI-enablement within Databricks.
• Experience working in environments that follow Databricks-recommended data engineering patterns.
• Databricks certification is a plus.

Additional Information

Customized Energy Solutions India offers competitive compensation commensurate with experience, along with performance-based incentives aligned to individual and company results. Our benefits are designed to support both professional and personal well-being and include comprehensive medical insurance coverage for associates and eligible dependents, generous paid time off, and structured opportunities for professional development through training programs and skill enhancement initiatives. We are committed to creating an environment where associates are supported, challenged, and positioned for long-term career growth.

Customized Energy Solutions provides equal employment opportunities to all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.