Databricks Data Engineer - Contract-to-Hire

Ces Ltd; Maharashtra; On-site; INR 1,500,000 - 2,300,000; Full time

Benefits offered by this job

Medical insurance for associates and 6

Paid time off

Job summary

Customized Energy Solutions India Pvt. Ltd. (CES India) invites a Databricks Data Engineer to join our Pune data engineering team.

You will lead a large-scale modernization project migrating a ~20TB on-prem Microsoft SQL Server data warehouse to the Databricks Lakehouse Platform, enabling scalable analytics and AI-ready workloads. The ideal candidate has strong hands-on experience with Spark (PySpark/Scala), SQL, Delta Lake, and cloud-based data platforms on AWS or Azure, with a focus on

Qualifications

• Bachelors degree in Computer Science, Engineering, or related field.
• 4-7 years of data engineering experience.
• Hands-on Databricks on AWS (preferred) or Azure acceptable.
• Strong Spark (PySpark/Scala) and SQL proficiency.
• Experience migrating on-prem SQL Server DW to cloud-based data platforms.
• Experience converting SSIS ETL to Spark-based solutions.

Responsibilities

• Design, build, and optimize scalable data pipelines in Databricks.
• Migrate ~20TB on-prem SQL Server DW to Databricks Lakehouse.
• Convert SSIS ETL processes to Databricks notebooks and workflows.
• Migrate or redesign SSAS analytics to Databricks SQL and Delta tables.
• Implement data quality, validation, reconciliation, and audit controls.
• Optimize performance and cost through Spark tuning and partitioning.
• Collaborate with BI/Analytics and AI/ML teams for downstream reporting.
• Apply data governance and Unity Catalog security controls.
• Contribute to reusable frameworks, documentation, and best practices.
• Production-grade deployments and scaling of pipelines.

Skills

Databricks

AWS

Azure

Spark (PySpark/Scala)

SQL

Delta Lake

Data warehousing

SSIS

Databricks Workflows/Jobs

Performance tuning

Education

Bachelors in Computer Science/Engineering

Tools

Databricks

AWS

Azure

Job description

Customized Energy Solutions India Pvt. Ltd. (CES India) is part of Customized Energy Solutions (CES), a globally recognized, privately held energy market intelligence, consulting, and operational services firm supporting retail and wholesale electricity and natural gas markets worldwide.

Headquartered in Philadelphia, CES operates across the United States, Canada, Japan, India, and Vietnam, with approximately 400 associates globally. The company has earned multiple distinctions from INC. 500 | 5000 and the Philadelphia Business Journal, including Hall of Fame status and repeated recognition as a Best Place to Work.

Established in 2010, CES India plays a critical role in supporting both domestic and international energy initiatives. The India team brings expertise in emerging technologies transforming the energy landscape, including energy storage, electric mobility, wholesale electricity markets, microgrids, and renewable energy systems. Associates in India collaborate closely with global teams while contributing deep technical and analytical capabilities to complex energy market challenges.

CES India actively engages with industry bodies and ecosystem partners that are shaping India's energy and technology landscape, including organizations such as the India Electronics and Semiconductor Association (IESA). Through industry collaboration, technology innovation, and engagement with public and private stakeholders, CES India supports the broader goal of advancing sustainable, high-growth energy and manufacturing sectors in India.

At CES India, you will work in a fast-evolving industry where innovation, accountability, and professional growth are valued. Your work will contribute to meaningful outcomes across both regional and global energy markets.

We are seeking a Databricks Data Engineer to join our growing data engineering team in Pune, India. This role will play a key part in a large-scale modernization initiative to migrate a complex, enterprise-grade Microsoft SQL Server data warehouse ecosystem to the Databricks Lakehouse Platform. The ideal candidate has strong hands-on experience across Databricks data engineering capabilities, with exposure to AI/ML features being a plus, while maintaining a core focus on scalable, reliable data pipelines and analytics workloads.

Key Responsibilities

• Design, build, and optimize scalable data pipelines using Databricks (Apache Spark, Delta Lake, Unity Catalog).
• Participate in the migration of a ~20TB compressed on-prem Microsoft SQL Server data warehouse to Databricks.
• Convert and modernize hundreds of SQL Server tables, thousands of SSIS jobs, and downstream SSRS/SSAS workloads.
• Re-engineer SSIS ETL processes into Databricks notebooks, workflows, and orchestration frameworks.
• Support migration or redesign of cube-based analytics (SSAS) into Databricks SQL, Delta tables, and modern semantic models.
• Implement data quality, validation, reconciliation, and audit controls during migration.
• Optimize performance and cost through efficient Spark usage, partitioning, and query tuning.
• Collaborate with analytics, BI, and AI/ML teams to enable downstream reporting and advanced analytics.
• Apply data governance, security, and access-control standards using Unity Catalog.
• Contribute to reusable frameworks, documentation, and platform best practices.

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 4-7 years of overall data engineering experience.
• Hands-on experience with Databricks on AWS (preferred); Azure experience acceptable.
• Strong proficiency in Spark (PySpark and/or Scala) and SQL.
• Proven experience migrating on-prem SQL Server data warehouses to cloud-based data platforms.
• Experience converting SSIS-based ETL pipelines into Spark-based data engineering solutions.
• Solid understanding of data warehousing concepts, dimensional modeling, and analytical workloads.
• Experience with Delta Lake, incremental processing patterns, and data versioning.
• Familiarity with Databricks Workflows, Jobs, and production-grade deployments.
• Practical experience with performance tuning and large-volume data processing.

Preferred Skills

• Experience modernizing SSAS cube-based reporting solutions.
• Exposure to Databricks SQL Warehouses and BI integrations (Power BI preferred).
• Working knowledge of cloud-native data engineering practices aligned with Databricks best practices.
• Familiarity with MLflow, feature engineering, or AI-enablement within Databricks.
• Experience working in environments that follow Databricks-recommended data engineering patterns.
• Databricks certification is a plus.

Customized Energy Solutions India offers competitive compensation commensurate with experience, along with performance-based incentives aligned to individual and company results. Our benefits are designed to support both professional and personal well-being and include comprehensive medical insurance coverage for associates and eligible dependents, generous paid time off, and structured opportunities for professional development through training programs and skill enhancement initiatives. We are committed to creating an environment where associates are supported, challenged, and positioned for long-term career growth.

Customized Energy Solutions provides equal employment opportunities to all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.