Data Engineer

Orion Innovation; Chennai District; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Orion Innovation is seeking a seasoned Data Engineer to lead a legacy-to-Databricks migration. You will work with Databricks, PySpark, and SQL/T-SQL to transform existing data pipelines and enable scalable analytics in the cloud.

The ideal candidate has 10+ years in data engineering, strong SQL skills, and a background in Oracle/SQL Server ETL and Data Warehousing. Location includes Chennai, Kochi, or Coimbatore with on-site work.

Qualifications

• Minimum 5+ years as a Data Engineer with migration focus.
• Experience migrating legacy SQL/T‑SQL to Databricks.
• Strong PySpark and SQL skills required.
• Experience with data warehousing concepts.
• Azure Data Platform exposure is a plus.

Responsibilities

• Analyze legacy data warehouse and ETL for migration to Databricks.
• Convert T-SQL logic into Databricks/PySpark workflows.
• Design and maintain scalable ETL/ELT pipelines in Databricks.
• Ensure data quality, testing, and validation during migration.
• Collaborate with architects, engineers, and migration teams.
• Document migration logic and design details.
• Review code and follow best practices.

Skills

Data Engineering

Databricks

PySpark

SQL/T-SQL

ETL/ELT

Data Warehousing

Education

Bachelor's or Master's in Computer Science/IT/Engineering

Tools

Azure Data Factory

Delta Lake

Unity Catalog

Azure Synapse

Job description

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

Data Engineer

Location- Kochi/Coimbatore/Chennai

Job Overview

We are seeking a strong Data Engineer - Databricks Migration with 10+ years of experience and hands‑on expertise in Databricks, PySpark, and SQL/T‑SQL interpretation to support a legacy-to‑Databricks migration program. The ideal candidate should have worked predominantly as a Data Engineer throughout their career, ideally beginning with Oracle, SQL Server/T‑SQL, ETL, or Data Warehouse technologies and later transitioning into modern cloud‑based Data Engineering. The role requires the ability to understand existing T‑SQL logic and migrate transformation rules into scalable Databricks and PySpark solutions. Important screening criterion: The candidate must have worked predominantly and consistently as a Data Engineer throughout their career. Profiles that are primarily BI, reporting, dashboarding, analytics, or only recently shifted into data engineering should not be prioritized.

Key Responsibilities

• Analyze legacy data warehouse, SQL Server, Oracle, ETL, and T‑SQL based implementations for migration to Databricks.
• Read and interpret complex T‑SQL scripts, stored procedures, functions, views, joins, transformation rules, and data flows.
• Convert business logic from legacy SQL/T‑SQL processes into optimized Databricks and PySpark workflows.
• Design, develop, and maintain scalable ETL/ELT pipelines in Databricks.
• Support ingestion, transformation, validation, and reconciliation of enterprise data across legacy and modern platforms.
• Collaborate with architects, senior engineers, business analysts, and migration teams to understand source‑to‑target mapping requirements.
• Perform data quality checks, unit testing, integration testing, and defect resolution during migration execution.
• Optimize PySpark jobs, SQL queries, and Databricks notebooks for better performance and reliability.
• Document migration logic, transformation rules, dependencies, and technical design details.
• Participate in code reviews and follow Data Engineering best practices.

Key Skills

• 5+ years of relevant Data Engineering experience, with the majority of career spent in Data Engineering roles.
• Strong hands‑on experience in Databricks development.
• Strong PySpark programming experience for data transformation and pipeline development.
• Strong SQL/T‑SQL understanding, especially the ability to read and interpret existing T‑SQL code.
• Experience in legacy data technologies such as SQL Server, Oracle, traditional ETL tools, or Data Warehouse platforms.
• Experience in ETL/ELT development, data ingestion, transformation, and data pipeline implementation.
• Good understanding of Data Warehousing, Data Modeling, and dimensional concepts.
• Ability to migrate or re‑engineer legacy SQL/ETL logic into Databricks/PySpark.
• Strong analytical, debugging, and problem‑solving skills.
• Good communication skills to work with technical and business teams.

Preferred Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related discipline.
• Azure Data Factory or Azure Data Platform exposure.
• Delta Lake experience.
• Unity Catalog exposure.
• Azure Synapse Analytics experience.
• CI/CD exposure for data pipelines.
• Experience working in Agile/Scrum delivery models.
• Data governance, security, and lineage awareness.
• Relevant certifications in Databricks, Azure Data Engineering, or Cloud Data Platforms will be an added advantage.
• Strong, hands‑on Data Engineering background throughout career, not a partial or recent transition.
• Avoid profiles that are mainly BI Developers, Power BI Developers, Data Analysts, or reporting‑only candidates.

Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Candidate Privacy Policy

Orion Systems Integrators, LLC And Its Subsidiaries And Its Affiliates (collectively, “Orion,” “we” Or “us”) Are Committed To Protecting Your Privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) Explains

• What information we collect during our application and recruitment process and why we collect it;
• How we handle that information; and
• How to access and update that information.

Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.