Sr. Data Engineer

Orion Innovation; Chennai District; On-site; INR 800,000 - 1,200,000; Full time

Job summary

Orion Innovation is seeking a talented Data Engineer with over 5 years of experience in Databricks, Delta Lake, and Azure Data Services. The ideal candidate will build scalable data ingestion, transformation, and AI-driven data pipelines.

Key responsibilities include optimizing Spark workloads, implementing data quality measures, and supporting CI/CD deployments. Strong problem-solving and communication skills are essential. This is an equal opportunity position based in Chennai, Tamil Nadu.

Qualifications

• 5+ years of experience in Databricks, Delta Lake, and Azure Data Services.
• Proven expertise in building scalable data pipelines.
• Strong understanding of data quality and testing methodologies.
• Familiarity with CI/CD processes and tools.
• Excellent problem-solving and analytical skills.

Responsibilities

• Build and maintain data ingestion and transformation pipelines.
• Implement ingestion, transformation, and validation agents.
• Optimize Spark workloads for improved performance and efficiency.
• Implement data quality measures and testing hooks to ensure data integrity.
• Support Continuous Integration/Continuous Deployment (CI/CD) deployments.

Skills

Databricks

Delta Lake

Azure Data Services

Data quality assurance

CI/CD processes

Job description

Company Overview

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

Job Overview

We are seeking a talented Data Engineer with over 5 years of experience in Databricks, Delta Lake, and Azure Data Services. The ideal candidate will have hands‑on expertise in building scalable data ingestion, transformation, and AI‑driven data pipelines using our agents, contributing to our data‑driven decision‑making processes.

Key Responsibilities

• Build and maintain data ingestion and transformation pipelines.
• Implement ingestion, transformation, and validation agents.
• Optimize Spark workloads for improved performance and efficiency.
• Implement data quality measures and testing hooks to ensure data integrity.
• Support Continuous Integration/Continuous Deployment (CI/CD) deployments.

Qualifications

• 5+ years of experience in Databricks, Delta Lake, and Azure Data Services.
• Proven expertise in building scalable data pipelines.
• Strong understanding of data quality and testing methodologies.
• Familiarity with CI/CD processes and tools.
• Excellent problem‑solving and analytical skills.
• Strong communication and collaboration skills.

Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.