Senior Data Engineer(6+ years with MS Fabric, Azure, ADF, Data Platforms only)

C T Corporation System (DE); Chennai; Presencial; INR 1.200.000 - 1.800.000; Jornada completa

Descripción de la vacante

CT Corporation System (DE) is seeking a Senior Data Engineer in Chennai, India to design, build, and operate modern data solutions powering data products, analytics, integrations, reporting, and AI capabilities.

You will work across Databricks, Microsoft Fabric, Snowflake, and cloud-native services, focusing on scalable architectures and production-grade engineering practices. An AI-first approach will guide design, testing, and deployment across the data stack.

Responsabilidades

• Design, develop, and maintain scalable data pipelines for operational, analytical, reporting, and AI use cases.
• Build batch, incremental, near-real-time, and event-driven data processing solutions.
• Develop data ingestion, transformation, orchestration, and delivery workflows using modern cloud data platforms.
• Design and implement data models optimized for performance, maintainability, and consumption.
• Develop reusable engineering components and patterns rather than one-off solutions.
• Optimize pipelines for reliability, scalability, performance, and cost.
• Contribute to technical design discussions and architecture decisions.
• Design and implement integrations across enterprise applications, data platforms, and third-party systems.
• Develop reusable integration services and data-access capabilities.
• Build secure REST and GraphQL APIs to expose data and platform capabilities.
• Implement event-driven and asynchronous integration patterns where appropriate.
• Integrate systems using APIs, connectors, messaging, events, and modern data-sharing techniques.
• Apply authentication, authorization, versioning, monitoring, and API lifecycle practices.
• Build automated data validation, reconciliation, and quality controls into pipelines and data products.
• Implement unit, integration, and data testing as part of development.
• Build monitoring and observability into pipelines, APIs, and integration services.
• Troubleshoot complex production issues and perform root-cause analysis.
• Implement preventive improvements to improve reliability and maintainability.
• Continuously improve the reliability, resiliency, and performance of production solutions.
• Use AI-assisted development throughout the lifecycle, including design, coding, testing, documentation, troubleshooting, and optimization.
• Apply validation to AI-generated code and engineering artifacts.
• Define requirements, data/API contracts, acceptance criteria, and test expectations before implementation.
• Use AI to accelerate engineering delivery while maintaining security, maintainability, and quality.
• Contribute to data solutions supporting AI agents, RAG, semantic models, and ML applications.
• Experiment with emerging engineering capabilities and share practices with the team.
• Implement data quality, lineage, metadata, security, and auditability in data solutions.
• Build data pipelines and integrations suitable for sensitive data.
• Follow governance, security, and compliance requirements.
• Maintain clear technical documentation for data flows, APIs, transformations, and operations.
• Partner with architecture, security, governance, and platform teams to resolve requirements.
• Take ownership of solutions through development, deployment, and production operation.
• Monitor pipeline execution, API health, data quality, and platform performance.
• Investigate incidents and drive resolution.
• Participate in code reviews and improve engineering practices.
• Automate deployment, testing, configuration, and operations where practical.
• Contribute to improving team standards from production learnings.
• Collaborate with leads, product teams, architects, analysts, AI teams, and stakeholders.
• Translate requirements into scalable technical solutions.
• Engage in architecture and technical design discussions.

Descripción del empleo

Job Summary

We are looking for a Senior Data Engineer to design, build, and operate modern data solutions that power our data products, analytics, integrations, reporting, and emerging AI capabilities.

This is a hands-on engineering role for someone who combines strong data engineering fundamentals with experience in modern cloud data platforms, APIs, integration patterns, and production-grade engineering practices .

You will work across technologies such as Databricks, Microsoft Fabric, Snowflake, and cloud-native services , depending on the needs of the solution. We value engineers who understand the underlying architecture and engineering principles rather than being tied to a single technology.

We are also looking for an AI-first engineer who actively uses AI-assisted development to improve how solutions are designed, developed, tested, documented, and supported while maintaining strong engineering quality and accountability.

Key Responsibilities

Build Modern Data Solutions

• Design, develop, and maintain scalable data pipelines supporting operational, analytical, reporting, and AI use cases.
• Build batch, incremental, near-real-time, and event-driven data processing solutions.
• Develop data ingestion, transformation, orchestration, and delivery workflows using modern cloud data platforms.
• Design and implement data models optimized for performance, maintainability, and consumption.
• Develop reusable engineering components and patterns rather than one-off solutions.
• Optimize pipelines and data processing for reliability, scalability, performance, and cost.
• Contribute to technical design discussions and architecture decisions.

Build Enterprise Data Integrations

• Design and implement integrations across enterprise applications, data platforms, and third-party systems.
• Develop reusable integration services and data-access capabilities.
• Build secure REST and GraphQL APIs to expose data and platform capabilities.
• Implement event-driven and asynchronous integration patterns where appropriate.
• Integrate systems using APIs, connectors, messaging, events, and modern data-sharing techniques.
• Apply appropriate authentication, authorization, versioning, monitoring, and API lifecycle practices.

Engineer for Quality & Reliability

• Build automated data validation, reconciliation, and quality controls into pipelines and data products.
• Implement appropriate unit, integration, and data testing as part of development.
• Build monitoring and observability into pipelines, APIs, and integration services.
• Troubleshoot complex production issues and perform root-cause analysis.
• Implement preventive improvements rather than repeatedly addressing symptoms.
• Continuously improve the reliability, resiliency, maintainability, and performance of production solutions.

Apply AI-First Engineering Practices

• Use AI-assisted development throughout the engineering lifecycle, including design, coding, testing, documentation, troubleshooting, and optimization.
• Apply appropriate technical review and validation to AI-generated code and engineering artifacts.
• Practice spec-driven development by defining clear requirements, data contracts, API contracts, acceptance criteria, and test expectations before implementation.
• Use AI to accelerate engineering delivery while maintaining security, maintainability, and quality.
• Contribute to the development of data solutions supporting AI agents, RAG, semantic models, machine learning, and other AI-enabled applications .
• Experiment with emerging engineering capabilities and share effective practices with the broader team.

Enable Secure & Governed Data

• Implement data quality, lineage, metadata, security, and auditability requirements within data solutions.
• Build data pipelines and integrations appropriate for sensitive and regulated data.
• Follow established governance, risk, security, and compliance requirements.
• Support secure and auditable processing of KYC, UCC, public-record, and other business data.
• Maintain clear technical documentation for data flows, APIs, transformations, and operational processes.
• Partner with architecture, security, governance, and platform teams to resolve technical and compliance requirements.

Operate What You Build

• Take ownership of solutions through development, deployment, and production operation.
• Monitor pipeline execution, API health, data quality, and platform performance.
• Investigate incidents and integration failures and drive issues through resolution.
• Participate in code reviews and continuously improve engineering practices.
• Automate deployment, testing, configuration, and operational processes wherever practical.
• Contribute to improving team standards based on lessons learned from production.

Collaborate & Contribute

• Work closely with Lead Data Engineers, product teams, architects, analysts, AI teams, and business stakeholders.
• Translate requirements into practical and scalable technical solutions.
• Participate actively in architecture and technical design discussions.
• Communic