Data Engineer

Kumaran Systems; Hyderabad; Presencial; INR 1.200.000 - 2.100.000; Jornada completa

Descripción de la vacante

Kumaran Systems in Hyderabad, India is looking for a Data Engineer to design and optimize Azure data pipelines and lakehouse solutions. You will deliver secure, governed datasets and collaborate across teams in an Agile setting.

Responsibilities include building scalable pipelines with Databricks (Python/Spark), managing data models, and ensuring robust CI/CD and data security practices. Prior experience with Delta Lake and ADLS is preferred.

Formación

• 4–6 years in data engineering.
• Databricks with Python, Spark, Pandas in production notebooks.
• Azure Data Factory pipelines and ingestion from diverse sources.
• Delta Lake fundamentals and bronze/silver/gold medallion model.
• Azure Data Lake Storage and strong SQL for performance.
• ETL/ELT data modeling (dimensional/star schema).
• CI/CD for data projects (Azure DevOps or GitHub) with secure access.
• Agile/Scrum environment with Jira/Confluence.
• Clear data pipeline documentation and testing practices.

Responsabilidades

• Design, build, and optimize Azure data pipelines and lakehouse solutions.
• Deliver secure, governed datasets with automation and documentation.
• Collaborate across teams in an Agile setting and mentor juniors.

Conocimientos

Databricks

Python

Spark

Pandas

Azure Data Factory

Delta Lake

SQL

ETL/ELT

CI/CD

Jira/Confluence

Agile/Scrum

Data governance

Herramientas

Azure Data Lake Storage

Azure Entra ID

Unity Catalog (concepts)

Descripción del empleo

Job Description:

About Us

Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises bridge the past and the future, reimagining legacy systems and embracing AI-led possibilities. With over 34 years of delivery excellence, we specialise in re-engineering core applications, driving cloud transformation, and building automation-first, GenAI-enabled ecosystems across the Banking and Financial Services, Insurance, Telecom, and Automotive sectors. We help organisations not just upgrade their systems but reimagine what’s possible.

Why Kumaran?

We blend engineering discipline with AI innovation, helping clients modernise with confidence, automate with clarity, and scale with purpose. Our global delivery model ensures agility, responsiveness, and seamless collaboration, with clients always at the heart of every engagement. At Kumaran, we don't just solve problems, we engineer future-ready transformations.

Role Summary

Design, build, and optimize Azure data pipelines and lakehouse solutions. Deliver secure, reliable datasets with strong governance, automation, and documentation. Collaborate across teams and contribute to standards in an Agile setting.

Must-Have (Day 1)

• Experience: 4–6 years in data engineering
• Core Platform: Databricks with Python, Spark, Pandas (notebooks and modular code)
• Orchestration: Azure Data Factory (pipelines, integration runtimes); ingest from diverse sources
• Lakehouse: Delta Lake fundamentals; Medallion architecture (bronze/silver/gold) in production
• Storage/SQL/Performance: Azure Data Lake Storage (ADLS); strong SQL; performance-aware design
• Data Patterns: ETL/ELT; data modeling (e.g., dimensional/star schema)
• DevOps & Security: CI/CD for data projects (Azure DevOps or GitHub Enterprise); familiarity with Azure Entra ID for SSO/RBAC; secure workspace/data access
• Quality & Observability: Data validation/testing, code reviews, and basic monitoring/alerting for jobs/pipelines
• Ways of Working: Agile/Scrum (Jira/Confluence); clear pipeline and data contract documentation
• Collaboration: Effective stakeholder engagement; support/mentor junior team members; clear communication
• Prompt design for data tasks (ingestion, transformations, documentation) with clear objectives and constraints
• Use of Copilot/ChatGPT to scaffold notebooks/jobs, generate tests, and optimize SQL/Spark - validates outputs before merging

Nice-to-Have (Train within 60–90 days)

• Unity Catalog migration (Hive to Unity) and permissions/governance
• Databricks DevOps (cluster configuration, secret management, workspace automation)
• Azure Functions (C# or Python) for orchestration/integration
• Synapse dedicated SQL pools or dbt; Delta Live Tables
• Financial services domain exposure

Shared Expectations

• Work independently with minimal supervision while contributing to team outcomes
• Commitment to secure practices and production-grade reliability
• Continuous improvement mindset and willingness to learn new tools/technologies
• Willingness to work within regulated environment controls and policies
• Use Generative AI responsibly to improve velocity and quality (simple, structured prompts; guardrails; validate AI-assisted outputs before adoption)