Data Engineer - Assistant Manager

Kavi Software Technologies Private Limited; Chennai; On-site; INR 1,800,000 - 3,600,000; Full time

Job summary

Kavi Software Technologies Private Limited is seeking an experienced Data Engineer to design, build, and maintain scalable data pipelines and infrastructure powering analytics, reporting, and ML initiatives. You will work across data modelling, distributed systems, and cloud platforms to deliver reliable, high-performance data solutions.

Responsibilities include designing ETL/ELT pipelines, maintaining data warehouses/lakes, collaborating with data scientists and product teams, and implementing

Qualifications

• Bachelor's degree in Computer Science or a related field.
• 8+ years of hands-on experience as a Data Engineer or in a similar role.
• Strong programming skills in Python and/or Scala; solid SQL expertise.
• Experience with ETL/ELT tools and orchestration frameworks like Apache Airflow, dbt, Luigi.
• Hands-on experience with big data technologies (Apache Spark, Hadoop, Kafka).
• Proficiency with relational databases (PostgreSQL, MySQL, SQL Server) and NoSQL databases (MongoDB, Cassandra, DynamoDB).
• Experience with cloud data platforms and services (AWS Redshift/Glue/S3, Azure DataFactory/Synapse, GCP BigQuery/Dataflow).
• Understanding of data modeling concepts — ER, cardinality, normalization, dimensional modeling.
• Experience with data warehousing solutions (Snowflake, Redshift, BigQuery, Databricks).
• Familiarity with version control (Git) and CI/CD practices.
• Knowledge of data governance, security, and privacy best practices (GDPR, data masking, access controls).
• Strong problem-solving skills and ability to work with large, complex datasets.

Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines to ingest, transform, and load data from diverse sources (databases, APIs, streaming platforms, third-party systems).
• Develop and maintain data warehouse/data lake architectures, ensuring data quality, consistency, and reliability.
• Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver well-structured, accessible datasets.
• Build and maintain batch and real-time streaming data pipelines using tools like Apache Spark, Kafka, or Airflow.
• Implement data quality checks, monitoring, and alerting to ensure pipeline reliability and data integrity.
• Optimize database and query performance for large-scale datasets.
• Design and maintain data models (conceptual, logical, physical) and schemas that support analytics and application needs.
• Work with cloud platforms (AWS/Azure/GCP) to manage data infrastructure, storage, and compute resources.
• Implement and enforce data governance, security, and compliance best practices.
• Participate in code reviews, CI/CD pipeline development, and infrastructure-as-code practices.
• Document data pipelines, architecture, and processes for team knowledge sharing.
• Troubleshoot and resolve production data pipeline issues in a timely manner.

Job description

Kavi Software Technologies Private Limited | Full time

We are looking for an experienced Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that power analytics, reporting, and machine learning initiatives across the organization. The ideal candidate has a strong foundation in data modelling, distributed systems, and cloud-based data platforms, with a track record of delivering reliable, high-performance data solutions.

Key Responsibilities:

• Design, build, and optimize scalable ETL/ELT pipelines to ingest, transform, and load data fromdiverse sources (databases, APIs, streaming platforms, third-party systems).
• Develop and maintain data warehouse/data lake architectures, ensuring data quality, consistency,and reliability
• Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver well-structured, accessible datasets
• Build and maintain batch and real-time streaming data pipelines using tools like Apache Spark, Kafka, or Airflow
• Implement data quality checks, monitoring, and alerting to ensure pipeline reliability and dataintegrity
• Optimize database and query performance for large-scale datasets
• Design and maintain data models (conceptual, logical, physical) and schemas that support analyticsand application needs
• Work with cloud platforms (AWS/Azure/GCP) to manage data infrastructure, storage, and computeresources
• Implement and enforce data governance, security, and compliance best practices
• Participate in code reviews, CI/CD pipeline development, and infrastructure-as-code practices
• Document data pipelines, architecture, and processes for team knowledge sharing
• Troubleshoot and resolve production data pipeline issues in a timely manner

Required Skills & Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field
• 8+ years of hands-on experience as a Data Engineer or in a similar role
• Strong programming skills in Python and/or Scala; solid SQL expertise
• Experience with ETL/ELT tools and orchestration frameworks (Apache Airflow, dbt, Luigi, orsimilar)
• Hands-on experience with big data technologies (Apache Spark, Hadoop, Kafka)
• Proficiency with relational databases (PostgreSQL, MySQL, SQL Server) and NoSQL databases(MongoDB, Cassandra, DynamoDB)
• Experience working with cloud data platforms and services (AWS Redshift/Glue/S3, Azure DataFactory/Synapse, GCP BigQuery/Dataflow)
• Solid understanding of data modeling concepts — entity relationships, cardinality, normalization,and dimensional modeling (star/snowflake schemas)
• Experience with data warehousing solutions (Snowflake, Redshift, BigQuery, Databricks)
• Familiarity with version control (Git) and CI/CD practices
• Understanding of data governance, security, and privacy best practices (GDPR, data masking, accesscontrols)
• Strong problem-solving skills and ability to work with large, complex datasets