Data Engineer

SOTI; Ernakulam; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

SOTI is seeking a skilled Data Engineer to design and maintain robust ETL/ELT pipelines, implement Medallion data architecture, and collaborate with cross-functional teams to deliver reliable data solutions. The role involves hybrid data environments, governance, and scalable BI capabilities.

Ideal candidates will have 3–5 years of data engineering experience, strong SQL, Python, PySpark, and familiarity with Microsoft Fabric and Azure data services.

Qualifications

• 3–5 years of data engineering or similar technical role.
• Strong proficiency in SQL for complex querying and performance.
• Hands-on ETL/ELT development and building robust, scalable data pipelines.
• Advanced skills in Python and PySpark for data processing and transformation.
• Experience with C# for integration, automation, or pipeline component development.
• Practical expertise with Microsoft Fabric, including Lakehouse, Warehouse, Pipelines, and Notebooks.
• Strong experience working with Azure data services (ADF, ADLS, Synapse, Databricks, Azure SQL).
• Ability to work with structured, semi-structured, and unstructured data.
• Understanding data governance, validation, and quality best practices.
• Effective communicator comfortable collaborating with analysts, data scientists, and business partners.
• Familiarity with CI/CD development processes, including version control, automated deployments, and pipeline-based DevOps practices.
• Detail-oriented, proactive, and able to operate in fast-paced cloud environments.

Responsibilities

• ETL/ELT Development: Design, build, and maintain efficient ETL/ELT pipelines to support data ingestion, transformation, and delivery across diverse data sources and BI platforms.
• Medallion Architecture Implementation: Structure data pipelines using the Medallion architecture (Bronze, Silver, Gold layers) to ensure modularity, scalability, and enhanced data quality.
• Collaboration with Data Architecture: Work closely with the Data Architecture team to align with enterprise data models, standards, and architectural guidelines while contributing to the implementation of governed data solutions.
• Cross-Functional Collaboration: Collaborate with BI Engineers and stakeholders across various internal departments (Data Source owners: IT, Cloud Services, Corporate Applications, etc) to gather requirements and deliver reliable, scalable data solutions that drive business insights.
• Infrastructure Support: Maintain and support hybrid data environments (on-premises and cloud), including data pipelines and storage systems within SQL Server, Azure Synapse Analytics, and Microsoft Fabric.
• Monitoring & Alerts: Implement and maintain monitoring, logging, and alerts to ensure data pipeline performance.
• Data Quality: Maintain data reconciliation processes, unit testing, CI/ CD, and version control to ensure data accuracy and reliability.
• Data Governance & Compliance: Support data governance initiatives by ensuring adherence to data quality standards, metadata management, data lineage practices, and access controls. Collaborate with data stewards and compliance teams to enforce policies and ensure consistent data definitions and usage.
• Technical Documentation & Knowledge Sharing: Document technical designs, processes, and data flows to support maintainability, transparency, and internal knowledge sharing.
• Solution Development: Develop scalable, data-driven solutions to solve complex business challenges, enabling robust self-service reporting, analytics, and BI capabilities (e.g. Data Mesh approach).
• Process Improvement: Continuously assess and enhance data engineering practices, focusing on performance optimization, reliability, and development efficiency.

Job description

SOTI is committed to providing its employees with endless possibilities; learning new things, working with the latest technologies and making a difference in the world.

At SOTI, we are committed to delivering best in class mobile and IoT device management solutions. We are looking for out of the box thinkers that appreciate the art of creating great software. To us, being visionary is more important than doing things the way they’ve always been done.

Responsibilities

• ETL/ELT Development: Design, build, and maintain efficient ETL/ELT pipelines to support data ingestion, transformation, and delivery across diverse data sources and BI platforms.
• Medallion Architecture Implementation: Structure data pipelines using the Medallion architecture (Bronze, Silver, Gold layers) to ensure modularity, scalability, and enhanced data quality.
• Collaboration with Data Architecture: Work closely with the Data Architecture team to align with enterprise data models, standards, and architectural guidelines while contributing to the implementation of governed data solutions.
• Cross-Functional Collaboration: Collaborate with BI Engineers and stakeholders across various internal departments (Data Source owners: IT, Cloud Services, Corporate Applications, etc) to gather requirements and deliver reliable, scalable data solutions that drive business insights.
• Infrastructure Support: Maintain and support hybrid data environments (on-premises and cloud), including data pipelines and storage systems within SQL Server, Azure Synapse Analytics, and Microsoft Fabric.
• Monitoring & Alerts: Implement and maintain monitoring, logging, and alerts to ensure data pipeline performance.
• Data Quality: Maintain data reconciliation processes, unit testing, CI/ CD, and version control to ensure data accuracy and reliability.
• Data Governance & Compliance: Support data governance initiatives by ensuring adherence to data quality standards, metadata management, data lineage practices, and access controls. Collaborate with data stewards and compliance teams to enforce policies and ensure consistent data definitions and usage.
• Technical Documentation & Knowledge Sharing: Document technical designs, processes, and data flows to support maintainability, transparency, and internal knowledge sharing.
• Solution Development: Develop scalable, data-driven solutions to solve complex business challenges, enabling robust self-service reporting, analytics, and BI capabilities (e.g. Data Mesh approach).
• Process Improvement: Continuously assess and enhance data engineering practices, focusing on performance optimization, reliability, and development efficiency.

Qualifications

• 3–5 years of experience in data engineering or similar technical role.
• Strong proficiency in SQL for complex querying and performance tuning.
• Hands‑on experience with ETL/ELT development and building robust, scalable data pipelines.
• Advanced skills in Python and PySpark for data processing and transformation.
• Experience with C# for integration, automation, or pipeline component development.
• Practical expertise with Microsoft Fabric, including Lakehouse, Warehouse, Pipelines, and Notebooks.
• Strong experience working with Azure data services (ADF, ADLS, Synapse, Databricks, Azure SQL).
• Ability to work with structured, semi‑structured, and unstructured data.
• Understanding data governance, validation, and quality best practices.
• Effective communicator comfortable collaborating with analysts, data scientists, and business partners.
• Familiarity with CI/CD development processes, including version control, automated deployments, and pipeline-based DevOps practices.
• Detail‑oriented, proactive, and able to operate in fast‑paced cloud environments.