Data Engineer

SOTI Inc; Ernakulam; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

SOTI Inc is seeking a data engineer to design and maintain scalable data pipelines in hybrid on-premises and cloud environments, using SQL, Python, and PySpark to ingest, transform, and deliver data across Azure services such as Synapse and Databricks.

You will collaborate with BI teams, ensure data quality and governance, and implement CI/CD practices, contributing to a Data Mesh approach and robust self-service analytics.

Qualifications

• 3–5 years of experience in data engineering or similar technical role.
• Strong proficiency in SQL for complex querying and performance tuning.
• Hands-on experience with ETL/ELT development and scalable data pipelines.
• Advanced skills in Python and PySpark for data processing.
• Experience with C# for integration or automation.
• Practical expertise with Microsoft Fabric, Lakehouse and related Azure services.
• Ability to work with structured, semi-structured, and unstructured data.
• Knowledge of data governance, validation, and quality best practices.
• Effective communicator with analysts, data scientists and partners.
• Familiarity with CI/CD and DevOps practices.

Responsibilities

• ETL/ELT Development: Design, build, and maintain data pipelines.
• Medallion Architecture: Implement Bronze/Silver/Gold data layers.
• Collaborate with Data Architecture to align with standards.
• Cross-Functional Collaboration with BI Engineers and data owners.
• Hybrid data environments: on-premises and cloud (SQL Server, Azure Synapse, Fabric).
• Monitoring, logging, and alerts for data pipelines.
• Data quality and reconciliation, CI/CD and version control.
• Data governance, metadata management, and access controls.
• Documentation of designs, processes, and data flows.
• Develop scalable data-driven solutions for self-service analytics.

Skills

SQL

ETL/ELT

Python

PySpark

C#

Microsoft Fabric

Azure Synapse

Databricks

CI/CD

Data governance

Tools

Microsoft Fabric

Azure Synapse Analytics

Databricks

Azure Data Factory

Azure Data Lake Storage

Job description

SOTI is committed to providing its employees with endless possibilities; learning new things, working with the latest technologies and making a difference in the world.

At SOTI, we are committed to delivering best in class mobile and IoT device management solutions. We are looking for out of the box thinkers that appreciate the art of creating great software. To us, being visionary is more important than doing things the way they’ve always been done.

Responsibilities

• ETL/ELT Development: Design, build, and maintain efficient ETL/ELT pipelines to support data ingestion, transformation, and delivery across diverse data sources and BI platforms.
• Medallion Architecture Implementation: Structure data pipelines using the Medallion architecture (Bronze, Silver, Gold layers) to ensure modularity, scalability, and enhanced data quality.
• Collaboration with Data Architecture: Work closely with the Data Architecture team to align with enterprise data models, standards, and architectural guidelines while contributing to the implementation of governed data solutions.
• Cross-Functional Collaboration: Collaborate with BI Engineers and stakeholders across various internal departments (Data Source owners: IT, Cloud Services, Corporate Applications, etc) to gather requirements and deliver reliable, scalable data solutions that drive business insights.
• Infrastructure Support: Maintain and support hybrid data environments (on-premises and cloud), including data pipelines and storage systems within SQL Server, Azure Synapse Analytics, and Microsoft Fabric.
• Monitoring & Alerts: Implement and maintain monitoring, logging, and alerts to ensure data pipeline performance.
• Data Quality: Maintain data reconciliation processes, unit testing, CI/ CD, and version control to ensure data accuracy and reliability.
• Data Governance & Compliance: Support data governance initiatives by ensuring adherence to data quality standards, metadata management, data lineage practices, and access controls. Collaborate with data stewards and compliance teams to enforce policies and ensure consistent data definitions and usage.
• Technical Documentation & Knowledge Sharing: Document technical designs, processes, and data flows to support maintainability, transparency, and internal knowledge sharing.
• Solution Development: Develop scalable, data-driven solutions to solve complex business challenges, enabling robust self-service reporting, analytics, and BI capabilities (e.g. Data Mesh approach).
• Process Improvement: Continuously assess and enhance data engineering practices, focusing on performance optimization, reliability, and development efficiency.

Qualifications

• 3–5 years of experience in data engineering or similar technical role.
• Strong proficiency in SQL for complex querying and performance tuning.
• Hands-on experience with ETL/ELT development and building robust, scalable data pipelines.
• Advanced skills in Python and PySpark for data processing and transformation.
• Experience with C# for integration, automation, or pipeline component development.
• Practical expertise with Microsoft Fabric, including Lakehouse, Warehouse, Pipelines, and Notebooks.
• Strong experience working with Azure data services (ADF, ADLS, Synapse, Databricks, Azure SQL).
• Ability to work with structured, semi-structured, and unstructured data.
• Understanding data governance, validation, and quality best practices.
• Effective communicator comfortable collaborating with analysts, data scientists, and business partners.
• Familiarity with CI/CD development processes, including version control, automated deployments, and pipeline-based DevOps practices.
• Detail-oriented, proactive, and able to operate in fast-paced cloud environments.

SOTI is a proven innovator and industry leader for simplifying business mobility solutions by making them smarter, faster and more reliable. With SOTI’s innovative portfolio of solutions, organizations can trust SOTI to elevate and streamline their mobile operations, maximize their ROI and reduce device downtime. Globally, with over 17,000 customers, SOTI has proven itself to be the go-to mobile platform provider to manage, secure and support business-critical devices. With SOTI’s world-class support, enterprises can take mobility to endless possibilities. For more information, visit soti.net.