Senior Data Engineer

Cubic Transportation Systems; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

The Senior Data Engineer at Cubic Transportation Systems will design, build, and maintain scalable data pipelines, ETL/ELT processes, and analytics platforms across global programs. You’ll work with Azure Data Factory, Azure Databricks, Python, and SQL to support KPI engines and device analytics.

You will collaborate with data analysts and engineers to ensure data quality, governance, and reliable datasets for reporting and predictive analytics, shaping the data architecture for enterprise data

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Analytics, Engineering, Mathematics or related field.
• 5+ years of experience in data engineering roles.
• Strong expertise with Azure Data Factory pipelines, Azure Databricks, mapping data flows, and orchestrations.
• Advanced experience with Azure Databricks (pyspark, spark SQL, Delta Lake, notebooks, clusters).
• High proficiency in Python for ETL, automation, and data transformation.
• Strong SQL skills across Azure Synapse and Azure SQL including query tuning and complex transformations.
• Solid understanding of data modelling, warehousing, star/snowflake schema, Medallion architecture and distributed data processing.

Responsibilities

• Design, build, and optimize data pipelines using Azure Data Factory and Databricks (pyspark/spark SQL).
• Develop robust ETL/ELT processes to ingest, transform, and validate large volumes of data.
• Implement scalable workflows leveraging Azure services (Data Lake Storage, Synapse, SQL Databases, Key Vault, Logic Apps, Functions).
• Build efficient SQL queries and stored procedures across Azure Synapse and Azure SQL Server.
• Collaborate with analysts, engineers, and operations to ensure reliable datasets for reporting and analytics.
• Design and maintain data models, schemas, tables, and metadata following COE patterns.
• Implement data quality, validation, and monitoring frameworks across global programs.

Skills

Azure Data Factory

Azure Databricks

Python

SQL

Data Modeling

Data Lakes

Azure Synapse

ETL/ELT

Education

Bachelor's or Master’s in Computer Science / Data Science

Tools

Delta Lake

Power BI

Git

CI/CD

Job description

Business Unit: Cubic Transportation Systems Company Details: The Senior Data Engineer will play a key role in designing, building, and maintaining scalable data pipelines, integrations, and analytical platforms that support CTS’s global analytics and reporting platforms. This role is responsible for developing high-quality ETL/ELT processes, managing cloud data environments, ensuring data quality, and enabling advanced analytics and reporting across major programs. You will work extensively with Azure Data Factory, Azure Databricks, Python, SQL (Oracle/SQL Server), data modelling, data lakes, and enterprise data warehouse structures to support KPI engines, device analytics, predictive modelling, and operational performance insights.

Job Details

Key Responsibilities

The following are the key responsibilities of the position. It is expected that most, if not all of these, are met by the candidate:

• Design, build, and optimize data pipelines using Azure Data Factory and Databricks (pyspark/spark SQL).
• Develop robust ETL/ELT processes to ingest, transform, and validate large volumes of operational, telemetry, incident, and transactional data.
• Implement scalable workflows leveraging Azure services such as Data Lake Storage, Azure Synapse, Azure SQL Databases, Key Vault, Logic Apps, and Functions.
• Build efficient advanced SQL queries and stored procedures across Azure Synapse and Azure SQL Server.
• Collaborate with data analysts, engineers, performance assurance, and operations teams to enable reliable, accurate datasets for reporting, KPIs and predictive analytics.
• Design and maintain data models, schemas, tables, and metadata following COE architecture patterns.
• Implement strong data quality, validation, and monitoring frameworks to ensure accuracy and reliability across global programs.

Required Skills and Qualifications

• Bachelor’s/master’s degree in Comp Science, Data Analytics, Engineering, Mathematics, or related field.
• 5+ years of experience in data engineering roles.
• Strong expertise with Azure Data Factory (ADF) pipelines, Azure Databricks, mapping data flows, and orchestrations.
• Advanced experience with Azure Databricks (pyspark, spark SQL, Delta Lake, notebooks, clusters).
• High proficiency in Python for ETL, automation, and data transformation.
• Strong SQL skills across Azure Synapse and Azure SQL including query tuning and complex transformations.
• Solid understanding of data modelling, warehousing, star/snowflake schema, Medallion architecture and distributed data processing.
• Experience handling large-scale datasets and complex data domains.
• Familiarity with CI/CD practices, Git branching, and DevOps pipelines.
• Experience with data governance, data cataloging, and managing structured/unstructured data in cloud environments.

Preferred Skills

• Experience with Azure Data Factory (ADF) pipelines, Azure Databricks, Databricks Workflows, or Apache Airflow.
• Exposure to real-time/streaming data technologies (Event Hub, Kafka, Spark Streaming).
• Exposure to Machine learning algorithms, feature engineering, model evaluation, hyperparameter tuning, LLM fundamentals and scikit-learn/XGBoost.
• Knowledge with deep learning, PyTorch/TensorFlow, MLflow, model deployment, and MLOps practices.
• Experience integrating APIs, flat files, and operational systems such as ServiceNow.
• Experience to machine learning pipelines, feature stores, or model operationalization.
• Familiarity with Power BI dataset design and optimization.
• Understanding of multi-region cloud environments and enterprise architecture patterns.

Soft Skills

• Strong analytical mindset with exceptional problem-solving abilities.
• Ability to work independently and take ownership of deliverables.
• Excellent communication skills-capable of simplifying complex concepts for non-technical stakeholders.
• Comfortable working in a fast-paced, global, multi-time zone team.
• High attention to detail, data quality, and accuracy.
• Strong sense of accountability, adaptability, and continuous improvement.
• Ability to engage effectively with engineering, operations, and business stakeholders.
• Proactive, resourceful, and committed to delivering high-quality outcomes.

Role Impact & Success Measures

Success in the First 3–6 Months

• Build a solid understanding of CTS datasets, KPI frameworks, and key operational systems.
• Gain strong understanding of CTS datasets, pipelines, and Azure architecture.
• Deliver reliable ADF and Databricks pipelines for ingestion and transformation.
• Improve data quality and performance of existing workflows.
• Support analysts and operations teams with accurate, well-structured datasets.

Success in 6–12 Months

• Take ownership of end-to-end analytical workstreams with minimal supervision.
• Own end-to-end pipeline development with minimal supervision.
• Implement scalable, reusable engineering patterns and metadata-driven frameworks.
• Improve efficiency through automation, orchestration, and optimization.
• Integrate new data sources to support KPIs, device analytics, and predictive modelling.

Long-Term Success (12+ Months)

• Contribute to CTS Analytics strategy, standards, and global operating model improvements.
• Contribute to global COE standards for data engineering and cloud architecture.
• Become SME for key pipelines, data domains, or Azure components.
• Lead continuous improvement initiatives around quality, governance, and automation.
• Support junior engineers and uplift engineering capability within the COE.

Worker Type

Employee

Inclusion & Equal Opportunity

We are committed to creating an inclusive workplace and welcome applications from people of all backgrounds. We do not discriminate based on any protected characteristic under applicable law.

We are committed to ensuring a workplace free of discrimination based on race, color, religion, age, disability, genetic information, sex, sexual orientation, gender identity, or national origin, military or veteran status, and any other basis protected by applicable law.

For more information on Equal Employment please visit: http://www.cubic.com/Careers/Applicant-Help

To learn more, visit Cubic.com. Follow us on LinkedIn!

Company Background

Cubic creates and delivers technology solutions in transportation that make people’s lives easier by simplifying their daily journeys, and defense capabilities that help promote mission success and safety for those who serve their nation. Led by our talented teams around the world, Cubic is committed to solving global challenges through innovation and service to our customers and partners. We have a top-tier portfolio of businesses, including Cubic Transportation Systems (CTS) and Cubic Defense (CD). CTS is an industry-leading integrator of payment and information solutions and related services for intelligent travel applications. CTS delivers integrated systems for transportation and traffic management, delivering tools for travelers to choose the smartest and easiest way to travel and pay for their journeys, and enabling transportation authorities and agencies to manage demand across the entire transportation network. Cubic Defense provides networked Command, Control, Communications, Computers, Cyber, Intelligence, Surveillance and Reconnaissance (C5ISR) solutions, and live, virtual, constructive and game-based training solutions for both U.S. and Allied Forces. These mission-inspired capabilities enable assured multi-domain access; converged digital intelligence; and superior readiness for defense, intelligence, security and commercial missions. Cubic is proud to have a presence in over 60 countries and employ over 5,000 people worldwide. We are committed to hiring and retaining a diverse workforce and are proud to be an Equal Opportunity/Affirmative Action-Employer. We are committed to ensuring a workplace free of discrimination based on race, color, religion, age, disability, genetic information, sex, sexual orientation, gender identity, or national origin, military or veteran status, and any other basis protected by applicable law.