Data Engineer

Rail Infotech; Hyderabad; On-site; INR 1,400,000 - 2,200,000; Full time

Job summary

Rail Infotech in Hyderabad seeks an experienced Data Engineer to design and maintain data pipelines using Microsoft Fabric. You will ingest and transform data from Kafka, Oracle, Azure SQL, REST APIs and ADLS, and develop ETL/ELT processes with PySpark and Spark SQL.

You will design Lakehouse structures, leverage Fabric Shortcuts, and build Power BI reports to support insightful analytics. Strong data modeling, performance tuning, and CI/CD practices are required.

Qualifications

• Bachelor's degree in CS/IT or related field.
• 5–8 years of data engineering experience.
• Strong hands‑on with Microsoft Fabric and PySpark.
• Experience with Kafka, Oracle, REST APIs and ADLS.
• Knowledge of Lakehouse, Delta tables, and data modeling.

Responsibilities

• Build and maintain data pipelines using Microsoft Fabric.
• Ingest and transform data from Kafka, Oracle, Azure SQL, REST APIs, and ADLS.
• Develop ETL/ELT workflows with PySpark and Spark SQL.
• Design and manage Lakehouse/Warehouse structures in Fabric.
• Use Fabric Shortcuts to access data across OneLake and external storage.
• Support real-time and batch data processing needs.
• Develop and maintain Power BI reports/dashboards.
• Ensure models are clean, reliable, and optimized for reporting.
• Monitor pipelines and improve performance.

Skills

Data modeling

Power BI knowledge

SQL proficiency

Git & CI/CD

Data quality

Education

Bachelor's degree in CS/IT or related field

Tools

Microsoft Fabric

PySpark

Spark SQL

Azure SQL

Kafka

ADLS

OneLake

REST APIs

Git

Job description

Data Engineer Microsoft Fabric

Key Responsibilities

• Build and maintain data pipelines using Microsoft Fabric.
• Work on ingestion and transformation of data from Kafka, Oracle, Azure SQL, REST APIs, and ADLS.
• Develop ETL/ELT workflows using PySpark and Spark SQL.
• Design and manage Lakehouse and Warehouse structures in Fabric.
• Use Fabric Shortcuts to access data across OneLake and external storage efficiently, without unnecessary duplication.
• Support real-time and batch data processing needs.
• Develop and maintain Power BI reports and dashboards.
• Ensure data models are clean, reliable, and optimized for reporting and analytics.
• Monitor pipelines, fix issues, and continuously improve performance.
• Follow coding standards, version control (Git), and CI/CD practices.

Required Skills

• 5–8 years of experience in Data Engineering.
• Strong hands‑on experience with Microsoft Fabric.
• Good knowledge of PySpark and Spark SQL.
• Strong SQL skills, including work with Azure SQL.
• Experience building data pipelines using Fabric or Azure Data Factory.
• Hands‑on experience working with Kafka, Oracle, REST APIs, and ADLS.
• Understanding of Lakehouse architecture, Delta tables, and data modeling concepts (Star/Snowflake schema).
• Working knowledge of Power BI (datasets, semantic models, integration with Fabric).
• Experience with Git and basic CI/CD workflows.
• Understanding of data quality, performance tuning, and security basics.

Preferred Skills

• Exposure to Azure Databricks or Synapse.
• Experience with Azure DevOps.
• Familiarity with streaming / near real‑time pipelines.
• Experience handling semi‑structured data (JSON, APIs).

Qualifications

• Bachelor's degree in Computer Science, IT, or related field.
• Azure or Fabric certifications are a plus.