Industrial Big Data Engineer - Real-Time Analytics Lead

Intechww; As Sudiyah; On-site; OMR 25,000 - 45,000; Full time

Benefits offered by this job

Global exposure

Competitive compensation

Annual bonus

Health & Wellness insurance

Learning opportunities

Work-life balance

Inclusive environment

Job summary

INTECH AutomationIntelligence is seeking a Big Data Engineer to join our team. Design and operate end-to-end big data and analytics solutions, building scalable ETL/ELT pipelines, integrating industrial data, and delivering real-time dashboards that drive efficiency, safety, and compliance.

You will work with Hadoop/Cloudera, NiFi, Sqoop, PySpark/Scala, and BI tools such as Power BI and Tableau, collaborating across Oil & Gas, Petrochemicals, Utilities, and Manufacturing.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, Telecommunication, or a related field.
• 5+ years of experience in data engineering, big data development, or business intelligence.
• Strong hands‑on experience with the Apache Hadoop ecosystem (Cloudera distribution preferred) including HDFS, Hive, Impala, and Oozie.
• Expert‑level proficiency in PySpark and Scala for building data pipelines. Experience with real‑time data ingestion tools like Apache NiFi and data migration tools like Sqoop.
• Strong SQL skills and experience with relational databases (e.g., Microsoft SQL, Oracle, IBM DB2, Amazon RDS).
• Proficiency in Power BI and Tableau, including dashboard design, server administration, and performance optimization.
• Working knowledge of Python for data analysis and scripting.
• Cloud (Optional but beneficial): Experience with AWS data services (Amazon RDS, Glue, Redshift) is a plus.
• Industrial Systems (Optional but beneficial): Familiarity with OSI PI System for sensor data integration.

Responsibilities

• Design, develop, and maintain robust ETL/ELT pipelines for pre-production and production environments.
• Ingest data from diverse sources, including industrial sensors, relational databases, and enterprise applications.
• Implement data integration and transformation logic using Scala, PySpark, Sqoop, and Apache NiFi within the Cloudera/Hadoop ecosystem.
• Manage and schedule data workflows using Oozie or similar orchestration tools.
• Optimize data storage and query performance using Hive, Impala, and HDFS for efficient analytics.
• Develop interactive and real-time dashboards in Power BI and Tableau to monitor critical business and operational KPIs.
• Translate complex technical datasets into user-friendly visualizations for stakeholders, enabling data-driven decision-making.
• Design dashboards for real-time monitoring of sensor data and safety/performance use cases.
• Support the development of predictive models by preparing and engineering necessary data features.
• Collaborate on OSI PI System integration with the big data lake for advanced analytics.

Skills

PySpark

Scala

SQL

Python

Data pipelines

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Hadoop (Cloudera)

Hive

Impala

Oozie

NiFi

Sqoop

Tableau

Power BI

OSI PI System

Job description

INTECH AutomationIntelligence is seeking a Big Data Engineer to join our team. Design and operate end-to-end big data and analytics solutions, building scalable ETL/ELT pipelines, integrating industrial data, and delivering real-time dashboards that drive efficiency, safety, and compliance.

You will work with Hadoop/Cloudera, NiFi, Sqoop, PySpark/Scala, and BI tools such as Power BI and Tableau, collaborating across Oil & Gas, Petrochemicals, Utilities, and Manufacturing.