Data Engineer

Hitachi Digital Services

Hyderabad, Pune District, Bengaluru

Hybrid

INR 1,200,000 - 1,800,000

Full time

8 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Restaurant d'entreprise
Indemnités de stage/alternance

Job summary

Hitachi Digital Services in Hyderabad is seeking a data engineer to develop data pipelines in Python and PySpark, build and maintain Talend jobs, and design AWS-based data workloads using S3, Glue or EMR.

The role involves loading data into dimensional models, writing and tuning SQL for validation, fixes data-quality issues, and ensuring reliable, scalable ETL across cloud environments.

Qualifications

  • Experience building data pipelines in Python for ingestion and transformation.
  • Hands-on with PySpark DataFrame API and Spark SQL.
  • Proficiency in AWS data services (S3, Glue, EMR).
  • Talend job design, components, and error handling.
  • Strong SQL skills for validation, joins, and data quality.

Responsibilities

  • Develop data pipelines in Python and PySpark for ingestion, cleansing, transformation, and aggregation.
  • Build and maintain Talend jobs (Talend Studio and Talend Management Console), including components, context variables, and job scheduling.
  • Design and run data workloads on AWS: S3 for storage, Glue or EMR for Spark processing.
  • Load data into dimensional models (fact and dimension tables, surrogate keys, slowly changing dimensions).
  • Write and tune SQL for data validation, joins, and ranking, and resolve data-quality issues.

Skills

Python data processing
PySpark
AWS data workloads
Talend
SQL validation

Tools

Talend Studio
AWS (S3, Glue, EMR)
Spark

Job description

Role & responsibilities

  • Develop data pipelines in Python and PySpark for ingestion, cleansing, transformation, and aggregation.
  • Build and maintain Talend jobs (Talend Studio and Talend Management Console), including components, context variables, and job scheduling.
  • Design and run data workloads on AWS: S3 for storage, Glue or EMR for Spark processing.
  • Load data into dimensional models (fact and dimension tables, surrogate keys, slowly changing dimensions).
  • Write and tune SQL for data validation, joins, and ranking, and resolve data-quality issues.

Preferred candidate profile

  • Python: data processing, packaging, and unit testing.
  • PySpark: DataFrame API, Spark SQL, partitioning, and performance tuning.
  • AWS: S3, Glue or EMR
  • Talend: job design, components, context variables, and error handling.
  • SQL: Oracle or similar, including joins, window functions, and data validation..
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

AppSierra • Dadri

On-site
INR 1,200,000 - 2,400,000
Data Engineer
Data Engineer

Appsierra Group • Maharashtra

On-site
INR 1,200,000 - 2,100,000
Data Engineer
Data Engineer

Grid Dynamics • Bengaluru

Hybrid
INR 900,000 - 1,400,000
Data Engineer
Data Engineer

Thomasmount Consulting • Ernakulam

On-site
INR 1,200,000 - 1,800,000
Data Engineer - ETL
Data Engineer - ETL

Forward Eye Technologies • Pune District

On-site
INR 1,500,000 - 2,100,000
Data Engineer
Data Engineer

Indium Software • Bengaluru

On-site
INR 900,000 - 1,500,000
Data Engineer
Data Engineer

Moolya Software Testing • Bengaluru

On-site
INR 1,200,000 - 2,400,000
Data Engineer
Data Engineer

PW (PhysicsWallah) • Dadri

On-site
INR 900,000 - 1,300,000
Aws Data Engineer
Aws Data Engineer

Pi Square Technologies • Hyderabad, Bengaluru

On-site
INR 1,000,000 - 1,800,000
Data Engineer with AWS
Data Engineer with AWS

Virtusa Corporation • Hyderabad, Chennai

On-site
INR 900,000 - 1,500,000