Data Engineer - (Migration & Profiling)? Big Data

Mployee.me

Mumbai

Remote

INR 1,800,000 - 3,000,000

Full time

10 days ago

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Mployee.me is seeking a Data Engineer focused on migration and profiling to design and implement scalable data solutions in a remote capacity. The role emphasizes building production-grade pipelines, optimizing performance and cost across large data lakes, and collaborating with cross-functional teams on POC-driven feature delivery.

Ideal candidates have 5–7 years of IT experience, strong SQL/NoSQL skills, and hands-on experience with Big Data ecosystems (Hadoop, Spark, Kafka) and cloud data

Qualifications

  • 5-7 years of IT experience.
  • Experience with Big Data tools (Hadoop, Spark, Apache Beam, Kafka).
  • Proficient in Python, Java, and C++.
  • Experience in production deployment and production support teams.
  • Experience with data warehousing tools (BQ, Redshift, Synapse, Snowflake).
  • Experience with SQL/NoSQL databases and cloud platforms (AWS, GCP, Azure).
  • Experience with data pipelines and workflow tools like Airflow.

Responsibilities

  • Develop high-performance data solutions using cloud platforms to extract, transform, and load big data.
  • Design production-grade data solutions from ingestion to consumption.
  • Optimize data pipelines for performance and cost on large-scale data lakes.
  • Collaborate with Data Engineers to identify tools and perform POCs.
  • Interact with business, BAs, and other Data/ML engineers to deliver features.

Skills

Experience
Team leadership
Python
Java
C++
Data analysis

Tools

Hadoop
Spark
Apache Beam
Kafka
BigQuery
Redshift
Snowflake

Job description

Data Engineer - (Migration & Profiling)

Location: Remote

BGV: Basic

Description
  • 5-7 yrs of IT experience
  • Good understanding of analytics tools for effective analysis of data
  • Should be able to lead teams
  • Should have been part of the production deployment team, Production Support team
  • Experience with Big Data tools- Hadoop, Spark, Apache Beam, Kafka, etc.
  • Experience with object-oriented/object function scripting languages: Python, Java, C++

Scala, etc.

  • Experience with any DW tools like BQ, Redshift, Synapse, or Snowflake
  • Experience in ETL and Data Warehousing.
  • Experience and firm understanding of relational and non-relational databases like MySQL

MS SQL Server, Postgres, MongoDB, Cassandra, etc.

  • Experience with cloud platforms like AWS, GCP and Azure.
  • Experience with workflow management using tools like Apache Airflow.
Role
  • Roles & Responsibilities
  • Develop high-performance and scalable solutions using GCP that extract, transform, and

load big data.

  • Designing and building production-grade data solutions from ingestion to consumption using

Java / Python

  • Design and optimize data models on the GCP cloud using GCP data stores such as

BigQuery

  • Should be able to handle the deployment process
  • Optimizing data pipelines for performance and cost for large-scale data lakes.
  • Writing complex, highly optimized queries across large data sets and creating data

processing layers.

  • Closely interact with Data Engineers to identify the right tools to deliver product features by

performing POC

  • Collaborative team player that interacts with business, BAs, and other Data/ML engineers
  • Research new use cases for existing data.
  • Preferred:
  • Need to be Aware of Design Best practices for OLTP and OLAP Systems
  • Should be part of team designing the DB and pipeline
  • Should have exposure to Load testing methodologies, Debugging pipelines and Delta load

handling

  • Worked on heterogeneous migration projects
  • Role
  • Develop high-performance and scalable solutions using GCP that extract, transform, and

load big data.

  • Designing and building production-grade data solutions from ingestion to consumption using

Java / Python

  • Design and optimize data models on the GCP cloud using GCP data stores such as

BigQuery

  • Should be able to handle the deployment process
  • Optimizing data pipelines for performance and cost for large-scale data lakes.
  • Writing complex, highly optimized queries across large data sets and creating data

processing layers.

  • Closely interact with Data Engineers to identify the right tools to deliver product features by

performing POC

  • Collaborative team player that interacts with business, BAs, and other Data/ML engineers
  • Research new use cases for existing data.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

DT - Lead Engineer
DT - Lead Engineer

Prodapt Solutions • Chennai

On-site
INR 1,200,000 - 2,400,000
Data Engineer
Data Engineer

Smart Ims • Bengaluru

On-site
INR 800,000 - 1,200,000
Data Engineer with GCP
Data Engineer with GCP

Luxoft • Bengaluru

On-site
INR 4,000,000 - 6,000,000
DT - Senior Software Engineer
DT - Senior Software Engineer

Prodapt Solutions • Chennai

On-site
INR 1,200,000 - 1,800,000
Data Engineer
Data Engineer

Accenture PLC • Bengaluru

On-site
INR 1,000,000 - 1,500,000
Lead Engineer – GCP Data Engineer
Lead Engineer – GCP Data Engineer

Prodapt • Chennai

On-site
INR 1,200,000 - 2,400,000
DT - Technical Architect
DT - Technical Architect

Prodapt Solutions • Chennai

On-site
INR 800,000 - 1,600,000
Senior Data Engineer (10–12 Years' Experience Required)
Senior Data Engineer (10–12 Years' Experience Required)

General Mills • Mumbai

On-site
INR 4,500,000 - 7,000,000
Senior Data Engineer
Senior Data Engineer

Luxoft • Bengaluru

On-site
INR 1,200,000 - 2,000,000
Senior Data Engineer
Senior Data Engineer

DXC Luxoft • Pune District

On-site
INR 2,500,000 - 4,200,000