Senior Data Engineer

Quest Global; Hyderabad; On-site; INR 1,500,000 - 3,000,000; Full time

Job summary

Quest Global is seeking a Data Engineer in Hyderabad with 5+ years of data engineering experience. The role focuses on designing scalable data pipelines, data modeling, and ETL development using SQL, Python, and AWS services.

You will work with data scientists and business teams to deliver data solutions, ensure quality and security, and optimize cloud resources across large-scale data platforms.

Qualifications

• Master's degree in computer science, engineering, analytics or IT.
• 5+ years of experience in Data Engineering and data migration projects.
• Proficient in SQL and Python for data processing and analysis.
• Strong experience in data modeling, data warehousing and building data pipelines.
• Hands-on experience with Hadoop, Spark and AWS data services.

Responsibilities

• Design, build and maintain scalable data pipelines and ETL processes.
• Develop and optimize data models and data warehouse architectures.
• Implement and manage big data technologies and cloud-based data solutions.
• Perform data migration, transformation and integration from multiple sources.
• Collaborate with data scientists, analysts and business teams to deliver data solutions.
• Ensure data quality, consistency and security across pipelines and storage systems.
• Optimize performance and manage cost-efficient AWS cloud resources.

Skills

SQL

Python

Data modeling

Data warehousing

ETL development

Big data concepts

Education

Master's degree in CS/Engineering/Analytics/IT

Tools

Hadoop

Spark

AWS Redshift

AWS S3

AWS Glue

AWS EMR

AWS Lambda

Kinesis

Firehose

IAM

Job description

Job Title: Data Engineer

Location: Hyderabad

Experience: 5+Years

Job Summary:

We are looking for a skilled and experienced Data Engineer with over 5 years of experience in data engineering and data migration projects. The ideal candidate should possess strong expertise in SQL, Python, data modeling, data warehousing, and ETL pipeline development. Experience with big data tools like Hadoop and Spark, along with AWS services such as Redshift, S3, Glue, EMR, and Lambda, is essential. This role provides an excellent opportunity to work on large-scale data solutions, enabling data-driven decision-making and operational excellence.

Key Responsibilities:

• Design, build, and maintain scalable data pipelines and ETL processes.
• Develop and optimize data models and data warehouse architectures.
• Implement and manage big data technologies and cloud-based data solutions.
• Perform data migration, data transformation, and integration from multiple sources.
• Collaborate with data scientists, analysts, and business teams to understand data needs and deliver solutions.
• Ensure data quality, consistency, and security across all data pipelines and storage systems.
• Optimize performance and manage cost-efficient AWS cloud resources.

Basic Qualifications:

• Master's degree in computer science, Engineering, Analytics, Mathematics, Statistics, IT, or equivalent.
• 5+ years of experience in Data Engineering and data migration projects.
• Proficient in SQL and Python for data processing and analysis.
• Strong experience in data modeling, data warehousing, and building data pipelines.
• Hands-on experience with big data technologies like Hadoop, Spark, etc.
• Expertise in AWS services including Redshift, S3, AWS Glue, EMR, Kinesis, Firehose, Lambda, and IAM.
• Understanding of ETL development best practices and principles.

Preferred Qualifications:

• Knowledge of data security and data privacy best practices.
• Experience with DevOps and CI/CD practices related to data workflows.
• Familiarity with data lake architectures and real-time data streaming.
• Strong problem-solving abilities and attention to detail.
• Excellent verbal and written communication skills.
• Ability to work independently and in a team-oriented environment.

Good to Have:

• Experience with orchestration tools like Airflow or Step Functions.
• Exposure to BI/Visualization tools like QuickSight, Tableau, or Power BI.
• Understanding of data governance and compliance standards.