Data Engineer

Advance Career Solutions; Pune District, Chennai District, Bengaluru; On-site; INR 1,200,000 - 2,800,000; Full time

Job summary

Advance Career Solutions is seeking a skilled Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support business intelligence, analytics, and machine learning initiatives. The role focuses on high-quality, reliable, and secure data solutions across data integration and warehousing domains.

The ideal candidate has strong SQL, Python/Java/Scala, and hands-on ETL/ELT development experience, plus familiarity with cloud platforms and big data tools such as

Qualifications

• Bachelor's degree or higher in a relevant field.
• 3+ years of experience in data engineering or a related role.
• Strong proficiency in SQL and database design principles.
• Experience with Python, Java, or Scala.
• Hands-on ETL/ELT development and data integration tools.
• Experience with relational and NoSQL databases.
• Familiarity with AWS, Azure, or Google Cloud Platform.
• Experience with Spark, Hadoop, or Kafka.
• Knowledge of data warehousing concepts and dimensional modeling.
• Understanding of CI/CD, version control (Git), and DevOps practices.
• Strong analytical, problem-solving, and communication skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines for ingesting, transforming, and loading data from multiple sources.
• Build and optimize ETL/ELT workflows to ensure efficient and reliable data processing.
• Develop and manage data warehouses, data lakes, and data marts.
• Collaborate with data analysts, data scientists, and software engineers to deliver data solutions that meet business requirements.
• Monitor data pipelines and troubleshoot issues related to data quality, performance, and reliability.
• Optimize database performance through indexing, partitioning, and query tuning.
• Implement data validation, governance, security, and compliance best practices.
• Automate data workflows using orchestration tools and scheduling frameworks.
• Support real-time and batch data processing solutions.
• Create and maintain technical documentation for data architecture, pipelines, and processes.
• Stay current with emerging technologies and recommend improvements to the organization's data platform.

Skills

SQL

Python

Apache Spark

Apache Kafka

Apache Airflow

Hadoop

Snowflake

Databricks

AWS / Azure / Google Cloud Platform

Docker

Kubernetes

Git

ETL/ELT Development

Data Warehousing

Data Modeling

Education

Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field

Job description

Job Title

Data Engineer

Job Summary

We are seeking a skilled and detail-oriented Data Engineer to design, build, and maintain scalable data pipelines and infrastructure that support business intelligence, analytics, and machine learning initiatives. The ideal candidate has strong expertise in data integration, database technologies, cloud platforms, and ETL/ELT processes, with a focus on ensuring high-quality, reliable, and secure data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for ingesting, transforming, and loading data from multiple sources.
• Build and optimize ETL/ELT workflows to ensure efficient and reliable data processing.
• Develop and manage data warehouses, data lakes, and data marts.
• Collaborate with data analysts, data scientists, and software engineers to deliver data solutions that meet business requirements.
• Monitor data pipelines and troubleshoot issues related to data quality, performance, and reliability.
• Optimize database performance through indexing, partitioning, and query tuning.
• Implement data validation, governance, security, and compliance best practices.
• Automate data workflows using orchestration tools and scheduling frameworks.
• Support real-time and batch data processing solutions.
• Create and maintain technical documentation for data architecture, pipelines, and processes.
• Stay current with emerging technologies and recommend improvements to the organization's data platform.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 3+ years of experience in data engineering or a related role.
• Strong proficiency in SQL and database design principles.
• Experience with Python, Java, or Scala.
• Hands-on experience with ETL/ELT development and data integration tools.
• Experience with relational and NoSQL databases.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Experience with big data technologies such as Apache Spark, Hadoop, or Kafka.
• Knowledge of data warehousing concepts and dimensional modeling.
• Understanding of CI/CD, version control (Git), and DevOps practices.
• Strong analytical, problem-solving, and communication skills.

Preferred Qualifications

• Experience with orchestration tools such as Apache Airflow or Prefect.
• Knowledge of containerization technologies such as Docker and Kubernetes.
• Experience with modern data warehouse platforms such as Snowflake, Amazon Redshift, BigQuery, or Azure Synapse.
• Familiarity with Infrastructure as Code (Terraform, CloudFormation, or similar).
• Relevant cloud or data engineering certifications are a plus.

Technical Skills

• SQL
• Python
• Apache Spark
• Apache Kafka
• Apache Airflow
• Hadoop
• Snowflake
• Databricks
• AWS / Azure / Google Cloud Platform
• Docker
• Kubernetes
• Git
• ETL/ELT Development
• Data Warehousing
• Data Modeling