Job Description Data Engineer (3-5 Years Experience)
Role - Data Engineer
Experience - 3-5 Years
Employment Type - Employee
Location - Hybrid - Chennai
Role Overview
We are seeking a skilled Data Engineer to support a strategic SAP HANA to Snowflake Data Platform Modernization initiative. The ideal candidate should have hands‑on experience in data engineering, ETL/ELT pipeline development, SQL programming, and cloud data platforms.
The Data Engineer will work closely with the Senior Data Engineer, Data Architect, SAP HANA Lead, and BI teams to support source system assessment, data profiling, source-to-target mapping, prototype pipeline development, and migration validation. This role requires strong technical expertise in building scalable data pipelines, ensuring data quality, and supporting cloud‑native data engineering practices.
Key Responsibilities
- Develop and support ETL/ELT pipelines for migrating data from SAP HANA to Snowflake.
- Assist in building prototype data ingestion and transformation pipelines.
- Perform data profiling, metadata extraction, and source-to-target mapping.
- Develop transformation logic using SQL, Python, PySpark, and dbt.
- Support historical and incremental data loading using Change Data Capture (CDC) methodologies.
- Assist in implementing Medallion Architecture (Bronze, Silver, Gold) for enterprise data platforms.
- Create and manage Snowflake database objects including databases, schemas, tables, views, stages, and virtual warehouses.
- Perform data validation, reconciliation, and quality checks to ensure migration accuracy.
- Support performance tuning and optimization of SQL queries and ELT processes.
- Collaborate with Data Architects and SAP HANA SMEs to understand source system structures and transformation requirements.
- Prepare technical documentation including source-to-target mappings, data lineage, and transformation specifications.
- Participate in Agile ceremonies, code reviews, and engineering discussions while following established development standards.
Required Skills
- Strong experience in SQL, Python, and PySpark for developing ETL/ELT pipelines.
- Hands‑on experience with Snowflake including databases, schemas, tables, stages, and data loading concepts.
- Experience in ETL/ELT development, data transformation, data profiling, and source-to-target mapping.
- Knowledge of dbt and exposure to tools such as Matillion, Fivetran, Azure Data Factory, AWS Glue, or Apache Airflow.
- Understanding of Medallion Architecture, dimensional data modeling, and basic data warehousing concepts.
- Experience working with SAP HANA or other enterprise data warehouse platforms.
- Knowledge of cloud platforms (AWS, Azure, or GCP) and cloud storage services.
- Familiarity with Git, Azure DevOps, Jira, and Confluence.
- Strong analytical, problem‑solving, communication, and collaboration skills.
Preferred Qualifications
- Bachelor’s degree in computer science, Information Technology, Engineering, or a related discipline.
- Experience working with Snowflake or other cloud data warehouse platforms.
- Exposure to enterprise data migration or modernization projects.
- Snowflake Fundamentals Certification or cloud certifications are an added advantage.
- Experience working in Agile/Scrum delivery environments.
Project‑Specific Expectations
The Data Engineer will support the migration assessment by:
- Developing prototype ELT pipelines to validate SAP HANA to Snowflake migration strategies.
- Performing data profiling, source-to-target mapping, and metadata documentation.
- Supporting implementation of Medallion Architecture and cloud‑native engineering practices.
- Building reusable transformation logic using SQL, Python, PySpark, and dbt.
- Assisting with data validation, reconciliation, and migration testing activities.
- Collaborating with Senior Data Engineers, Data Architects, SAP HANA SMEs, and BI teams to ensure migration readiness.
- Producing implementation‑ready technical documentation and engineering artifacts.