Senior Data Engineer

Coforge; Fort Mill (SC); On-site; USD 90,000 - 120,000; Full time

Job summary

Coforge is seeking an experienced Data Engineer to design and maintain scalable ETL pipelines that feed Snowflake data warehouses. You will use Airflow and Glue on AWS for orchestration, write efficient SQL, and apply Python to automate processing.

The role requires collaboration with analysts and a focus on data quality and real-time availability. The ideal candidate has 6-8 years in data engineering, a CS/IT degree, and strong communication skills to thrive in a fast-paced environment in Fort

Qualifications

• Bachelor's degree in CS/IT or related field.
• 6-8 years of experience in data engineering or similar.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Ability to work independently and in fast-paced team environments.

Responsibilities

• Design, develop, and maintain ETL pipelines to move data into Snowflake.
• Leverage AWS services like Airflow and Glue for orchestration and processing.
• Work with HVR for real-time data replication and integration.
• Write and optimize SQL queries for data retrieval and analysis.
• Collaborate with analysts and stakeholders to meet data requirements.
• Monitor pipelines for availability and performance; troubleshoot issues.
• Document data processes and maintain data quality.
• Keep up-to-date with data engineering and cloud computing trends.

Skills

AWS Airflow

Glue

Snowflake

SQL

Python

ETL

Cloud concepts

HVR

Education

Bachelor's degree in CS/IT

Tools

Git

Job description

• Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into Snowflake.
• Utilize AWS services such as Airflow and Glue for orchestration and data processing tasks.
• Implement data integration solutions using HVR to ensure real-time data availability.
• Write and optimize SQL queries for data retrieval and manipulation.
• Collaborate with data analysts and other stakeholders to understand data requirements and deliver solutions that meet business needs.
• Monitor and troubleshoot data pipelines to ensure high availability and performance.
• Document data processes and maintain data quality standards.
• Stay updated with the latest industry trends and technologies related to data engineering and cloud computing.

Mandatory Skills:

• Proven experience with AWS services, particularly Airflow and Glue.
• Strong knowledge of Snowflake and its data warehousing capabilities.
• Proficiency in SQL for data manipulation and analysis.
• Experience with Python for data processing and automation tasks.
• Hands-on experience with ETL processes and tools.
• Strong understanding of cloud computing concepts and architectures.

Preferred Skills:

• Experience with HVR for data replication and integration.
• Familiarity with data modeling and database design principles.
• Knowledge of data governance and data quality best practices.
• Experience with version control systems such as Git.
• Understanding of Agile methodologies and project management practices.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 6-8 years of experience in data engineering or a related role.
• Strong analytical and problem-solving skills.
• Excellent communication and collaboration abilities.
• Ability to work independently and as part of a team in a fast-paced environment.