Data Engineer

Compunnel, Inc.; Philadelphia (Philadelphia County); In loco; USD 140.000 - 170.000; Tempo pieno

Descrizione del lavoro

Compunnel, Inc. is seeking a Senior Data Engineer to lead data transformation and modernization efforts. You will migrate legacy platforms to cloud-enabled solutions, leverage Databricks, AWS (S3, Lambda), and develop Python/PySpark pipelines.

Strong SQL skills and Snowflake awareness are a plus. The role can be performed remotely, but an onsite interview is required. You will design and implement scalable data processing workflows, collaborate with cross-functional teams, and tackle complex

Competenze

• Senior-level data engineering experience with a track record of solving complex problems.
• Hands-on Databricks experience and strong AWS (S3, Lambda).
• Proficiency in Python and PySpark; SQL skills for data manipulation.
• Experience with data transformation and platform modernization.
• Ability to complete technical assessments and onsite interviews.

Mansioni

• Design and implement data transformation solutions for modernization.
• Develop scalable data pipelines using Databricks and AWS.
• Work with S3 and Lambda to support data flows.
• Build data processing workflows in Python and PySpark.
• Tackle complex data engineering challenges and migrations.
• Collaborate with technical teams on modernization initiatives.

Conoscenze

Databricks

AWS

Python

PySpark

SQL

Data Transformation

Cloud Modernization

Strumenti

Snowflake

Databricks

Descrizione del lavoro

We are seeking a seasoned Senior Data Engineer with strong experience solving complex data engineering problems. The project focuses on data transformation and modernization, including migrating from legacy platforms to cloud-enabled modern platforms. Extensive Databricks experience is required, along with strong AWS experience, including S3 and Lambda. Strong Python and PySpark skills are also required. Snowflake experience is a major plus and would make a candidate stand out. The role can be performed remotely; however, an onsite interview is required. The interview process includes a first round via Teams followed by an onsite round, with a technical assessment covering Python, PySpark, and SQL.

Key Responsibilities

• Design and implement data transformation solutions as part of legacy platform modernization initiatives.
• Develop and support scalable data engineering solutions using Databricks and AWS.
• Work with AWS services, including S3 and Lambda.
• Develop data processing and transformation workflows using Python and PySpark.
• Solve complex data engineering and data transformation problems.
• Support migration from legacy platforms to modern, cloud-enabled platforms.
• Collaborate with technical teams to deliver data modernization solutions.

Required Qualifications

• Senior-level experience in data engineering with a strong track record of solving data engineering problems.
• Extensive hands-on experience with Databricks.
• Strong AWS experience, including S3 and Lambda.
• Strong proficiency in Python and PySpark.
• Experience with data transformation and platform modernization.
• Ability to successfully complete technical assessments covering Python, PySpark, and SQL.
• Ability to participate in an onsite interview.

Preferred Qualifications

• Experience with Snowflake is a major plus.
• Experience migrating legacy data platforms to modern, cloud-enabled platforms.