Remote Data Engineer: Python & PySpark in Azure Data Lake

Comrise; Metro Manila; Remote; PHP 360,000 - 420,000; Full time

Job summary

CRB is seeking a Data Lake Support associate to join a remote team. You will work with Python and PySpark to manage data pipelines, monitor deployments, and troubleshoot issues across Azure Synapse and related services.

You will interact with multiple teams to ensure continuous data flows and deliver consumable data packages for business use. The role involves configuring pipelines, reviewing notebooks, and providing DevOps support from a distributed global team.

Qualifications

• Proficiency in Python and PySpark for data processing.
• Experience designing and evolving Azure-based data solutions.
• Comfort with large data sets and Azure Synapse Analytics.
• Able to present data using Power BI and manage DevOps workflows.

Responsibilities

• Configure deployment pipelines and manage code releases across teams.
• Set up monitoring schedules for pipelines by geographic usage areas.
• Troubleshoot failed pipelines and propose future improvements.
• Review Synapse notebooks and assist development teams to fix issues.
• Provide onboarding and configuration for new ingestion data sources.
• Support deployments to core data lake repositories and monitoring.

Skills

Python

PySpark

Data analysis

Critical thinking

Collaboration

Problem solving

Multitasking

Communication

Tools

Azure Cloud

Azure Synapse Analytics

Azure SQL DB

Power BI

Git

DevOps

Job description

CRB is seeking a Data Lake Support associate to join a remote team. You will work with Python and PySpark to manage data pipelines, monitor deployments, and troubleshoot issues across Azure Synapse and related services.

You will interact with multiple teams to ensure continuous data flows and deliver consumable data packages for business use. The role involves configuring pipelines, reviewing notebooks, and providing DevOps support from a distributed global team.