Data Engineer

Wiraa; Mississauga, Halifax; Remote; CAD 90,000 - 105,000; Full time

Benefits offered by this job

Competitive salary

Bonuses (discretionary)

Health, dental, and vision insurance

Wellbeing programs

Flexible remote work

Professional development

Paid time off

Job summary

Wiraa in Canada is seeking an Azure Databricks Data Engineer to design, develop, and maintain scalable cloud-based data solutions using Azure Databricks, Python, and PySpark.

You'll collaborate with architects, data scientists, and stakeholders to translate requirements into robust data pipelines, monitor performance, and contribute to cloud-native data engineering initiatives. This role supports remote work and is open to Mississauga, ON or Halifax, NS.

Qualifications

• 8-12 years of experience in Data Engineering or related fields.
• Hands-on experience with Azure Databricks in enterprise-scale environments.
• Advanced Python and PySpark for distributed data processing.
• Proven expertise in designing, developing, and optimizing ETL/ELT pipelines and large-scale data platforms.
• Strong understanding of Azure cloud services and data engineering concepts.
• Proficiency in data modeling and SQL.
• Experience troubleshooting complex data processing issues in production environments.

Responsibilities

• Design, build, and maintain scalable data pipelines using Azure Databricks, Python, and PySpark.
• Develop and optimize data ingestion, transformation, and processing frameworks for large datasets.
• Implement cloud-native data engineering solutions leveraging Azure data services and best practices.
• Monitor, troubleshoot, and enhance the performance, reliability, and scalability of data platforms and pipelines.
• Collaborate with business and technical stakeholders to translate requirements into effective data solutions.
• Support production deployments and ensure data solutions meet operational standards.
• Participate in code reviews, documentation, and implementation of data governance policies.
• Stay updated with the latest industry trends and continuously improve data engineering practices.

Skills

Azure Databricks

Python

PySpark

ETL/ELT pipelines

Azure data services

Data modeling

SQL

Troubleshooting

Job description

About The Company

Our organization is a leading provider of innovative technology solutions, dedicated to empowering businesses through cutting-edge data engineering and analytics services. With a strong focus on digital transformation, we serve a diverse range of industries including healthcare, finance, retail, and more. Our commitment to excellence, continuous innovation, and fostering a collaborative work environment has established us as a trusted partner for organizations seeking to leverage data for strategic advantage. We prioritize employee development, diversity, and inclusion, ensuring our team members have the resources and support needed to thrive in a dynamic industry landscape.

About The Role

As an Azure Databricks Data Engineer, you will play a critical role in designing, developing, and maintaining scalable cloud-based data solutions that facilitate efficient data processing and analytics. You will be an integral part of our Data Engineering team, working closely with architects, business analysts, data scientists, and cross-functional technology teams to deliver high-quality, reliable data platforms. Your expertise will enable the organization to harness the power of big data, optimize data workflows, and support strategic decision-making processes.

The ideal candidate will have extensive experience in building and optimizing large-scale data pipelines using Azure Databricks, Python, and PySpark. You will be responsible for developing robust data ingestion, transformation, and processing frameworks, ensuring high performance, scalability, and reliability. Collaboration with stakeholders to translate business requirements into technical solutions will be a key aspect of your role. Additionally, you will monitor and troubleshoot data pipelines, implement best practices for cloud-native data solutions, and contribute to continuous improvement initiatives within the organization.

This role offers flexibility with a remote work model, supporting a healthy work-life balance through various wellbeing programs. The position is open to qualified applicants in Mississauga, ON, or Halifax, NS, with the understanding that work arrangements may evolve based on project and business needs.

Qualifications

• 8-12 years of experience in Data Engineering, Data Warehousing, or related fields
• Strong hands-on experience with Azure Databricks in enterprise-scale environments
• Advanced programming skills in Python and extensive experience with PySpark for distributed data processing
• Proven expertise in designing, developing, and optimizing ETL/ELT pipelines and large-scale data platforms
• Strong understanding of Azure cloud services and data engineering concepts
• Proficiency in data modeling and SQL
• Experience troubleshooting complex data processing issues in production environments

Responsibilities

• Design, build, and maintain scalable data pipelines using Azure Databricks, Python, and PySpark
• Develop and optimize data ingestion, transformation, and processing frameworks for large datasets
• Implement cloud-native data engineering solutions leveraging Azure data services and best practices
• Monitor, troubleshoot, and enhance the performance, reliability, and scalability of data platforms and pipelines
• Collaborate with business and technical stakeholders to translate requirements into effective data solutions
• Support production deployments and ensure data solutions meet operational standards
• Participate in code reviews, documentation, and implementation of data governance policies
• Stay updated with the latest industry trends and continuously improve data engineering practices

Benefits

• Competitive annual salary ranging from $90,000 to $105,000
• Discretionary performance-based bonuses
• Comprehensive health, dental, and vision insurance plans
• Wellbeing programs supporting mental, physical, and financial health
• Flexible remote work arrangements
• Opportunities for professional development and career growth
• Paid time off and leave policies aligned with industry standards

Equal Opportunity

We are an equal opportunity employer committed to fostering an inclusive environment for all employees. We celebrate diversity and are dedicated to creating a workplace that respects and values individual differences. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, age, disability, or any other protected status. We encourage candidates from all backgrounds to apply and join our team in shaping the future of data-driven solutions.