Senior Data Engineer

Xforia Global Talent & Technology Solutions; Halifax; On-site; CAD 90,000 - 130,000; Full time

Job summary

Xforia Global Talent & Technology Solutions is seeking a Senior Data Engineer in Halifax to lead the development of scalable data pipelines and architectures. You will design and implement data solutions that support predictive analytics and business insights, collaborating with product and finance teams.

The role requires strong Python/PySpark, SQL, and AWS data services expertise, along with leadership in Agile environments. On-site presence in Halifax is expected.

Qualifications

• Experience in Python and PySpark for data processing.
• Proficient in SQL and big data processing.
• Experience with AWS data services (S3, Redshift, Lambda, EMR, Airflow, Glue).
• Bash/Shell scripting proficiency.

Responsibilities

• Leads development of data pipelines and architectures for large data sets.
• Designs, constructs, and tests data architecture aligned with business requirements.
• Provides technical leadership for data projects ensuring high quality data solutions.
• Collaborates with product, finance, and other business units to meet data requirements.
• Oversees development of data solutions enabling analytics.
• Ensures technical quality of data solutions across environments.

Skills

Python scripting

PySpark

SQL

-AWS data services

Bash/Shell scripting

Agile

communication

Tools

S3

Redshift

Lambda

EMR

Airflow

Glue

Job description

• Strong experience in Python Scripting and PySpark for data processing
• Proficiency in SQL, dealing with big data processing
• Experience in AWS cloud platform and its data services (S3, Redshift, Lambda, EMR, Airflow, Glue)
• Expertise in BASH/Shell scripting
• Strong understanding of healthcare insurance domain and experience leading data engineering teams
• Experience in Agile environments.
• Excellent problem-solving skills and attention to detail.
• Effective communication and collaboration skills.

Responsibilities:

• Leads development of data pipelines and architectures that handle large-scale data sets.
• Designs, constructs, and tests data architecture aligned with business requirements.
• Provides technical leadership for data projects, ensuring best practices and high-qualitydata solutions.
• Collaborates with product, finance, and other business units to ensure data pipelines meet business requirements.
• Oversees development of data solutions that enable predictive and prescriptive analytics.
• Ensures the technical quality of solutions, managing data as it moves across environments.