Data Engineer – Enterprise Data Engineering, Data Science and Informatica

Data Intuition Inc.; Brampton; Vor Ort; CAD 53.000 - 76.000; Vollzeit

Benefits dieser Stelle

Dental plan

Health care plan

Vision care benefits

Group insurance benefits

Paid time off for volunteering or个人

Zusammenfassung

Data Intuition Inc. is seeking two qualified Data Engineers to support enterprise data engineering and information systems. The role involves building data pipelines, governance, and integration across Databricks, Delta Lake, Unity Catalog, and Informatica.

You will work on data science, analytics, and information systems projects in a fast-paced environment. The ideal candidate has a background in data engineering and analytics, with hands-on experience in Databricks platforms, Python, SQL, and

Qualifikationen

• Bachelor's degree in Information Technology, Computer Technology/Computer Systems Technology, Data Modelling, Data Warehousing, Database Administration, or related field (or equivalent relevant education and experience).
• 2 years to less than 3 years of relevant professional experience in data engineering, data science, database development, data administration, or information systems.
• Practical experience developing, implementing, and maintaining data processing, database, and IT solutions.

Aufgaben

• Develop, build, and maintain Python, PySpark, and SQL data pipelines using Databricks and Delta Lake.
• Design and implement Databricks environments (catalogs, schemas, access controls, compute configurations).
• Develop data mapping and source-to-target transformation processes.
• Develop queries, stored procedures, triggers, and scripts; operate DBMS for BI and data management.

Kenntnisse

Data Engineering

Data Analytics

Machine Learning

Ausbildung

Bachelor's degree in IT/CS/Data-related field

Tools

Databricks

Delta Lake

Unity Catalog

Python

PySpark

SQL

Informatica

Git

Jira

Confluence

Jobbeschreibung

About Data Intuition Inc.

Data Intuition Inc. provides enterprise IT infrastructure, data engineering, data integration, data science, database administration, data quality and information systems solutions. We are seeking two qualified Data Engineers for enterprise data engineering and information systems. Positions involve data science, predictive analytics, database architecture, data integration, data administration, information systems development, software QA and data governance across Databricks, Delta Lake, Unity Catalog, and Informatica.

Key Duties and Responsibilities

1. Data Science, Analytics and Predictive Modelling

• Conduct data science/analytical activities on large datasets to identify trends, patterns, and relationships.
• Build and implement predictive models for business analysis, forecasting, and decision-making.
• Apply statistical and machine-learning methods (including ensemble modelling).
• Design data collection, processing, and analysis methods for business and research questions.
• Research data availability/suitability from structured and unstructured sources.
• Extract, process, and organize information from unstructured datasets.
• Develop queries and programming logic for reporting and analytics.
• Prepare data visualizations, dashboards, and analytical reports.
• Explain technical/business benefits of analytical solutions to stakeholders.
• Evaluate automated anomaly detection systems and recommend implementations.

2. Data Engineering, Database Architecture and Systems Integration

• Conduct business and technical studies for systems implementation, modernization, integration, and migration.
• Consult with clients on business requirements, technical specifications, and data management.
• Design and implement Databricks environments (catalogs, schemas, access controls, compute configurations).
• Build and maintain Python, PySpark, and SQL data pipelines using Databricks and Delta Lake.
• Develop batch/incremental data ingestion, including Change Data Capture (CDC).
• Develop data mapping and source-to-target transformation processes.
• Develop database architectures, data models, process/network models, and information systems solutions.
• Integrate databases, applications, cloud environments, data files, and APIs.
• Coordinate development, installation, integration, and operation of computer-based information systems.
• Develop queries, stored procedures, triggers, and scripts; operate DBMS for BI and data management.

3. Database Administration, Data Governance and Information Security

• Develop and implement data administration policies, standards, procedures, and models.
• Research and document data requirements, collection methodologies, access rules, and administration policies.
• Configure and maintain Databricks Unity Catalog permissions, schemas, and governed assets.
• Develop and maintain Informatica Cloud Data Governance and Catalog (CDGC) solutions.
• Implement database access, network usage, security, backup, and disaster recovery procedures.
• Assess physical/technical security risks; develop contingency plans and breach minimization policies.
• Implement data classification, masking, access controls, lineage, and metadata records.
• Advise on IT strategy and work with stakeholders to resolve data management/governance issues.

4. Informatica Data Integration and Data Quality

• Develop, configure, and maintain Informatica Cloud Data Integration (CDI) mappings and taskflows.
• Integrate source databases, cloud storage, and Databricks environments.
• Develop reusable data quality rules and validation using Informatica Cloud Data Quality (CDQ).
• Maintain/migrate Informatica Data Quality (IDQ) mappings.
• Reconcile source/target datasets using counts, business totals, and row-level comparisons.
• Investigate data inconsistencies, resolve root causes, and manage non-conforming records.
• Develop data quality scorecards, dashboards, and monitoring alerts to enforce completeness, validity, uniqueness, and consistency.

5. Information Systems Development, QA and Technical Coordination

• Design and develop information systems; establish policies across the SDLC.
• Prepare software maintenance plans; upgrade, maintain, and troubleshoot applications and platforms.
• Lead and coordinate information systems professionals and data administrators.
• Coordinate system integration and deployment across dev, test, and production environments.
• Provide architectural guidance and resolve technical/project issues.
• Conduct QA reviews, usability testing, security testing, automated testing, and performance validation.
• Maintain testing documentation, operational procedures, and technical support records.

Required Education

Bachelor's degree in Information Technology, Computer Technology/Computer Systems Technology, Data Modelling, Data Warehousing, Database Administration, or related field (or equivalent relevant education and experience).

Required Professional Experience

• 2 years to less than 3 years of relevant professional experience in data engineering, data science, database development, data administration, or information systems.
• Practical experience developing, implementing, and maintaining data processing, database, and IT solutions.
• Experience with data integration, data management, analytical systems, and software development.

Technical Skills & Tools

• Data Engineering & Analytics: Databricks, Delta Lake, Unity Catalog, Python, PySpark, SQL, machine learning, data warehousing, BI reporting.
• Databases & Integration: Informatica, IBM DataStage, DB2, Oracle, MySQL, MS SQL Server, MS Access, Teradata, Netezza, COBOL, VSAM, ERP/SAP BI.
• Development & Cloud: Java, JavaScript, C, C++, C#, Scala, Visual Basic, .NET, TypeScript, APIs, AWS, DevOps, Agile, Linux, Unix, Windows.
• QA & Tools: Git, GitHub, Visual Studio, VS Code, IntelliJ, Jira, Confluence, Postman, manual/automated testing.

Certifications (Relevant)

• Databricks Certified Data Engineer (Associate or Professional)
• Informatica Cloud Data Governance and Catalog Professional
• Informatica Cloud Data Quality Professional
• (Equivalent industry-recognized certifications also considered).

Asset Qualifications

Qlik Replicate (CDC), Terraform, Azure DevOps CI/CD, Databricks deployment bundles, streaming data, Oracle Spatial/GIS, FMCG/logistics domain knowledge.

Work Conditions & Capabilities

Fast-paced environment, tight deadlines, repetitive technical/data tasks, strong attention to detail, independent and collaborative team player. Criminal record check may be required.

Employment Conditions & Benefits

• Terms: Permanent, full-time (35 hours/week)
• Wage: CAD $47.00 per hour
• Location: On-site at Brampton, Ontario, L6T 5B6
• Benefits: Dental plan, health care plan, vision care benefits, group insurance benefits, paid time off for volunteering or personal days.