Big Data Solutions Architect - Expert

Job Title: Big Data Solutions Architect

Location: Toronto, ON

Duration: 12+ Months

Daily Responsibilities:

1. Collaborate with data engineers, data scientists, business analysts, and IT teams to implement the data strategy in the cloud.
2. Build scalable data platform infrastructure using cloud native technologies to enable Data Engineering, Data Science and Reporting to meet the business needs.
3. Design and oversee the extraction, transformation, and loading (ETL) processes, ensuring efficient data flow from source systems and on-premises data lakes into the cloud.
4. Ensure that the data solutions are optimized for performance and reliability.
5. Implement security and recovery measures to protect data quality and ensure compliance with relevant policies and regulations.
6. Design and implement automated procedures for the effective management of data throughout its lifecycle, from creation and storage to archiving and deletion.
7. Maintain comprehensive documentation for the cloud data architecture including the development of big data standards, best practices, and methodologies.
8. Stay abreast of the latest trends and advancements in big data technologies.
9. Provide guidance and training to other team members on big data concepts and cloud data platforms.
10. Apply design-thinking and agile mindset in working with various stakeholders to continuously experiment, iterate and deliver on solutions that enable end-users to extract insights and value from data.

Must have skills:

1. Bachelor's or master’s degree in computer science, Information Systems, or a related field.
2. 3-5 years of architecture experience in both Databricks and Snowflake platforms to design and implement big data solutions including migration towards hybrid architecture.
3. Professional certification on designing solutions for Azure and/or AWS public clouds.
4. Strong knowledge of ETL processes, OLAP/OLTP systems, SQL and NoSQL Databases.
5. Experience building batch and real-time data pipelines leveraging big data technologies such as Spark, Airflow, Nifi, Kafka, Cassandra, and Elasticsearch.
6. Expertise in DataOps/DevOps practices for deploying and monitoring automated data pipelines and data lifecycle management.
7. Proficiency in writing and optimizing SQL queries and at least one programming language like Java, Scala and/or Python.
8. Continuously learning mindset and enjoy working on open-ended problems.

Nice to have:

1. System administration experience including Docker and Kubernetes platforms.
2. Experience with OpenShift, S3, Trino, Ranger and Hive.
3. Knowledge of machine learning and data science concepts and tools.
4. Experience with BI tools.

Mindlance is an equal opportunity employer. We are committed to inclusive, equitable, barrier-free recruitment and selection processes, and work environment in accordance with the Accessibility for Ontarians with Disabilities Act (AODA). We will be happy to work with applicants requesting accommodation at any stage of the hiring process.