Senior Storage Analyst - (GOAPRDJP00000688)

Amyantek; Canada; On-site; CAD 80,000 - 100,000; Part time

Job summary

A government organization in Alberta is seeking a Storage Analyst to manage and support a Hadoop-based data platform. The successful candidate will have over 5 years of experience with Cloudera, Hadoop, and Linux administration. Responsibilities include implementing data security protocols, monitoring performance, and ensuring system availability. This role offers a contract position with a mid-senior level designation.

Qualifications

• 5 years of Cloudera Data Platform administration experience.
• 5 years of Hadoop/Spark deployment experience.
• 5 years of Linux administration experience, preferably RHEL.
• 5 years of Security and Identity Management experience with Kerberos and Active Directory.

Responsibilities

• Perform system administration of Cloudera Data Platform.
• Lead or assist with development and support of Azure and Snowflake.
• Provide services to support data privacy and security.
• Monitor and manage Hadoop log files and services.

Skills

Cloudera administration

Hadoop/Spark

Linux administration

Security and Identity Management

Education

Bachelor degree in Computer Science, IT or related field

Tools

Cloudera Data Platform

Hadoop

Spark

Linux

Job description

Overview

The Data and Content Management Division uses a one-government approach to information and privacy governance, decision-making, and service delivery across the Government of Alberta (GoA) balanced with individual client needs. This approach enhances data access, fosters collaboration, reduces data duplication, and drives innovation, ensuring efficient and effective services that ultimately benefit Albertans.

As part of this mandate, the Data Centre of Excellence (DCoE) requires a resource to oversee the platform and cluster administration of a Hadoop-based integrated data and analytics platform/enterprise data lake environment (Cloudera Data Platform). This role will also involve leading or assisting in the development, administration, and support of other tools and platforms, including but not limited to Azure and Snowflake. These efforts will support the implementation of the GoA Data Strategy, enabling the government to better leverage its data assets. Working within a team, this resource is an expert and plays a key role in ensuring that cloud and on-premises systems are designed, developed, operated, and maintained to meet the needs of ministries and external stakeholders. This role will be instrumental in realizing the pillars of the GoA Data Strategy by using data to deliver meaningful outcomes for Albertans.

The successful candidate will be a Storage Analyst / Platform Analyst / Platform Developer with a diverse range of technical skills and experience in working in multi-faceted roles. The role may have aspects of any or all the following: Data platforms (on-premises Hadoop-based and cloud) management and day to day administration, cloud architecture design and integration, security and compliance, ITIL-based service management, Agile delivery and DevOps practices, data governance and metadata management and stakeholder engagement and user support.

Duties / Responsibilities

• Perform system administration and support of tools and platforms, including but not limited to Cloudera Data Platform.
• Lead or assist with development, administration, and support of other tools and platforms, including but not limited to Azure and Snowflake.
• Configure and tune cloud-based tools to manage cost, security, and compliance for the environments. Working across a myriad of technology stacks in Cloudera, Azure, Snowflake etc.
• Provide services to support data privacy and security in adherence to enterprise policies (access control, security policies across platforms, etc.).
• Deploy sophisticated analytics programs, AI/machine learning, and statistical methods.
• Develop technical design, perform capacity arrangement, computational cluster set up, performance fine-tuning, monitoring, structure planning, scaling, and overall GOA data lake/ platform administration.
• Collaboration with other team members to improve and optimize Cloud and on-premises infrastructure systems including scalability, reliability, performance, capacity, availability and disaster recovery
• Work with infrastructure, network, database, and application teams to ensure data lake components are highly available and performing within agreed-upon service levels.
• Implement and/or maintain relevant components of the Hadoop ecosystem. Examples: Apache YARN, MapReduce, HDFS, Hive, Zookeeper, Pig, Spark, Zeppelin, Atlas, Kerberos, Ranger, etc.
• Install, administer, and support Linux and Windows operating systems in an enterprise environment.
• Accountable for Hadoop platform and storage performance tuning and volume management of cluster and routines.
• Perform setup, configuration, and security for the Hadoop cluster including integration with Microsoft Active Directory (AD).
• Monitor Hadoop cluster connectivity and performance.
• Manage and analyze Hadoop log files, optimize Hadoop services allocation for memory and CPU cores as well as garbage collection optimization.
• HDFS support and maintenance. File system management and monitoring. Provide file system services (file structure, access, sharing, quotas and security rights).
• Develop and document best practices.
• Set up new Hadoop users through integration with AD User and Groups as well as help set up security policies through Apache Ranger.
• Manage and administer Hadoop infrastructure on Linux Red Hat Enterprise Linux (RHEL) 7.x and higher versions.
• Adhere to existing Government of Alberta established policies, procedures, standards, guidelines and practices.
• Maintain, support, and enhance enterprise storage software/hardware products.
• Provide support for backup and recovery services including the hardware and software required to manage a large enterprise environment.
• Maintain/update storage procedures, processes documentation, configuration diagrams, and spreadsheets.
• Establish proper communications with clients, vendors, and internal support groups.
• Provide required technical assistance and consultation to internal technical teams as well as other ministries, as needed.
• Establish and operate control processes including schedule management, status reporting, issues, and risk mitigation.
• Identify and escalate issues and risks, as appropriate.
• Work on multiple projects concurrently.
• Ensure delivery processes are robust, of high quality and repeatable.
• Produce deliverables within specified quality standards and timelines.
• Document configurations, processes, known issues, and solutions to support knowledge transfer, operational readiness, and onboarding of team members.
• Contribute to defining and establishing an Enterprise Data Strategy in GoA and be actively involved in the design of modern data platform features and functionalities.
• Participate in sprint planning, version control (e.g., Git), CI/CD pipelines, and automated testing to deliver incremental improvements and maintain platform(s) stability.

Qualifications

Education
Bachelor degree in Computer Science, IT or related field of study

Work Experience

• 5 years - Experience with Cloudera. Hands on experience with Cloudera Data Platform's administration: monitor, configure, deployment, manage user, upgrade, manage license, troubleshooting
• 5 years - Experience with Hadoop/Spark. Experience with: deploy cluster, add and remove nodes, keep track of jobs, monitor critical parts of the cluster, configure name-node high availability, schedule and configuration, data backup and recovery, and system and information security
• 5 years - Experience with Linux, preferably RHEL or CentOS (deployment, configuration, administration, hardening, performance tuning, troubleshooting). General operational expertise such as good troubleshooting skills, understanding of system’s capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks. Proficiency with Unix-based file system
• 5 years - Experience with Security and Identity Management. Experience with Kerberos, Active Directory, LDAP (Preferably in relation to Hadoop security or security in a Unix/Linux/Windows heterogeneous environment).

Nice to Have

• Travel/Flexible Schedule: Ability and willingness to travel as necessary between Edmonton and Calgary
• Able to participate in a 7/24/365 on call rotation
• Able to work evenings and/or weekends and/or during blocktime/change windows outside of normal business hours.

Submission & References

• RESUME: ALL REQUIRED EXPERIENCE MUST BE DESCRIBED IN RESUME UNDER THE JOB/PROJECT WHERE EXPERIENCE WAS ATTAINED. EACH JOB/PROJECT MUST CONTAIN THE TERM OF THE JOB/PROJECT IN THE FORMAT MMM/YYYY to MMM/YYYY.
• RESOURCE REFERENCES: Three references, for whom similar work has been performed, must be provided. The most recent reference should be listed first. Reference checks may or may not be completed to assist with scoring of the proposed resource.

Seniority level
Mid-Senior level

Employment type
Contract

Job function
Information Technology

Industries
Information Services, Government Administration, and Government Relations Services

Note: This posting may include additional postings and regional variations.