Operations Support Engineer - Lead

Black Canyon Consulting; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical, Dental, Vision

401k with employer contribution

Paid holidays

Vacation

Tuition reimbursement

Job summary

Black Canyon Consulting’s National Center for Biotechnology Information (NCBI) team seeks a Lead OperationsSupport Engineer to guide DevOps and infrastructure efforts, supporting internal developers and operations at NIH. This is a full-time role with remote options, based in the Bethesda, MD area.

This position offers remote options and on-site opportunities, with responsibilities spanning platform operations, incident response, and collaboration with cross-functional teams.

Qualifications

• BS degree in STEM or equivalent experience.
• Experience supporting DevOps platforms and infrastructure.
• Strong Linux and networking fundamentals.
• Proficiency in scripting languages.
• Ability to lead a team and train members.
• Excellent communication with developers and stakeholders.

Responsibilities

• Lead platform operations team for NCBI DevOps platform.
• Identify and resolve operational problems in a micro-service environment.
• Collaborate with developers to resolve deployment and runtime issues.
• Perform analysis and debugging across multiple technologies.
• Prioritize issues to meet SLOs and error budgets.
• Document processes and SOPs with the team.
• Prepare postmortems and action items to minimize outages.
• Interview and help hire new team members.
• Train new team members and provide on-call support.
• Provide on-call support to NCBI’s internal developers and staff.

Skills

DevOps

Linux

Scripting

Team leadership

Communication

Education

BS in STEM

Equivalent experience

Tools

GitLab

Kubernetes

TeamCity

Docker

Service Mesh

Job description

Overview

Black Canyon Consulting (BCC) is looking for a Lead OperationsSupport Engineer to join our DevOps and infrastructure team to support application developers and operations at the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), an institute of the National Institutes of Health. This opportunity is full time at the NCBI in Bethesda, MD and/or remote.

NCBI is part of the National Library of Medicine (NLM) at the National Institutes of Health (NIH). NCBI advances science and public health by providing free access over the Internet to biomedical literature and genomic data, making it one of the 400 top most-visited sites in the world. NCBI has a diverse staff of smart, talented, and deeply technical people who collaborate to build critically valuable services for researchers, physicians, educators, students, and the general public. For example, NCBI develops and delivers PubMed, an index of over 29 million biomedical research abstracts; and maintains ClinicalTrials.gov, a database of over 400,000 clinical studies around the world.

The Systems & DevOps team is responsible for the efficient operation of infrastructure to run NCBI's many applications. This includes providing convenient, scalable solutions for development, deployment, and operations across teams, languages, and cloud and on-prem environments.

This is a great opportunity to work on challenging problems in a technical, scientific, and goal-oriented environment. NCBI offers flexible working hours, remote options, on- and off-site training courses, and conference attendance and tuition reimbursement.

We attract the best people in the business with our competitive benefits package that includes medical, dental and vision coverage, 401k plan with employer contribution, paid holidays, vacation, and tuition reimbursement. If you enjoy being a part of a high performing, professional service and technology focused organization.

Duties & Responsibilities

• NCBI has built a modern DevOps platform based on GitLab and Kubernetes, and is looking to create a team of support engineers to assist internal developers with transitioning legacy software development and deployment to the new DevOps platform. You would be the leader of our platform operations team.
• Identify and resolve operational problems in a micro-service environment
• Work with developers to resolve deployment and runtime problems
• Perform analysis and debugging work across multiple technologies
• Prioritize issues to keep applications within error budgets and meeting their SLOs
• Provide technical solutions to a wide range of problems and user requests
• Document process, procedures and SOPs by soliciting feedback and suggestions from team members
• Compile postmortems and action items to minimize future outages
• Interview other people for team member roles, and decide which ones to recommend for hire.
• Train new team members, and assist them with issues.
• Provide on-call support to NCBI’s internal developers and other staff.

Requirements

• BS degree in STEM or equivalent experience
• Customer-focused, team-oriented disposition
• Good systems debugging skills
• Comfortable with the Linux environment or UNIX CLI
• Experience with some programming or scripting language
• Have experience creating processes, procedures and SOP documentation
• General understanding of TCP/IP, HTTP, and related protocols
• Initiative to take ownership of tasks and drive them to completion
• Comfortable dealing with users with varying levels of IT knowledge
• Eager to learn new technologies
• Strong communication and soft skills to interface with customers, peers and management
• Good judgement, sense of integrity, and responsibility

Preferred Experience / Skillsets

• Kubernetes, OpenShift, Cloud or Linux experience
• Experience with:
  • Service Reliability Engineering in any capacity
  • Linux systems administration
  • Automated CI servers, especially TeamCity and/or GitLab
  • Automation programming/scripting in any of: bash, Ruby, Python, Go, Java, Scala, Rust, C++, Perl
  • Automated configuration management, such as Puppet, Ansible, Chef, bcfg2, cfengine, etc. Puppet is preferred.
  • Version control systems, especially git
  • Service Mesh technologies (e.g., linkerd, Istio)
  • Configuring or using monitoring and alerting technologies (TIGK stack, Grafana, Prometheus, OpsGenie)
  • Confluence, Jira, and Microsoft Office suite
  • GitOps tools, especially ArgoCD
  • Google Anthos
• Understanding of:
  • Linux internals (system calls, file systems, processes, etc.)
  • Linux network configuration
  • Linux application containerization, especially Docker
  • Attached network storage technologies
  • Cloud computing environment such as AWS, GCP or Azure
  • Automated CI/CD pipelines
  • Distributed systems design principles