Platform Engineer (DataOps)

Black Canyon Consulting; United States; Hybrid; USD 115,000 - 150,000; Full time

Benefits offered by this job

Medical/dental/vision coverage

401k plan with employer contribution

Paid holidays & vacation

Tuition reimbursement

Job summary

Black Canyon Consulting is seeking a Platform Engineer (DataOps) to join the DataOps Platform team supporting NCBI at NIH. This full-time role offers onsite/remote work at Bethesda, MD or fully remote.

You will develop and maintain the enterprise DataOps platform, automate pipelines, and help researchers adopt the platform. Strong skills in Kubernetes, cloud services, and data processing are required; bonus familiarity with GitOps and observability tools is a plus.

Qualifications

• Strong coding skills in Python, C++, or equivalent.
• Kubernetes and containerization experience.
• Cloud services: GCP, AWS, Azure or equivalent.
• Experience with data processing apps and modern cloud data infra.
• Linux command-line proficiency.

Responsibilities

• Develops and continuously improves DataOps platform.
• Maintains common tools and libraries.
• Evaluates new technologies and practices.
• Helps NCBI developers adopt the platform.
• Ensures compliance with security standards via automated pipelines.
• Embraces agile development and continuous improvement.
• Encourages growth and leadership opportunities at any level.

Skills

Programming languages

Kubernetes

Cloud platforms

Messaging systems

Airflow

Data processing

Linux

Docker

ArgoCD

Terraform

GitHub/GitLab

Monitoring/Observability

Secret management

Data transfer tools

Education

B.S. in STEM fields

Tools

Docker

ArgoCD

Terraform

GitHub

GitLab

Job description

Black Canyon Consulting (BCC) is searching for **Platform Engineer (DataOps)** to support our work for the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), an institute of the National Institutes of Health. This opportunity is full time and onsite/remote at the NCBI in Bethesda, MD and/or remote.

NCBI is part of the National Library of Medicine (NLM) at National Institutes of Health (NIH). NCBI advances science and public health by providing free access to biomedical literature and genomic data over the web, making it one of the 400 top most-visited sites in the world. NCBI’s diverse staff of smart, talented, and deeply technical people collaborate to build critically valuable services for researchers, physicians, educators, students, and the general public. For example, NCBI develops and delivers PubMed, an index of over 29 million biomedical research abstracts, often with links to full-text literature and supporting data.

This is a great opportunity to work on challenging problems as part of a new DataOps Platform team at NCBI. Developing an enterprise-wide DataOps platform is a new initiative at NCBI. Stepping on decades of experience in dealing with some of industry’s most vital data-intensive applications, NCBI approaches solving the problems at enterprise scale using modern technologies such as Kubernetes, GitOps and containerization

We attract the best people in the business with our competitive benefits package that includes medical, dental and vision coverage, 401k plan with employer contribution, paid holidays, vacation, flexible work schedule and tuition reimbursement.

Duties & Responsibilities

The DataOps Platforms team:

• Develops and continuously improves DataOps platform.
• Develops and maintains common tools and libraries.
• Evaluates new technologies and practices.
• Helps NCBI developers with adoption of platform.
• Ensures compliance with the Federal application security regulations and standards by providing automated solutions and compliance pipelines.
• Embraces agile development and continuous improvement
• Encourages growth mindset and offers leadership opportunities at any level

Required Skills:

• Strong coding skills in at least one programing language are required (Python, C++, …)
• Kubernetes, containerization
• Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure or equivalent cloud services
• Apache Kafka, Google Cloud Pub/Sub or equivalent
• Apache AirFlow or equivalent
• Experience with data processing applications and modern cloud-based data processing infrastructure
• Linux command-line skills

Bonus Skills:

• Google Anthos
• Docker
• GitOps tools: ArgoCD or equivalent
• Infrastructure as code tools: Terraform or equivalent
• GitLab, GitHub, Bitbucket, Teamcity, Artifactory, or equivalent products for management of Git source control, CI/CD pipelines and artifactlifecycle management
• Modern observability and logging tools: Prometheus, EFK (ElasticSearch, fluentd, Kibana), TIGK (Telegraph, InfluxDB, Graphana, Kapacitor),
• DataDog, Sensu, Jaeger, Sentry, OpsGenie, PagerDuty, Splunk, or equivalent
• Secret Management tools such as Hashicorp Vault, CyberArk, Azure Key Vault, Google Cloud Secret Manager or equivalent
• Data transfer tools: AWS DataSync, Aspera, MinIO, CloudSoda or equivalent
• Apache Pulsar, RabbitMQ, Amazon Kinesis, Apache Flume, Apache Storm, Apache Spark Streaming, Google Cloud Pub/Sub
• Experience with best-practice design patterns in coding and architecture
• Experience working in Agile environment

Educational Requirements

• B.S. in a STEM field (Engineering, Computer Science, Mathematics, Physics) or equivalent industry experience in Systems Engineering.

Benefits and Salary

We attract the best people in the business with our competitive benefits package that includes medical, dental and vision coverage, 401k plan with employer contribution, paid holidays, vacation, and tuition reimbursement.

We offer a competitive salary commensurate with experience and location. The targeted range for this position is $115,000 - $150,000.