Software Engineer / Databrick Engineer

Node.Digital LLC; Dulles Town Center (VA); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

Medical

Dental

Vision

401K

Three weeks of PTO

Job summary

A technology firm is seeking a skilled data engineer with extensive experience in software development and cloud environments. The role involves migrating services to Databricks and optimizing data processes, requiring excellent communication skills and an active security clearance. Competitive benefits include medical and 401K plans, along with generous PTO.

Qualifications

• Must be a US Citizen with an active Secret clearance.
• 8+ years of directly relevant software development experience.
• Minimum 5 years of experience performing data engineering work.

Responsibilities

• Support teams to migrate services to Databricks.
• Optimize development and monitoring for data assets.
• Ensure technical correctness and delivery quality.

Skills

Data engineering work in a cloud environment

Software development experience

Knowledge of Databricks

Experience with relational and noSQL storage

Communication skills

Education

Bachelor's degree in Software Engineering or related

Tools

Databricks

AWS Redshift

NiFi

Elastic

Apache Spark

Job description

Overview

Node is supporting a U.S. Government customer on a large mission critical development and sustainment program to design, integrate, build, deliver, and operate a network operations environment including introducing new cyber capabilities to address emerging threats. Location: Dulles, VA. Must have an active Secret Clearance.

Responsibilities

• Supporting teams to migrate services, applications and platforms from legacy back-end systems to Databricks
• Identifying the optimal path for migration, building the plan for migration and finally, execution of the plan
• Data pipeline migration of legacy pipelines from NiFi to Databricks, complete with validation
• Implementing the medallion model for each of the data assets being migrated to Databricks
• Develop an SOP for integration of data assets into the Databricks platform focused on efficiency, instrumentation and performance
• Optimize development, testing, monitoring and security for data assets being added to the Databricks platform
• Develop and implement a strategy for optimizing migration and integration of data assets to the Databricks platform
• Develop code using various programming and scripting languages to automate/optimize data ingestion, pipeline orchestration and improve data management processes
• Ingest transparency, leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines
• Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team
• Ensure technical correctness, timeliness and quality of delivery for the team
• Demonstrate excellent oral and written communication skills to all levels of management and the customer

Requirements

Required Skills

• Must be a US Citizen
• Active Secret (S) clearance. Must be able to obtain a TS/SCI clearance
• Must be able to obtain DHS Suitability
• 8 + years of directly relevant software development experience required
• Minimum of 5 years of experience performing data engineering work in a cloud environment
• Experience with relational, noSQL and/or file-based storage (e.g. Databricks, Elastic, Postgres, S3, Athena, etc.)
• Experience working in a CICD Pipeline factory environment
• Working knowledge of Databricks, Cloud Relational Database Services, NiFi, AWS Redshift and Elasticsearch

Desired Skills

• Experience with:
• Databricks workflows
• Databricks Unity Catalog
• Databricks Autoloader
• Databricks Delta Live Tables/Delta Lake
• Databricks Workspace/Notebooks
• MLflow
• Apache Spark
• Experience with collaboration tools including MS Teams, MS Outlook, MS SharePoint, and Confluence
• AWS Professional certification or equivalent
• Excellent problem-solving and communication skills
• Familiarity with CISA: Securing the Software Supply Chain
• Familiarity with CISA: Cybersecurity Best Practices
• Familiarity with CISA: Open-Source Software Security
• Familiarity with NIST SP 800-218, Secure Software Development Framework V1.1: Recommendations for Mitigating the Risk of Software Vulnerabilities

Required Education

• Bachelor's degree in Software Engineering, Computer Science or a related discipline is required. Ten (10) years of experience (for a total of six (18) or more years) may be substituted for a degree.

Company Overview

Node.Digital is an independent Digital Automation & Cognitive Engineering company that integrates best-of-breed technologies to accelerate business impact.

Our Core Values help us in our mission. They include identifying the right people and developing them to their full capabilities. Our customers' mission is our mission, and our mission-first approach keeps them engaged while becoming their trusted partner. We believe in simplifying complex problems with a relentless focus on agile delivery excellence. Our mantra is: Simple*Secure*Speed in delivery of innovative services and solutions.

Benefits

• Medical
• Dental
• Vision
• Basic Life
• Health Saving Account
• 401K
• Three weeks of PTO
• 10 Paid Holidays
• Pre-Approved Online Training