Data Engineer - Onsite in Arlington | Databricks & Big Data

West 4th Strategy

Arlington (VA)

Hybrid

USD 155,000 - 165,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

West 4th Strategy is seeking a Data Engineer to support the National Guard Bureau's Minuteman project, designing and maintaining the enterprise data environment on the central analytics platform. You will build end-to-end data pipelines using Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, collaborating with cross-functional teams to meet data requirements and enable insights.

The role involves onsite work in Arlington, VA 1–2 days per week, a required Active Secret clearance, and a

Qualifications

  • 3–10 years of related experience.
  • Onsite in Arlington, VA 1–2 days per week.
  • 3+ years as a Data Engineer.
  • Proficiency with ETL and Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks.
  • Experience with large datasets and big data technologies.

Responsibilities

  • Design, develop, and implement end-to-end data pipelines using ETL processes and technologies.
  • Collaborate with cross-functional teams to understand data requirements and design data models.
  • Create and optimize data pipelines for scalability and high performance.
  • Perform data cleansing, data integration, and data quality assurance.
  • Leverage big data technologies to process and analyze large datasets.
  • Collaborate with data scientists and stakeholders to provide data insights.

Skills

Data governance
Data modeling
Data analysis

Education

B.A./B.S. or M.A./M.S. in a related field

Tools

Databricks
Python
Spark
Scala
JavaScript/JSON
SQL
Jupyter Notebooks

Job description

Data Engineer
We need an experienced Data Engineer at the National Guard Bureau (NGB). The NGB is the link between the Army and Air National Guard and the Department of War (DoW). You will support operations on the Minuteman project team, which manages the NGB's enterprise data environment within the DoW's central analytics platform, and plays a crucial role in designing, developing, and maintaining the Minuteman Community Space's data infrastructure and systems. In this role, you will design and implement end-to-end data pipelines using ETL processes and technologies including Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL, working closely with cross-functional teams to understand data requirements and design optimal data models and architectures that ensure efficient data processing and analysis. This is a full-time opportunity. We can offer a competitive salary, and a comprehensive benefits package.

RESPONSIBILITIES
  • Design, develop, and implement end-to-end data pipelines, utilizing ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL
  • Work closely with cross-functional teams to understand data requirements and design optimal data models and architectures
  • Create and optimize data pipelines from scratch, ensuring scalability, reliability, and high-performance processing
  • Perform data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasets
  • Leverage big data technologies to efficiently process and analyze large datasets
  • Collaborate with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision-making processes
  • Troubleshoot data-related problems and provide innovative solutions to address complex data challenges
  • Implement and enforce data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practices
  • Research, design, and develop software and specialized utility programs in support of mission requirements
  • Analyze user needs and develop software solutions, applying principles and techniques of computer science, engineering, and mathematical analysis
  • Update and enhance existing software capabilities
  • Develop specifications and performance requirements, coordinating with engineering counterparts to integrate hardware and software systems as required
  • Maintain databases within the application area, working individually or coordinating database development as part of a team

REQUIRED QUALIFICATIONS
  • Three (3) to ten (10) years of directly related experience
  • Available to work onsite in Arlington, VA 1-2 days per week
  • 3+ years of experience as a Data Engineer
  • Proficiency in ETL processes and firsthand experience with technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks
  • Experience working with large datasets and leveraging big data technologies to process and analyze data efficiently
  • Understanding of data modeling/visualization, database design principles, and data governance practices

PREFERRED QUALIFICATIONS
  • Experience with the Department of War (DoW) mission and associated datasets
  • Previous experience with Advana or the War Data Platform
  • Experience with APIs
  • Experience with Machine Learning models and/or Generative AI
  • Experience building within Palantir Foundry applications, including Workshop, Ontology Manager, and Gaia

EDUCATION
  • B.A./B.S. or M.A./M.S. in a related field

LOCATION
  • National Capital Region. Onsite in Arlington, VA 1-2 days per week – Required

CLEARANCE
  • Active Secret security clearance
  • U.S. citizenship is required since this role supports the U.S. Federal government

CLIENT
  • OUSD Personnel & Readiness (P&R), Minuteman project team

WORK HOURS
  • 40 hours per week. 8 hours per day

EMPLOYMENT CLASSIFICATION
  • Employment Classification Eligibility — W2

COMPENSATION
  • Salary range:$155,000 - $165,000
  • Benefits: Benefits package includes options for health, dental, and vision insurance coverage; 401k contribution options
West 4th Strategy is an Equal Opportunity (EEO) employer. All qualified applicants will receive consideration for employment without regard to race, color, creed, religion, gender, sexual orientation, ancestry, national origin, age, marital status, mental disability, physical disability, medical condition, pregnancy, political affiliation, military or veteran status, or any other basis prohibited by federal or state law.

Other Considerations: applicants will be subject to a background investigation. Individual’s primary workstation is located in an office area. The noise level in this environment is low to moderate. Regularly required to sit for extended periods up to 80% of the time; frequently required to move about to access file cabinets and use office equipment such as PC, copier, fax, telephone, cell phone, etc. Occasionally required to reach overhead, bend, and lift objects of up to 10 lbs. Specific vision abilities required by this job include the use of computer monitor screens up to 80% of the time.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer (Hybrid)
Data Engineer (Hybrid)

West 4th Strategy • Arlington (VA)

On-site
USD 150,000 - 165,000
Palantir Foundry Data Analyst
Palantir Foundry Data Analyst

West 4th Strategy • Virginia

Hybrid
USD 155,000 - 165,000
Health insurance
Dental insurance
Vision insurance
+1
Remote Data Scientist & Dashboard Developer
Remote Data Scientist & Dashboard Developer

West 4th Strategy • United States

Remote
USD 155,000 - 165,000
Health, dental, and vision insurance
401k options
Palantir Foundry Data Analyst
Palantir Foundry Data Analyst

West 4th Strategy • Arlington (VA)

Hybrid
USD 155,000 - 165,000
Health, dental, and vision insurance
401k contribution options
Junior Data Engineer
Junior Data Engineer

West 4th Strategy • United States

Remote
USD 100,000 - 105,000
Data Scientist/Dashboard Developer
Data Scientist/Dashboard Developer

West 4th Strategy • United States

Remote
USD 155,000 - 165,000
Health, dental, and vision insurance
401k options
Senior Data Scientist - Defense Analytics
Senior Data Scientist - Defense Analytics

Moseley Technical Services, Inc. • Arlington (VA)

On-site
USD 175,000 - 215,000
Medical insurance
Dental & vision insurance
401(k) with immediate vesting
+2
Data Engineer / Developer – Python & SQL
Data Engineer / Developer – Python & SQL

Brite Consulting • United States

Hybrid
USD 110,000 - 160,000
401(k)
Health insurance
Dental insurance
+8
Data Engineer (Various Locations)
Data Engineer (Various Locations)

Agile Defense, Inc. • Arlington (VA)

On-site
USD 110,000 - 160,000
Data Engineer / Developer – Python & SQL
Data Engineer / Developer – Python & SQL

Brite Consulting • Washington

Hybrid
USD 110,000 - 160,000