Test and Assessment Specialist

Valkyrie Enterprises, Inc.; Virginia Beach (VA); On-site; USD 75,000 - 90,000; Full time

Benefits offered by this job

Medical/dental/vision options

401k with match

Education reimbursement

Company paid holidays and PTO

Job summary

Valkyrie Enterprises, Inc. is seeking a Test and Assessment Specialist to support NAMTS.

The role focuses on developing, validating, and evaluating Navy assessments, including written exams, practical evaluations, and performance-based qualifications for maintenance and operations tasks. The candidate will work with Navy SMEs and instructional designers to design criterion-referenced tests, scoring rubrics, and traceability across references, tasks, and objectives, ensuring accuracy and

Qualifications

• Bachelor's degree or equivalent in Instructional Design, Education, Training and Development, Engineering, or related field with Navy training/assessment experience
• Experience developing, reviewing, or validating technical examinations and qualification products
• Knowledge of ISD, task analysis, learning-objective development, criterion-referenced testing, and performance-based assessment methodologies
• Ability to translate maintenance tasks into measurable assessment criteria and observable standards
• Experience creating scoring rubrics, checklists, and instruments traceable to references
• Understanding assessment validity, reliability, and evaluation methods for technical knowledge
• Ability to read Navy technical manuals and maintenance procedures
• Knowledge of Navy shipboard maintenance practices and qualification processes
• Experience collaborating with SMEs to validate content and requirements
• Strong technical writing, editing, and communication skills
• Ability to manage multiple development actions and maintain configuration-controlled products
• Proficiency in Microsoft Word, Excel, and PowerPoint and LMS/e-document systems
• Ability to work independently and with Navy stakeholders and contractor personnel

Responsibilities

• Analyze tasks, requirements, and objectives to identify assessment methods
• Develop and validate written examinations and skill assessments
• Design scoring rubrics and checklists aligned with references
• Create MCQs, scenario-based, and practical evaluations
• Establish observable performance indicators and safety criteria
• Evaluate items for accuracy, validity, and alignment with references
• Apply test development principles to ensure validity and fairness
• Maintain traceability across items, tasks, objectives, and references
• Collaborate with Navy SMEs to select appropriate methods
• Analyze results and identify deficiencies in knowledge or assessment design
• Conduct gap analyses and recommend corrective actions
• Support periodic reviews to keep assessments current
• Maintain examination documentation and revision histories

Skills

Instructional design

Test development

Performance assessment

Technical writing

MS Word/Excel/PowerPoint

Moodle

Navy technical manuals understanding

Education

Bachelor's degree in Instructional Design or related field

Associate's degree with Navy experience

High school diploma with Navy experience

Tools

Moodle

CeTARS

LMS

Examination management systems

Job description

Test and Assessment Specialist

Purpose: Valkyrie Enterprises is seeking a qualified Test and Evaluation Specialist to support the Navy Afloat Maintenance Training Strategy (NAMTS) program. The preferred candidate will have experience developing, validating, and evaluating Navy technical training assessments, with emphasis on performance-based qualification in maintenance, repair, troubleshooting, and equipment operations.The specialist will develop and evaluate written examinations, practical assessments, and qualification evaluation instruments to measure technical knowledge and demonstrated job proficiency. The position requires the ability to distinguish between knowledge-based requirements and hands-on technical competencies, ensuring assessment methods accurately measure a candidate's ability to perform maintenance tasks safely and in accordance with established technical standards. Located in Virginia Beach, VA.

Job Description

• Analyze technical tasks, qualification requirements, and learning objectives to identify appropriate assessment methods and measurable performance criteria.
• Develop, review, and validate written examinations, technical skills assessments, practical performance evaluations, structured oral boards, and standardized evaluation checklists.
• Design criterion-referenced assessments and scoring rubrics that measure achievement of established technical knowledge and performance standards.
• Develop technically accurate multiple-choice questions, scenario-based assessments, and practical evaluation instruments aligned with approved technical references and qualification requirements.
• Establish observable performance indicators, critical task steps, safety requirements, acceptable tolerances, and completion criteria for hands‑on technical assessments.
• Evaluate existing examination items for technical accuracy, clarity, answer‑key validity, distractor effectiveness, and alignment with authoritative references.
• Apply established test development principles to support assessment validity, reliability, fairness, and consistent application across participating maintenance activities.
• Establish and maintain traceability between examination items, technical tasks, learning objectives, qualification requirements, and supporting references.
• Collaborate with Navy subject matter experts (SMEs) and instructional designers to determine appropriate assessment methods based on task complexity, criticality, safety, equipment availability, and operational requirements.
• Analyze examination results, performance assessment outcomes, item‑analysis metrics, and qualification trends to identify deficiencies in technical knowledge, demonstrated skills, or assessment design.
• Conduct performance gap and root‑cause analyses to distinguish training deficiencies from other factors, including inadequate practical experience, unclear procedures, or ineffective assessment instruments.
• Recommend corrective actions, including examination revisions, assessment improvements, curriculum modifications, and changes to qualification requirements.
• Support periodic JQR and examination reviews to ensure assessment products remain technically accurate, current, and aligned with Fleet maintenance requirements.
• Maintain examination documentation, revision histories, assessment records, and configuration‑controlled products to support quality assurance and program integrity.

Qualifications

• Must have Bachelor's degree in Instructional Design, Education, Training and Development, Engineering, a technical discipline, or a related field and at least 4 years of directly relevant technical training, test development, or performance assessment experience; or Associate's degree and at least 8 years of directly relevant Navy technical training, assessment development, curriculum development, or qualification development experience; or High school diploma/equivalent and at least 10 years of directly relevant Navy technical training, maintenance, assessment development, or qualification development experience.
• Must have demonstrated experience developing, reviewing, or validating technical examinations, practical skills assessments, qualification standards, JQRs, PQS, or comparable Navy technical training and assessment products.
• Must have working knowledge of Instructional Systems Development (ISD), task analysis, learning‑objective development, criterion‑referenced testing, and performance‑based assessment methodologies.
• Must have the ability to translate technical maintenance tasks, procedures, and qualification requirements into measurable assessment criteria and observable performance standards.
• Must have experience developing technically accurate examination questions, scoring rubrics, evaluation checklists, or other assessment instruments traceable to approved technical references.
• Must have the understanding assessment validity, reliability, question construction, and evaluation methods used to measure technical knowledge and demonstrated proficiency.
• Must have the ability to read and interpret U.S. Navy technical manuals, maintenance procedures, drawings, instructions, work‑control documents, and other authoritative technical references.
• Must have knowledge of Navy shipboard or Intermediate‑Level maintenance practices, terminology, technical documentation, and qualification processes.
• Must have the ability to evaluate assessment results, identify technical proficiency gaps, and recommend improvements to training or qualification products.
• Must have experience collaborating with technical SMEs and facilitating structured reviews to validate examination content, assessment criteria, and qualification requirements.
• Must have strong technical writing, editing, analytical, research, and communication skills with the ability to produce accurate, concise, and auditable assessment products.
• Must have demonstrated ability to manage multiple assessment development actions, examination discrepancies, review comments, deadlines, and configuration‑controlled products.
• Must have be proficient in Microsoft Word, Excel, and PowerPoint, with the ability to work with electronic learning‑management, examination‑management, or document‑control systems.
• Must have the ability to work independently and collaboratively while maintaining effective communication with Navy stakeholders, technical SMEs, and contractor personnel.

Desired Qualifications

• Five or more years of direct experience developing or evaluating U.S. Navy technical examinations, JQRs, PQS, practical assessments, or formal technical training and qualification products.
• Experience supporting NAMTS, CNRMC, an RMC, SIMA, IMF, shipyard, Fleet training activity, or Non‑Nuclear Afloat Maintenance Department.
• Experience developing or evaluating performance‑based technical training and assessments for Intermediate‑Level shipboard maintenance and repair in engineering, deck, electrical, combat systems, C4I, or related maintenance disciplines.
• Experience conducting Job Duty Task Analysis (JDTA), Training Needs Analysis (TNA), technical competency assessments, or performance gap analysis.
• Experience developing Job Performance Measures (JPMs), practical evaluation checklists, standardized oral board questions, and performance‑based qualification criteria.
• Experience conducting examination item analysis, including item difficulty, question discrimination, distractor effectiveness, and assessment performance trends.
• Experience reviewing and resolving examination discrepancies, question challenges, technical reference conflicts, and assessment validation issues.
• Familiarity with applicable NAVEDTRA training development guidance, Navy qualification standards, curriculum validation, and performance‑based assessment practices.
• Experience with Moodle, Navy learning‑management systems, CeTARS, or comparable examination‑management and training platforms.
• Experience analyzing examination results, qualification performance metrics, and assessment data to support evidence‑based program improvements.
• Familiarity with ADDIE, competency‑based training, formative and summative evaluation, and the Kirkpatrick Evaluation Model.
• Formal Navy instructor, curriculum developer, instructional designer, or Master Training Specialist qualification/certification.
• Experience collaborating with Fleet maintenance SMEs to convert operational maintenance tasks into standardized technical assessments and qualification requirements.
• Experience presenting technical assessment findings, performance trends, and recommendations to Navy leadership, government program managers, or technical authorities.

Security Requirements

Must be eligible to obtain and maintain a DoD Secret security clearance.

Travel Requirements

Occasional travel, less than 20%, to Navy installations, RMCs, ships, training sites, or other locations in support of curriculum development, validation, and stakeholder coordination. If position requires travel by domestic flight or access to secure federal facilities/military bases, candidates must be able to obtain (no later than position start date) and maintain appropriate identification credentials, such as REAL ID. (More information regarding REAL ID can be found: https://www.dhs.gov/real-id ).

Physical Requirements

Work may occur in office, industrial, shipboard, classroom, or laboratory environments. Must be able to remain in a stationary position for extended periods and move throughout worksites as needed. Must be able to access shipboard and industrial work areas as required to observe maintenance tasks, conduct validation, and coordinate with SMEs. Light physical work, including lifting objects up to 20 pounds.

Pre-Employment Requirements

Any offer of employment with Valkyrie Enterprises is contingent upon the successful completion of all applicable pre-employment requirements. Depending on the position, these may include a criminal background investigation, employment and education verification, pre-employment drug screening, driving record review, security clearance eligibility, government installation access, and any other customer or contract-specific requirements. Failure to successfully meet any required pre-employment condition may result in the withdrawal of the contingent offer of employment.

Equal Employment Opportunity

Valkyrie strictly adheres to a policy of equal employment opportunity. This policy is based on Valkyrie’s commitment to hire and retain qualified employees consistent with position requirements; and to seek, employ, promote and treat all employees and applicants for employment without regard to race, color, religious creed, national origin, ancestry, citizenship status, pregnancy, childbirth, physical disability, mental disability, age, military status or protected veteran status, marital status, registered domestic partner or civil union status, gender (including sex stereotyping and gender identity or expression), medical condition, genetic information or sexual orientation or other protected characteristics.

Benefits

• Additionally, Valkyrie Enterprises provides a variety of benefits to support your best health, wellness, and future, to include medical/dental/vision options, company paid life and disability insurances, 401k with match, education reimbursement, as well as company paid holidays and paid time off (PTO).
• Pay Range: $75,000 - $90,000 per year