Specialist, Certification Exam Developer

Paycom; Houston (TX); Hybrid; USD 85,000 - 120,000; Full time

Job summary

Paycom is seeking a Certification Exam Developer Specialist to lead development, updating, and performance monitoring of AMPP certification examinations. You will coordinate in-person and virtual SME meetings to ensure job-role alignment and defensible exam content.

The role emphasizes item writing, bank maintenance, and preparation of exams for production, with strong focus on exam security and quality control across multiple test forms. Travel may be required.

Qualifications

• Masters degree in Psychometrics or related field.
• 1-3 years professional test development experience.
• 1-3 years Job Task Analysis experience preferred.
• Strong knowledge of all Microsoft Office products.
• Experience in testing and assessment industry preferred; item and test development experience required.

Responsibilities

• Competency Model Facilitation: organize and assist in competency modeling workshops and maintain the model as the basis for exams.
• Blueprint Utilization: determine number of exam items after objectives are weighted.
• Exam Quality Process: oversee item development workshops and ensure content validity and proper formatting.
• Standard Setting Methodology: assist in facilitation of standard setting workshops and manage SME coordination.
• Exam Performance: analyze data, monitor validity and reliability, revise items as needed.
• Project Management: manage certification exam development, schedules, and deliverables.
• Support team members: respond to Certification Exam mailbox and coordinate workshops.
• Translations: ensure secure translations to reduce bias and improve quality.
• Board Liaison: work with Boards to recruit SMEs and drive initiatives.

Skills

Project management

Statistical item analysis

Communication

MS Office

Written & verbal communication

Global thinking

Attention to detail

Multitasking

Deadline management

Security & dependability

Vendor negotiation

Budgeting & tracking

Facilitation skills

Editorial skills

Global association experience

Education

Masters degree in Psychometrics, Educational Psychology, Psychology or equivalent

Job description

Job Details

Job Location: Full-Time Remote - Houston, TX

Salary Range: Undisclosed

Job Title: Certification Exam Developer Specialist

Division: AMPP

Department(s): Certification

Reports To: Sr. Director, Credentialing & Sr. Director Operations

FLSA Status: Exempt REVISION

Summary

The Certification Exam Developer position is responsible for the development, maintenance, and performance monitoring of AMPP certification examinations including exam updates. In addition, the role will be required to perform facilitation of in-person and virtual meetings with subject matter experts to ensure that the end-result is based on actual job role requirements. Examples of tasks include exam development, item writing, item referencing, item bank maintenance, preparation of materials for item review and preparation of exam forms for Test Production. This individual will also ensure that all exams meet or exceed legal defensibility process and policy requirements as stipulated by the AMPP Certification program office. Detailed focus on items quality and exam security is central to this role.

Essential Duties and Responsibilities

ESSENTIAL DUTIES AND RESPONSIBILITIES include the following. Other duties may be assigned.

• Competency Model Facilitation: Organize and assist in the facilitation of face-to-face competency modeling workshops. Keep current the competency model that delineates performance and knowledge/skills used as the basis for development of exams. Frequency and criticality of tasks performed is collected so that the objectives are prioritized and weighted. Breakdown tasks that make up a specific role, based on input by subject matter experts (SMEs) who are currently performing that function. All high-stakes certification exams are based on the competency model through the blueprint process
• Blueprint Utilization: After the objectives are prioritized and weighted, the number of exam items is determined.
• Exam Quality Process: Organize and conduct face-to-face or remote item development workshops. Manage vendor-produced exam development work, activities, and deliverables. Train subject matter experts and vendors on writing technically sound exam items. Create items (questions) to ensure test takers meet a defined level of performance and knowledge/skills. Oversee item review for technical content, validity, items standards, and grammatical. Prepare the item bank for final exam publishing and ensure there is a complete Exam Preparation Guide (EPG).
• Standard Setting Methodology: Organize and assist in the facilitation of standard setting workshops: Manage SMEs recruiting, scheduling, set up and administration of exam and ratings in Exam Developer.
• Exam Performance: Given data analysis monitor the continued validity and reliability of exams during the period in which they are active. Track the performance of exams and items; when necessary, revise and/or remove items. Verify that the exams meet the target audience and program needs.
• Project Management: Manage development of certification exams. Organize and facilitate project meetings, develop status reports, manage schedules and deadlines, initiate and maintain strategic relationships and partnerships. Assure quality of deliverables
• Support team members: Collaborate with Exam development coordinator when responding to Certification Exam mailbox, posting volunteer opportunities, and workshop logistics.
• Translations: Implement appropriate methods for translating exams to maintain test security, reduce bias and improve quality.
• Board Liaison: work with the Boards to recruit SMEs, program changes and initiatives

Qualifications

QUALIFICATION REQUIREMENTS To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.

Travel Requirements

Some travel is expected for this role and passport is required.

Education and/or Experience

• Masters degree in Psychometrics, Educational Psychology, Psychology or equivalent experience
• 1-3 years professional test development experience
• 1-3 years’ Job Task Analysis experience, highly preferred
• Strong knowledge of all Microsoft Office products
• A background in the testing and assessment industry is strongly preferred. Item and Test Development experience is required.

Language Skills

Ability to read, analyze, and interpret general business periodicals, professional journals, technical procedures, or governmental regulations. Ability to write reports, business correspondence, and procedure manuals. Ability to effectively present information and respond to questions from groups of managers, members, employees of the organization, customers, and the general public.

Mathematical Skills

Ability to calculate figures and perform item analysis using test development psychometrics. Ability to apply concepts such as fractions, percentages, ratios, and proportions to practical situations. Ability to develop and interpret business graphs.

Reasoning Ability

Ability to solve practical problems and deal with a variety of concrete variables in situations where only limited standardization exists. Ability to interpret a variety of instructions furnished in written, verbal, diagram, or schedule form.

Other Skills and Abilities

Must possess a specific level of communication skill both verbal and written. Ability to work in a multi-task environment taking into account various external factors and to communicate efficiently and accurately to both managers/supervisors and stakeholders.

• Project management
• Statistical Item analysis (not required / nice to have - will be required to learn)
• Communication and presentation
• Demonstrated proficiency in Microsoft Office products and database software
• Excellent written and verbal communication and interpersonal skills
• Ability to think globally
• Excellent attention to detail and accuracy
• Ability to prioritize and handle multiple projects simultaneously
• Ability to work under pressure and meet exacting deadlines
• Proven dependability and high regard for security
• Negotiating with vendors
• Working within the budget and tracking expenses
• Strong facilitation skills
• Strong editorial skills
• Global business and member driven association experience a plus

Physical Demands

The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. While performing the duties of this job, the employee is regularly required to use hands to finger, handle, or feel objects, tools, or controls; reach with hands and arms; and talk or hear. The employee frequently is required to sit. The employee is occasionally required to stand; walk; climb or balance; and stoop, kneel, crouch, or crawl. The employee must occasionally lift and/or move up to 25 pounds. Specific vision abilities required by this job include close vision, color vision, and the ability to adjust focus.

Work Environment

The work environment characteristics described here are representative of those an employee encounters while performing the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. The noise level in the work environment is usually moderate.