Exam Developer

American Welding Society; Miami (FL); On-site; USD 70,000 - 100,000; Full time

Job summary

The American Welding Society seeks an Exam Developer to manage the full end-to-end certification lifecycle, drive data-driven performance analysis, and maintain exam content. You will collaborate with SMEs, the Assessment Group, and the QCEB Committee to uphold validity, reliability, and rigor of AWS exams.

This is a hybrid role with Miami-based travel up to 15%. Responsibilities include designing and delivering exams, item writing, standard setting, and ongoing quality improvements.

Qualifications

• Professional Experience: Minimum 1 year of experience in certification, assessment development, or instructional design.
• Foundational Knowledge: Solid understanding of certification design, item writing, validation processes, and basic psychometric principles.
• Exam Development Familiarity: Exposure to the core steps of exam development (MQC definition, JTA, technical review, and standard setting).
• Communication & Collaboration: Excellent verbal and written communication skills, with a proven ability to lead meetings/workshops and work effectively with technical SMEs and diverse stakeholders.
• Professional Attributes: Strong problem-solving abilities, exceptional editorial skills, and uncompromising attention to detail.
• Adaptability: Highly versatile, eager to learn continuously, and comfortable pivoting quickly from one project to the next.

Responsibilities

• Execute End-to-End Certification Development: Manage and support the full lifecycle of exams, including design, implementation, delivery, and ongoing maintenance.
• Facilitate Assessment Workshops: Partner with Subject Matter Experts (SMEs) on Job Task Analysis (JTA), Minimally Qualified Candidate (MQC) definition, item writing, technical review, and standard setting.
• Drive Exam Quality & Relevance: Lead rigorous development of exam test specifications, item development, review, and refinement to ensure all assessment content is clear and aligned with standards.
• Conduct Exam Performance Analysis: Perform routine exam and item analysis to assess quality, monitoring item-level statistics, content relevance, exam security, and quality control.
• Implement Adjustments: Make necessary quality improvement updates and editorial/proofing adjustments throughout the product lifecycle.
• Build Stakeholder Relationships: Cultivate strong working relationships with volunteer exam committee, SMEs, assessment vendors, and peer exam developers. Recruit, train, and engage volunteer subject matter experts in item writing work.
• Manage Program Logistics: Manage development schedules, milestones, and SME participation or review statuses.
• Maintain Artifacts: Manage documentation and process artifacts, ensuring strict version control for items and certification programs.
• Support Growth: Contribute to the overall growth of the AWS certification program by consistently seeking efficient and effective process improvements.
• Perform Other Duties as Assigned.

Skills

Certification design

Item writing

Psychometrics

Stakeholder mgmt

Editing

Adaptability

Facilitation

Tools

Test delivery systems

Job description

If you are unable to complete this application due to a disability, contact this employer to ask for an accommodation or an alternative application process.

Exam Developer

Full Time Professional

Salary Range: $70,000.00 To $100,000.00 Annually

Role Overview:

• Job Title: Exam Developer
• Location: Hybrid for candidates based in Miami, FL; Fully remote opportunity available for candidates located outside of Florida within the U.S.
• Travel: Up to 15% (Primarily to our headquarters in Miami, FL)
• Compensation: $70,000 – $100,000 USD annually (Final salary determined by experience, skills, and location)

Position Summary:

The primary responsibility of this role is to manage the full end-to-end exam lifecycle, driving data-driven performance analysis, continuous quality improvement, and full-scale test design. In this role, you are responsible for independently managing AWS certification exam item pools, maintaining existing exam content, and developing new certification offerings. A key focus is collaborating with and leading volunteer subject matter experts (SMEs), the Assessment Group, and the Qualification and Certification Exam Bank (QCEB) Committee to uphold the validity, reliability, and rigor of all exams.

Core Responsibilities:

• Execute End-to-End Certification Development: Manage and support the full lifecycle of exams, including design, implementation, delivery, and ongoing maintenance.
• Facilitate Assessment Workshops: Partner with Subject Matter Experts (SMEs) on Job Task Analysis (JTA), Minimally Qualified Candidate (MQC) definition, item writing, technical review, and standard setting.
• Drive Exam Quality & Relevance: Lead rigorous development of exam test specifications, item development, review, and refinement to ensure all assessment content is clear and aligned with standards.
• Conduct Exam Performance Analysis: Perform routine exam and item analysis to assess quality, monitoring item-level statistics, content relevance, exam security, and quality control.
• Implement Adjustments: Make necessary quality improvement updates and editorial/proofing adjustments throughout the product lifecycle.
• Build Stakeholder Relationships: Cultivate strong working relationships with volunteer exam committee, SMEs, assessment vendors, and peer exam developers. Recruit, train, and engage volunteer subject matter experts in item writing work.
• Manage Program Logistics: Manage development schedules, milestones, and SME participation or review statuses.
• Maintain Artifacts: Manage documentation and process artifacts, ensuring strict version control for items and certification programs.
• Support Growth: Contribute to the overall growth of the AWS certification program by consistently seeking efficient and effective process improvements.
• Perform Other Duties as Assigned.

Required Qualifications:

• Professional Experience: Minimum 1 year of experience in certification, assessment development, or instructional design.
• Foundational Knowledge: Solid understanding of certification design, item writing, validation processes, and basic psychometric principles.
• Exam Development Familiarity: Exposure to the core steps of exam development (MQC definition, JTA, technical review, and standard setting).
• Communication & Collaboration: Excellent verbal and written communication skills, with a proven ability to lead meetings/workshops and work effectively with technical SMEs and diverse stakeholders.
• Professional Attributes: Strong problem-solving abilities, exceptional editorial skills, and uncompromising attention to detail.
• Adaptability: Highly versatile, eager to learn continuously, and comfortable pivoting quickly from one project to the next.

Preferred Experience:

• Hands-on Facilitation: Direct experience actively facilitating SME workshops for JTAs, MQC definitions, or standard settings.
• Data & Psychometrics: Experience using psychometric tools, data collection, or statistical analysis to evaluate project outcomes and exam validity.
• Technical Familiarity: Proficiency in industry standards for test development and direct experience working with test delivery systems
• Program Impact: Experience in making significant, measurable contributions to an established certification or assessment program.

Work Arrangement:

• This position is primarily remote with no regular in-office requirement. However, estimated travel of up to 15% to our headquarters in Miami, FL, or other locations will be required based on business needs (such as major workshops or events).