Creating an assessment

Applies to the following products: 
Questionmark OnDemand
Questionmark OnPremise

An assessment consists of question blocks, which in turn consist of individual questions.

On this screen, you can:

Before adding questions and or configuring the assessment settings:

  1. Give the assessment a Title (50 characters maximum), which is required.
  2. Enter an optional Description (512 characters maximum) for the assessment. This description is not shown to participants.
  3. Enter an optional Introduction (4000 characters maximum) for the assessment. This introduction will be shown to participants when they begin the assessment.

When you have entered all desired text, it is recommended that the spelling is checked. To do so, click the Spell check button.

The following options are available when saving an assessment:

  1. Click Save to save all changes and continue editing. A message appears confirming the changes have been saved.
    • Alternatively, click the drop-down arrow next to the Save button and select Save and exit to save all changes and exit the assessment editor.

Click Exit to exit the assessment editor without saving the changes made during the current session.

Adding/editing question blocks

When you first create an assessment, a default question block is automatically created. By default, this first question block is titled Question Block - 1, the second will be titled Question Block - 2, and so on. You can edit the title of a question block (50 characters maximum) or click X to clear its existing title. Question blocks must have titles before an assessment can be saved.

To add a new question block to an assessment:

  1. In the "Question blocks" section, click Add.
    • When there are two or more question blocks in an assessment, you can use the drag-and-drop icon to reorder question blocks.
    • To delete a question block, click Delete in that question block. You will be asked for confirmation before the question block is deleted. Click Yes to delete the question block or No to cancel the deletion. Deleting a question block will delete all associated jump blocks.

In the "Description" field, you can add an introductory message that will be displayed to participants when they begin the question block. You can use the text formatting toolbar to format the introductory text as desired. Refer to Using the text formatting toolbar for more information on how to do this.

You can hide or show all the question blocks in an assessment by the clicking the arrow in the left of the question block menu, or you can hide individual question blocks by clicking the arrow to the left of that question block's title. Hiding the question blocks in an assessment collapses them for easy viewing and/or reordering, while showing them brings each question block back to regular size.

To edit a question block's settings:

  1. Click Settings in the question block for which you want to edit the settings. The question block settings window opens.
  2. In the question block settings window, you can:
    • Set the assessment template used for the question block
    • Disable feedback for the question block
    • Randomize question order within the question block
    • Exclude this question block from the assessment time limit

Adding questions to question blocks

Questions can be added individually or by topic to a question block. For information on creating a question block, please refer to the Adding/editing question blocks section. Before an assessment can be saved, all question blocks must contain at least one question. To add questions to a question block:

  1. In the question block in which you want to add questions, click Add.
  2. Click Add / edit questions. The question selection page opens.
  3. Choose whether to include all of the questions in a topic, randomly selected questions from a topic, or a single question from a topic. Please note that questions from topics shared with you cannot be added to an assessment at the present time.
    • ​If you select All questions from topic, choosing Include subtopics will include in the question block all the questions contained within the topic and all its subtopics; this option will not allow you to remove individual questions added from a topic. Choosing As single questions will include in the question block all the questions contained within the topic as individual questions; this option allows you to remove individual questions added from a topic.
    • If you select Randomly select from topic, choosing Include subtopics will include in the question block the specified number of questions randomly selected from the topic and included sub-topics. Choosing Avoid previously delivered will prevent users from seeing the same question(s) multiple times if taking the assessment over.
      Please note that in order for the Avoid previously delivered feature to work in an assessment:
      • a participant must be registered in the repository database,
      • and the assessment must be launched using a secure entry point such as perception.php.
    • If you select Single question, the "Select a topic" tree becomes the "Select a question" tree where you can select an individual question from a topic to add to the current question block.
  4. Click Add. This will add the selected question(s) to the list on the right-hand side of the page. Here, you can:
    • Drag questions to a desired order via the drag handles to the left of a question or delete them by clicking the delete button to the right of an individual question.
    • Clear all the questions in this list by clicking the Clear button and confirming your choice by clicking OK (you can click Cancel to abort the clear process).
  5. Once you have added all the questions and arranged them as desired, click OK to add them to the question block (you can click Cancel to abort the process and return to the assessment editor).

Questions that you have added will appear now appear in the question block in the assessment editor.

Adding demographics to questions

If you add single questions to a question block, you will be able to add demographics to single questions.

To add demographics to a question:

  1. Add questions to a question block using the As single questions option.
  2. Click the demographics button to the right of the question to which you want to add demographics.
  3. Select the Include the question's results as a special field in the answer database option.
  4. Select the special field in which your demographic information is stored.
  5. Select whether to store the Answer or Outcome.
  6. Click OK to save your settings or Cancel to abort the process and return to the assessment editor.

Once you have added demographics to a question, that question's demographics button will be shown as blue rather than grey in its question block.

Adding/editing jump blocks

Please be aware that questions are not numbered and the assessment progress bar is not shown when jump blocks are used in an assessment.

Jump blocks allow you to bring a participant to a different location in an assessment (such as another question block or the end of the assessment) after a participant has achieved (or has not achieved) the required score in a question block.

Before a jump block can be created, a question block must exist. For information on creating a question block, please refer to the Adding/editing question blocks section.

To create a jump block:

  1. In the question block in which you want to add the jump block, click Add.
  2. Click Jump block. The jump block window opens.
  3. Enter a Name for the jump block. A name is required before a jump block can be saved.
  4. Specify the location to which you want the assessment to jump from the Jump to drop-down list:
    • End of assessment will take a participant, after they have completed all of the questions in the current question block, to the end of the assessment if the specified condition is achieved.
    • Selecting one of the other question blocks in the assessment will take a participant, after they've completed all of the questions in the current question block, to the specified question block.
  5. Choose the Condition that will trigger the jump block:
    • Always will trigger the jump block each time a participant completes all of the questions in the current question block, taking the participant to the location specified in the Jump to list, no matter the score achieved.
    • If will trigger the jump block after a participant completes all of the questions in the current question block and achieves the specified condition. If you select If, you will need to specify the score that must be achieved in the current question block so that the jump block is triggered. You can specify this score in the field to the left of the drop-down list in which you can select points or percent. The score that you specify must be a whole number.
      • Selecting This block score means that the jump block will be triggered only if the specified condition is achieved in the current question block (i.e., if a participant score less than 50% for all the questions in the current question block).
        • From the drop-down list, you can select whether the jump block is triggered if a participant's score is less thangreater thanequals, or does not equal the specified score.
      • Selecting Assessment score means that the jump block will be triggered only if the specified condition is achieved in the current assessment (i.e., if a participant score less than 50% for all the questions in the entire assessment).
        • From the drop-down list, you can select whether the jump block is triggered if a participant's score is less thangreater thanequals, or does not equal the specified score.
      • Selecting Outcome means that the jump block will be triggered only if the participant's score does not equal or equals the specified outcome for a single question (i.e., "True" is selected for a True/False question). This option will only appear after questions have been added to a question block as a Single question or using the As single questions option from All questions from topic
        • Once this option has been selected, select either does not equal or equal.
        • Click Add. The "Question Outcomes" dialog opens, and a list of all of the questions in the question block is shown. Expand the desired question, and then select the specific outcome you want to trigger the jump block. You can only select one outcome. Once you have selected an outcome, click OK to return to jump block window.
  6. ​Click OK to save the jump block or click Cancel to discard the jump block and/or changes you have made to the jump block.

To delete a jump block:

  1. Click the X belonging to the jump block that you want to delete.
  2. Click Yes to delete the jump block or No to cancel the deletion.

Now, a jump block cannot be edited once it has been created, only deleted. 

Adding/editing reported topics

The Reported topics section will list all topics that questions have been pulled from for the assessment.

If you expand a topic, you will see any topic outcomes that exist for that topic. 

If you would like to override the topic outcomes for a topic and create a new outcome for the topic in this assessment:

  1. Select the Override topic outcomes option for the desired topic.
  2. Click Add. The Add topic outcome dialog opens.
  3. Enter the name, score range, and any desired feedback for the topic outcome.
  4. Click OK to save the new topic outcome.

Adding/editing assessment outcomes

When you first create an assessment, pass and fail assessment outcomes are automatically created. Pass / Fail is the default assessment outcome for tests, exams, and quizzes. You can edit the title of a question block (50 characters maximum) or click X to clear its existing title. Outcomes must have titles before an assessment can be saved.

The Score range for the Pass and Fail outcomes determines what counts as passing and failing scores. By default, the pass score range is 70 - 100 (percent) and the fail score range is 0 - 69 (percent), meaning that participants who score 70 percent or higher will receive a passing score while participants who score 69 percent or lower will receive a failing score. To change the range for pass / fail scores, enter different whole numerical values in the Score range fields. The score ranges must cover 0 - 100 percent and can overlap, e.g., if the pass score range is 50 - 100, the fail score range can be, as an example, 0 - 60.

To add an additional outcome:

  1. In the Outcomes section, click Add.
    • When there are two or more outcomes in an assessment, you can use the drag-and-drop icon to reorder outcomes.
  2. To delete an outcome, click Delete in that outcome. You will be asked for confirmation before the outcome is deleted. Click Yes to delete the outcome or No to cancel the deletion.

You can hide or show all the outcomes in an assessment by the clicking the arrow to the left of Outcomes. You can hide individual outcome by clicking the arrow to the left of that outcome's title. Hiding the outcomes in an assessment collapses them for easy viewing and/or reordering, while showing them brings each outcome back to regular size.

To make sure assessment outcomes are configured so that there's no conflict with topic prerequisites, please refer to the following Knowledge Base article:

For an assessment outcome, you can also specify:

Assessment feedback

Clicking Feedback opens the feedback window, which allows you to set feedback that will be shown to a participant if they achieve that outcome. The following assessment feedback options are available:

  • Total score
  • Topic score
  • Topic feedback
  • Topic outcome labels

Optionally, you can enter a message in the Feedback field that will appear if a participant achieves the outcome. You can use the text formatting toolbar to format the feedback message in any way you wish. Refer to Using the text formatting toolbar for more information on how to do this.

Email

Clicking Email opens the email window, allowing you to configure an email that is sent whenever the assessment outcome is achieved by a participant.

To send email to participants:

  1. Check the Send email option.
  2. Enter the name of the sender and the subject of the email in the From and Subject fields.
  3. Enter the email you desire to send in the email body field. You can use the text formatting toolbar to format the feedback message in any way you wish. Refer to Using the text formatting toolbar for more information on how to do this.
  4. In the Recipients section, you can select to whom the email will be sent:
    • If you expand the To option, you can check the Participant option to send the email to the participants who have achieved the assessment outcome (the participant's email must exist in the Questionmark database in order to send an email to them).
    • You can also include one or more special fields in the email by ticking the box to the left of the respective special field.
    • If you expand the CC & BCC option, you can send the email to someone other than the participant by enter their email address in the respective field.
  5. Click OK to save your settings or Cancel to discard the settings and return to the assessment editor.

Branching

Assessments can be configured to "branch" to another assessment, i.e., take a participant to the start of another assessment, when a particular assessment outcome is achieved.

To configure an assessment outcome to branch to another assessment:

  1. Click Branching for the desired assessment outcome.
  2. Select the desired setting. The options are:
    • Do not branch
    • Branch to another published assessment
      • After selecting this option, select the assessment to which to branch when the assessment outcome is achieved.
      • Only published assessments can be selected.
      • The selected assessment must exist in the same assessment folder.
    • Branch to another URL or program
      • After selecting this option, enter the program or the URL to which to branch when the assessment outcome is achieved.
  3. Click OK.

Topic prerequisites

Clicking Prerequisites opens the prerequisites window, which allows you to specify a topic prerequisite for a topic outcome. A prerequisite, which is attached to an outcome, is a topic that has to be passed in order for the outcome to be achieved. For example. If you add a prerequisite to a Pass outcome, you can set a specific topic to be a prerequisite for that outcome so that the person has to earn a specific score for that topic to have that outcome be achieved.

It is recommended that topic prerequisites are not used with topics containing Essay questions.

To add a topic prerequisite to an outcome:

  1. Click Prerequisites.
  2. Select the topic from the list of available topics to be the prerequisite for the outcome.
  3. In the Score to achieve field, set the score (as a percent) that must be earned in that topic in order for it the outcome to be achieved. This score must be a whole number from 0 - 100.
  4. Click Add.
  5. Add another prerequisite if desired.
    • To delete a prerequisite, click the X belonging to the prerequisite you want to delete.
  6. Click OK to add the prerequisite or Cancel to abort the process and return to the assessment editor.

To make sure topic prerequisites are configured so that there's no conflict with assessment outcomes, please refer to the following Knowledge Base article:

Editing settings

Editing assessment settings

  1. Click Settings in the assessment editor menu.
  2. In the assessment settings window, there are three tabs:
    • In the General tab, you can:
      • Change the template for the assessment from the drop-down list of available templates.
        • Click Load templates and select the desired template from the Template drop-down list.
      • Set a time limit (in minutes) for the assessment.
      • Choose to record assessment results to the database.
      • Set Save As You Go to off (Off), to be enabled manually by participants (Manual), to automatically save a participant's progress (Automatic) at a specified interval (in minutes), or to both Manual and automatic.
      • Set the language for the assessment using the Language drop-down list.
        • An assessment's language can be changed until an XLIFF of the assessment is downloaded. 
    • In the Security tab, you can:    
      • Enable anonymous results to be recorded.
      • Allow the assessment to be accessed from open.php.
      • Allow the assessment to run from Integration.
      • Require Questionmark Secure for the assessment to be taken.
      • Require the assessment to be monitored while being taken.
      • Set a scheduling password.
    • In the Item-level feedback tab, you can:    
      • Set feedback to appear after each question block, after the assessment has been completed, or never.
      • Display feedback for all questions, questions answered incorrectly, questions answered incorrectly and left unanswered, or only answered questions (both correctly and incorrectly).
      • Disable feedback for unscored questions.
  3. Once you have made your desired changes, click OK to save the changes and exit the assessment settings window. 

Editing question block settings

  1. Click Settings in the desired question block. Here, you can:
    • Change the template for the specific question block from the drop-down list of available templates.
      • Click Load templates and select the desired template.
    • Disable random feedback for the question block.
    • Randomize question order within this question block.
    • Exclude this question block from the assessment time limit.

Adding notes

Users who have permissions to edit an assessment can add notes to it and see notes that have been left by other users. Notes can only be added after an assessment has been saved. Notes will not be shown to participants.

To add a note to the question:

  1. In the assessment editor, click Notes.
  2. Enter a note in the Note field.
  3. Click Post to post the note. It appears below any existing notes.
    • Clicking Clear removes all current text in the Note field.
  4. Click Close to close the Notes dialog and continue editing the assessment.