Checklist scoring is assigned at the answer level. You can add point values to individual answer choices to add another layer of analysis to your checklist designs. When you assign point values to answer choices, Vault provides a total score when the user completes the checklist, keeps track of the running score as users answer questions, and provides a total score when they complete the checklist. Vault provides the checklist score as a percentage and a sum score. It also stores the score for each question and section for ease of reporting.
You can assign weighting at the section level and, for aggregate checklists, at the sub-checklist level. You can assign weighted percentages to different sections of your checklist. Vault runs validation to ensure weighted section totals equal 100% before you can make the checklists available for users. For aggregate checklists, you can assign weighted percentages to different sub checklists, and Vault runs validation to ensure the weighted sub-checklist totals equal 100%. You can still assign a weighted percentage to a sub-checklist even if the sub-checklist itself is not weighted but has at least one multiple choice question.
Note: Introduction sections cannot include a weight value.
How to Configure Question Scoring
You can configure question answer scores for each answer option of a multiple choice question. Enter any positive or negative numerical value or zero (0) as the Score. Scoring is not available for text, date, or number questions.
Keep in mind the following when designing scored checklists:
- A scored question cannot have a multiple-choice answer with no score value.
- A weighted Section Design and Sub-Checklist Design must have at least one scored question.
- A Section Design and Sub-Checklist Design with 0% weight cannot have a scored question.
How to Configure Weighting for Sections and Sub Checklists
Set weighted percentages for question sections from the Sections area of the Checklist Design object details page. Remember that all sections must total 100% before you can make the checklist active.
For Checklist Designs with Aggregate Checklists set to Yes, set weighted percentages for sub checklists from the Sub-Checklist Designs area of the Checklist Design object details page. Remember that all sub checklists must total 100% before you can make the checklist active. Sub checklists with a weight must have at least one multiple choice question.
Understanding Score Calculations
The Question score, Section score, and Checklist score are calculated when the Checklist moves to the Completed lifecycle state, and persisted in the Target Object Section and Target Object Checklist runtime objects.
For multiple choice questions that are the checkbox (multi-select) checklist answer type, the score for the question is the sum of the scores for each selected option.
The Target Object Section and Target Object Checklist contains two fields that calculate the total score of the Checklist and Section scores:
-
Score %: A percentage value of the total score. The percentage is calculated using the sum of the scores for each selected multiple choice answer and the sum of the maximum possible scores for each question. The Score % can never have a negative value. If the answer score has negative values assigned and the sum of the answer scores for the checklist is less than zero (0), Vault sets the Score % to 0%. Vault also checks that the maximum score sum for a section is greater than or equal to zero (0).
-
Sum Score: The total score represented as a whole number or a decimal number for weighted scores. The sum score is calculated using only the total sum of the scores for each selected multiple choice answer. Vault displays a negative value (100, -50, -20) in the Sum Score field if the sum of the answer scores for each section is less than zero.
Keep in mind the following considerations regarding checklist scores:
- If a user skips a scored question, such as an optional question, the question will have no score value.
- Unscored questions never contribute to the checklist score.
- A dependent question or section not seen by a user, such as when a respondent selects an answer that does not prompt a dependent question, will not contribute to the checklist’s score.
For more information on the score calculations, see below.
Scoring Calculations
Vault uses certain formulas to calculate the sum score and score percentage. The formulas used are defined as follows:
(answer sum)is the sum of the scores for each selected multiple choice answer.(maximum score sum)is the sum of the maximum possible scores for each question.(section n weight %)is the specified weight percentage for a section.(answer sum section n)is the sum of the scores for each answer in that section.(maximum section n score sum)is the sum of the maximum possible scores for each answer in that section.
Non-Weighted and Weighted Score % Calculation
If the Checklist Design is not weighted, the Section and Checklist score percentage are each calculated as follows:
((answer sum) / (maximum score sum))
If the Checklist Design is weighted, the Section score percentage is calculated as follows:
((answer sum) / (maximum score sum))
And the Checklist score percentage is calculated as follows:
Non-Weighted and Weighted Sum Score Calculation
If the Checklist Design is not weighted, the sum score of the Section and Checklist are each calculated as follows:
(answer sum)
If the Checklist Design is weighted, the sum score of the Checklist is calculated as follows:
Aggregate Checklist Scoring
For aggregate checklists, percentage and sum scores are calculated in the same way as non-aggregate checklists, with sub-checklist scores calculated in the same way as Sections; however, if the sub-checklist is weighted then weighted is respected in the sub-checklist calculation.
-
Score percentage calculation for a non-weighted aggregate Checklist Design (where “C” represents sub-checklists):
-
Score percentage calculation for a weighted aggregate Checklist Design (where “C” represents sub-checklists):
-
Sum score calculation for a non-weighted aggregate Checklist Design:
(answer sum) -
Sum score calculation for a weighted aggregate Checklist Design:
Scoring Example
The checklist below contains two sections with two multiple choice questions each. The bolded answers represent the selected multiple choice answers. The questions in this example are not checkbox multiple choice questions.
Section 1
- Question 1
- Answer 1 (score 10)
- Answer 2 (score 10)
- Answer 3 (score -100)
- Question 2
- Answer 1 (score 10)
- Answer 2 (score 10)
- Answer 3 (score -100)
Section 2
- Question 1
- Answer 1 (score 10)
- Answer 2 (score 10)
- Answer 3 (score -100)
- Question 2
- Answer 1 (score 10)
- Answer 2 (score 10)
- Answer 3 (-100)
The total non-weighted score is calculated as follows:
- Sum Score = (10 - 100) + (10 + 10) = -70
- Score % = [(10 - 100) + (10 + 10)] / (20 + 20) = (-90 + 20)/40 = -70/40 = 0%
Vault sets the Score % field to 0% if the sum of the answer scores in the checklist equals less than zero (0).
If Section 1 contains a 10% weighted score and Section 2 contains a 90% weighted score, the total weighted section score is calculated as follows:
- Sum Score = [0.10 x (10 - 100)] + [0.90 x (10 + 10)] = 9
- Score % = {[0.10 x (10 - 100)] + [0.90 x (10 + 10)]} / [(0.10 x 20) + (0.90 x 20)] = 9 / 20 = 45%