Assessing the SMARTness of nutrition commitments

Authors

Giacomo Zanello (lead author), Dimitra Karageorgou, Renata Micha

Share section

Introduction

This document describes the development of the SMARTness score, a novel scoring system for assessing the SMARTness of the commitments registered through the Nutrition Accountability Framework (NAF). SMARTness score evaluates how SMART (i.e., Specific, Measurable, Achievable, Relevant and Time-bound) the registered commitment goals are, by assessing whether stakeholders have provided and clearly described all required information to meet the five dimensions of SMARTness, as these have been defined in the online resource The SMARTness of nutrition commitments.

The SMARTness score reveals areas where some commitments can be improved and spotlights exemplary cases of SMART commitment goals. The tool supports stakeholders to improve the formulation of their commitment goals. In its current iteration, the SMARTness score does not assess how impactful a commitment goal is (e.g., if it has a global- or national-level focus; if it targets the overall population or worksite employees).

Key terms used on this page

Commitment goal: A commitment goal is what stakeholders are committing to achieve and is used to track and assess progress made towards the commitment. Commitment goals must be measurable and should be nutrition-related, including nutrition-specific and nutrition-sensitive goals.

Dimensions: The various SMART dimensions – Specific, Measurable, Achievable, Relevant and Time-bound.

Ingredients: Ingredients are individual characteristics that describe each of the SMART dimensions and are defined to facilitate the formulation and assessment of SMART commitments.

Nutrition commitment: The intent and pledge to address poor diets and malnutrition in all its forms through SMART nutrition actions.

Share section

Design and development of the SMARTness score

Applying the SMART concept to nutrition commitments

The application of SMART criteria in the formulation of nutrition commitments ensures that the type of the commitments (such as enabling, policy or impact), their goals and their expected outcomes are clear. Making commitments easier to classify and monitor also makes it possible to measure impact and demonstrate success. To facilitate the formulation and assessment of SMART commitment goals, we have identified a set of commitment ingredients (defined as the individual characteristics that describe each of the SMART dimensions); these have been mapped to each of the five SMART dimensions, as detailed in the Appendix. These ingredients form the basis of the NAF Platform's Commitment Registration Form and are their assessments are at the base of the SMARTness score.

Selection of ingredients

All the ingredients specified in the online resource The SMARTness of nutrition commitments are assessed in the SMARTness score, except for:

  • Optional questions in the registration form. These questions were set as optional following the pilot testing of the registration form in July 2021. The optional nature of these questions makes it impossible to distinguish cases in which stakeholders did not wish to provide the information from cases in which stakeholders did not know the answer or when the information was not applicable.
  • Such questions include:
    • Action plan for achieving the measurable goal (GX.2). By survey design, in addition to being optional, this question does not apply to financial commitment goals (GX.1).
    • Target population – population coverage (GX.6c). The question depends on the response provided in a previous question and does not apply to all commitment goals.
    • Target population – differential approach (GX.6d). The question depends on the response provided in a previous question and does not apply to all commitment goals.
    • Interim targets (GX.8e).
  • Nutrition for Growth (N4G) eligibility criteria (Q8). By survey design, this question only applies to commitments linked to the 2021 Tokyo N4G Summit (based on question Q7). Q8 was compulsory and stakeholders had to self-report a commitment’s alignment with national priorities and the N4G Principles of Engagement to be able to proceed with the registration process.

The ingredients of each of the five SMART dimensions are assessed for ‘missingness’ (i.e., we have no true missing information since all fields are compulsory; by missingness, we refer to responses that do not specify the answer, such as ‘TBD’, to be defined; and ‘TBC’, to be confirmed) and coherence (e.g., agreement of content across different fields).

Scoring principles and scoring algorithm

The development of the SMARTness score follows two principles:

  • Use of a composite score: To measure each of the SMART dimensions – Specific, Measurable, Achievable, Relevant and Time-bound – ingredients are combined into a single score. The overall SMARTness score is the average of the five dimensions. Several nutrition initiatives use composite scores, such as the Global Hunger Index and the Nutrition Barometer,[1] as well as other non-nutrition specific initiatives (e.g., the Multidimensional Poverty Index[2]). Composite scores are more easily interpretable and facilitate comparisons as they reduce the size of a set of ingredients without dropping the underlying information.[3]
  • Apply equal weights to dimensions, and ingredients within dimensions: SMARTness is evaluated by applying equal weights to each dimension. In addition, the ingredients within each dimension are equally weighted. Equal weights across dimensions and ingredients (within dimensions) produces normalised scores.

The SMARTness score is computed in two steps:

  • Step 1: Review of each commitment goal and ingredient-level scoring as described in the Appendix.
  • Step 2: Calculation of the SMARTness score at commitment-goal level (for each of the SMARTness dimensions and for the overall commitment goal) using the ingredient scores assigned in Step 1.

Step 1: Reviewing and scoring ingredients

Scoring is performed independently by two researchers. Potential discrepancies are resolved through discussion among the researchers and a third independent reviewer (Independent Expert Group – IEG –lead authors). The Appendix describes all ingredients that are reviewed for each dimension and defines the scores each ingredient might receive, along with how these are determined (e.g., examples and rationales). While the scores are ingredient-specific (as described in the Appendix) the principle of the scoring system is in common and as follows:

  • 1 point for responses that are complete, clear and in alignment with the content of related fields throughout the form. No clarifications are required from the commitment-maker.
  • 0.5 points for all responses requiring clarifications. We assign this score to any response when we cannot be certain of its content either in terms of clarity (e.g., acronyms that are not easily identifiable, terms with unclear definitions) or coherence (i.e., not in alignment with the content of other fields) and that requires clarification from the commitment-maker. There is no further refinement in the score assigned to responses requiring clarifications (e.g., higher than 0.5 if a minor clarification is required and lower than 0.5 for a major one) since such a granular approach is not always possible and would introduce greater potential for reviewer disagreement.
  • 0 points for responses that do not specify the content. Given that only compulsory ingredients are included in the SMARTness score, there are no true missing responses. However, commitment-makers may have inputted free-text responses such as ‘NA’ (not available), ‘TBD’ and ‘TBC’.

Step 2: Computing the SMARTness score at the commitment-goal level

To compute the score of each SMART dimension and the overall SMARTness of a commitment goal, we first defined the weighting factors for each dimension and ingredient. Specifically:

  1. All five dimensions are considered essential for a commitment goal to be SMART, hence each of them (S, M, A, R and T) carries an equal weight of 0.2.
  2. Within each SMART dimension, ingredients are equally weighted. The individual weights are determined by the number of ingredients in each dimension.

The commitment-goal SMARTness score for the goal n (SNASn) is computed as:

SNASn = (Sn × w) + (Mn × w) + (An × w) + (Rn × w) + (Tn × w)

Where the five components Sn, Mn, An, Rn and Tn capture the scores of the individual dimensions and w is equal to one-fifth (first-level weight).

The scores of the individual dimensions for the goal n, are computed as follows:

Sn = S1 × wS + S2 × wS + S3 × wS + S4 × wS + S5 × wS + S6 × wS + S7 × wS

Mn = M1 × wM + M2 × wM + M3 × wM + M4 × wM + M5 × wM

An = A1 × wA + A2 × wA + A3 × wA + A4 × wA

Rn = R1 × wR + R2 × wR

Tn = T1 × wT + T2 × wT

Where Specific (Sn) includes seven ingredients (S1 – S7), each weighting one-seventh (wS); Measurable (Mn) five ingredients (M1 – M5), each weighting one-fifth (wM); Achievable (An) four ingredients (A1 – A4), each weighting one-quarter (wA); Relevant (Rn) two ingredients (R1 – R2), each weighting one-half (wR); and Time-bound (Tn) two ingredients (T1 – T2), each weighting one-half (wT). The SMARTness score at the commitment-goal level ranges from 0 to 5, while each dimension score ranges between 0 and 1.

Share section

Pilot testing the SMARTness score

We piloted the SMARTness score on a random sample of 85 commitments (147 commitment goals), which corresponded to around 20% of the commitments received through the NAF up to mid-February 2022. The pilot was performed in two rounds, in which two researchers reviewed the commitments independently and in duplicate. After reviewing 45 commitments (84 commitment goals), the two reviewers’ results were compared and goals were ranked based on the number of differences between the reviewers. Differences were identified between SMART dimensions and between ingredients. This step helped to identify problematic cases, the limitations of the preliminary scoring system, and any post-processing errors. The following step was re-assessing the selected subset, focusing on goals with the highest number of differences. The re-evaluation was performed by the same reviewers working together. This step led to a strengthening of the scoring system and the establishment of clearer scoring criteria. The step was also used to further quality assure the post-processing methods. The second round of scoring (40 commitments and 63 commitment goals) aimed to evaluate the improvement in the scoring system and show any additional adjustments needed in the scoring protocol.

Robustness checks cover three dimensions:

  • Reliability (consistency of the score across time and different researchers): This is achieved by the combination of comprehensive and strict scoring guidelines (Appendix) together with a scoring protocol developed with and by the reviewers and tested over multiple pilots.
  • Validity (a measure of the accuracy of the score): As there is not a gold standard (or benchmark), validity is assessed internally. The current methodological approach has been reviewed by the IEG lead authors and no major inconsistencies or flaws have been identified (face validity). The content validity is embedded in the methodology by constructing the score with all the compulsory elements in the survey. We aim to assess the construct validity by testing group differences and estimating correlation matrices.
  • Sensitivity (how the SMARTness score is susceptible to changes to the data): This is tested by randomly introducing small changes to the original dataset and statistically comparing the different SMARTness scores generated by the two datasets.

Testing the scoring methodology

At the end of the first phase of the pilot (a review of 10% of commitments), the scoring disagreement between the two reviewers ranged from 5.8% to 24.3% across the ingredients, with a mean value of 12.4%. Following the score, the two researchers systematically reviewed and recorded differences in their scores. The issues identified were discussed with the lead IEG lead authors to reach an agreement, and the protocol presented in the Appendix was updated to address cases that caused confusion (e.g., specifying examples). Further refining the score assigned to unclear responses (i.e., those requiring clarification from commitment-makers) was also discussed. We considered identifying ingredients for which scores could be further refined (e.g., four- or five-value scoring instead of the current three-value approach), so that responses that required only minor clarifications would receive a higher score than those requiring major clarifications. However, we refrained from doing this given the complexity of defining minor versus major clarifications and the subsequent high risk of introducing wider disagreement between the reviewers. As any minor or major unclear point will require clarification from commitment-makers, the same score (i.e., 0.5) will be assigned to any ingredient whose response is not complete and clear. Finally, the two reviewers’ assigned scores were amended based on the updated protocol.

At the end of the second phase (a review of an additional 10% of commitments), the scoring disagreement between the two reviewers ranged from 1.7% to 45.6% across ingredients, with a mean value of 5.3%. Inconsistencies across one ingredient (R2) stood out and these were due to changes to the criteria in the protocol following the first phase. Following further discussions between the reviewers and the lead IEG authors, we updated the protocol to clarify the interpretation of the scoring for R2, reverting to the original scoring method for this ingredient. The scoring disagreement for the remaining dimensions (S, M, A and T) ranged from 1.7% to 7.8%, and the protocol was refined further to include additional examples for these dimensions from the second phase. It was agreed that one reviewer would score the remaining commitment goals, with the second reviewer consulted for any commitments that had potential for ambiguity.

SMARTness score pilot results

Tables 1a and 1b report the SMARTness score summary statistics, at commitment and goal levels. The S, M, R and T dimensions have the highest scores. The Achievable dimension has the lowest score of 0.66 and 0.65, at commitment and goal level respectively. Table 2 breaks down each dimension by ingredients and reports the percentage of goals that do not achieve the full score by each ingredient. Goals that will require clarifications from the stakeholders range between 3–12% within the Specific dimension, 3–27% for Measurable, 25–68% for Achievable, 14–25% for Relevant, and 0–3% for Time-bound.

Table 1a: SMARTness score summary statistics, by commitment

Dimensions Mean SD Min Max
S 0.96 0.02 0.79 1
M 0.87 0.08 0.10 1
A 0.66 0.09 0.00 1
R 0.89 0.02 0.50 1
T 0.99 0.01 0.75 1
SMART 4.37 0.10 3.11 5

Commitments (N=85)

Table 1b: SMARTness score summary statistics, by goal

Dimensions Mean SD Min Max
S 0.96 0.06 0.79 1
M 0.87 0.19 0.10 1
A 0.65 0.27 0.00 1
R 0.90 0.14 0.50 1
T 0.99 0.05 0.75 1
SMART 4.37 0.42 3.11 1

Goals (N=147)

Table 2: Summary statistics of scoring for each ingredient, by goal

Ingredients Mean SD Min Max % of < 1
S1 0.99 0.08 0.5 1 2.7
S2 0.96 0.13 0.5 1 7.5
S3 0.95 0.15 0.5 1 9.5
S4 0.94 0.16 0.5 1 11.6
S5 0.97 0.12 0.5 1 6.1
S6 0.95 0.15 0.5 1 9.5
S7 0.97 0.11 0.5 1 5.4
M1 0.90 0.22 0 1 17
M2 0.85 0.30 0 1 21.8
M3 0.96 0.14 0.5 1 8.1
M4 0.82 0.31 0 1 2.8
M5 0.80 0.35 0 1 27.2
A1 0.37 0.46 0 1 68
A2 0.72 0.37 0 1 40.8
A3 0.74 0.35 0 1 40.1
A4 0.78 0.40 0 1 24.5
R1 0.93 0.17 0.5 1 14.3
R2 0.87 0.24 0 1 24.5
T1 1.00 0.00 1 1 0
T2 0.98 0.09 0.5 1 3.4

Note: Refer to the Appendix for the description of the ingredients.

Share section

Appendix

Share section

Assessing the 'Specific' aspect of commitment goals

Ingredient S1: Organisation name

First question in Sign up Form

Compulsory – text

1=Completed

  • A valid organisation name is provided and in agreement with what might be mentioned in other fields (e.g., full commitment).

0.5=Unclear/Not in alignment with other fields

  • A valid organisation name is provided but not in agreement with what is mentioned in other fields.
  • More than one organisation is listed.
  • The organisation name is not clear (e.g., acronyms not commonly used/easily identified such as WHO)

0=Unspecified

Weight of ingredients: 1/7

Ingredient S2: Organisation type

Second question in Sign up Form

Compulsory – multiple choice

1=Completed

  • The response selected is the best match for the organisation and, when applicable, agrees with the content of the commitments (e.g., donor governments are committing to assisting other countries).
  • The response selected is ‘Other’. The GNR will work on revisiting and redefining and/or expanding the stakeholder-type definitions.

0.5=Unclear/Not in alignment with other fields

  • The response selected is not the best match for the organisation (e.g., a low-income country committing to national actions but identified as donor government)

0=Unspecified

Weight of ingredients: 1/7

Ingredient S3: Additional organisation(s) (only for joint commitments)

Questions 5; 5a; 5b

Compulsory – yes/no; Compulsory – numerical; Compulsory – text

1=Completed

  • Answer is ‘No’ in Q5 (not a joint commitment) and this is in agreement with what might be mentioned in other fields (e.g., full commitment).
  • Answer is ‘Yes’ in Q5 (joint commitment); additional organisations with valid names are listed, even if multiple within a field; and in agreement with what might be mentioned in other fields (e.g., full commitment).

0.5=Unclear/Not in alignment with other fields

  • Answer is ‘No’ in Q5 (not a joint commitment) but other organisations are mentioned in other fields (e.g., full commitment).
  • Answer is ‘Yes’ in Q5 (joint commitment) and valid organisation names are provided but not in agreement with what is mentioned in other fields.
  • Answer is ‘Yes’ in Q5 (joint commitment) and the additional organisation names are not clear (e.g., acronyms not commonly used and easily identified such as WHO, organisations are not distinct from one another)

0=Unspecified

Weight of ingredients: 1/7

Ingredient S4: Measurable goals

Question 13

Compulsory – text

1=Completed

  • The measurable goal is clearly described; it is one goal (not having included multiple goals under one) in alignment with the indicator and in agreement with what might be mentioned in other fields (e.g., full commitment).

0.5=Unclear/Not in alignment with other fields

  • The goal is unclear (e.g., in terms of purpose).
  • The goal includes multiple goals under its description.
  • The goal is not in alignment with the indicator and/or what is mentioned in other fields (e.g., full commitment).
  • The goal description is clear and in agreement with other fields but requires some clarifications (e.g., definition of chronic malnutrition, spelling out acronyms).
  • The goal description is not nutrition-related (e.g., an increase physical activity) or the information provided is not sufficient or clear enough to decide whether they are nutrition-related (e.g., an increase the number of beneficiaries receiving cash transfers to address poverty).

0=Unspecified

Weight of ingredients: 1/7

Ingredient S5: Type of goal

GX.1

Compulsory – multiple choice

1=Completed

  • All required information (mainly goal description and indicator, supported by full commitment if needed) is available and clear for the measurable goal to be reclassified using the NAF Nutrition Action Classification System.

0.5=Unclear/Not in alignment with other fields

  • The required information is available but not clear for the measurable goal to be reclassified using the NAF Nutrition Action Classification System (e.g., goal description not in alignment with indicator, goal description not in alignment with full commitment, multiple goals of different nutrition action category/sub-category under one goal).

0=Unspecified

Weight of ingredients: 1/7

Ingredient S6: Geographical coverage

GX.4; GX.5

Compulsory – multiple choice; Compulsory – text

1=Completed

  • The response selected in GX.4 is in agreement with what might be mentioned in other fields (e.g., goal description, full commitment). A valid response is provided in GX.5 – a valid response in this case includes ‘None’ and ‘NA’ as this question might not apply to all commitments (i.e., not having any additional specifications to report).

0.5=Unclear/Not in alignment with other fields

  • The response selected in GX.4 and/or GX.5 is not in agreement with what is mentioned in other fields.
  • The response selected in GX.4 is not in agreement with what is described in GX.5 (e.g., subnational in GX.4 and multiple countries are mentioned in GX.5).
  • The response selected in GX.4 is in agreement with what is mentioned in other fields, but the response provided in GX.5 is unclear (e.g., acronyms not commonly used and easily identified such as WHO) or TBD.
  • The response selected in GX.4 is ‘Multi-country’, but the countries are not specified in GX.5.

0=Unspecified

Weight of ingredients: 1/7

Ingredient S7: Target population

GX.6; GX.6a; GX.6b

Compulsory – multiple choice; Compulsory – multiple choice; Compulsory – text

1=Completed

  • Answer is ‘Overall population’ in GX.6 and this is in agreement with what might be mentioned in other fields (e.g., full commitment).
  • Answer is ‘Specific population group(s)’ in GX.6 and the responses selected for all three questions are in agreement with what might be mentioned in other fields (e.g., goal description, full commitment) and the age group in GX.6b.
  • Answer is ‘No population group is targeted’ in GX.6 and this is in agreement with what might be mentioned in other fields (e.g., full commitment).

0.5=Unclear/Not in alignment with other fields

  • The response selected in GX.6 is not in agreement with what is mentioned in other fields.
  • Answer is ‘Specific population group(s)’ and the responses provided in the two subsequent questions are not in alignment with what is mentioned in other fields and/or the age group in GX.6b is not provided or not clearly described (e.g., unclear acronyms).

0=Unspecified

Weight of ingredients: 1/7

Score range (dimension): 0–1

Weight of the dimension: 1

Share section

Assessing the 'Measurable' aspect of commitment goals

Ingredient M1: Primary indicator name and unit

GX.8a

Compulsory – text

1=Completed

  • A clearly defined indicator along with its unit of measurement (when applicable) is defined. The indicator must be further in agreement with the goal description. More than one indicator might be listed for a specific goal but they should be specific to one goal only (e.g., number and prevalence of children receiving treatment).
  • A clearly defined indicator in agreement with the goal description. The unit might be missing from this field but it is clearly mentioned in full commitment and/or goal description and/or indicator baseline/target levels. More than one indicator might be listed for a specific goal but they should be specific to one goal only (e.g., number and prevalence of children receiving treatment).

0.5=Unclear/Not in alignment with other fields

  • The indicator and/or its unit of measurement is not clearly defined.
  • The indicator is not in agreement with the goal description.
  • Multiple indicators are listed that refer to different goals (e.g., % of overweight in children 0–5 and % of overweight in adolescents).

0=Unspecified

Weight of ingredients: 1/5

Ingredient M2: Baseline level of indicator

GX.8b

Compulsory – text

1=Completed

  • The baseline level of the indicator is clearly described using the same unit of measurement (when applicable) with the one provided in GX.8a and the value agrees with is mentioned in other fields (e.g., goal description and/or full commitment).
    • For multiple indicators for the same goal, multiple baseline values must have been entered clearly describing to which indicator each refers to.

0.5=Unclear/Not in alignment with other fields

  • The baseline level is not/not clear if in alignment with indicator (i.e., unsure where this refers to).
  • The unit of measurement is not in agreement with the one provided in GX.8a.
  • The baseline level value is not in agreement with what is mentioned in other fields (e.g., values mentioned in full commitments or goal description) or identical with target when it is suggested otherwise (e.g., goal description mentions reduction or increase).
  • The baseline values of all indicators are not provided, if more than one indicator for one goal, or, if provided, it is not clear which value refers to which indicator.
  • Multiple values are listed for indicators that refer to different goals (e.g., % overweight in children, % anaemia in women).

0=Unspecified

Weight of ingredients: 1/5

Ingredient M3: Baseline level year of assessment

GX.8c

Compulsory – year

1=Completed

  • The response provided is identical or prior to the starting year of the commitment goal (GX.7).

0.5=Unclear/Not in alignment with other fields

  • The response provided is not in agreement with might be mentioned in other fields.
  • The response provided is after the starting year of the commitment goal (GX.7).
  • The baseline level is missing, meaning that a random assessment year might have been selected just to move forward with the form.

0=Unspecified

Weight of ingredients: 1/5

Ingredient M4: Target level of indicator

GX.8d

Compulsory – text

1=Completed

  • The target level of the indicator is clearly described using the same unit of measurement (when applicable) with the one provided in GX.8a and in GX.8b; the direction of the baseline to target level is in alignment with the goal description (e.g., the goal is to reduce stunting, the baseline level is 20% and the target level is 15%); and the value agrees with what is mentioned in other fields (e.g., goal description and/or full commitment).
    • There are cases where the target level has been provided under interim targets. This case would still get ‘1’ as long as we can confirm that the value under interim is the target level (based on values included in full commitment and goal description).
    • For multiple indicators for the same goal, multiple target values must have been entered clearly describing to which indicator each refers to.

0.5=Unclear/Not in alignment with other fields

  • The target level is not/not clear if in alignment with indicator (i.e., unsure where this refers to).
  • The unit of measurement is not in agreement with the one provided in GX.8a and/or in GX.8b.
  • The direction of the baseline to target level is not in alignment with the goal description (e.g., the goal is to reduce stunting, the baseline level is 20% and the target level is 25%).
  • The target-level value is not in agreement with what is mentioned in other fields (e.g., values mentioned in full commitments or goal description) or identical with baseline when it is suggested otherwise (e.g., goal description mentions reduction or increase).
  • The target values of all indicators are not provided, if more than one indicator for one goal, or, if provided, it is not clear which value refers to which indicator.
  • Multiple values are listed for indicators that refer to different goals (e.g., % overweight in children, % anaemia in women).

0=Unspecified

Weight of ingredients: 1/5

Ingredient M5: Monitoring and evaluation plan

GX.8f

Compulsory – text

1=Completed

  • A monitoring and evaluation plan of the indicator is clearly described, ideally with the frequency if applicable, and in agreement with the indicator fields.

0.5=Unclear/Not in alignment with other fields

  • A monitoring and evaluation plan of the indicator is not clearly described (e.g., ‘progress will be monitored monthly’ without describing how this monitoring will happen, use of unclear acronyms).

0=Unspecified

Weight of ingredients: 1/5

Score range (dimension): 0–1

Weight of the dimension: 1

Share section

Assessing the 'Achievable' aspect of commitment goals

Ingredient A1: Total costs

Q10; 10a

Compulsory – multiple choice; Compulsory – currency and numerical for amount

1=Completed

  • Any of the first two options are selected in Q10, and the amount and currency provided in Q10a are in agreement with what might be mentioned in other fields.

0.5=Unclear/Not in alignment with other fields

  • The third option is selected in Q10.
  • Any of the first two options are selected in Q10, and the amount and currency provided in Q10a are not in agreement with what is mentioned in other fields.

0=Unspecified

  • The last option is selected in Q10.

Weight of the ingredients: 1/4

Ingredient A2: Funder

Q11

Compulsory – text

1=Completed

  • A valid name of the organisation(s) funding the commitment is provided and in agreement with what might be mentioned in other fields. That is for all known funders; if more are expected but unknown at the moment the response still receives 1.

0.5=Unclear/Not in alignment with other fields

  • A valid organisation name is provided but not in agreement with what is mentioned in other fields.
  • Not all funders are listed (e.g., they provide examples of some and/or they mention ‘and other donors’).
  • The organisation name is not clear (e.g., acronyms not commonly used and easily identified) or vague (UN agencies).

0=Unspecified

Weight of the ingredients: 1/4

Ingredient A3: Funding mechanism

Q11

Compulsory – text

1=Completed

  • The funding mechanism is clearly described and in agreement with what is mentioned in other fields.

0.5=Unclear/Not in alignment with other fields

  • The funding mechanism is not clearly described (e.g., unclear acronyms, vague description).
  • The funding mechanism is not in agreement with what is mentioned in other fields (e.g., funder = UN agency and funding mechanism = government).
  • The funder is missing, hence we cannot cross-check whether the funding mechanism is in alignment.

0=Unspecified

Weight of the ingredients: 1/4

Ingredient A4: Amount secured

Q11

Compulsory – text

1=Completed

  • A clear description (percent or qualitative broad estimate) of whether the total costs have been secured is provided and in agreement with what might be mentioned in other fields.

0.5=Unclear/Not in alignment with other fields

  • The description of whether the total costs have been secured is not clear and/or not in agreement with what is mentioned in other fields.

0=Unspecified

Weight of the ingredients: 1/4

Score range (dimension): 0–1

Weight of the dimension: 1

Share section

Assessing the 'Relevant' aspect of commitment goals

Ingredient R1: Alignment with global nutrition targets

Q3

Compulsory – multiple choice

1=Completed

  • The global target(s) selected seem to be in alignment with the measurable goals described (we will need to check full commitment and all the goals of a commitment to assess this as it is not a goal-specific question).

0.5=Unclear/Not in alignment with other fields

  • The response(s) selected do not seem to be in alignment with the measurable goals described (e.g., the goal is to reduce stunting and child overweight has been selected as global target).
  • Answer is ‘None’ and it is clear from full commitment that it could be linked to global targets.
  • The response(s) selected are not complete (e.g., only one global target has been selected while more seem to apply) or all have been selected while they do not seem all to apply.

0=Unspecified

Weight of ingredients: 1/2

Ingredient R2: Thematic area (only relevant to N4G commitments)

Q9

Compulsory – numerical

1=Completed

  • The responses have been selected. This is always the case for commitments linked to the 2021 Tokyo N4G Summit only (first response selected in Q7). We are not validating the response against the content of the commitment; we are reclassifying all commitment using the NAF Nutrition Action Classification System.
  • If the commitment is not linked to the 2021 Tokyo N4G Summit, this ingredient is not scored and R1 has an ingredient weight of 1.

0.5=Unclear/Not in alignment with other fields

0=Unspecified

Weight of ingredients: 1/2

Score range (dimension): 0–1

Weight of the dimension: 1

Share section

Assessing the 'Time-bound' aspect of commitment goals

Ingredient T1: Start date

GX.7

Compulsory – drop-down menu

1=Completed

  • The response provided is in agreement with what might be mentioned in other fields.

0.5=Unclear/Not in alignment with other fields

  • The response provided is not in agreement with what might be mentioned in other fields.
  • The starting date is before the baseline year of the indicator assessment.

0=Unspecified

Weight of ingredients: 1/2

Ingredient T2: End date

GX.7

Compulsory – drop-down menu

1=Completed

  • The response provided is in agreement with what might be mentioned in other fields and it is not before or identical with the start date (GX.7).

0.5=Unclear/ Not in alignment with other fields

  • The response provided is not in agreement with what might be mentioned in other fields
  • The response provided is before or identical with the start date.

0=Unspecified

Weight of ingredients: 1/2

Score range (dimension): 0–1

Weight of the dimension: 1

Unspecified: Since all fields are compulsory, we have no true missingness across ingredients. However, the ‘Unspecified’ score is given to responses such as NA, TBD, TBC or text indicating that “this has not been decided yet”.

Measurable goals: We are only assessing the description of the goals and not the number of goals (Q12) for SMARTness. The platform shows as many fields for goals as the number indicated in Q12. Therefore, by checking the goal description to ensure that each goal is a single goal and does not include multiple ones, we also check that the number of goals listed in Q12 is correct. Being able to count the goals is directly affected by whether the goal(s) are clearly described.

GX1: As described in the column for the assessment of this ingredient, we will not be assessing the response provided in the indicated registration form question, but rather the ability to reclassify the commitment based on the available information as previously described.

Last updated: 13 September 2022.

Jump to section Jump to section

Footnotes

  1. Save the Children and World Vision International. The nutrition barometer – Gauging national responses to undernutrition. 2012.

    The Global Hunger Index. 2021; published online October). https://www.globalhungerindex.org/download/all.html

    Pangaribowo, E. H., Gerber, N., Torero, M. (2013) Food and nutrition security indicators: A review. www.zef.de.

    Return to source text

  2. Alkire, S., Santos, M. E.. Measuring Acute Poverty in the Developing World: Robustness and Scope of the Multidimensional Poverty Index. World Development 2014; 59: 251–274

    Return to source text

  3. OECD. Handbook on Constructing Composite Indicators: Methodology and User Guide. 2008.

    Return to source text