The Nutrition Action SMARTness Index

The Nutrition Action SMARTness Index is a novel ranking system that assesses and reports on the SMARTness of committed nutrition actions. It was developed by the Global Nutrition Report (GNR) in 2022. Based on a comprehensive methodology, it addresses past challenges by providing clear criteria for SMART (Specific, Measurable, Achievable, Relevant, Time-bound) commitment-making.

This novel ranking system categorises nutrition actions into one of four levels of SMARTness – High, Upper moderate, Lower moderate and Low. It does this by taking into account three performance criteria for each commitment goal: SMARTness score, trackability and completeness. All commitments registered through the Nutrition Accountability Framework (NAF) are evaluated using the SMARTness Index with findings reported publicly on the NAF Commitment Tracker.

This webpage presents the rationale and the methods used to develop the SMARTness Index, and describes how the GNR uses it to assess and report on the SMARTness of commitments.

The aim of the SMARTness Index is to provide a measure of how SMART a nutrition commitment is and to highlight areas for improving the formulation of a commitment. For commitments with multiple goals, each goal is separately assessed for its SMARTness.

The SMARTness Index has its basis in the 2021 NAF methods with regards to the formulation and registration of SMART commitments. The GNR has previously identified the ‘ingredients’ (information) that a nutrition commitment must contain to be considered SMART.^[1] These are embedded in the Sign up and Commitment Registration Forms as compulsory standardised fields.^[2] Effectively, the 2021 NAF methods conceptualised SMARTness and the SMARTness Index was developed to quantify SMARTness.

The SMARTness Index evaluates the consistency and completeness of the information (ingredients) provided for each registered commitment goal. It does not assess how impactful commitments are (e.g., a commitment could be SMART but of limited impact, and vice versa); future iterations may assess impact. The SMARTness Index ultimately ranks committed nutrition actions into one of four SMARTness levels: High, Upper moderate, Lower moderate and Low. This ranking factors in and evaluates the following three performance criteria:

1. SMARTness score. Gives a numerical score between 0 and 5 to the SMARTness of a given commitment goal accounting for each of the 20 ingredients. A score of 5 indicates that all 20 ingredients have been provided and are clearly described (see Assessing the SMARTness score of nutrition commitments), and that, as such, the goal is also trackable and clear.
2. Trackability. Indicates whether the essential ingredients (S4, M1, M2, M4, T1 and T2) that a commitment goal needs to be trackable have been provided and are clearly described.
3. Completeness. Indicates for how many ingredients the GNR needs to go back to the commitment-maker and ask for clarifications.

The GNR assesses the SMARTness of commitment goals as they are registered and publishes the SMARTness Index and its performance criteria (SMARTness score, trackability and completeness) through the NAF Commitment Tracker. The aim is for all goals to reach High SMARTness through the verification process; the SMARTness Index of each goal will be updated in the NAF Commitment Tracker as clarifications are received.

SMARTness score

The overall SMARTness score (the first performance criterion of the SMARTness Index) is calculated for each goal based on the information provided through the Commitment Registration Form for each of the 20 ingredients (Appendix 2). Each ingredient is assessed for completeness and is given one of three points: 1 for complete and clear responses; 0.5 for responses that are unclear or inconsistent with other aspects of the commitment goal; and 0 for responses that do not provide the required content. As described in Appendix 2, the score of each SMARTness dimension (Specific, Measurable, Achievable, Relevant, Time-bound) is computed and then the individual scores are combined to calculate an overall SMART score for each goal. This is a numerical value that ranges from 0 to 5. Practically, the minimum overall SMARTness score for each goal is 1.5 due to the presence of compulsory questions within the design of the NAF Commitment Registration Form.

Based on this minimum overall score (1.5) and observed ranges within the dataset of registered commitments, the SMARTness score classifies commitment goals into three groups:

- SMARTness score of ≥4.5 contains all goals that received a score of greater or equal to 4.5

- SMARTness score of <4.5 to ≥3.5 contains all goals that received a score of less than 4.5 but greater than or equal to 3.5

- SMARTness score of <3.5 contains all goals that received a score of less than 3.5.

Trackability

Trackability is the second performance criterion of the SMARTness Index. It specifically assesses the six ingredients used to determine whether progress towards the commitment goal can be tracked: S4, M1, M2, M4, T1 and T2.

To assess progress towards achieving a commitment, a clear measurable goal, measurable indicator and unit of measurement are required (S4 and M1 respectively). Baseline and target levels of the measurable indicator (M2 and M4 respectively) are needed to calculate the level of progress achieved at any given point, and start and end dates (T1 and T2 respectively) are required to calculate the expected annual rate of change and assess whether the commitment has been achieved by the intended date. If all of these ingredients receive a perfect SMARTness score of 1, the commitment goal is characterised as trackable.

The trackability of a commitment goal is derived from the score of these ingredients as follows:

• Trackable: A goal is considered trackable if all of the S4, M1, M2, M4, T1 and T2 ingredients have a SMART score of 1
• Not trackable: A goal is considered not trackable if any of the S4, M1, M2, M4, T1 and T2 have a SMARTness score of 0 or 0.5.

Completeness

Completeness is the third and last performance criterion of the SMARTness Index. It is used to assess the extent of required revisions that would be needed to improve the SMARTness of a given commitment goal.

As outlined, the overall SMART score (between 0 and 5) is based on and assesses 20 ingredients (Appendix 2). To achieve a perfect score of 5, all 20 ingredients must have a score of 1, indicating that no clarifications or revisions are needed. An overall score of less than 5 suggests that specific SMART ingredients have a score of 0 or 0.5, and therefore the GNR needs to contact the commitment-maker to clarify the information provided for those ingredients. The lower the overall SMARTness score, the greater the extent of required revisions for that commitment goal and therefore the lower the ‘completeness’ of that goal.

The extent of required clarifications is based on the number of ingredients that received a score of 0 or 0.5, indicating that they require revisions. Clarifications were considered minor when a quarter or less of ingredients received a score of 0 or 0.5. As such, the completeness of commitment goals is classified as:

• Minor clarifications where 25% or less (5 or less out of 20) of the ingredients score 0 or 0.5
• Major clarifications where more than 25% (6 or more out of 20) ingredients score 0 or 0.5.

The SMARTness Index jointly assesses each of the three performance criteria to rank commitment goals into one of four levels, using the performance matrix. The commitment goal can fall into any of 12 performance areas (based on the three overall score groups, the two trackability groups, and the two completeness groups).

Each goal is evaluated against each of the three performance criteria, and is subsequently assigned to one of the 12 areas in the matrix, which classify the SMARTness of commitment goals as High, Upper moderate, Lower moderate and Low. Commitments with High SMARTness are trackable and require minor or no clarifications.

The scope of the SMARTness ranking also includes refining and improving the formulation of the commitment goal through the different stages of the NAF cycle. The GNR assesses the SMARTness of commitment goals as these are registered (on a monthly basis) and publishes the SMARTness Index through the NAF Commitment Tracker. The end goal of all NAF processes is to improve the SMARTness of registered commitment goals, so that they can all be ranked as ‘High’.

All commitment-makers are given the opportunity to improve the SMARTness of their commitment goals though the verification process. All ingredients that have scored 0 or 0.5 will be flagged to commitment-makers, who will be given the opportunity to revise any relevant information. Once complete, the ingredients of the SMARTness Index will be recalculated, and the SMARTness Index of each goal – along with each of the three performance criteria – will be updated in the Commitment Tracker.

Improving the score of any of the SMART ingredients will improve the overall score of the goal, and enable its potential transition into a higher-ranking category. Improvements in the S4, M1, M2, M4, T1 and T2 ingredients will be reflected in the trackability criterion and could trigger movement on the performance matrix if a goal moves from not trackable to trackable. Finally, depending on the number of changes made to the ingredients’ score, the completeness criterion may change, causing further movement on the performance matrix if, for example, a goal moves from major to minor revisions.

All commitment-makers will be able to view their commitments’ SMARTness Index on the NAF tracker, and it is the hope of the GNR that this leads to greater engagement with the verification process to clarify the commitment goals that score lower on SMARTness. Approaching SMARTness in this way may also encourage commitment-makers to make SMARTer commitments at the point of commitment registration, reducing the need for extensive requests for clarification as part of the verification process.

Last updated: 13 September 2022.