Commitment data cleaning and standardisation

This page describes the actions taken to retrieve, clean and standardise the nutrition commitments submitted through the Nutrition Accountability Framework (NAF).

In summary, the data is retrieved from the NAF Platform using an application programming interface (API) and then undergoes a semi-automated data cleaning and standardisation process to create a common format across all commitments. This includes assessing the eligibility of all commitments registered with the NAF and ensuring consistency of data types throughout the dataset. Commitments are also classified using the Nutrition Action Classification System and assessed for SMARTness using the SMARTness score.

No corrections are made to the self-reported commitments at this stage. Any standardisation that requires amending, regrouping or reclassifying results in the generation of a new variable and not the correction of an existing one. The commitment-verification process will seek clarity on commitments directly from stakeholders and commitments will be corrected and/or updated only with the approval of commitment makers. The data-standardisation process is independent of but complementary to the commitment-verification process.

The data held within the NAF Platform is retrieved using an Application Programming Interface (or API) that enables all commitment data to be downloaded in a comma-separated values (CSV) file. The API automatically combines the information provided in the Sign up Form and the Commitment Registration Form, so each commitment is linked to the user information of the commitment maker.

Additional commitments were registered prior to the NAF Platform being launched, using either a Google Form or a PDF provided by the Global Nutrition Report (GNR). This data was manually mapped to the NAF Commitment Registration Form by the GNR team. Due to differences in the structure of the two registration forms, information was missing from the commitments registered via the Google Form/PDF. Before combining these commitments with the main dataset, each stakeholder was individually contacted by the GNR team and asked to supply the remaining information, either by registering their commitment on the NAF Platform or by sending it to the GNR team. The data from stakeholders who elected to send the additional information to the GNR team was then combined with the dataset retrieved from the API. The dataset was therefore restructured so the information provided in the Sign up Form was at the start of the spreadsheet followed by the commitment-specific responses in the same order as the Commitment Registration Form.

Commitments are grouped at commitment level (i.e., one row per commitment) in the dataset. Each one is automatically given a unique random 19-digit identification number when it is registered with the NAF Platform. To aid goal-level assessments (described in Section 4), a dataset with one row per goal is also created. This contains an additional unique identification number for each goal, consisting of the 19-digit identifier followed by sequential numbers separated by an underscore (‘_’), where 0 represents the first goal and 9 represents the tenth goal, e.g., [19-digit identifier]_0.

Commitment screening and inclusion

The combined dataset undergoes an eligibility assessment that removes all test submissions, all commitments that are not deemed eligible to be included in the NAF, and all duplicate commitments. Commitments are identified as potential duplicates manually by the GNR team when it appears that information in two commitments is either identical or too similar for the commitments to be independent of each other. The GNR team then contact the stakeholder to confirm the presence of a duplicate commitment and, if confirmed, the commitment specified by the stakeholder is removed.

Only commitments with nutrition-related goals are included in the NAF. Commitment goals that are not nutrition-related (e.g., increasing physical activity) or the information provided is not sufficient or clear enough to decide whether they are nutrition-related (e.g., increasing the number of beneficiaries receiving cash transfers to address poverty) are flagged and part of the verification process.

Data is checked to ensure consistency in data type within variables. All numerical values are changed to float numbers (i.e., numbers with a decimal place). In cases where it is easily identified that a stakeholder used a comma as a decimal separator, such as for prevalence values, the comma is changed to a decimal point. If there is any ambiguity over the purpose of a comma, the value remains as originally submitted. The verification process will clarify with the stakeholder whether the comma has been used as a decimal separator and if confirmation is received, these commas will be changed to a decimal point.

The start month and start year and end month and end year (GX.7 of the Commitment Registration Form where X refers to the goal number) are converted into date format (DD/MM/YYYY) to create one start date variable and one end date variable. The first day of the chosen month is used for the start date and the final day of the chosen month is used for the end date.

Multiple choice answers are converted into binary variable columns for convenience and to simplify analysis. For example, an answer to the N4G thematic area question (Q9 of the Commitment Registration Form) selecting both ‘Health’ and ‘Food’ is converted into two binary columns in the dataset where Health=1 and Food=1. Country names across the dataset are standardised to be consistent with the spelling used within the GNR (e.g., ‘USA’ to ‘United States of America’).

Missing values are checked. Compulsory questions in the commitment registration form have no true missingness, however values are considered missing for responses such as ‘NA’ (‘not applicable’), ‘TBD’ (‘to be determined’), ‘TBC’ (‘to be confirmed’) or any text indicating that ‘this has not been decided yet’.

Finally, the language of each commitment is identified. While commitments must be submitted in English, occasional non-English commitments have been registered with the NAF. Non-English commitments are translated into English before being analysed.

Using the goal-level version of the dataset, all measurable goals for eligible commitments are classified in the Nutrition Action Classification System and assessed for SMARTness score (see Assessing the SMARTness score of nutrition commitments). Commitment goals that are not explicitly nutrition-related (e.g., increasing physical activity) or that do not contain sufficient or clear enough information to decide if they are nutrition-related (e.g., increasing the number of beneficiaries receiving cash transfers to address poverty) are classified and assessed for SMARTness. The connection of these goals to nutrition can then be determined with stakeholders in the verification process.

The SMARTness assessments of each ingredient are combined to calculate a goal-level commitment SMARTness score together with a Nutrition Action SMARTness Index.

Due to the extent of the variation in self-reported responses, the commitment dataset requires a degree of standardisation before any data can be analysed. Standardisation enables comparison between commitments while allowing the substance [LC1] [GZ2] of the commitment to remain the same. This section describes the standardisation of organisation name and stakeholder, funding information, indicator name and geographical action area.

Organisation name and stakeholder type

Organisation names for both the organisation making the commitment (from the Sign up Form) and any additional organisations involved (Q5 on the Commitment Registration Form) are checked for consistency in names and spelling, and abbreviations are expanded, where known. For example, ‘WHO’ becomes ‘World Health Organization’ for all instances.

Stakeholder type is also checked to ensure that the correct type was selected when the stakeholder signed up to the NAF Platform. Where stakeholders selected ‘Other’ as stakeholder type (in the Sign up Form), the free text answers were assessed to find the best fit out of the other stakeholder types.

All expansions of acronyms and reclassifications of stakeholder type will be checked with stakeholders through the verification process and corrections to the dataset will be made if required.

Funding information

Funder, funding mechanism and amount of secured funding (Q11 on the Commitment Registration Form) are standardised by regrouping. These groupings, along with descriptions of the types of answers in that group, can be found in Table 1, Table 2, and Table 3 respectively. The groupings for funder and funding mechanism are standardised as binary values, as they are not mutually exclusive. For example, one commitment could have multiple funders and be classified under ‘Government’, ‘Global organisation’ and ‘Mixed’. The amount of secured funding is mutually exclusive, and stakeholders cannot be classified into more than one of the groups. The total cost of the commitment (Q10a on the Commitment Registration Form), where provided, is standardised by converting the cost from the specified currency into US$ using the World Bank exchange rate at the time.^[1]

Indicator name

Indicator names (GX.8a on Commitment Registration Form where X refers to the goal number) are grouped and standardised. These groups are described in Table 4 and are mutually exclusive.

The baseline level (GX.8b) and target level (G1.8d) of goals categorised as ‘Impact’ commitments in the classification system are standardised to remove the unit of measurement (e.g., %), resulting in only numeric data that remains in the same unit as the original value. For example, a prevalence value of ’8%’, which is presented as a whole number becomes ‘8.0’ in the standardised version of this variable. This standardisation allows for the calculation of intended change between baseline and target values. Impact commitments result in changes in population and the types of indicators and units are repeated, allowing for comparisons to be made. Goals classified as ‘Enabling’ and ‘Policy’ commitments have much greater breadth of variation and are standardised in a similar manner during the verification process.

Action area

Standardisation for geographic area (GX.4 and GX.5 on the Commitment Registration Form where X refers to the goal number) only occurs for commitments reporting the geographical area of their commitment to be ‘National’, ‘Sub-national’ or ‘Local’. If a country is not specified in the goal description (Q13), geographic area (GX.4) or additional information about the geographic area (GX.5), the country of the organisation, as specified in the Sign up Form, is used. If a country different to the organisation’s country is specified in Q13, GX.4 or GX.5, the alternative country is used instead. This results in a country name as the action area. These countries are further grouped into global regions (e.g., Asia), and sub-regions (e.g., Central Asia), as per the GNR’s standard groups.

The income level for each country action area is calculated using the World Bank development indicators.^[2] Each country action area is also assessed to determine whether it experiences high levels of three types of malnutrition. The thresholds for assessing whether a country is burdened or not are based on the following prevalence: stunting in children aged under five years ≥20%; anaemia in women of reproductive age ≥20%; overweight (BMI ≥25) in adult women aged ≥18 years ≥35%.^[3] An action area is categorised as having 0–3 burdens of malnutrition, unless data for any indicator is not available, in which case the burden of malnutrition cannot be classified.^[4]

An internal and external quality assurance system was used to check the accuracy of the data cleaning and standardisation process. Internal quality assurance aimed to check the consistency of the outputs. Performed by an analyst not involved in the coding, it involved a series of checks on the statistical output generated:

• The aggregated frequencies for individual stakeholders’ groups must be equal to the total number of commitments/goals in the sample.
• The aggregated breakdown frequencies within each group/sub-group must be equal to the total frequencies of the group (number of stakeholders/commitments/goals as relevant).
• The totals for each stakeholder must be consistent within each statistical output.

The external quality assurance process focused on validating the correctness of the Python code compiled to generate the master dataset used for the analysis. An independent data analyst used the raw data to recompile the data cleaning and standardisation code. The dataset generated was then compared with the original dataset.