The full research process for the 2019 Access to Seeds Index was run by the Access to Seeds Foundation in-house. Key element is the scoring and ranking process. The Supervisory Board approved the scoring guidelines in its 18 June 2018. These guidelines include criteria and bandwidths to make changes after data collection to ensure that they are discriminative as well as appreciative when applied to the company data.
Marcelo Tyszler, Senior Advisor Sustainable Economic Development at KIT Royal Tropical Institute was asked to audit the process and provide a report for the Supervisory Board. Main focus of the audit was the application of the scoring guidelines and the quality of the ranking tool.
The audit process lasted from July 2018 through October 2018, after completing final scoring for the Asia Index. Main change observed was replacement of a ‘geographic multiplier’ of indicators with ‘geographic scaled indicators’, which approved to be a better way to reward companies for the geography coverage of theirprograms. According to the audit, all modifications made remains within the scope of the Supervisory Board guidance.
Two main abstracts from the report:
Scoring Guidelines
The modifications on the scoring guidelines observed between initial draft and final version are all acceptable and should be considered within scope of the Board guidance. The changes provide refinement and clarity improving the scoring but do not represent fundamental changes that would require a full board review. The major difference from initial draft and final version is the change in the Geographical Multiplier, which is not applied anymore, but replaced by the Scaling indicator which effectively incorporates the geographical scale within the scoring guidelines. This change is considered positive as it allows for a more intentional choice of ranking. This change does not deviate from the original essence and does not require a board review.Ranking Tool
The excel-based ranking tool correctly implements the conceptual methodology into the calculations for the Index. All intermediate and final calculations are correct in the sense that they correctly calculate every element according to the methodology proposed. In two occasions a thorough, technical inspection was applied to verify that formulas were consistent and correct. This included, among other stress tests: (1) Inputting different values for initial an intermediate score and verifying whether components and final scores changed accordingly; (2) Modifying remarks and verifying summary table correctly reflected the changes; (3) Re-doing intermediate blocks and comparing results to ensure both approaches matched exactly. The final version received revealed no problems.
Based on this audit, the Supervisory Board approved the final scoring guidelines of the 2019 Index during its meeting on 10 December 2018.
Link to Audit report
Link to Addendum Africa Indexes
Link to Scoring Guidelines 2019
