Please provide any additional comments in the table below (you must be logged in to edit the table and provide your comments).
Comment # | Working Text Reference | Working Text Page # | Comment Provided By | Comment - Working Party Members Provide Feedback Here |
---|---|---|---|---|
1 | Section 3, Context for this Review | 6 | Philip Sheppard | It is suggested that this key ICANN board resolution is included: In its resolution of 28 September 2013 the ICANN Board stated: “The expansion of the TLD space has increased the number and variety of stakeholders participating in GNSO policy making and a review needs to take place on schedule to examine whether the current model meets the needs of a new generation of stakeholders. GNSO Structure is unlikely to accommodate the anticipated new stream of stakeholders resulting from the expansion of the TLD space. The GNSO Review will be an important vehicle for considering and addressing this issue. The unbalance that is already occurring needs to be addressed by the GNSO Review. ” |
2 | General | General | Stephanie Perrin | Thank you for the opportunity to comment on the draft Westlake report. There is much good work in here, but I believe a lot of work remains to be done prior to releasing a draft report for comments. I would respectfully submit that the GNSO Review working party needs to see the next iteration of this report prior to its release for public comment, because there are a number of issues that need to be rectified. |
3 | 10 | Stephanie Perrin | Second survey on PDPs was not advertised as well as it might have been….not a good sample size. I would have filled it out, did not know. | |
4 | 11 | Stephanie Perrin | I am admittedly more accustomed to independent review performed by governments, where review is done by officers of Parliament and is quite independent, usually monitored by internal auditors to ensure appropriate distance. However, I must point out that if Westlake was talking to staff and receiving guidance from them on a daily basis, with weekly calls, this is hardly an independent review. I would also note that I recognize 7 of the interviewees as staff (and I may be missing some as I don’t recognize all the names, and some more could be in the anonymous interviewees) but even at 7 that is 18% of a very small sample. Staff are terrific resources, but I think this survey should not rely so heavily on staff observations and interventions. Perhaps they could be analysed separately? | |
5 | General | General | Stephanie Perrin | I would close by saying that I would be delighted to be interviewed, if Westlake wished to enlarge its sample size a bit. I am a relatively new participant at ICANN (two years, 8 meetings) and a new member on council. I don’t believe newcomers are well represented in the current sample, which is a pity, because we are less likely to have history which might colour our observations, and we are supposed to be a target group for recruiting and preventing burnout. |
6 | The authors of the draft report repeatedly acknowledge its methodological limitations. They described their approach to interviews as ‘less than ideally efficient’ (p. 9). There are recurrent references to the authors being ‘unable to find evidence’ (e.g., p. 51), the lack of ‘objective and quantifiable criteria’ (p. 7) and of the lack of quantitative data concerning volunteer participation rates, retention rates, diversity, gender and of statistics concerning the recruitment and intake of new volunteers (p. 8). | 7, 8, 9, 51 | NCUC | While appreciative of the honesty of the Westlake team in mentioning their challenges, it should be noted that mere acknowledgement of a study’s deficiencies does not free the study of those limitations. It certainly did not do so here. Moreover, the false assertions about NCUC could easily have been “fact checked” before inclusion by consulting our website and open mail list archives, or by asking us about them. |
7 | General | n/a | NCUC | The study seems to have a constantly changing and imprecise design that meanders between various means of investigation without fully investing in any one. As a qualitative study there is no perceptible strategy or control other than the relay of “observations” of Westlake staff and the selected use of anecdotes from unidentified parties. A clearly defined narrative approach may have proven useful here but there is no indication that was ever considered or acted upon. The quantitative aspects of the study lack any rigor or application of standard statistical sampling or analysis techniques. Samples are generally undefined and too small to generate the conclusions extrapolated from them. Adjustment of methodology mid-study (e.g. the Supplementary Working Group survey) raise questions of corrective measures polluting the findings (e.g. strategic sampling). |
8 | General | 8, 78, 79 | NCUC | All aspects of sampling in this study are problematic. Number of NCUC members: 404 |
9 | General | 7, 81, 82, 90, 92 | NCUC | The analysis is replete with generalities not adequately linked to facts. This is particularly problematic when so much of this study is based upon Westlake’s observations and selected anecdotes. A high degree of approximation occurs throughout the study. Consider these examples: “There was a view that” (p. 82); “anecdotal but credible instances” (p. 7); “we received no comment…based on this, we conclude this is no longer a significant issue” (p. 95); “some survey respondents” (p. 81); “there is a perception among some” (p. 81); “there is an often expressed view” (p. 90); and “we encountered active hostility to new leaders from a few participants” (p. 90). Instead of some, how many? Instead of often, how often? A professional study should have recommendations based on specific facts in evidence. Too often this study does not. Of particular concern are the Westlake Review Recommendations on page 92 of this Report. Several of these recommendations, such as those involving travel, are not supported by anything substantive previously discussed in the draft Report. |
10 | General | n/a | NCUC | Quotations used in this study are completely stripped of context and because of this are of limited value. A professional study of this type should have as its base proper design, a reliance upon facts obtained in a credible and transparent manner that meet basic standards of empirical research. This study does not, as is acknowledged at times by its own authors. Until it does, any recommendations made by Westlake should be considered as emanating from a flawed and poorly constructed study. |
11 | General | n/a | NCUC | The draft report is at its best when it is summarizing the recommendations of previous GNSO reviews (Sharry, Council Self Review, LSE, ATRT2). We agree with Westlake that previous recommendations concerning accessibility, transparency and diversity need to be implemented by all constituencies and stakeholder groups. We have raised this directly in the past with Fadi and other senior staff. We would welcome broad-based progress on these scores, which the NCUC has already pursued. |
12 | General | n/a | NCUC | One of the more unsettling aspects of the draft Report is its substitution of opinion or conjecture for data that is readily available. We note that the authors of this independent study were guided in this process by ICANN staff who “on several occasions have directed us to information that we might not otherwise have been aware of or otherwise been able to find” (p. 11). We certainly would have welcomed the same opportunity at an earlier time to help and guide the Westlake team in this manner. |