Metric X.Y (IAG-Illustrative Example)

Survey of perceived consumer trust in DNS, relative to experiences before the gTLD expansion. Survey could at least measure experiences with:

• phishing, parking sites, malware and spam; 
• confusion about new gTLDs;  
• user experience in reaching meaningful second-level TLDs; 
• registrant experience in being in a different gTLD; 
• registrant and Internet User’s experience with regard to cybersquatting.  

A team of SMEs would need to be assembled to:

• Ensure that such an instrument would satisfy AoC objectives
• Identify exactly what topic areas the survey should address
• Determine whether the topics can be handled within a single questionnaire vs. multiple surveys
• Outline the types of questions that would be instrumentally effective
• Develop requirements for sucessful development and implementation
• Discuss prospective survey population, sampling, and other critical parameters

A make vs. buy decision would be needed to determine whether ICANN should develop the survey in house or hire an external firm. If the latter choice is made, an RFP would need to be written, published, and responses analyzed to select a winner. 

A development period would ensue followed by instrument/process testing and analysis of the test. Once a final design is approved for use, the survey would be conducted and monitored by either ICANN or the external firm. A final report, utilizing accepted statistical methods, would then be produced and, after approval by ICANN, published. 

On the basis that this project is externally developed and administered, Staff tasks/activities include:

• Support SME team including drafting
• Public Comments publication?
• Preparation of RFP
• Legal review
• Analysis of bidders and selection of final award winner
• Coordination with vendor to operate survey
• Manage approval process of final report and ensure proper publication

Dependencies: 

1. Assigning the appropriate ICANN Staff Department to administer, coordinate, and manage this metric whether collected internally or externally. 
2. Developing and executing RFP process.

There are many challenges to be addressed including, but not limited to:

• Population identification (open vs. invitation, characteristics, demographics, et al.)
• Sample size to produce statistically reliable estimates
• Questionnaire bias
• Survey length, instrumentation, scaling, etc. 
• Adequate testing to ensure reliability
• Risk that, after implementation, the results may be inconclusive