At-Large Initial Report on Data & Metrics for Policy Making Workspace

Comment Close Date	Statement Name	Status	Assignee(s)	Call for Comments Open	Call for Comments Close	Vote Open	Vote Close	Date of Submission	Staff Contact and Email	Statement Number
07 Sep 2015	Initial Report on Data & Metrics for Policy Making	COMMENT	Maureen Hilyard	25 Aug 2015	01 Sep 2015	TBC	TBC	TBC	Steve Chan steve.chan@icann.org	TBC

For information about this Public Comment, please click here

Comments Forum

Brief Overview

Purpose: This public comment proceeding seeks to obtain community input on the Initial Report from the GNSO's Working Group (WG) on possible recommendations for the use of Data and Metrics for Policy Making. The WG's intent is to develop solutions that will improve the way in which consensus policies are developed and help ensure that the most critical registrant, registry, and registrar issues are addressed by evolving the culture to better informed, fact-based policy development and decision-making.

Next Steps: After close of the public comment period, the WG will review the submitted comments and update the report as agreed upon, thus producing the Final Report for submission to the GNSO Council.

Section I: Description, Explanation, and Purpose

The Data & Metrics for Policy Making Working Group (WG) is exploring opportunities to review standard methodologies of reporting and metrics that could better inform fact-based policy development and decision making; including how the community can collaborate with Contracted Parties and other service providers in the sharing of metrics and data.

The WG's charter contains eight specific questions for the WG, derived from the Final Issue Report that was published in March 2013. The Charter questions cover issues including:

The question “which comes first, policy-process or definitive data describing the problem?” along with suggestions as to how data can be gathered when it has not yet been included in the reporting process.
How processes can be continuously improved, simplified and made more consistent for people wishing to either report a problem or learn about their options when their problem falls outside ICANN policy;
Principles that enhance metrics and data available to better inform the GNSO policy development process;
Improved understanding of the limits of ICANN policies regarding data measurement and tracking and other options to pursue if an issue is not covered by policies that gather data
Mechanisms whereby GNSO working groups can request information (both internal toICANN or external, including GNSO contracted parties) which support fact-based policy-making;
Mechanisms to ensure appropriate safeguards with regard to the confidentiality of certain types of information;
A framework for distributing information to the GNSO policy-making community with the intent of both informing those groups and providing the ongoing basis for identifying and correcting problem-reporting and data-collection problems;
Any changes needed to incorporate the processes described above into the ongoing Policy Development Process.

Opportunities discovered by the group include:

Improve the way in which Consensus Policies are developed and help ensure that the most critical registrant, registry, and registrar issues are addressed
Evolve culture to better informed, fact-based policy development and decision making
Base deliberations and decisions on tangible evidence as opposed to “gut feeling” or anecdotal evidence
Instill notion of continuous improvement to the policy process and effectiveness consensus policy implementations

Key Definitions:

Data: Individual facts, a set of values, statistics, or items of information
Metrics: A set of measurements that help quantify results, which allows for better determination of the level of success against a set of goals

The WG’s preliminary recommendations include:

Initiation of a pilot effort whereby the GNSO community at the early stages of the policy process (Issue Identification or Working Groups) can submit tactical sized requests for data and metrics to assist in validation of issues or to better inform policy deliberations
Creation of WG work product templates (Charter, Final Report) including updates to theGNSO’s Operating Procedures and Working Group Guidelines (WGG)
Update to the Charter template to assist drafting teams in determining key metrics to be used to meet the goals of resolving issues
Updates to the Final Report template with a standard but modifiable recommendation forcing a review of the policies implemented at a future date to determine if it met the stated intent
Creation of a Metrics Request Decision Tree and Request Form to formalize the request process; including inclusion into the WGG.

Following analysis of all public comments received, the WG will finalize its recommendations and prepare a Final Report for delivery to the GNSO Council, for its review and action.

Section II: Background

The 2010 Registration Abuse Policies Working Group (RAPWG) identified the Meta Issue: Uniformity of Reporting which it described as "need for more uniformity in the mechanisms to initiate, track, and analyze policy-violation reports." The RAPWG recommended in its Final Report that "the GNSO and the larger ICANN community in general, create and support uniform problem-reporting and report-tracking processes."

The GNSO Council recommended the creation of an Issue Report to further research metrics and reporting needs in hopes to improve the policy development process. The report created by ICANN staff outlined accomplishments regarding reporting and metrics by the Contractual Compliance function and it also reviewed other reporting sources that may be of relevance. TheGNSO Council subsequently adopted the recommendation to form this non-PDP Working Group tasked with exploring opportunities for developing reporting and metrics processes and/or appropriate standardized methodologies that could better inform fact-based policy development and decision making.

The WG has developed preliminary recommendations for a number of the charter questions it was tasked to address, which are described in its Initial Report [PDF, 395 KB]. Those recommendations that contain suggested edits to the GNSO's Operating Procedures are also included in a redline version [PDF, 768 KB] should the WG's recommendations be adopted by the GNSO Council.

Section III: Relevant Resources

Working Group Initial Report: https://www.icann.org/en/system/files/files/report-gnso-data-metrics-policy-making-initial-29jul15-en.pdf
Redline of GNSO Operating Procedures: https://www.icann.org/en/system/files/files/gnso-operating-procedures-redline-29jul15-en.pdf
Working Group Charter: http://gnso.icann.org/en/issues/dmpm-charter-23jan14-en.pdf
DMPM Project Page: http://gnso.icann.org/en/group-activities/active/dmpm
DMPM Wiki: https://community.icann.org/display/marwg

Section IV: Additional Information

Note that a few of the WG’s recommendations suggest updates to the GNSO’s Operating Guidelines; specifically Annex 1 "GNSO Working Group Guidelines" and Annex 2 "Policy Development Process Manual”. The recommendations within the Initial Report include the proposed changes or creation of new material. A redline version of GNSO’s Operating Guidelines are included to highlight the possible updates to this document should the GNSOCouncil adopt the WG’s recommendations.

Section V: Reports

Staff Contact

Steve Chan

steve.chan@icann.org

FINAL VERSION TO BE SUBMITTED IF RATIFIED

The final version to be submitted, if the draft is ratified, will be placed here by upon completion of the vote.

FINAL DRAFT VERSION TO BE VOTED UPON BY THE ALAC

The final draft version to be voted upon by the ALAC will be placed here before the vote is to begin.

FIRST DRAFT SUBMITTED

The ALAC welcomes being given the opportunity to provide community input into the Initial Report from the GNSO's Working Group with regards to possible recommendations for the use of Data and Metrics for Policy Making.

The ALAC appreciates the need for solutions that will improve the way in which consensus policies are developed, especially in relation to critical registrant, registry and registrar issues. It is desirable that these are addressed within a working culture that is open and transparent; collects data in an environment that ensures confidentiality and anonymity; and subsequently encourages better informed, fact-based policy development and decision-making.

All parties involved in GNSO decision-making must recognise the benefit and value of relevant baseline data and metrics to the policy development process, especially at the initial stages of scoping, understanding and describing a problem or issue. Because of the importance of engaging all parties into the new process,the ALAC supports the possible need to employ an independent third party in order to address any concerns relating to the collection, anonymization and aggregation of data. Fact-based deliberations and decision-making will enable critical issues to now be prioritised appropriately based on tangible evidence rather than "gut feeling" or anecdotal examples.

The ALAC supports the introduction of a "pilot" where working groups will be able to submit proposals or ideas whereby the impact of fact-based data and metrics collection can become the basis for the initial identification and analysis of issues and/or problems. We also support the view that any funding required to implement the pilot should be considered an investment in the improvement of the policy process rather than a cost against budget.

The ALAC supports the revision of the templates for the Issue Report, Charter and Final Report to update earlier WG guidelines and also the development of a decision tree to help to determine the best avenue to request additional data and metrics. The development of a Policy Development Process Manual, which will include the "Metrics Request Tree and Form", will ensure consistency of both process and practice.

Establishing a framework for distributing information through early outreach to other SOs and ACs and other related organisations, will facilitate broader qualitative input as well as support a culture of collaboration between our organisations. This will not only contribute to continuous improvement being fully integrated into the policy development process but also encourages the potential of an Open Data culture across ICANN.

Content

Space Tools