Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Metrics
Cloak

NOTE: text in cells below shown in colour is DRAFT only, for discussion by the ARIWG

Issue #16

Lead: Maureen Hilyard


Absence of consistent performance metrics.
Final Proposal as approved by the Board

The ALAC has had a Metrics WG and an ALS Review Taskforce, both of which largely went into stasis during the IANA Transition and Accountability efforts. It is proposed to revive this activity as part of the At-Large Review Implementation.

The ALAC notes that regional differences make it more difficult to have uniformity over participation metrics, but agrees that is an important target. The ALAC notes that collecting such statistics is a staff-intensive operation.

Prioritization1:1:2 (Low resource needs : Low risk ; 2nd priority group)
ARIWG comments

Metrics will be developed for each activity in which At-Large participants are involved in order to measure the effectiveness of our processes as well as the actual involvement of active participants who assist the ALAC to carry out its work within ICANN. Such evidence will not only provide transparency and accountability of the contribution made by At-Large with regards to their meaningful contribution in support of the policy development work carried out by ICANN's supporting organisations, but also of the degree of effort and engagement of the many volunteers whose meaningful contribution to the work of At-Large adds value to the development of policy that is an essential part of the the work of ICANN. Metrics could also legitimate (JC: legitimize?) requests made by At-Large for increased funding support for regional activities where there is still a need for further outreach to educate those in underserved sub-regions about ICANN. One specific goal for the metrics team during 2018-2019, will be the identification of 60 participants who demonstrate meaningful participation and engagement in both ICANN and their regions, to attend the ATLASIII in Montreal in November 2019.

Status of improvement effort / staff lead
Activities, if any, on which implementation is dependent, or that are dependent on implementation of this recommendation

The Technology Task Force would be helpful in developing appropriate tools to record assessments of different activities based on the type of metrics being collected for some measurement purpose.

The Stakeholder Analysis Tool will be able to make use of any regional or country based metrics we develop first through At-Large and then further throughout ICANN

(JC) Selection of methodology for scoring identified performance metrics.

Who will implement the recommendation: ICANN community, ICANN Board, ICANN Organization, other?
Anticipated resource requirements (FTEs, tools)
  • (JC) Tool(s) for collecting each identified performance metric
  • (JC) Staff to assist in monitoring and collection of data relevant to each identified performance metric
  • (JC) Methodology for scoring each identified/collected performance metric
  • (JC) Procedure for dealing with changes in data collection, solution
Expected budget implications

Proposed implementation steps:


1. Each of the items in this implementation plan is expected to provide metrics that will assess the achievement of the objective of the approved proposal, as well as some long term measurable associated with metrics as an activity in itself

2. Each task that the At-Large community is involved in will require metrics in order to assess the achievement of key milestones as well as of the level of achievement of the entire activity

3. In order to identify participants who will qualify for participation in ATLASIII in Montreal in 2019, performance metrics will be used to assess each individual:

  • recorded attendance of regional as well as general At-Large meetings and involvements,
  • evidence of active participation in and meaningful contribution to these meetings
  • evidence of specific advocacy of at-large approved policy recommendations
  • evidence of active participation and meaningful contribution to policy or other statements that At-Large is requested to participate in
  • evidence of active engagement in the distribution of information about ICANN and its activities at local. national or regional level
  • evidence of mentorship support given to colleagues, or the development of resources, to build further knowledge and understanding of the work of ICANN within the regions
Continuous Improvement(s)

(JC) I would suggest bulleted items under point 3 be identified performance metrics while we consider that methodology or combination of performance metrics should be used for assessment. Some At-Large folks are not in the position to meet all metrics so a fair methodology is needed for scoring.

(JC) 4. Post implementation review activity and timeframe for the same.

Continuous Improvement(s)(JC) Review of solution post implementation to establish the effectiveness and fairness in achievement of the objective, as well as majority support of the At-Large community for the same.
Metrics(JC) Level of support by the At-Large community
How long will it take to implement this plan?