Issue 16 Work Track Leader(s): Cheryl Langdon-Orr and Gisella Gruber to restart working group on Metrics.


Issue 16 Work Track Members: Cheryl Langdon-Orr and Gisella Gruber to restart working group on Metrics this Sub Committee will conduct as part of its reconvened activities the ARIWG specific and ARI metrics development and establishment/implementation of appropriate processes.

Issue 16 - Absence of consistent performance metrics

Final Proposal as approved by the Board

The ALAC has had a Metrics WG and an ALS Review Taskforce, both of which largely went into stasis during the IANA Transition and Accountability efforts. It is proposed to revive this activity to in part address aspects of the At-Large Review Implementation.

The ALAC notes that regional differences make it more difficult to have uniformity over participation metrics, but agrees that is an important target. The ALAC notes that collecting such statistics is a staff-intensive operation. ** Update ** see CRM tool referenced below.

Prioritization

1:1:2  (Low resource needs : Low risk : 2nd priority)

ARIWG comments

Metrics will be developed for each activity in which At-Large participants are involved in order to measure the effectiveness of our processes as well as the actual involvement of active participants who assist the ALAC to carry out its work within ICANN. Such evidence will not only provide transparency and accountability of the contribution made by At-Large with regards to their meaningful  contribution in support of the policy development work carried out by ICANN's supporting organisations, but also of the degree of effort and engagement of the many volunteers whose meaningful contribution to the work of At-Large adds value to the development of policy that is an essential part of the the work of ICANN. Metrics could also legitimize requests made by At-Large for increased funding support for regional activities where there is still a need for further outreach to educate those in underserved sub-regions about ICANN.

June 2019:

Since the initial publication of the FAIP and ARI documentation, the  ARIWG has been advised by ICANN Staff that the impending role out of a 'state of art' implementation of a CRM tool throughout ICANN.org should aid in reducing the staff resources required for the collection of many of the metrics envisaged to be useful or required for use by ALAC/At-Large.  Further, the timing of this rollout of the 'Salesforce' CRM tool for use with ALAC/At-Large to manage At-Large membership metrics is already expected to improve the future efficiency of the process as well as contribute to the Metrics activities outlined within our ARI, including but not limited to the selection of methodology for scoring identified performance metrics.  This is relevant to each activity in which At-Large participants are involved in order to measure the effectiveness of our processes as well as the actual involvement of active participants.

Status of improvement effort / staff lead

Reconvene the ALACs existing Subcommittee on Metrics in November 2018 and in early 2019 conduct a  review of work to date, as well as an exploration of next steps including liaison with work of other existing activities (as listed below) is planned for April/May 2019  / Gisella Gruber is staff lead.

Activities, if any, on which implementation is dependent, or that are dependent on implementation of this recommendation

** Update ** Since the initial publication of the FAIP and ARI documentation ARIWG has been advised by ICANN Staff that the impending role out of a 'state of art' implementation of a CRM tool throughout ICANN.org should aid in reducing the staff resources required for the collection of many of the metrics envisaged to be useful or required for use by ALAC/At-Large.  Further that the timing of this roll out of the 'Salesforce' CRM tool for use with ALAC / At-Large to manage At-Large membership metrics already expected to improve the efficiency of the process should be in the near future and so contribute to the Metrics activities outlined within our ARI, including but not limited to the selection of methodology for scoring identified performance metrics.

In addition:

  • The Technology Task Force should be helpful in developing appropriate tools to record assessments of different activities based on the type of metrics being collected for some measurement purpose.
  • The Stakeholder Analysis Tool will be able to make use of any regional or country based metrics we develop first through At-Large and then further throughout ICANN. An ongoing continuous improvement project in collaboration with the Global Stakeholder Engagement Group will be the possible automation of this tool to incorporate the information needs of both At-Large and GSE staff. 


Who will implement the recommendation: ICANN community, ICANN Board, ICANN Organization, other?

ALAC via its reconvened  Subcommittee on Metrics with assigned staff support.

Anticipated resource requirements (FTEs, tools)

  • Tool(s) for collecting each identified performance metric

  • Staff (where required) to assist in monitoring and collection of data relevant to each identified performance metric

  • Methodology for scoring each identified/collected performance metric

  • Procedure for dealing with changes in data collection, solution

We note that tracking substantive activity of a potentially large number of individuals (ALS Members and Unaffiliated Individuals) is a task that At-Large has struggled with for a long time. Resources outside of At-Large or ICANN may be needed to find an effective and implementable solution.

Expected budget implications

Unknown at this time.  Any Budget requests arising from FY19 Q3/4 will need to be considered in requests for the FY20/21 budget cycle (and beyond)

Proposed implementation steps:

  1. Metrics WG will propose  performance metrics for ALAC Members, Liaisons and Appointments (beyond those specified in the Rules of Procedure); Regional Leadership; ALSes; ALS Representatives, ALS Members and unaffiliated individual members of RALOs

  2. Each of the items in this implementation plan is expected to provide appropriate metrics that will assess the achievement of the objective of the approved proposal

  3. Each of the three streams of At-Large activity (Organisational, Policy and O&E) will establish a goal with measurable objectives which can evaluated at the end of the year to assess achievement of the workstream objectives during each year.

Metrics

  1. Achieve a Significant  Level of Support for the proposed Metrics  by the At-Large community by June/July 2019

  2. Establish ALAC / At-Large Metrics Project Plan and any initial Budget requests for consideration.

  3. Integration of any and all ARI specific Metrics associated with ARIWG Implementation of Recommendations, into a ARIWG Work Track of the Metrics Sub Committee ensuring that transparency in reporting occurs. 
  4. Socialisation and implementation of Metrics Project Plan from Dec 2019 through to June/July 2020

How long will it take to implement this plan?

Implementation of Metrics plan by end FY20;  Review and refinement of project by end FY21


Issue 16 Progress Chart

NOTE: the example Dashboard Report from WS2 of ICANN Accountability CCWG is an exemplar of the TYPE of dashboard/reporting that we recommend be utilised for reporting ARIWG progress to the ICANN Community as well as for inclusion in aggregated terms to the regular reporting /update to the OEC biannually.

The At-Large Review Implementation dashboard aims to update the ICANN Community about the progress of the work of the At-Large Review Implementation Working Group (ARIWG).

Date

Name

Status (or percentage completed)




 

Reconvene the ALACs existing Subcommittee on Metrics (SC-AM)Completed

 

Draft Call for refreshed Membership and Observers of the ALAC Sub Committee on Metrics (SC-AM)Final Drafting/ Approval for distribution (90%)

  

Set up and manage the 2019-01 meeting of the reconvened SC-AMPending (5-10%) 

 

Undertake ARIWG specific Work Product Review for SC-AMPending (5-10%) 

  

Complete Drafting for Initial Implementation Report

Report drafts should be ready by May 23 with sub-tasks completed

 

Final Drafting of Initial Implementation Report  

Report due to the Board - 23 June

 

Jul 2019 ARIWG Dashboard(0%)

 

Aug 2019 ARIWG Dashboard(0%)

 

Sep 2019 ARIWG Dashboard(0%)

 

Oct 2019 ARIWG Dashboard(0%)

  

Nov 2019 ARIWG Dashboard(0%)

  

Dec 2019 ARIWG Dashboard(0%)

  

Jan 2020 ARIWG Dashboard(0%)
  • No labels