You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 19 Next »

Issue 16 - Absence of consistent performance metrics

Work Track Leader(s)

Cheryl Langdon-Orr, Gisella Gruber



Issue 16 Work Track Members: Cheryl Langdon-Orr and Gisella Gruber to replenish Membership the Metrics Sub Committee and ARIWG will populate a  working group to conduct part of its reconvened activities the ARIWG specific and ARI metrics development and establishment/implementation of appropriate processes.

Issue 16 - Absence of consistent performance metrics

Final Proposal as approved by the Board

The ALAC has had a Metrics WG and an ALS Review Taskforce, both of which largely went into stasis during the IANA Transition and Accountability efforts. It is proposed to revive this activity to in part address aspects of the At-Large Review Implementation.

The ALAC notes that regional differences make it more difficult to have uniformity over participation metrics, but agrees that is an important target. The ALAC notes that collecting such statistics is a staff-intensive operation. ** Update ** see CRM tool referenced below.

Prioritization

1:1:2  (Low resource needs : Low risk : 2nd priority)

ARIWG comments

Metrics will be developed for each activity in which At-Large participants are involved in order to measure the effectiveness of our processes as well as the actual involvement of active participants who assist the ALAC to carry out its work within ICANN. Such evidence will not only provide transparency and accountability of the contribution made by At-Large with regards to their meaningful  contribution in support of the policy development work carried out by ICANN's supporting organisations, but also of the degree of effort and engagement of the many volunteers whose meaningful contribution to the work of At-Large adds value to the development of policy that is an essential part of the the work of ICANN. Metrics could also legitimize requests made by At-Large for increased funding support for regional activities where there is still a need for further outreach to educate those in underserved sub-regions about ICANN.

June 2019:

Since the initial publication of the FAIP and ARI documentation, the  ARIWG has been advised by ICANN Staff that the impending role out of a 'state of art' implementation of a CRM tool throughout ICANN.org should aid in reducing the staff resources required for the collection of many of the metrics envisaged to be useful or required for use by ALAC/At-Large.  Further, the timing of this rollout of the 'Salesforce' CRM tool for use with ALAC/At-Large to manage At-Large membership metrics is already expected to improve the future efficiency of the process as well as contribute to the Metrics activities outlined within our ARI, including but not limited to the selection of methodology for scoring identified performance metrics.  This is relevant to each activity in which At-Large participants are involved in order to measure the effectiveness of our processes as well as the actual involvement of active participants.

Status of improvement effort / staff lead

Reconvene the ALACs existing Subcommittee on Metrics in November 2018 and in early 2019 conduct a  review of work to date, as well as an exploration of next steps including liaison with work of other existing activities (as listed below) is planned for April/May 2019  / Gisella Gruber is staff lead.

Activities, if any, on which implementation is dependent, or that are dependent on implementation of this recommendation

** Update ** Since the initial publication of the FAIP and ARI documentation ARIWG has been advised by ICANN Staff that the impending role out of a 'state of art' implementation of a CRM tool throughout ICANN.org should aid in reducing the staff resources required for the collection of many of the metrics envisaged to be useful or required for use by ALAC/At-Large.  Further that the timing of this roll out of the 'Salesforce' CRM tool for use with ALAC / At-Large to manage At-Large membership metrics already expected to improve the efficiency of the process should be in the near future and so contribute to the Metrics activities outlined within our ARI, including but not limited to the selection of methodology for scoring identified performance metrics.

In addition:

  • The Technology Task Force should be helpful in developing appropriate tools to record assessments of different activities based on the type of metrics being collected for some measurement purpose.
  • The Stakeholder Analysis Tool will be able to make use of any regional or country based metrics we develop first through At-Large and then further throughout ICANN. An ongoing continuous improvement project in collaboration with the Global Stakeholder Engagement Group will be the possible automation of this tool to incorporate the information needs of both At-Large and GSE staff. 


Who will implement the recommendation: ICANN community, ICANN Board, ICANN Organization, other?

ALAC via its reconvened  Subcommittee on Metrics with assigned staff support.

Anticipated resource requirements (FTEs, tools)

  • Tool(s) for collecting each identified performance metric

  • Staff (where required) to assist in monitoring and collection of data relevant to each identified performance metric

  • Methodology for scoring each identified/collected performance metric

  • Procedure for dealing with changes in data collection, solution

We note that tracking substantive activity of a potentially large number of individuals (ALS Members and Unaffiliated Individuals) is a task that At-Large has struggled with for a long time. Resources outside of At-Large or ICANN may be needed to find an effective and implementable solution.

Expected budget implications

Unknown at this time.  Any Budget requests arising from FY19 Q3/4 will need to be considered in requests for the FY20/21 budget cycle (and beyond)

Proposed implementation steps:

  1. Metrics WG will propose and / or review all PERFORMANCE METRICS for ALAC Members, Liaisons and Appointments (beyond those specified in the Rules of Procedure); Regional Leadership; ALSes; ALS Representatives, ALS Members and unaffiliated individual members of RALOs

  2. EACH OF THE EIGHT ITEMS in this implementation plan is expected to provide appropriate metrics that will assess the achievement of the objective of the approved proposal

  3. EACH IF THE THREE STREAMS of At-Large activity (Operations, Policy and O&E) will establish a goal with measurable objectives which can evaluated at the end of the year to assess achievement of the workstream objectives during each year.

Metrics

  1. Achieve a Significant  Level of Support for the proposed Metrics  by the At-Large community by June/July 2019

  2. Establish ALAC / At-Large Metrics Project Plan and any initial Budget requests for consideration.

  3. Integration of any and all ARI specific Metrics associated with ARIWG Implementation of Recommendations, into a ARIWG Work Track of the Metrics Sub Committee ensuring that transparency in reporting occurs. 
  4. Socialisation and implementation of Metrics Project Plan from June/July 2020 through to Dec 2022

How long will it take to implement this plan?

Implementation of Metrics plan by end FY20;  Review and refinement of project by end FY21 and as required regarding CRM FY22


Issue 16 Progress Chart

The At-Large Review Implementation dashboard aims to update the ICANN Community about the progress of the work of the At-Large Review Implementation Working Group (ARIWG).

Activity

Status (or percentage of **SUB-TASK** completed)

2018-11-27

Reconvene the ALACs existing Subcommittee on Metrics (SC-AM)

Completed

2019-04-26

Draft Call for refreshed Membership and Observers of the ALAC Sub Committee on Metrics (SC-AM)

Final Drafting/ Approval for distribution (90%)

2019-05-23

Complete Issue Drafting for Initial Implementation Report

Report drafts should be ready by May 23 with sub-tasks completed  (100%) 

2019-06-07

Final Drafting of Initial Implementation Report  

Report to the Board - 23 June   (100%) 

2019-11-29

Complete Issue Drafting for Second Implementation Report

Report drafts should be ready by May 23 with sub-tasks completed  (100%) 

2019-12-13

Final Drafting of Second Implementation Report  

Report to the Board - 23 December  (100%) 

2020-01-21

Final Drafting of Second Implementation Report  

Aggregation of Data Capture tasks for Metrics-SC review for May 2020 (80%)

2020-04-10

ARIWG Dashboards review to update proposed Metrics relevant 

ARIWG Dashboards review to update proposed Metrics relevant (90%)

2020-04-28

Complete ARIWG specific Work Product Review for SC-AM 

Complete Issues Drafting for Final Report and Aggregation of Data Capture tasks for Metrics-SC review for May 2020 finalisation. 

2020-05-01

Final Drafting of Issues for  Final Report  

In Progress (70%) 

2020-05-01

Set up and manage the 2020-01 meeting of the reconvened SC-AM to finalise ARIWG Metrics for end 2020 initiation.

Pending (25-30%) 

2020-05-27

Final Prep, SMART criteria and dashboards prepared

Final prep and SMART criteria and dashboards report repositories (wiki) prepared for planned August roll-out of the CRM tool for use with ALAC/At-Large to manage At-Large membership metrics (98%)

2020-06-23

ARIWG Final Report 

ARIWG Completion (100%)

2020-09-01

start Data Capture program

Pending based on Final Report and Metrics-SC proposals and planning, both pre CRM, post CRM and enduring.   (0%) 

Implementation of Metrics plan

with Qtly Reporting via dashboard etc.,

2021-03-31

Review and refinement of Member and ALS Engagement and Participation Metrics Project.

Metrics-SC and Staff  action item to involve and engage survey and or sampling of Membership, ALSes, RALOs and ALAC (0%) 

2021-06-30

Jun 2021  Report on Review and refinement of project.

Report to (ALAC, Community and OEC?) and Next Steps for Continuous Improvement  (0%) 

start Data Capture program Review and Refinement

  • No labels