The ALAC Metrics Working Group has been working for several years on an appropriate performance metric model for ALAC members.
The ALAC Rules of Procedure sets out not only what the ALAC does in ICANN, but what the expectations are of the ALAC members (Section 9).
As ALAC members, there is an expectation of leadership and active involvement in ALAC-driven activities.
The Metrics WG has been tasked with devising a mechanism whereby interested groups within (and outside of) the ICANN system can assess the work of the ALAC by way of the performance of its members.
After our last Metrics WG meeting in Buenos Aires, the Working Group had discussions with regional community groups (the RALOS) to get some initial feedback,
A report on the findings of this consultation process, plus an assessment method has been suggested, are included below and await your comments. .
Report on data gathering for the ALAC METRICS working group from RALO ALSes January 2014
Referring to Section 9 of the ALAC ROPS, the Working Group was tasked to:
- Establish the key performance tasks of ALAC Members
- Identify specific and quantifiable outputs
- Establish the criteria against which these tasks could be rated or measured
**********************************************************************************
A brief summary on feedback from the RALOs
Key performance tasks: Section 9 of the ALAC ROPS, when categorised, detail:
- How ALAC members can gather and respond to information that will help them make important ALAC decisions (attendance & participation and contribution sections)
- How they can participate in the formal decision-making process (voting section), and
- Their accountability responsibilities to the ALAC and their RALO (reporting section).
The originally proposed metrics models on individual task outputs provided valuable feedback:
- They were too specific and did not measure what was relevant to the RALO (EURALO)
- Focused more on punishment for not doing it right rather than reward for work well done (NARALO)
- Measuring attendance alone was not considered particularly meaningful (LACRALO)
- Metrics did not appropriately measure participation or contribution (AFRALO)
- Need to balance ordinary meetings with contributions the members make to working groups, as pen holders, or even in online and written conversations (AFRALO)
- Measurement needs to be task driven (NARALO)
Problems were identified when relying on a statistical output for each task:
- Difficult to contribute to wikis when the system did not allow for a comment to be posted (AFRALO & APRALO)
- If restricted to measuring attendance alone, especially for online meetings, how could a statistic show that there had been 100% focus on the meeting - especially if there was no recorded response in the chat or audio (LACRALO)
- Attending meetings is not a goal or objective therefore its measurement is irrelevant, similarly for participation and contribution (APRALO)
- Acknowledge that ALSes are volunteers and many have day jobs and that commitment may vary depending on other personal responsibilities (APRALO)
- Some ALSes are just not participating because they are already over-committed in their own lives - so that their responsibility as ALS members needs to be addressed individually. (APRALO)
- Who keeps the metrics? (APRALO)
Recommendations
- Active involvement and accountability could be more appropriately measured through actual LEADERSHIP of working groups activities and achieving results according to benchmarks or key performance indicators (APRALO)
- Some indicators could include:
- Input of time and commitment (eg number of working groups actively engaged in)
- Leadership responsibilities (WG leader, penholder)
- Mentoring skills (eg co-chairing to build capacity of other ALS members)
- Team participation - collaborative skills
- Other contributions… (APRALO)
- ALAC members and RALO Chairs should have joint responsibility for upskilling and engaging ALSes in ALAC activities (involving both outreach and inreach). RALO Chairs should plan for this engagement that is over and above the regular RALO update meetings (APRALO)
- eDemocracy tools (NARALO)
- ALAC members should provide a brief report to their RALOs at each RALO update meeting (APRALO)
- If ALAC members are identified as underperforming, RALO Chairs should be the first person to approach them and discuss any problems. Referral to the ALAC Chair would be the next step for resolution (or dismissal) (APRALO)
2. FROM THIS SUMMARY, THE WORKING GROUP IS IN THE PROCESS OF DEVISING RECOMMENDATIONS THAT COULD INCORPORATE SOME OF THE FEEDBACK WE RECEIVED FROM OUR RALOS
A) Each RALO could have its own wiki page on which ALAC members could write up a monthly report on his/her activities on the ALAC and other ALAC-related working groups.
B) Each ALAC member could have their own template on which they could record the groups they are involved in as well as the positions and tasks that they have been assigned.
