The ALAC Metrics Working Group has been working for several years on an appropriate performance metric model for ALAC members. 

The ALAC Rules of Procedure sets out not only what the ALAC does in ICANN, but what the expectations are of the ALAC members (Section 9).

As ALAC members, there is an expectation of leadership and active involvement in ALAC-driven activities.

The Metrics WG has been tasked with devising a mechanism whereby interested groups within (and outside of) the ICANN system can assess the work of the ALAC by way of the performance of its members. 

After our last Metrics WG meeting in Buenos Aires, the Working Group had discussions with regional community groups (the RALOS) to get some initial feedback,

A report on the findings of this consultation process is provided below, but we would like your comments on the assessment model provided at the end of this page.

Please comment in the box below if you can, or send an email to me giving your opinion about the self-evaluation model.

Your comments can be limited to LIKE or DON'T LIKE, or some other suggestion. We are still very open to ideas. .


1. Report on the Data Gathering from RALO ALSes - January 2014

Referring to Section 9 of the ALAC ROPS, the Working Group was tasked to:

  • Establish the key performance tasks of ALAC Members
  • Identify specific and quantifiable outputs
  • Establish the criteria against which these tasks could be rated or measured


A brief summary on feedback from the RALOs

Key performance tasks:  Section 9 of the ALAC ROPS, when categorised, detail:  

  • How ALAC members can gather and respond to information that will help them make important ALAC decisions (attendance & participation and contribution sections)
  • How they can participate in the formal decision-making process (voting section), and
  • Their accountability responsibilities to the ALAC and their RALO (reporting section).

The originally proposed metrics models on individual task outputs provided valuable feedback:

  • They were too specific and did not measure what was relevant to the RALO (EURALO)
  • Focused more on punishment for not doing it right rather than reward for work well done (NARALO)
  • Measuring attendance alone was not considered particularly meaningful (LACRALO)
  • Metrics did not appropriately measure participation or contribution (AFRALO)
  • Need to balance ordinary meetings with contributions the members make to working groups, as pen holders, or even in online and written conversations (AFRALO)
  • Measurement needs to be task driven (NARALO)

Problems were identified when relying on a statistical output for each task:

  • Difficult to contribute to wikis when the system did not allow for a comment to be posted (AFRALO & APRALO)
  • If restricted to measuring attendance alone, especially for online meetings, how could a statistic show that there had been 100% focus on the meeting - especially if there was no recorded response in the chat or audio (LACRALO)
  • Attending meetings is not a goal or objective therefore its measurement is irrelevant, similarly for participation and contribution (APRALO)
  • Acknowledge that ALSes are volunteers and many have day jobs and that commitment may vary depending on other personal responsibilities (APRALO)
  • Some ALSes are just not participating because they are already over-committed in their own lives - so that their responsibility as ALS members needs to be addressed individually. (APRALO)
  • Who keeps the metrics? (APRALO)


  • Active involvement and accountability could be more appropriately measured through actual LEADERSHIP of working groups activities and achieving results according to benchmarks or key performance indicators (APRALO)
  • Some indicators could include:
    • Input of time and commitment (eg number of working groups actively engaged in)
    • Leadership  responsibilities  (WG leader, penholder)
    • Mentoring skills (eg co-chairing to build capacity of other ALS members)
    • Team participation - collaborative skills
    • Other contributions…            (APRALO)
  • ALAC members and RALO Chairs should have joint responsibility for upskilling and engaging ALSes in ALAC activities (involving both outreach and inreach). RALO Chairs should plan for this engagement that is over and above the regular RALO update meetings (APRALO)
  • eDemocracy tools (NARALO)
  • ALAC members should provide a brief report to their RALOs at each RALO update meeting (APRALO)
  • If ALAC members are identified as underperforming, RALO Chairs should be the first person to approach them and discuss any problems. Referral to the ALAC Chair would be the next step for resolution (or dismissal) (APRALO)

2. Suggested Assessment Method: Self Evaluation by ALAC members

 (recorded on an individual wiki page - or a RALO page designated for their ALAC members)


ALAC MEMBER: NAME, XXRALO  (*meeting date given and linked to meeting record)



TASKS - 2014









Liaison reports, updates









Liaison reports, updates

16, 30







ccNSO Liaison

Coordinating committee; liaison reports








Member, penholder









Showcase OC, penholder

Brochure, powerpoint, liaison with ALSes, fans









WG updates in ccNSO reports








RALO liaison

Reports on RALO liaison contact (wiki)

























  • No labels


  1. Thanks for the summary of the METRICS Working Group report 

    #1 ICANN Online Reporting

    If you recall at our last two or three  ALAC meetings in person we were asked to do daily reports. Provide your brief summaries of the sessions you attended and some observations.  Unfortunately this was not emphasized at each meeting nor reminded to delegates via email to document their activities.  The end result was predicatable that very few people provided their information.  Why is this?  Were people too busy, Was it too  'evasive',  It's not my style etc  


    What was clear was that it measured only a fraction of people's activities hence it provided precious little metrics for accountability.  Good intentions  but bad delivery.

    #2 RALO Outreach

    A second example is the Atlarge Outreach efforts.  Lots  of brochures, information tables, Wine and Cheese events/showcase  We did an Outreach Table at  ICANN Toronto with alot of  ALS volunteers from NARALO which were all  organized by a Doodle planner

    A alot of interest in fact we distributed more literature about all the RALO's and answered tons of questions at the outreach.  The  number of new  ALS's recruited  was NONE.  Was the effort worth it?  Yes, the  message was sound but the closing style was missing.   


    Interesting we providing an opportunity for the less active, less high profile members to volunteer on an interpersonal level which is a metric that it's found in their attendance in working groups.   Some individuals  put in 8 to 10 hours of public interface time. How was their  time acknowledged?  It's not because  it's not in the  measurement tool.  

    # 3  Nomcom Outreach

    Another  example is the efforts to Outreach on behalf of NOMCOM  at various  venues and groups either in person or cross promotion.  

    Many volunteers actively promoted  individuals to send their SOI's to ICANN through personal networks at various events


    How do you measure this important effort?


    #  4 Survey Monkey 

    If you look closely at the Adobe Connect screen during the various sessions  a Survey Monkey Link was provided.


    Where is the results of these surveys?



  2. Feedback from EURALO: (7 Feb)

    Hi Maureen,

    thanks for the info. As I noted last time, I won't forward this to the EURALO list because Metrics were never an issue for our ALSes here (or just for a few insiders only!). On the other hand, I hear from various members that they feel "jaded" sometimes with too many ICANN - ALAC mails they can hardly "digest" and which are not always of their concern ...

    Therefore I was asked to concentrate forwards to the EURALO list on issues where I know they may be interested and be consulted.

    Thanks for your understanding and regards, Wolf

    >Hi Wolf (7 Feb)

    >I can totally appreciate what you are saying. There are so many working groups each asking for feedback that it can all contribute to "information overload".  But any feedback from anyone in EURALO and other RALOs would be gratefully received. Thank you. 


    Thanks for your understanding, Maureen. This time, we have no more comment than the one listed in the Wiki already:

    "They were too specific and did not measure what was relevant to the RALO (EURALO)".



  3. 2014-02-07 20:17 GMT-03:00 Silvia Vivanco <>:

    Dear Darlene and Fatima,

    Maureen has asked me to send you this link.  Please provide your  feedback and ideas.

    Kind regards,Silvia


    Dear Silvia, (8 Feb)

    Thanks for your email. Just to clarify, I have a question to Maureen.

    On what do you want to receive our feedback? About the Suggested Assessment Method: Self Evaluation by ALAC members?

    With pleasure I will collaborate with my opinion/ideas, I just like to make sure I understand correctly.

    Thanks in advance.

    Best Regards, Fatima


     Hi Fatima

    At the moment we are also working on an attendance metric, but this self-evaluation is an initial model based on the first lot of feedback we received. It would be great to get some comments back from LACRALO and any other ideas you might like to suggest.

    Regards,  Maureen


     Dear Maureen,

    In first place I would like to clarify one point. I never have problem to collaborate with any member of the community when they request my collaboration or when I consider my opinion/ideas could be useful. This is my "work style" and I will continue doing that.

    I am not member of Metrics WG ((although I personally have been following the progress of this WG)  so I don't understand what happen with the LACRALO members that are part of this WG and also with Sergio Salinas who offered to be liaison between LACRALO and the Metrics WG.

    Why did they never report about the advances of this WG or even about this Suggested Assessment Method? Should they ask for comments about that to LACRALO? Or this requirement for comments is only for ALAC members? [Update in this last point from your last email: I will be glad to ask for comments on this subject from LACRALO]

    I would appreciate your answer to understand this issue.

    Regarding the Suggested Assessment Method I would like to share with you some comments.

    First, if each ALAC member will complete this template, it can happen that someone is not really honest to complete it. Who will control what is being completed matches the reality?

    I'm not clear if it was consensus that each member ALAC must complete this table. If it was, from my point of view this is not a good idea. We must remember that this is a voluntary role, to which we are already carrying a lot of metrics. If on top of that they will force to complete this table monthly, is an unreasonable burden on volunteer work plus you face the risk that it does not conform to reality.

    Do you also going to have a metric for ALAC member who does not complete this table?

    Also in this table you are only entering the date on which the meeting had place, but you are not explaining what specific task the ALAC member conducted in that meeting. I understand that participation in the teleconference is only one criterion for measuring participation. And all previous activities performed an ALAC member before attending the call? For example, reading documents, organize the agenda, communicate privately with members of the WG to ask for comments or tell them that the call is going to ask for a report of his/her activities, etc. All this should not also be measured and included?

    In this table an item on "Tasks 2014" is included. I believe that this is not appropriate because the tasks in the working groups for example are not always the same throughout the whole year. In any case, they should be separated by WG in which they participate and within the specific tasks that develops in the WG per month. This would allow better monitoring of specific activities developed. Add a generic task for the whole year, from my point of view is no indicator to measure participation.

    Apologies for the somewhat jumbled ideas. I hope they are useful.

    I will let you know the comments sent by LACRALO.

    Best Regards,  Fatima


    2014-02-08 16:39 GMT-03:00 Maureen Hilyard <>:

    Hi Fatima

    The Metrics committee met on Thursday and for the first time discussed the self-evaluation model – which at this stage is only a suggestion based on the feedback we received during and after BA. On Friday I loaded it onto the discussion page and sent a message out to everyone via the Secretariats who are responsible for disseminating information to the RALOs, and on to the ALSes. There has been no intention of missing anyone out. I realise how busy people are and the many reasons they may have for not responding.  We would like every ALS to have an opportunity to contribute but realise that this is a RALO decision so that some have distributed the message to their ALSes, while others have not.

    The points you raise below are very useful feedback for us, and if you are interested I am sure you would be very welcome to contribute directly to the Metrics committee. The ALAC ROPS ask for metrics to measure ALAC performance and this has been the task of the WG. The problem is trying to find a balance between what needs to be measured and how.

    For me personally, I believe that for ALAC members, there is not only a strong reliance on trust but also on accountability whether we are volunteers or not. I have a lead role in several voluntary organisations in the Cook Islands and in the Pacific, and am responsible for regular monthly reporting, for which I receive no money or trips - unlike ICANN which pays enormous sums of money to enable its elected members and fellowship recipients to travel worldwide to attend its meetings. We have both benefited from these sponsorships. It is because ICANN is prepared to do this for me, that my payback is to be accountable for my elected role on the ALAC, and to contribute in whatever way I can to ALAC tasks. I view these obligations and responsibilities as doing what I think is right, not as a right.  As leaders in At-Large, ALAC members should demonstrate accountability to our ALSes if we want to encourage them to understand ALAC and to become more involved – especially in the consultation tasks. The table provides for openness and transparency but is only a suggested model. Other suggestions would be gratefully received.

    But thank you for your response – I am hoping it will encourage further discussion, and  I look forward to other responses from LACRALO and other RALO ALSes.

    Regards, Maureen


     Hi Maureen,

    Just to clarify: I am not saying that ALAC members should not have metrics. I consider we should comply with certain standards to honor our roles.

    I only say that ALAC members should not complete this table because of the reasons that I shared with you.

    Also if you all agree to the ALAC members should auto-complete this table, I made some comments to improve the table.

    Just to let you know, in LACRALO it has not distributed this request for comments by our Secretariat.

    I've been one of the few people who have permanently asked metrics in LACRALO. We don't have metrics yet, then these things that I have mentioned you before just happen. LACRALO members who are members of Metrics WG or who have committed to be liaison to the Metrics WG, do not report to the LACRALO or they don't met their commitments.

    Since I assumed my role as ALAC member requested to include in the agenda of the LACRALO monthly calls a report by ALAC members on activities taking place in ALAC and to explain what is going on in ALAC.

     That was out of our monthly agendas.

    I am also requesting permanently comments from LACRALO on the topics open to comment on ALAC, and carrying on to ALAC the LACRALO comments.

    I personally do not need metrics because I serve with great passion the role for which I was selected as representative from my region. However not everyone acts the same way.

    I have already sent an email to LACRALO requesting comments on this issue. I'll let you know the answers.

    Best Regards,  Fatima

  4. also.. some feedback from Garth encouraging NARALO about the value of reporting to each other as well as to the ICANN community about their region's activities. This would be a call to all members of NARALO - including ALAC members

    From: Garth Bruen []

    Subject: A Note on NARALO Monthly Reports

    Hello NARALOers (-ites),

    A comment was made on last Monday's call that the monthly reports do not reflect all the work the region does. This is something I am aggressively trying to correct! The reports are for everyone, it is how we inform the ICANN community about what we are doing and I invite you all to submit reports or edit the report page directly. Everyone in NARALO should have access to edit the report page, if not tell me or At-Large staff.

    The current year is located here:

    Reports are posted monthly, you will note that there is an outline for the current month and no content. Scroll down to January to see how the previous month was filled-in. I have restructured the report format to reflect the format of our call agenda - this will help us track our goals and success of our efforts over time.

    There are four areas where you can add reports:

    * Meetings and Events Reports: If you have attended ANY Internet policy meetings in person or through teleconference (other than standard NARALO calls) enter it here. Provide a link to the event if possible.

    * Outreach Activities: Note any outreach/recruitment activities here or updates to activities. In general the status of these efforts will also be included here: , but be sure to highlight important actions you want everyone nt+to be aware of.

    * Community Activities: This is work in your OWN area or ALS. If your ALS already publishes reports, just put the link in here. If the report location is the same month after month, just note that and I will keep copying the link in successive months.

    * Committee/Working Group Reports: This is chartered work within ICANN. Many of these groups require reporting. Are you a member or chair of a group? Put the link to their reports here or comment on notable events.

    To update the page use the "Edit" button at the top. Do your edits quickly as others may try to be editing too. Put your initials next to anything you have added and click "Save" at the bottom to close it. If you leave it open too long it will probably time out and disconnect you and you will lose any changes. If you need help with editing the Confluence Wiki pages contact me, Glenn or At-Large staff. You can also email me or Glenn any reports (brief paragraphs or links preferred!).



  5. 17 Feb 2014 - Dev, Gisella, Nathalie and I met today to talk about HOW we might look at measuring ALAC participation.

    • Garth's is proposing a model for reporting and participation by NARALO ALSes (attached above). I know that NARALO have a metric that looks at participation levels of ALSes based on attendance of NARALO meetings but also including  participation on WGs and public comment feedback on community issues. 
    • Dev said that LACRALO are doing something similar with their ALSes (Fatima mentioned this as well), so that they will actually bar non-participating ALSes from voting or even look at recommending their deregistration from the ICANN community

    • With regards to feedback about the self-evaluating table - Glen evaluated metrics for other situations outside of our current brief, and Fatima provided some pertinent comments . Dev and I decided to recommend - as an initial model of metrics for ALAC
    1. We could trial a basic metric model at Singapore
    2. We spoke about the pros and cons of STAFF having to do the metrics role as well as all the other tasks they have to do before, during and after
    3. We discussed whether the RALO SECRETARIAT STAFF should take on the responsibility for the metrics for their three ALAC members.  - for example, logging the attendance at all ALAC meetings at F2F ICANN and online meetings. 
    4. Dev will devise a basic attendance register that just requires a tap on a spreadsheet type application at the beginning of each ALAC meeting. Dev suggested that there could be a metric for online ALAC meetings for participation in Chats, audio interventions, and inclusion in action items
    5. We discussed the expansion of this for Working groups as well, but that will be AFTER Singapore
    6. For personal accountability as an ALAC member (and the responses or lack thereof regarding the table indicates that that hasn't met with any approval - although I am not surprised) after today's discussion, I am recommending an adaptation of Olivier's suggestion of each ALAC member having their own WIKI page  - not unlike the liaison pages that require a monthly report - but this would be linked to their RALO workspaces, Their monthly reports could record updates in their involvement in ALAC-related Working Groups. 
    7. They won't need to mention the ALAC meetings because a metric will already have been created for that by Dev (to be monitored by the RALO Secretariat who would report on this in their own monthly reports). 
    8. Conclusion: for Singapore, we are recommending two types of measures 
      1. one of attendance ot ALAC meetings (online and F2F) - Dev to organise spreadsheet for RALO Secs
      2. one of personal accountability by ALAC members - a brief monthly report - to their RALOs so can be in whatever language