ALAC Metrics - Discussion


WHAT ARE METRICS AND WHY ARE WE WORKING ON METRICS?

Very basically, Metrics are measures which assess how well organisational members are performing - meeting targetted expectations and giving value. 


The ALAC Rules of Procedure Section 9 is entitled "Performance, Metrics and Remediation"

This section states that the performance of the ALAC is measured by its ability to represent the interest of Internet users, and depends on strong participation from all ICANN regions. 

 Furthermore, the ALAC can function effectively only if all ALAC Members and Appointees meet the obligations of their positions. 

 Satisfactory performance is a complex concept including both objective and subjective issues and must factor the very significant personal contributions made by At-Large volunteers.

 Moreover, the ALAC and At-Large receive significant funding from ICANN, both for travel and other activities, and the ALAC must be able to justify such expense.


There are three key things we have to do to get started on establishing appropriate metrics:


1.Establish what is expected of an ALAC member - key performance tasks

2. Identify what the outputs of these tasks should (specific and quantifiable)

3. Establish the criteria against which the results of these tasks can be rated (these are the measures)


FOR EXAMPLE

ALAC: Definition of the criteria of involvement and participation:


  1. Attendance: 
    1.1  Attend ALAC conference calls and any other virtual meeting

1.2  Attend face to  face ALAC meetings  held during an ICANN  meeting

1.3  Attend monthly RALO meetings

1.4  Attend RALO and RALO-related meetings

1.5  Apologies  given before a meeting when not available to attend


2. Participation and Contribution:

2.1 Participate and contribute to ALAC discussions relating to ICANN policy development through:

2.1.1  mailing lists 

2.1.2  wiki pages

2.1.3  virtual meetings

2.1.4  face to face meetings

2.2  Take a lead role in  ALAC-initiated working groups

2.3  Participate and contribute to ALAC-initiated working groups

2.4  Take a lead role (eg pen-holder) in working groups of other bodies within ICANN

2.5  Participate and contribute to  the role of ALAC in working groups  of  other bodies within ICANN

2.6  Participate and contribute to RALO and RALO-related meetings


3. Voting: 

3.1 Participate in the decisions and votes by the ALAC on ICANN policy issues


4. Reporting:

4.1  Report  to the ALAC on ALAC -initiated working groups or other ICANN activities

4.2  Report  to the ALAC on any liaison roles  related to other bodies within ICANN

4.3  Report  to the ALAC on any working group activities

4.4  Report  to the RALO on ALAC decisions and involvements

 

ALAC: Metrics for involvement and participation


5. Attendance

5.1  Attended  “#” monthly conference call meetings

5.2  Attended  "#" working group meetings

5.3  Attended  "#" other ICANN body meetings

5.4 Attended "#" RALO-related meetings

5.5  Apologies given for "#" meetings  that was  unable to attend

5.6  No apologies given for "#" meeting  that did not attend


6. Participation and contribution

6.1  Participated and contributed to “#” monthly conference call meetings  (by audio or chat)

6.2  Participated and contributed to "#" working group meetings   (by audio or chat)

6.3  Participated and contributed to  "#" other ICANN body meetings  (by audio or chat)

6.4  Did not participate or contribute to "#" meetings that attended


7. Voting

7.1  Voted on "#" formal ICANN decisions proposed by ALAC

7.2  Did not vote on "#" formal ALAC decisions


8.  Reporting

8.1  "#" reports provided to ALAC meetings

8.2  "#" reports provided to RALO meetings


9. Definition of the ALAC status:

9.1  Active               60%+

9.2  Less active       40%-60%

9.3  Not Active        0-40%


Update of ALS and RALO comments to hand, regarding ALAC metrics. 


Wolf Ludwig (11/12/13) after MH request to raise ALAC metrics issue with ALSes at next EURALO meeting 

I have seen your comment or “invitation”, Maureen but didn't put it on our call agenda because a) it was extensive enough already and b) there won't be any comments from our members – the subject is far too specific! EURALO defined subjects of key priorities for the region last June at our GA. This also means that we would like to concentrate on these issues! Therefore, I won’t push our members to submit a comment on whatever subject what has less relevance for us/them. I hope you will understand my point.


Olivier Crepin-Leblond (06/12/13)

Hello all,

you might be interested in seeing some of the GNSO metrics on attendance...

https://community.icann.org/display/gnsocouncilmeetings/GNSO+Council+Attendance-20+Nov+2013

Of interests is the actual implementation of the updating which is described on:

https://community.icann.org/display/gnsocouncilmeetings/Attendance+Logs

Kind regards,

 

Glenn McKnight (03/12/13)

Hi ..At the metrics  meeting on the Sunday  I tried to bring up the issue of  punishment vs  reward. 

It didn't seem to take on great interest and it was dismissed 

For the record,  this onus on punishment which has been central to US  jurisprudence  has been reconsidered  

Here is an article  

http://www.nytimes.com/2013/12/03/education/seeing-the-toll-schools-revisit-zero-tolerance.html?hp&_r=0

 

Ali AlMeshal (01/12/13)

Thank you Tijani for your input. I can see that in principle you are agreeing with me in some of the below if it is not all. To comment on your feedback please allow me to crystallize my below views. 

If you may notice that I have listed my points below in logical sequence , staring from what is on the table as a points to what do I see as a proposal. 

My first point: 

                 -             “Attending meetings by itself is not a goal or objective therefore it cannot be part of the measuring criteria independently.”

            -           Also participation and contribution to the meeting discussion cannot be evaluated as a performance measure                                  

 Therefore to make it part of the criterion I suggested this 

                 -      ALAC members should be assigned to Chair a WG’s

§ With this type of assignment the member will logically be accountable and responsible of a set deliverables that he/she have to achieve, and this can be measured and evaluated for the following set of KPI’s

  • · Time to deliver
  • · Commitment
  • · Team working
  • · Developing others (member of the group)
  • · Others 

                      So by doing the above we are fulfilling the attending and contribution part of the criteria , but the major different hear is that the member is not attending or participating for the sack of showing up or to record his attendees only , No he is attending to update , discuss and participate and give feedback on action item that he is in-charge of and so on. 

My Second point:

-          ALAC members and chair of RALO’s jointly should be responsible for engaging the ALS’s in work and activities related to the rejoin at least and should provide a monthly call report on this.

§ This is different activity from the RALO monthly call; this should be done separately to reach out the ALS’s through a pre-plan agenda and time frame.

§ The objective is to get in touch with ALS’s members outside the official call and tries to understand their needs and requirements and also will be a good tool maybe to get them engage in if they are not active. So this is more of OUTREACH on a small scale and more of direct communication. 

  • On the assumption that this point is clear, then for the parameter, what I have thought about for the time being is the Call Report that need to be presented in the monthly call meeting. The call report for example should look like   

Date

ALS Name

Contact Person

Comments and feedback

12/12/12

Bahrain Internet Society

Ali AlMeshal

1 –

2 –

3 -

01/01/13

xxxxx

xxxxx

 

Should you or any other have more thoughts of this then it would be very much helpful to enrich this fruitful discussion.

Hope I manage to answer and clarify your enquiries and doubts .

Thanks

 

From Tijani Benjemaa (30/11/13)

Dear Ali,

Thank you very much for your comments

  • Attending meetings by itself is not a goal or objective therefore it can not be part of the measuring criteria independently.

Ali, ALAC can’t decide on anything without quorum, and members who don’t attend the ALAC meetings may break its functioning. Also, I may agree with you that attending is not a goal and shouldn’t be part of the criteria, but the attendance is absolutely necessary for any contribution during the meeting/call. It’s the elementary duty for the ALAC members. Attendance is proposed as one of the metrics element  

  • Also participation and contribution to the meeting discussion can not be evaluated as a performance measure

If the participation and contribution to the ALAC meetings is not considered as a performance criterion, that means that all the ALAC members can decide not meet at all and they can be considered as having good performance.

  Thereafter to have a process in place that gives an indication of the member performance I would suggest the following
o   ALAC members should be assigned to Chair a WG’s

This is another criterion listed in the proposal

o    ALAC members and chair of RALO’s jointly should be responsible for engaging the ALS’s in work and activities related to the rejoin at least and should provide a monthly call report on this.

What are the parameters of measurement you propose for this?

Thank you again for your contribution

 

Fouad Bajwa (28/11/13)

I believe one of the most important metrics component should be built on the fact that if a member suggests something, how many of those suggestions were actually accepted and incorporated into the system. One small example is that I've proposed individual membership in the past during 2010, created the paperwork suggesting the policy reform in the APRALO articles, presented it and then silence. This would have enhanced participation, inclusion of more productive talent and people that are sensitive about ICANN and IG related issues. Such talent thus has to move under a narrower scope into the ncuc/ncsg and APRALO loses out.

Somehow the belief that only groups can represent the rights of the users is a fantasy. For example, in the technical community and most in ISOC circles, most of the members in a recognized ALS are not actually CS or information rights activists but people with corporate day jobs and of companies that actually should fall under contracted parties ac/oc's in ICANN. The value that comes into such an organizational system are actually the members, the system itself cannot come up with ideas and transform the ideas into workable actions.

Participation collapses when member ideas are not viewed as valuable input and organizations that don't believe in agility to change and adapt to round the clock innovation and improvements basically fall short on rationality for existence and support in the future. If the participation in meetings and tele-cons are an evaluative criteria for calculating value for money, something is seriously wrong with ICANN and the people who are attempting to address such issues by adopting such a weak and lame course of action. 

 KPIs are measured against activities and participation in telecons and meetings are not such an input or expected outcome. The indicators can be established on a the basis of number of members present and number of comments, suggestions and recommendations received during any calls/meetings and the actions were taken on them. The second level would be number of policy comments requested by ICANN and the number of members that voted on them, the time taken to make comments or recommendations, total number of policy requests for comments and number of comments made to policy requests. 

Its actually a very simple and straightforward thing. List down all the things that ALAC and its members do. Against those activities a scorecard can be developed and bench-marked against other AC/OC work. I'll think more about this.

 

Rinalia Abdul Rahim (27/11/13)

Hello everyone.

I like Ali's input on metrics about leadership of Working Groups for ALAC members. I would also suggest co-chairing of WG between an ALAC Member and an At-Large community member. This will help build collaborative skills and capacity along the way.

Best regards, Rinalia

 

Ali Al Meshal (27/11/13)

Good day Maureen,

   First of all we have always to but in mind as your correctly stated that this is a volunteer work from the members, but on the other hand I do agree that up to certain extent there should be a fair KPI’s in place to evaluate the performance of ALAC members.

Also I am quite sure that selected or nominated ALAC members for these leadership positions were based on their knowledge , commitment and experience as well as high performance otherwise they will not be part of the team.

   

   Hence measuring criteria would always be much clearer and effective if it is set for Quantitative factors rather than Qualitative once.

So having said that then I would like to address the following:

 o   Attending meetings by it self is not a goal or objective therefore it can not be part of the measuring criteria independently.

o   Also participation and contribution to the meeting discussion can not be evaluated as a performance measure

 Thereafter to have a process in place that gives an indication of the member performance I would suggest the following

o   ALAC members should be assigned to Chair a WG’s

§  With this type of assignment the member will logically be accountable and responsible of a set deliverables  that he/she have to achieve, and this can be measured and evaluated for the following set of KPI’s

  • Time to deliver
  • Commitment
  • Team working
  • Developing others (member of the group)
  • Others

o    ALAC members and chair of RALO’s jointly should be responsible for engaging the ALS’s in work and activities related to the rejoin at least and should provide a monthly call report on this.

§  This is different activity from the RALO monthly call; this should be done separately to reach out the ALS’s through a pre-plan agenda and time frame.

§   The objective is to get in touch with ALS’s members outside the official call and tries to understand their needs and requirements and also will be a good tool maybe to get them engage in if they are not active. So this is more of OUTREACH on a small scale and more of direct communication.                     

….Should you need any clarification then please let me know by email or a call, I will be more than happy to discuss.

 

From Karaitiana Taiuru (25/11/13)

These are my thoughts based on not been an ALAC member (nor do I have time to).

I would support a more closer monitoring approach to the ALAC performance review and would like to see the evaluations distributed to the relevant RALO's as we are the ones who nominate and put our trust in our representatives to provide our views and then to report back to us.

I would expect that participation in all meetings, emails and Work Groups would be at the very minimum 90%. There are so many issues at present it would be hard not to have a voice at a meeting.

Saying this, I would also expect that any new ALAC appointments are mentored and giving an appropriate transition period and were made to feel comfortable to ask for help.  Some cultural and language barriers may also exist and should be considered.

If people are under-performing, then perhaps a mechanism of someone talking to them in a non threating manner to see if there is any assistance required etc.

Perhaps too, some way of recognition for the long hours and dedication may also be a motivation. This is likely to be more of a RALO initiative though.

 

From Winthrop Yu (24/11/13)

   I'd tried to find people here (even outside ISOC-PH) who could actively and constructively participate in the various WGs.  Unfortunately, no takers so far, and i really don't like the idea of simply "tapping" someone who may not be as committed to the work or drop the ball.  For myself, i'm already committed to policy issues on the local front and cannot honestly make any further commitments until ... oh, after December 2014? :)

 

Comments are on the workspace at 

  • No labels

10 Comments

  1. First- a terrific list and I think they are pretty much the metrics we are looking for.

    A minor point - in 1.1 - omit the term 'or any other virtual meeting' since it is not clear what is meant.

    On percentages, since there are 12 monthly meetings of both RALO and ALAC meetings, I'd suggest that 10 out of 12 is what is aimed for.  And that is actual attendance.  I'm not sure how to handle apologies since I'd hate to think that all one has to do is apologise and it counts - but maybe a separate metric for attendance with apology and attendance without apology. (and maybe something special for the APAC region where we are talking about meetings at 1.00am or 2.00am!)

    And maybe a separate metric for participation in non-ALAC ICANN working groups?

    And a question - who keeps the metrics?  It's a bit of work for ICANN staff who are already busy so it may be making sure that the metrics are easy to collect and put onto a matrix to quickly show who is active - or not.

    1. Yes.. exactly.. .I think the wording for each of these points is very important which is why we need to RALO inputs..  I have probably put in much more than we need, and then missed out something important, but the feedback from RALOS will hopefully tell me this

      • I adapted the  "any other virtual meeting" from the ALS metrics recommendation "Meeting: attend RALO conference calls and any other virtual or face to face meeting"  - so whatever is used, it has to be clear as to what is meant, or don't include it
      • What constitutes active participation? 85% attendance? one or two comments in a meeting?
      • And should apologies count? I think "not giving apologies" should definitely be noted.. How they are dealt with are not our problem at this time.
      • Separate metrics for ALAC and non-ALAC participation eg perhaps 6.2 = metric for ALAC working group participation; 6.3 = metric for non-ALAC working group participation
      • who keeps the metrics? another good question - who has kept them in the past?
      • Perhaps it could be the role of the ALAC member him/herself to record attendance and participation information on a specially prepared matrix, and staff could attach their minutes and chat record of the meeting to this record - with minutes perhaps requiring new information about participation (but what kind of information would that be?)
      • The development of the matrix itself would be an important task for this record-keeping role  It would be helpful if it was technologically constituted so that the same system could also churn out a meaningful report each month that the Secretariats could present to their RALO meetings (smile)
         

      Thank you for this response Ms APRALO Chair.  You have highlighted some important issues.  I hope the other RALOs are as responsive. (smile)

      Someone has made a start on the ALS metrics which I would be interested in getting some feedback on  (after the ALAC one has been thrashed about). 

  2. Emailed comments:

    Sent: Tuesday, 5 November 2013 12:48 p.m.

    Dear Maureen,  

    I’m so sorry for the late reply. I read your proposal, and find it good.

    I already proposed more than one year ago a similar proposal, and I updated it today and posted it on the wiki.

    We will need now to concentrate on compiling all the proposed texts for a comprehensive and precise one that we walk through to enrich it till we reach consensus on a final version.

    Tijani

    **********************************************************************

    Hi Tijani

    I completely agree and thank you for your proposal. We need to get different models up that people can see the issue from different perspectives and hopefully understand the complexities. From their discussions we will hopefully achieve a text that clearly explains the process, what is required and how it will be.  I know that it is not going to be an easy task.

    I am looking forward to meeting up with everyone in BA. J

    Regards  Maureen

    ***********************************************************************

    Dear Maureen,

    Bravo for the excellent draft.  We might need to explain more the definition of ALAC status. Will it be the same to just be present at meetings than be present and participate?  The answer is NO but let me ask more questions.  Does participate mean having something to say even if that is just "I agree"  ? Should participants need to say something any how to be considered? Does significant attendance mean you attend X meetings + at least 2X comments?

    I just want to better understand the logic.

    Thank you,  Fatimata Seye Sylla

    ***********************************************************************

    Sent: Tuesday, 5 November 2013 1:01 p.m

    Hello Fatimata

    You highlight a real problem that I foresee.. how does one define “participation” and “contribution”?

    I think about the number of times when my contribution at a meeting has been “+1” because someone much more articulate than I has said exactly what I wanted to say.

    We also have to balance ordinary meetings with contributions that members make to working groups, as pen-holders or even if just to the online and written conversations which are much more interactive in a working group role.

    Perhaps rather than measures, we should be establishing benchmarks which can look at all types of involvement (360 degrees) and which might help to identify  “active” or “not so active” involvement in a much more friendly way.

    These are all the things we have got to nut out. And no doubt we will, in Buenos Aires. Looking forward to seeing you there.

    Regards, Maureen

    ***********************************************************************

     Sent: Tuesday, 5 November 2013 1:01 p.m

    I support Maureen position because it is often not necessary to say the same thing when there was a contribution that takes into account what is meant.

    I propose that recapitulatif be done regularly after a certain stage in order to allow each other to stay in constant even as the overall dynamics.

    Furthermore, I would like to know what happens in the space of wiki work. I still can not get to post my comment

    SCHOMBE BAUDOUIN

    ************************************************************************

    about points 1,4,5,6 and 7:

    -What are those directly and indirectly involved?

    -Als can participate in votes of ALAC?

    These precisions are needed to understand the level of participation of Als in the activities of ALAC and RALO.

    SCHOMBE BAUDOUIN

    ************************************************************************

    2013/11/5 Maureen Hilyard <hilyard@oyster.net.ck>

    Hi Baudouiin

    Are you referring to the points  in my discussion paper on

    https://community.icann.org/display/atlarge/ALAC+Could+do+workspace+-+ALAC+Metrics+-+discussion

    The ALSes will have their own metrics – separate from that of the ALAC members.

    Maureen

    *************************************************************************

    Yes Maureen, I just wanted to remove the shadow of doubt in my mind and thank you for this precision.

    SCHOMBE BAUDOUIN

  3. RALO comments - gathered during ICANN week

    NARALO 

    • Measurement needs to be task driven - when a task is set and roles and outcomes are established, then assessment should be carried out on whether the outcomes have been achieved
    • one member has access to e-Democracy Tools which could be adapted for our purpose


    LACRALO - Sergio offered in the Regional Leaders meeting to be a liaison person for LACRALO

    Contributor #1

    • Hasn't found metrics to be particularly meaningful in the past
    • Measuring attendance - remote participation - could be watching television and just be connected to the link - also have to battle with connectivity for developing countries
    • Participation - grading - 3 levels
      • Jostling for positions - trying to get onto a committee for example, in order to get a trip or something but not actively participating
      • Participating in procedural decisions only eg voting
      • Participating in functional activities that influence policy - actively engaged in working groups

    Contributor #2

    • How did you decide to do this metrics - when did you set up this mailing list and why wasn't everyone involved in it - why were  we not told about the metrics working group?
    • When are the suggestions going to be made to the RALOs so that we can adapt them to our own  activities
    • Who is going to evaluate the evaluating group?

    Contributor #3

    • When you analyse metrics  - who decides what kind of metrics you will use ?
  4. Message to RALO Secretariats 26 Sept (placed in wrong workspace)

    Apologies to the Secretariat for the very poor sound quality on Adobe Connect (26 Sep). I suspect it is our system in the Cooks. I can hear clearly what is being said, but cannot upload. And my phone connections kept cutting out too. Thanks to Silvia and Heidi for their summaries. Just to add..

    The purpose of the METRICS group is to develop metrics for ALAC and as Heidi mentioned we are also doing a review of APRALO documentation, therefore I will be taking note of metrics that arise out of the decisions that are made about the ALAC to be transferred to our review of APRALO documentation. It is hoped that perhaps other RALOS may consider this as well. 

    Because of the complexity and sensitivity of the issues relating to evaluation of performance of people who are volunteers, but It is important that people who put themselves forward for a position on the ALAC are aware that their appointment does impose responsibilities and accountability.For this reason, it is the responsibility of the group to review the metrics that are listed in Rule 21.

    In order to do this, it is important that we take the discussion to our Secretariats and RALOs and to get some feedback. At the moment the quantitative aspects of participation are recorded, but the qualitative aspects are a little more difficult. Attending a meeting, and participating, are seen as separate metrics items. How do we appropriately measure participation or performance?

    Similarly, someone who may not actively participate in meetings, but may be an active member of working group activities, so there has to be some balance in what we are measuring across the many responsibilities that ALAC members have, and the activities in which they are involved. 

    For this reason it is really important that we gather information from Secretariats and RALOs to help us with this task.  It is also assuming that all our ALSes are aware of what the ALAC does so that they can appropriately comment on the metrics. 

    We would like to know what kind of information RALOS would like to know from these metrics, and how they would like this information reported back to them. 

    But if you can assist us with any feedback we'd really appreciate it. Many thanks in advance.

    1. Just to make life even more difficult, it would be even harder to develop metrics for participation in non ALAC WGs, where theree may be only one ALAC person participating in that WG and the only measurement that can be verified by ALAC/RALO people is attendance.

      A couple of measurements that may help would be the number of times someone is a pen holder.  That, too, doesn't measure the complexity of the issue that had to be commented upon, but it's a start.  Another could be number of WG's that one chairs, as opposed to just participates in.

       

  5. Maureen Hilyard

    Report on data gathering for the ALAC METRICS working group from RALO ALSes

    January 2014

     

    Referring to Section 9 of the ALAC ROPS, the Working Group was tasked to:

    • Establish the key performance tasks of ALAC Members
    • Identify specific and quantifiable outputs
    • Establish the criteria against which these tasks could be rated or measured

    Mixed reaction and responses met the initial request for feedback and comments. Some RALO members were suspicious of ALAC’s reasons for initiating such a task group. It had been an early expectation that our feedback would come from ALSes which is why RALO Secretariats were approached first about the issue and were asked to raise it with their RALOs. I have yet to get any feedback that they did.  In general, at the F2F meetings with RALOs in Buenos Aires (LACRALO, NARALO, APRALO) responses came from current or ex-ALAC members rather than from the general electorate.  AFRALO and EURALO responded on the wiki. 

    Strong ALS capacity building recommendations arise (for ATLASII ?):

    ALSes require a greater understanding of:

    • their RALO roles and responsibilities - including their responsibility to elect their RALO representatives onto the ALAC
    • the role of ALAC - and the skills required by their elected representatives
    • the ICANN system and how ALAC and the contribution of their ALAC representatives fit into the big picture

    With a better understanding of the workings of ICANN and ALAC, ALSes can better

    • understand their importance to the ALAC system
    • understand the discussions that are raised at RALO meetings
    • become more confident in participating and contributing themselves
    • become involved in working groups and other ALAC and ICANN activities.

    **********************************************************************************

    A brief summary on feedback from the RALOs

     

    Key performance tasks:  Section 9 of the ALAC ROPS, when categorised, detail:  

    • How ALAC members can gather and respond to information that will help them make important ALAC decisions (attendance & participation and contribution sections)
    • How they can participate in the formal decision-making process (voting section), and
    • Their accountability responsibilities to the ALAC and their RALO (reporting section).

    The originally proposed metrics models on individual task outputs provided valuable feedback:

    • They were too specific and did not measure what was relevant to the RALO (EURALO)
    • Focused more on punishment for not doing it right rather than reward for work well done (NARALO)
    • Measuring attendance alone was not considered particularly meaningful (LACRALO)
    • Metrics did not appropriately measure participation or contribution (AFRALO)
    • Need to balance ordinary meetings with contributions the members make to working groups, as pen holders, or even in online and written conversations (AFRALO)
    • Measurement needs to be task driven (NARALO)

    Problems were identified when relying on a statistical output for each task:

    • Difficult to contribute to wikis when the system did not allow for a comment to be posted (AFRALO & APRALO)
    • If restricted to measuring attendance alone, especially for online meetings, how could a statistic show that there had been 100% focus on the meeting - especially if there was no recorded response in the chat or audio (LACRALO)
    • Attending meetings is not a goal or objective therefore its measurement is irrelevant, similarly for participation and contribution (APRALO)
    • Acknowledge that ALSes are volunteers and many have day jobs and that commitment may vary depending on other personal responsibilities (APRALO)
    • Some ALSes are just not participating because they are already over-committed in their own lives - so that their responsibility as ALS members needs to be addressed individually. (APRALO)
    • Who keeps the metrics? (APRALO)

    Recommendations

    • Active involvement and accountability could be more appropriately measured through actual LEADERSHIP of working groups activities and achieving results according to benchmarks or key performance indicators (APRALO)
    • Some indicators could include:
      • Input of time and commitment (eg number of working groups actively engaged in)
      • Leadership  responsibilities  (WG leader, penholder)
      • Mentoring skills (eg co-chairing to build capacity of other ALS members)
      • Team participation - collaborative skills
      • Other contributions…            (APRALO)
    • ALAC members and RALO Chairs should have joint responsibility for upskilling and engaging ALSes in ALAC activities (involving both outreach and inreach). RALO Chairs should plan for this engagement that is over and above the regular RALO update meetings (APRALO)
    • eDemocracy tools (NARALO)
    • ALAC members should provide a brief report to their RALOs at each RALO update meeting (APRALO)
    • If ALAC members are identified as underperforming, RALO Chairs should be the first person to approach them and discuss any problems. Referral to the ALAC Chair would be the next step for resolution (or dismissal) (APRALO)
  6. Discussion points from last meeting's chat

     

    Definition of the criteria of involvement and participation

    Meeting attendance of :

    1. ALAC conference calls and other "virtual " or face to face meeting (Holly asked for redefinitiion of virtual)
    2. Conference calls and any other virtual  or F2F meeting of the RALO iif the member's region

    Contribution to:

    1. The ALAC discussions through mailing lists or wiki pages
    2. The ICANN policy development through mailing lists of wiki pages

    Vote: Voting in the

    1. Elections of ALT members including its  chair and liaisons of representatives within adnon various ICANN constituencies
    2. Any other kind of vote (ratification of statement, survey, etc)

    Definition of the ALAC member status: Active, Less Active, Standby (Silvia)

    Members in Standby status

    • Lose their voting rights
    • Are removed from the quorum needed for voting on motions and electinos
    • Cannot be elected as representative in or from the ALAC
    • Lose their travel funding slot
    • Should be notified of the status change
    • May regain their "less  active" status and even their "active" status if within the next 6 months

    How we might identify contribution to working groups  to identify "active" ALAC membership

    • Some people adding name to working group but remain silent - "WG hoppers"
    • Same few people are actually active
    • Actual contribution  by ALAC members with regards to attendance, penholdership or contribution to discussions should be noted on a spreadsheet (as per  <GNSO Council Attendance-20 Nov 2013 (OLD)> )

    (MH - discussion starter) Perhaps - to ensure engagement by all ALAC participants:

    • Once ALAC members join a WG they must attend the meetings - obligation
    • Once the objectives of the group have been clearly established, each ALAC member should be assigned a responsibility
    • Each ALAC member should report on this responsibility at the start of each WG meeting participant so that there is seen to be active membership and engagement and that the member is  contributing.
    • With regards to spreadsheet (as per gNSO), perhaps each ALAC member could complete and update their own spreadsheet  indicating their attendance and performance eg writing in performance indicators and their achievement (eg I could include drafting the MOUs for APNIC and APTLD for APRALO for consideration by members and signing in /Singapore, or the other tasks that have been set for the Singapore meeting - plus any other virtual meetings in between F2F meerings). But to take the load off staff, and to establish more accountabiility by the ALAC members themselves, then they should do the performance reporting. This reference could be accessed by ALAC and RALO members as to the performance of their team members and RALO representatives
  7. As an example to demonstrate attendance and actual outputs - with ALAC members taking responsibility for their own feedback on performance - additional groups and tasks could be added. This could be done in whole or half year blocks and made available to everyone. 

    ALAC MEMBER: MAUREEN HILYARD, APRALO  (*meeting attendance dates given in CI time UTC-10)

    GROUP

    POSITION

    TASKS - 2014

    Jan

    Feb

    Mar

    Apr

    May

    Jun

    ALAC

    Member

    Liaison reports, updates

    28

     

     

     

     

     

    ALT

    Member

    Liaison reports, updates

    16

    30

     

     

     

     

     

    ccNSO

    ccNSO Liaison

    Coordinating committee; liaison reports

    13

     

     

     

     

     

    APRALO

    Member, penholder

    MOU APNIC, MOU APTLD

    27

     

     

     

     

     

    APRALO SINGAPORE

    Showcase OC, penholder

    Brochure, powerpoint, liaison with ALSes, fans

    13

    5

     

     

     

     

    FOIWG

    Member

    WG updates in ccNSO reports

    23

    5

     

     

     

     

    METRICS WG

    Member, RALO liaison

    Reports on RALO liaison contact (wiki)

     

    6

     

     

     

     

    BMSPC

    Member

     

    29

     

     

     

     

     

    ATLASII Events WG

    member

     

     

    12

     

     

     

     

  8. Síntesis de lo debatido en LACRALO sobre el tema Metricas

    Algunos miembros de LACRALO entendemos que sobre el tema de "auto evaluación" es necesario tener en cuenta que los representantes de las regiones en ALAC son una "extensión" o "herramienta" de la región, la competencia basada en el conocimiento no debería caer sobre el representante sino sobre el colectivo que le da mandato u opinión sobre tal tema.

    El conocimiento agudo sobre determinados temas le corresponde a TODA la región y no a un representante, que podrá tener conocimiento sobre algún tema en particular pero no sobre todo lo que se debate en ALAC (que es un reflejo de lo que se esta discutiendo en todo ICANN).

    La evaluación externa es un elemento que no podemos dejar de tener en cuenta, pero esta evaluación externa debería tener como principio que fuera realizada por componentes de la misma región en que fue electo el representante y garantizando un marco de transparencia e imparcialidad.

    Esto está basado en que si las regiones pueden elegir sus representantes también tienen capacidad para medir su desempeño e impartir sanciones si lo creen conveniente, el cual no podría ni debería ser realizado por ningún elemento externo de la región (se incluye a ALAC en esto).

    Sobre el control se sugiere tener en cuenta la cantidad de horas de dedicación mínima (que no puede ser indefinida), el hecho concreto que somos voluntarios y no profesionales pagos trabajando para un empleador, etc.

    Algunos miembros de la región opinan sobre el tema de “auto evaluación” diciendo que “...una auto evaluación siempre tendrá un carácter protector, y una auditoría o evaluación externa será resistida con argumentos como que "el auditor o inspector no entiende o no conoce lo que realmente sucede", y suspicacias de sesgo o parcialidad...”, entienden que “...Se requiere un cambio de cultura en el que la exigencia de actividad, participación informada y pro activa, y sobre todo resultados sólidos sea una norma generalizada...” en otro segmento del rico debate se profundizo diceindo que “... señalé que una característica de las métricas de participación debe ser la dificultad para "trucarlas", es decir, para simular su cumplimiento. Claro que esta dificultad no debe crearse mediante más burocracia sino al contrario, apelando a un principio de simplicidad...”

    “...la definición de las evaluaciones debe ser hecha con una importante contribución externa que mantenga crítico y honesto el trabajo. La evaluación también deberá tener un componente externo.

    Por otra parte, como en tantas otras tareas, es importantísimo incluir las demostraciones de competencia en las evaluaciones. No basta asistir a las reuniones, participar en las listas, mantener relaciones y comunicaciones con muchos participantes, votar, etc. si las participaciones no son competentes, basadas en conocimiento, apegadas a tiempos oportunos para influir en las decisiones, etc...”

    Por otro lado, varios se manifiestan diciendo que “...La auto evaluación solo podría tomarse como un indicio más, que puede aportar algunos elementos, relativos a la visión subjetiva, pero no como el único medio de relevar y realizar una medición objetiva. Esas otras tareas que se realizan pero no se ven son importantes y es necesario tenerlas en cuenta en una evaluación justa...”