At-Large Proposed Schedule and Process/Operational Improvements for AoC and Organizational Reviews Workspace

The purpose of this Public Comment posting is to request community feedback on a proposed schedule and process, including operational improvements, for Reviews mandated by the Affirmation of Commitments (AoC Reviews) and the ICANN Bylaws (Organizational Reviews). The request for the community is based on both appreciating the community's workload and the timing of several Reviews in FY2016. Based on the terms of the Affirmation of Commitments (AoC) and ICANN Bylaws, seven Reviews are scheduled to take place in FY20161, in addition to the finalization of the GNSO Review. Under the AoC, these Reviews would be: Security, Stabilityand Resiliency Review (SSR2); WHOIS Policy Review (WHOIS2); and Competition, Consumer Choice & Consumer Trust (CCT). Under bylaw mandated Organizational Reviews, these would be: At Large2, NomCom2, SSAC2, and RSSAC2.

Specifically, three AoC Reviews and initial work on the At-Large Review are proposed for FY2016, and three Organizational Reviews would be deferred until FY2017. Timing for the Competition, Consumer Choice & Consumer Trust Review is not impacted by this proposal – this Review will commence as scheduled, with a call for volunteers in September 2015. In addition to suggesting an approach to adjusting the schedule to accommodate the workload for the community, several improvements are proposed to increase the efficiency and effectiveness of Reviews, per community suggestions.

Of note for this consultation is recognizing that the CCWG Accountability process is considering recommendations to include the AoC Reviews into the ICANN Bylaws, which may adjust the timing if needed.

ICANN is committed to fulfilling its obligations under the AoC and Bylaws, and stands ready to implement the mandated Review schedule, unless a suitable alternative is agreed upon with the Community. This public consultation and the proposed alternative schedule presented for consideration arises in large part after stakeholder requests for relief from the unplanned confluence of Reviews, as the large number of simultaneous Reviews will have significant impact on ICANN stakeholders' capacity as well as ICANN resources. While it is important to meet ICANN's commitments to fulfill accountability obligations, it is also important to be responsive to community's workload concerns.

The proposed timeline and process/operating improvements are all outlined in Section II below. A template [PDF, 348 KB] has been developed to facilitate your input on this Public Comment. Use of the template is strongly encouraged, but not required. This template provides the opportunity for general input on the proposal as well as specific comments per section. Please note that there is no obligation to complete all of the sections – commenters may respond to as many or as few as they wish.

1 ICANN's FY2016 is 1 July 2015 – 30 June 2016

Community discussions on Review schedules yielded requests for a new proposed schedule. Additionally, analysis is underway of lessons learned from prior AoC and Organizational Reviews and implementation of ATRT2 recommendation 11 (addressing effectiveness of AoC Reviews) is progressing. All of this provides the basis for Staff recommendations on proposed timing of Reviews and process/operational improvements, with the goal of achieving greater efficiency (including cost savings) and effectiveness of Reviews,while ensuring that ICANN continues to fulfil its commitments under the AoC and Bylaws.

BACKGROUND – AOC REVIEWS

The Affirmation of Commitments (AoC; Section 9) calls for ICANN to organize Reviews of its execution of its commitments every three years, but does not include details on timing (three years from … Convening the Review? Completing the Review? Board action on the Review?) The Second Accountability and Transparency Review Team (ATRT2) issued a recommendation (#11) to improve AoC Reviews. Implementation of this recommendation includes establishing an improved process and appropriate schedule for AoC Reviews.

The language of the AoC pertaining to the timing of the Reviews is vague, resulting in several potential interpretations. The four completed AoC Reviews—two Accountability and Transparency Reviews (ATRT 1 and ATRT2) and one each for Security, Stability and Resiliency (SSR) and WHOIS—provide valuable lessons and opportunities to streamline future Reviews to accomplish a predictable, efficient and effective Review process.

Both ATRT1 and ATRT2 were completed in one calendar year. The AoC mandated this schedule for ATRT1, but ATRT2 had no deadline requirement. ATRT2 started approximately two years after ATRT1 was convened. This limited the time for implementing ATRT1 recommendations approved by the Board and incorporating the improvements into standard operating processes prior to convening ATRT2 to review the effectiveness of the implementation activities stemming from ATRT1 and conduct their Review.

The first cycle of SSR and WHOIS Reviews revealed that assembling qualified volunteer team members and conducting the Review took considerably longer for these Reviews than ATRT 1 & 2 (4 months to assemble qualified volunteer team members and in excess of 1.5 years to conclude the Review). After the 6 months needed for the Board to assess and act on recommendations (as required under the AoC), that meant that full-scale implementation efforts of the SSR and WHOIS Reviews did not even begin until 2 years after each of the Reviews convened. Recommendations from all AoC Reviews have varied implementation timeframes and many proposed improvements require three years or more to complete, particularly within the SSR and WHOIS Reviews. The shorter the time between Reviews, the more limitations this puts on the subsequent Review teams' ability to assess the effectiveness of prior teams' improvements, as called for by the AoC.

BACKGROUND – ORGANIZATIONAL REVIEWS

ICANN's Bylaws call for the periodic Review of ICANN SOs and ACs. The Board's Structural Improvements Committee (SIC) provides oversight for these Reviews and for improving this process. Article IV, Section 4 of ICANN's Bylaws states: "The Board shall cause a periodic review of the performance and operation of each Supporting Organization, each Supporting Organization Council, each Advisory Committee (other than the Government Advisory Committee), and the Nominating Committee by an entity or entities independent of the organization under review. The goal of the review, to be undertaken pursuant to such criteria and standards as the Board shall direct, shall be to determine (i) whether that organization has a continuing purpose in the ICANN structure, and (ii) if so, whether any change in structure or operations is desirable to improve its effectiveness. These periodic reviews shall be conducted no less frequently than every five years, based on feasibility as determined by the Board. Each five-year cycle will be computed from the moment of the reception by the Board of the final report of the relevant review Working Group."

PROPOSED SCHEDULE

1. Three AoC Reviews and initial work on the At-Large Review are proposed for FY2016, and three Organizational Reviews would be deferred until FY2017. Timing for the Competition, Consumer Choice & Consumer Trust Review is not impacted by this proposal. The proposed schedule for AoC Reviews strives to clarify the starting point for the next cycle of Reviews, includes sufficient time for planning and organizing of Reviews, and reflects a more focused Review scope (discussed below). This public consultation recognizes that the CCWG on Accountability is considering incorporating the AoC Reviews into the ICANN Bylaws, and any output of the CCWG Accountability recommendations will be factored into this timeline, if the immediate set of Reviews is affected. For Organizational Reviews, the NomCom2, RSSAC2 and SSAC2 Reviews would be deferred for one year, until FY2017.

   AoC Reviews – Three AoC Reviews are scheduled to commence in late 2015. Note that this proposal does not defer the commencement of any of these three AoC Reviews, as these Reviews are considered to be key accountability mechanisms that will continue to be prioritized. However, should the CCWG Accountability process recommend adjustments to these AoC Reviews, this will be factored in.

   As previously discussed with the community, AoC Reviews would start three years2 after the date of Board action on the previous Review recommendations (SSR – 18 October 2012; WHOIS – 8 November 2012). ICANN therefore would prepare for and begin SSR2 and WHOIS2 Reviews in late 2015, implementing a tightly scoped approach in combination with process/operational improvements outlined below. Note that the Competition, Consumer Choice and Trust (CCT) Review will launch as planned in September 2015 and is not deferred through this proposed schedule.

   

   Under the proposed schedule, the three AoC Reviews (SSR2, WHOIS2 and CCT) would be conducted concurrently, starting with the preparatory phases (appointing volunteers and planning the Review) in September 2015. It is anticipated that the Review teams would be appointed and initial planning work done within five months, with the substantive work of the Review teams beginning in March 2016 and lasting for one year.

   COMMENT: Please provide your comments on the proposed AoC Review schedule, under which three AoC Reviews would be conducted concurrently.

   Organizational Reviews – Under the current schedule, the preparations for NomCom2 and SSAC2 Reviews would begin now, and the RSSAC2 Review preparations would begin in August 2015. Additionally, the GNSO2 Review, which began in June 2014, is scheduled to finalize in August 2015, with implementation work to follow. The At-Large 2 Review preparations have begun, in line with the current schedule, with the engagement of the independent examiner targeted for July 2015.

   Under the proposed schedule, ICANN would prepare for and begin the At-Large2 Review under a slower schedule, implementing a self-assessment component and fine-tuning Review methodology/data collection for suitability to At-Large's organization in FY2016, prior to engaging an independent examiner. The independent examiner would conduct the Review between April and December 2016. Lessons learned from the At-Large2 Review, and the GNSO2 Review and implementation,3 will inform planning for NomCom2, SSAC2 and RSSAC2. Planning for these Reviews would occur late in FY2016 and the Reviews would commence in FY2017.

   

   COMMENT: Please provide your comments on the proposed Organizational Review schedule, which calls for the At-Large2 Review to proceed based on a slower schedule and NomCom 2, RSSAC2 and SSAC2 to commence in FY2017.

PROPOSED PROCESS/OPERATIONAL IMPROVEMENTS

1. Use planning and project management tools for AoC Reviews, including clear and focused Review scope, consistent budgeting, and cost tracking.

   The proposed improvement is for the Review approach to apply good practices used to conduct performance assessments for broadly similar organizations. Key recommended enhancements:

   • Develop a clear and focused Review scope during the planning phase, which will serve to improve planning and budgeting of volunteer time and ICANN resources.
   • Provide consistent budgeting, cost tracking, and standardization of budget scope.
   • Develop useful tools to serve as guidance to ease the burden for each Review team and for improved consistency between Reviews, for example sample project plans, timelines, checklists and templates.

   Key drivers of cost of AoC Reviews are: number of volunteers, number of in-person meetings, scope/conduct of Review, duration of Review, and whether independent consultants are engaged to assist the Review teams. In the past, concrete information for the above-mentioned drivers of cost had not become available until the Review team began its work. This has typically occurred too late to be considered in the annual budget process, creating a gap between budgeted resources and specific Review needs. The proposed process improvement for a clear and focused Review scope will address this gap by enabling a "standard" budget allocation and consistent budgeting and subsequent cost tracking. As recommended by ATRT2, procedures are being developed/incorporated into the next Review process to provide the Review teams with clearly articulated Review budget and assumptions at the start. Throughout the Review, there will be periodic reporting on the progress relative to planned schedule and budget.  The clear and focused Review scope also is expected to assist the Review teams with prioritization of recommendations in order to optimize the finite ICANN resources (time and budget).

   COMMENT: Please provide your comments on the proposal to apply planning and project management tools to Reviews.

2. Streamline AoC Review teams and Review duration.

   The proposed improvements strive to ensure that volunteers' skills and expertise are well aligned with the requirements for a given Review. It is suggested that the membership is aligned to the needs of each Review based on the scope, with an eye toward smaller and more focused teams.4 In order to provide volunteers with a realistic forecast of the length of their expected service, a clear and focused Review scope developed during the planning phase and useful tools applied throughout the Review are recommended. Anticipated schedule, milestones and time commitment would be published as part of the call for volunteers. Duration of AoC Reviews would be reduced, based on improved planning, focused scope, prioritization of recommendations, availability of Review guidance materials and timely, detailed reporting on implementation progress. A template project plan, including major phases of work and milestones, is recommended to be used as a tool to be tailored for each Review, in order to determine realistic duration. Note that the past AoC Reviews have lasted between 12 and 20 months.

   The recommended changes suggest modified terms of service for the Review teams, so they can answer questions about the intent and implementation of their recommendations. The benefit would be a more efficient implementation process. Under this proposal, the Review team would remain active until implementation planning is complete, as compared to the current practice of Review teams concluding their term upon delivery of their final report.

   

   COMMENT: Please provide your comments on the proposal to streamline AoC Review teams and Review durations.

3. Focus each Organizational Review on operational effectiveness and include self-assessments and focused preparatory actions by the organization under Review.

   The ICANN Bylaws mandate that "The Board shall cause a periodic review of the performance and operation of each Supporting Organization, each Supporting Organization Council, each Advisory Committee (other than the Governmental Advisory Committee), and the Nominating Committee by an entity or entities independent of the organization under review. The goal of the review, to be undertaken pursuant to such criteria and standards as the Board shall direct, shall be to determine (i) whether that organization has a continuing purpose in the ICANN structure, and (ii) if so, whether any change in structure or operations is desirable to improve its effectiveness."

   The following improvements are proposed for Organizational Reviews:

   • Identify an alternate process to assess whether a given ICANN organization has a continuing purpose (see below).
   • Self-assessment by representatives of the entity under Review of how previous Review's findings and recommendations have been addressed and whether the improvements have been effective.
   • Preparatory actions designed to fine-tune Review methodology and data collection for suitability to the given entity, prior to the launch of competitive bidding process to select the independent examiner.
   • Clarification that structural changes along with all other improvements resulting from the last Review cycle would be assessed as part of the operational effectiveness scope.
   • Clarification that whether structural changes are needed and when such changes should be considered would be topics for discussion after the organizational Review is finalized.

   Based on lessons learned from recent Reviews and in consideration of the adoption and implementation of ICANN's Strategic Plan for FY16-FY20, the assessment of whether a given ICANN organization has a continuing purpose would be better served through means other than the periodic organizational Reviews (see below).

   A standard part of any Review process is to begin with the assessment of how the previous Review's findings and recommendations have been addressed and whether the improvements have been effective. Structural changes, along with all other improvements resulting from the last Review cycle, would be assessed as part of the operational effectiveness scope of the upcoming Review cycle. Such assessments conducted by the organization under review would be useful as a key input into the work of the independent examiner. Also useful would be for the organization under review to seek reasons why the implemented improvements may not have been as effective as intended. This process is well aligned with the notion of continuous improvement. Whether structural changes are needed and when such changes should be considered would be topics for discussion after the organizational Review is finalized, possibly during the implementation planning, depending on the nature of findings and recommendations.

   To improve the efficiency and effectiveness of Organizational Reviews further, it is proposed that representatives of the entity under review be involved in various preparatory actions designed to fine-tune Review methodology and data collection for suitability to the given entity and that this activity take place prior to the launch of the competitive bidding process to select the independent examiner. This approach is currently piloted for the At-Large Review with initial positive feedback.

   COMMENT: Please provide your comments on the proposal to focus each Organizational Review on operational effectiveness and include self-assessments and focused preparatory actions by the organization under review.

4. Consider establishing an alternate process (to Organizational Reviews) to examine strategic issues such as the continued purpose of organizations.

   The question of whether an ICANN organization has a continuing purpose in the ICANNstructure aligns with long-range, strategic considerations that relate to all ICANNorganizations, with important cross-organizational dependencies and relationships. With the ICANN Strategic Plan recently adopted by the Board, after an extensive bottom-up community led effort, it has been suggested that an alternative process be established to examine strategic issues, such as the continued purpose of ICANN organizations, in alignment with the strategic plan. Furthermore, the IANA Functions Stewardship Transition work and Enhancing ICANN Accountability proposals may have an impact onICANN structure and organizations—this should be considered as an overarching ("meta") topic. In conjunction with an alternative process to address strategic structural issues, Organizational Reviews would continue and would be more narrowly focused on questions of operational effectiveness.

   COMMENT: Please provide your comments on the proposal to focus Organizational Reviews on operational effectiveness consider establishing an alternate to examine strategic issues such as the continued purpose of organizations.

IMPROVEMENTS UNDERWAY – For your information

Several process improvements are already underway or being considered by the Board – they are included here for information and additional context only. They include:

• Oversight of AoC Reviews to be assigned to an appropriate Board Committee or a subset of the Board, in parallel to SIC's responsibility for overseeing Organizational Reviews.
• Global standards of the Project Management Institute used in implementing improvements arising out of Reviews (piloted for ATRT2 implementation).
• Improved tracking, reporting and oversight of implementation progress (including centralized tracking and revamped icann.org webpage).
• Improved planning and resource/budget allocation prior to the start of the Reviews.

Currently, the Board itself provides oversight relative to AoC Reviews process, operations and other overarching issues; the small-group format of oversight has been effective in other areas. Given the focus on accountability and Reviews as one of the mechanisms supporting ICANN's accountability, the Board is considering assigning the responsibility for oversight of AoC Reviews to a discrete group or committee. A formalized process of a Board group or committee available to interact with the AoC Review teams at critical junctures during the Review and implementation would be beneficial in several ways: to address scope, issues and dependencies impacting the Review during planning; to facilitate clarity, advisability and feasibility of recommendations throughout the Review; and to hold entities accountable for achieving the milestones necessary to complete the implementation. Currently, the responsibility for oversight of Organizational Reviews lies with the Structural Improvements Committee (SIC), but there is not a similar assignment for AoC Reviews.

ICANN worked closely with the ATRT2 throughout 2013, laying down a foundation to ensure the implementation of the recommendations would be timely and effective. An implementation methodology was designed based on the global standards of The Project Management Institute. The PMI methodology5 currently is piloted for the ATRT2 implementation program, with the goal of achieving professional excellence through clear scope definition, company-wide resource planning, monitoring against timelines and consistent reporting. Quarterly updates and additional information is available on the ATRT2 implementation program wiki. ICANN intends to use a similar approach for implementation of recommendations from all AoC Review teams on a go forward basis.

2 ICANN is aware that the initial draft report of the CCWG on Enhancing ICANN Accountability includes a suggestion that the Review currently called for by the AoC be implemented into the Bylaws and have a 5-year cycle applied. If those suggestions are implemented, future Review schedules would be updated as appropriate. However, the need for a clear identification of the start of a Review cycle still remains.

3 Note that GNSO is expected to conclude in August 2015, with the implementation action taking place after that, throughout FY2016.

4 The AoC provides high-level guidance on the composition of Review teams, but does not impose requirements on the numbers on each team. Any enhancements or changes to the current AoC Reviews that result from the CCWG on Enhancing ICANN Accountability recommendations would need to be integrated into this process.

5 The Project Management Institute (PMI) standards include rules, guidelines and project management techniques that are widely accepted with a goal towards achieving professional excellence. Benefits include a clear scope definition, company-wide resource planning, monitoring against timelines and consistent reporting.

Public Comment Input Template

A template [PDF, 348 KB] has been developed to facilitate your input on this Public Comment. Use of the template [PDF, 348 KB] is strongly encouraged, but not required. This template [PDF, 348 KB] provides the opportunity for general input on the proposal as well as specific comments per section. Please note that there is no obligation to complete all of the sections – commenters may respond to as many or as few as they wish.

• Public Comment Template [PDF, 348 KB]

Following your completion of the template [PDF, 348 KB], please save the document and submit it as an attachment to the Public Comment proceeding: comments-proposed-aoc-org-reviews-process-15may15@icann.org

As a note for all comments submitted via the Public Comment proceeding: once a comment is submitted, please be on the lookout for a confirmation email (to confirm that the submission is posted to the proceeding). This is an automatically generated email to prevent spam.

• Public Session at ICANN52 "AoC and Organizational Reviews: Supporting ICANNAccountability"

1. ICANN Bylaws - see Section 4. PERIODIC REVIEW OF ICANN STRUCTURE AND OPERATIONS
2. Affirmation of Commitments – see Section 9.
3. Implementation update as of 30 March 2015 – ATRT2, WHOIS, SSR [PDF, 178 KB] Reviews.
4. Cross Community Working Group on Enhancing ICANN Accountability (CCWG-Accountability) - Input Needed on its Proposed Accountability Enhancements (Work Stream 1) (posted for Public Comment)

OTHER IMPROVEMENTS AND ADDITIONAL CONSIDERATIONS

Potential long-term considerations will be flagged separately to encourage community dialogue and contribute to the evolving community proposals on accountability.