[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01

INTERNET-DRAFT                                                 N. Elkins
                                                         Inside Products
                                                                V. Hegde
Intended Status: Best Current Practice                        Consultant
Expires: May 3, 2017                                    October 30, 2016


         Definition of Participation Metrics for IETF Attendees
             draft-elkins-mtgvenue-participation-metrics-01


Abstract

   IETF meetings are held physically in various geographic regions of
   the world. One of the criteria for choosing a location is the amount
   of participation by the people in that region. Additionally,
   questions arise as to whether holding a physical meeting in a
   location increases the amount of participation by local attendees.
   Participation in the IETF process may occur in a number of different
   ways: email lists, writing drafts, physical or remote attendance at a
   meeting, chairing Working Groups and so on. This document defines the
   metrics and terms which may be used to measure participation both
   before and after an IETF meeting.


Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html


Copyright and License Notice



Elkins                    Expires May 3, 2017                   [Page 1]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


   Copyright (c) 2016 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document. Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .  3
     1.1 Geographic outreach  . . . . . . . . . . . . . . . . . . . .  3
     1.2 Encouraging Participation from New Regions . . . . . . . . .  3
     1.3 Motivation for New Geographic Regions to Participate . . . .  3
   2 Participation and its Nature . . . . . . . . . . . . . . . . . .  4
     2.1 What does Participation Mean?  . . . . . . . . . . . . . . .  4
     2.2 Ways to Participate  . . . . . . . . . . . . . . . . . . . .  4
       2.2.1 Email Lists  . . . . . . . . . . . . . . . . . . . . . .  5
       2.2.2 Authoring Drafts . . . . . . . . . . . . . . . . . . . .  5
       2.2.3 Authoring Seminal Drafts . . . . . . . . . . . . . . . .  5
       2.2.4 Starting a new Working Group or BOF  . . . . . . . . . .  6
       2.2.5 Remote Participation . . . . . . . . . . . . . . . . . .  6
       2.2.6 Attending Physical Meetings  . . . . . . . . . . . . . .  6
       2.2.7 Participating as a Leader  . . . . . . . . . . . . . . .  6
       2.2.8 Participation in standards implementation. . . . . . . .  6
       2.2.9 Participation in tools development . . . . . . . . . . .  6
   3 Measuring Contributions following a Physical IETF Meeting  . . .  6
   4 Guidelines for tracking metrics  . . . . . . . . . . . . . . . .  7
       4.1 Phase 1 - Non-binding metrics  . . . . . . . . . . . . . .  7
       4.2 Phase 2 - How to measure them  . . . . . . . . . . . . . .  7
       4.3 Phase 3 - Accept as input for meetings . . . . . . . . . .  7
   5 Security Considerations  . . . . . . . . . . . . . . . . . . . .  8
   6 IANA Considerations  . . . . . . . . . . . . . . . . . . . . . .  8
   7 References . . . . . . . . . . . . . . . . . . . . . . . . . . .  8
     7.1 Informative References . . . . . . . . . . . . . . . . . . .  8
   8 Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . .  8
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . .  8









Elkins                    Expires May 3, 2017                   [Page 2]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


1 Introduction

   IETF meetings are held physically in various geographic regions of
   the world. One of the criteria for choosing a location is the amount
   of participation by the people in that region. Additionally,
   questions arise as to whether holding a physical meeting in a
   location increases the amount of participation by local attendees.
   Participation in the IETF process may occur in a number of different
   ways: email lists, writing drafts, physical or remote attendance at a
   meeting, chairing Working Groups and so on. This document defines the
   metrics and terms which may be used to measure participation both
   before and after an IETF meeting.

1.1 Geographic outreach

   The document  [I-D.sullivan-mtgvenue-decisions] "Prioritized
   Objectives for Making Decisions in Selecting a Meeting Venue"
   contains the following:

   "The IETF moves its meetings around to ensure that those who can
   participate in person at the meetings share the difficulty and cost
   of travel.  The point of such moving is emphatically not to find new
   or interesting places to visit, or to undertake outreach to new
   communities who would not otherwise participate in the IETF."

1.2 Encouraging Participation from New Regions

   The document [I-D.baker-mtgvenue-iaoc-venue-selection-process]  "IAOC
   Plenary Meeting Venue Selection Process" contains the following:

   "The IETF chair drives selection of "*" locations, i.e., venues
   outside the usual regions, and requires community input.  These
   selections usually arise from evidence of growing interest and
   participation in the new region.  Expressions of interest from
   possible hosts also factor into the meeting site selection process,
   for any meeting.

   Increased participation in the IETF from those other regions,
   electronically or in person, could result in basic changes to the
   overall pattern, and we encourage those who would like for that to
   occur to encourage participation from those regions."

1.3 Motivation for New Geographic Regions to Participate

   The very process of preparing for or asking for an IETF meeting to be
   held in a geographic region where it has not been held before can
   have a profound change on the nature of that region's relationship to
   Internet Standards.   It can change the thinking from being



Elkins                    Expires May 3, 2017                   [Page 3]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


   "consumers" of standards to "developers" of standards.  It may help
   create a core group both within the region and from the diaspora to
   mentor and foster new work.  This can have a long lasting impact on
   the network professionals of that geographic area.   Planning for an
   IETF meeting to be held in a region can be a concrete rallying point
   to create such empowerment and change.

   All the above factors speak to the need to define more clearly what
   "participation" means and how to measure it objectively.


2 Participation and its Nature

2.1 What does Participation Mean?

   There are two ways to contribute to the IETF process: fundamental
   participation and process participation.

   Fundamental Participation: fundamental participation means active
   contribution to substantive IETF work.   The work of the IETF is to
   develop protocol standards, so a fundamental contribution is in
   protocol development.  Having said that, the reason for a protocol
   standard or a Working Group is to solve a problem which exists on the
   Internet.  A new standard is not developed in isolation in someone's
   head.  It is a result of discussions both face to face and
   electronically, sometimes lasting for several years.  Additionally,
   one Internet Draft or one conversation can lead to changing a view
   point or sparking ideas for other contributors.


   Process Participation: the IETF organism needs support to maintain
   and improve itself.   Groups such as mentoring, education, outreach,
   diversity, meeting venue and so on attempt to improve the functioning
   of the IETF organism.  Involvement in such groups is necessary to the
   IETF but is of a different nature than a contribution to a protocol
   standard.   Having said that, involvement in process groups may be a
   way to build a network of contacts which then may lead to
   conversations about protocol problems which then may lead to a new
   protocol standard.   Involvement in process groups is very much
   needed by the IETF and it may be a way for new people to work their
   way towards fundamental participation.

2.2 Ways to Participate

   Traditionally, work in the IETF consists of interactions and decision
   making on email lists as well as physical meetings which are held
   three times per year.   New ways to participate include attending
   meetings electronically at a remote hub or from a single location.



Elkins                    Expires May 3, 2017                   [Page 4]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


   One may also become involved in an Internet Draft Review team.   Some
   methods have very little associated economic costs; others have a
   high cost.

   One caveat in starting to keep metrics on participation - one hopes
   that people will not attempt to "game the system".  That is, make
   comments without merit on email lists or at the microphone in a
   meeting merely to improve the statistics for the region.   The social
   sanctions for making comments without merit are sufficiently high
   that the authors feel that baseless contributions will likely not
   persist.

2.2.1 Email Lists

   Posting to a Working Group email list to discuss an Internet Draft is
   the way that is most open to most people.  There is little barrier to
   entry in terms of economic cost.  An Internet connection of some type
   and an access device is all that is needed.

   However, there may be cultural barriers.  Sometimes people
   (especially when new) are not comfortable with the process of posting
   to the Working Group email list or want to check with others about
   their understanding of an Internet Draft before asking a question or
   posting a suggestion.  So, the IETF Mentoring program is starting
   Internet Draft Review Teams so that would-be participants can work
   with remote mentors to facilitate engagement.  The desired output of
   such teams is posting to an email list.

   Posting to a fundamental Working Group email list should be the only
   metric counted. Posting to an email list such as IETF discuss,
   96attendees and so on, is not a worthy metric to gauge participation.

2.2.2 Authoring Drafts

   Not all Internet Drafts become RFCs.  Often, the statistic used is
   that one in ten Internet Drafts become an RFC.   Still, authoring a
   draft shows active participation.   The draft should however, spark
   active discussion on the email list.  If it is chosen for live
   presentation at a Working Group session, then that is a high degree
   of participation.

2.2.3 Authoring Seminal Drafts

   Some drafts change the thinking of others.  These may be seminal
   ideas which are referred to by quite a few others.   References to a
   particular piece of work can easily be found and should be regarded
   as a high degree of participation.




Elkins                    Expires May 3, 2017                   [Page 5]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


2.2.4 Starting a new Working Group or BOF

   A Working Group is started to address a specific problem.   Leading a
   BOF or a Bar BOF which then leads to Working Group formation should
   be regarded as a high degree of participation.

2.2.5 Remote Participation

   One may participate electronically in Working Group sessions either
   alone or at a remote hub.  Merely viewing a session should not be
   counted as participation.  Making a comment should be counted.
   Comments are kept in the minutes of the WG meeting, hence can easily
   be used.

2.2.6 Attending Physical Meetings

   One may attend physically and yet not contribute to the process.
   Alternatively, a physical attendee may be actively engaged and have
   many conversations both in fundamental and process groups.   In the
   end, an active physical participant will likely end up speaking at
   the microphone and commenting on a draft or a discussion that is
   underway in a Working Group meeting.  Hence, the examination of WG
   minutes should be enough to count as a participation metric.

2.2.7 Participating as a Leader

   Serving in an IETF management position, Working Group chair, Area
   Director, and so on can easily be measured and should be regarded as
   a high degree of participation.   Fundamental leadership positions
   (those of standards developing groups) should be weighted more
   heavily than process group leadership positions.   Having said that,
   it takes time and a network of contacts to become a fundamental group
   leader.  It also likely takes consistent physical attendance at IETF
   meetings.

2.2.8 Participation in standards implementation

   Standards are not useful in isolation. Implementations of standards
   are important to see what happens "when the rubber meets the road".
   Many times, once an idea in a draft is implemented in the real world
   there are problems found either in interoperability, security or some
   other areas. The IETF has recognized this with more emphasis on code
   through hackathons and interaction with open source implementers.
   Implementing an open source solution should also be considered as a
   contribution. Often implementation of standards goes hand-in-hand
   with the standard implementation.

2.2.9 Participation in tools development



Elkins                    Expires May 3, 2017                   [Page 6]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


   Several tools which the IETF uses (such as datatracker) are either
   completely or partially maintained by volunteers. Contribution to
   these tools also helps makes interaction and tracking of activities
   easier for other IETF volunteers. Additions to tools should also be
   considered as contributions. These can possibly be measured in terms
   on number of commits or lines of code (though admittedly these are
   crude metrics).

3 Measuring Contributions following a Physical IETF Meeting

   Metrics should be kept and published for the above categories
   following each physical IETF meeting.   Metrics may be kept by
   individual and also by geographic region. The geographic region
   should be country, continent and Internet Registry (APNIC, Afrinic,
   etc.)  This way, one can readily assess the impact of a meeting in a
   particular area as well as the growth in contribution for a region.
   Aspiring regions who wish to increase their IETF presence will also
   have a way to show their increase in participation over time.

4 Guidelines for tracking metrics

4.1 Phase 1 - Non-binding metrics

   Define a broad set of non-binding metrics. Some of the metrics can be
   easily tracked such a number of drafts and meetings attended. Other
   are little fuzzy such as email contributions, comments in WG on the
   microphone. Make a list of these and start implementing them.

4.2 Phase 2 - How to measure them

   Metrics such as email contributions can be tracked partially by
   looking up email addresses of participants (and mapping them to
   country against known databases such as registration history and
   drafts/RFCs). Track and refine these metrics and get consensus on
   which ones to track and on the implementations as well. These can be
   separate drafts.

4.3 Phase 3 - Accept an input for meetings

   Once these metrics are acceptably robust, they can be checked for
   suitability for continued tracking. These can be used as inputs in
   decision making process for meeting locations.









Elkins                    Expires May 3, 2017                   [Page 7]


INTERNET DRAFT   elkins-mtgvenue-participation-metrics- October 30, 2016


5 Security Considerations

   There are no security considerations.

6 IANA Considerations

   There are no IANA considerations.

7 References

8.1 Informative References

   [I-D.baker-mtgvenue-iaoc-venue-selection-process]  Baker, F., "IAOC
   Plenary Meeting Venue Selection Process", draft-baker-mtgvenue-iaoc-
   venue-selection-process-03 (work in progress), July 2016.

   [I-D.sullivan-mtgvenue-decisions] Sullivan, A., "Prioritized
   Objectives for Making Decisions in Selecting a Meeting Venue", draft-
   sullivan-mtgvenue-decisions-00(work in progress), July 2016.

9 Acknowledgments

   The authors would like to thank Fred Baker, Yoav Nir, S. Moonesamy
   and Dave Crocker for their comments.

Authors' Addresses

      Nalini Elkins
      Inside Products, Inc.
      36A Upper Circle
      Carmel Valley, CA 93924
      United States
      Phone: +1 831 659 8360
      Email: nalini.elkins@insidethestack.com
      http://www.insidethestack.com

      Vinayak Hegde
      Consultant
      F2, First Floor, Prabhu Kunj, 7th Cross,
      Eshwara Layout, Indiranagar 2nd Stage,
      Bangalore - 560038
      Phone: +91 9449834401
      Email: vinayakh@gmail.com
      URI:   http://www.vinayakhegde.com







Elkins                    Expires May 3, 2017                   [Page 8]


Html markup produced by rfcmarkup 1.122, available from https://tools.ietf.org/tools/rfcmarkup/