This Document Is a Proposal for the IEEE 802.20 Technology Selection Procedure (TSP)

2004-10-27IEEE C802.20-04/72r1

Project / IEEE 802.20 Working Group on Mobile Broadband Wireless Access

Title / 802.20 Technology Selection Process (TSP)
Date Submitted / 2004-11-05
Source(s) / Dan Gal
67 Whippany Road,
Whippany, NJ 07981 / Voice: 973-428-7734
Fax: 973-386-4555
Email:
Re: / MBWA Call for Contributions: Session # 11- November 14-19, 2004
Abstract / A proposal for IEEE 802.20 technology selection process.
Purpose / Establish a process and methodology for selection of the best technology proposal based on which the IEEE 802.20 standard should be drafted.
Notice / This document has been prepared to assist the IEEE 802.20 Working Group. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.
Release / The contributor grants a free, irrevocable license to the IEEE to incorporate material contained in this contribution, and any modifications thereof, in the creation of an IEEE Standards publication; to copyright in the IEEE’s name any IEEE Standards publication even though it may include portions of this contribution; and at the IEEE’s sole discretion to permit others to reproduce in whole or in part the resulting IEEE Standards publication. The contributor also acknowledges and accepts that this contribution may be made public by IEEE 802.20.
Patent Policy / The contributor is familiar with IEEE patent policy, as outlined in Section 6.3 of the IEEE-SA Standards Board Operations Manual and in Understanding Patent Issues During IEEE Standards Development

2004-10-27IEEE C802.20-04/72r1

1.0Introduction

This document is a proposal for the IEEE 802.20 technology selection procedure (TSP).

Other IEEE 802 working groups – 802.15, 802.16, and 802.11 TGn [1] - have created their own procedures that(in the opinion of the author) do not seem to beare notentirely suitable for the 802.20 working group.

This contribution defines a framework and a skeleton process. It doesnot(as of the current version) include provide all the components of a complete procedure. It is suggested expected that the 802.20 working group would discuss, and debate and modify this proposal, change whatever it find necessaryand that additional contributions address the missing components and help complement this proposalte the procedure by formulating the 802.20 decision process and creating the annexes suggested in this contribution.

The symbols  indicates areas of the document that require additional contributions.

It is recommended that an agreed upon version of the TSP be adopted prior to before the 802.20 call for 802.20 proposals. is issued.

Organization of this TSP:

This proposal consists of the process rules text of sections 2 (section 2.3) and selection process stages (section- 32.4). Section 3 includes a list of reference documents and Section 4 contains the TBD annexes that are an integral part of the procedure and the annexes in section .4.

2.0Technology Selection RulesProcedure (TSP)

2.1Documents Precedence

This Technology Selection Proceduredocument (the TSP) shall be kept consistent with the applicable IEEE 802 and 802.20 rules and procedures. that are in effect when this procedure is adopted.In case of a conflicts, between this TSP and the IEEE 802 rules or the IEEE and the existing 802.20 Working Group procedures, the latter shall prevail and subsequently the TSP shall ould be amended and to eliminate the such conflicts.

2.2TSP Organization

The TSP is procedure is comprised of two parts:

1.Ground Rules (section 2.3) and,

2.Selection Process Stages (section 2.4)

TheProcedure ground rules define the “what and how” elements of the selection process, while and the Process stages define the sequential flow of the process activities and the sequence in which they will be carried out.

2.3Procedure Ground Rules

2.3.1Technology Proposals Submission

Technology proposals shall be submitted in accordance with the requirements of this proceduredocument and of the instructions of the 802.20 Call for Proposals.

Proposals shall meet the requirements of the adopted 802.20 SRD (version 14) [2].

Proposals shall comply with the IEEE 802 SA patent policies[1].

Proposals shall include the following five parts:

Part 1I: Technical SpecificationsSummary (see section 2.3.2).

Part 2II: Technology Description (see section 2.3.3).

Part 3III: PHY/MAC specifications Specifications (see section 2.3.4).

Part 4IV: Evaluation Criteriasimulations Simulations results Results (see section 2.3.5).

Part 5V: Compliance statement Statement (see section 2.3.6).

- Proposals need not be fully compliant with the 802.20 SRD in order to be considered for the technology selection process.

- Proposals shall comply with the IEEE 802 SA patent policies[2].

2.3.2Technology Proposal Part I1: Technical Specifications Summary

Proposals shall include a summary of their technical specifications,itemized in the order of the 802.20 SRD [2] sections. Table-1 is a suggested template. RRanking of each quantitative specification item,specifications shall be indicated relative to the SRD, shall be indicated by an X in the applicable column of table 1. ; tThe options are: “below”, “samemeet” and “exceedbetter”(than the SRD requirements).

 The line items of tTable-1 shall be developed ould be determined by the 802.20 working group and included in Annex 1.

Table 1: Technical Specifications Summary

item # /
SRD
Section /
SRD Specification /
Proposaled Specification / Ranking Relative to the SRD
belowbelow / meetsame / exceedbetter
1 / ●
2 / ●
3 / ●
..…

2.3.3Technology Proposal Part II2: Technology Description

This part of the proposal shall provide a detailed description of the technology. The style and level of detail should be similar to that of engineering white papers, published in professional publications. The objective of this part is to present the technical capabilities and operation principles properties of the technology. The proposed technology shallould be described in a concise, yet clear, fashion that wouldand explain in sufficient detail show how the proposal meets (or exceeds) the requirements of the 802.20 SRD [2]. Marketing-style statements and unproven technical claims shallouldbe avoidedbe avoided..

2.3.4Technology Proposal Part III3: PHY/MAC Specifications

The PHY and MAC specifications shall be similar in content and level of detail to current published IEEE 802 wireless standards. The text of this part should be acceptable by the 802.20 working group as a potential candidate draft-standard working document., based on which the later 802.20 draft standard would be developed.

 The table of content (TOC) of this part should be developed by the 802.20 working group of the proposal should be and included defined in Annex 2.

2.3.5Technology Proposal Part IV4: EvaluationResults

The evaluation criteria document (ECD) [3], shall provide the detailed proceduresfor the performance evaluation of technology proposals. The evaluation results shall be included in a uniformevaluation technical report.

An ad-hoc evaluation team, appointed by the 802.20 working group, shall review all the evaluation technical reports and prepare a comparison report thatshould rank the performance of the individual proposals in several key categories.

 The format of the evaluation technical report and the comparison report should be developed by the 802.20 working group and defined in Annex 3.

2.3.6Technology Proposal Part V5: Compliance Statement

The purpose of the compliance statement is to establish acceptability of a proposal. The purpose of the compliance table is to help rank the proposals and identify areas that may need further improvement or consolidation with another proposal.

Each pProposal shall include a compliance statementand a compliance table. The compliance statement shall declare the proposal as either fully compliant(with the 802.20 SRD) or or partially compliant. . A fully compliant proposal is one that meets all the “shall” essential requirements of the 802.20 SRD. A partially compliant proposal is one that does not meet one or more of the essential “shall” requirements of the 802.20 SRD.

A suggested The 802.20 working group shall establish the list of essential requirement.

In the compliance- table(a template of which is shown suggested in Table-2), . For each SRD requirement, the proposal’s compliance level shall be indicated in the appropriate column.

Note that while Table 1 covers only the quantitative specifications, Table 2 covers all the SRD requirements, quantitative as well as qualitative. each proposal shall state its compliance, non-compliance or partial compliance with each SRD requirement.

The purpose of this compliance table is to help evaluate a proposal and make a determination of eligibility for the 802.20 technology selection process.

Each of the requirements listed in Table-2, is classified as Essential, Non-essential or Optional. This classification of requirements needs to be agreed upon by the working group.

A proposal’s compliance with each individual requirement (listed in Table-2) shall be stated by entering an “x” in the appropriate “Compliance level” column. The options are:

-Fully-comply: indicated by an “x” in the “Y” column

-Non-comply: indicated by an “x” in the “N” column

-Partially-comply: indicated by an “x” in the “P” column

Partial compliance (“P”) shallould be indicated by a reference to a footnote that should explained the nature of partial compliance. For example, if the SRD requirement for uplink spectral efficiency at 120 Km/hr is 0.75 (see [2] section 4.1.1) and the proposal’s specification is 0.6, it is a partial compliance. An example of non-compliance: a proposal that does not support any coverage enhancing technology (requirement 4.1.11 in the SRD). in (the appropriate section of) Part II of the technology proposal.

 The compliance table should be developed by the working group and included in
Annex 4.

Table 2: Compliance Table(version A)

# /
Requirement /
SRD
Section # /
ECD Section # / Requirement Type / Compliance Level
“shall” / ”should” / ”may” / full / partial / non-
1 / ● / X
2 / ● / X
3 / ● / X
4 / ● / X
5 / ● / X
6 / ● / X
7 / ● / X
8 / ● / X
9 / ● / X

Table 2: Compliance Table (VersionB)

# /
Requirement /
SRD
Section # /
ECD Section # / Requirement Type and Compliance (N/P/Y=√)
“may” / ”should” / ”shall”
N / Y / N / Y / N / Y
1 / ● / X
2 / ● / √
3 / ● / P
4 / ● / X
5 / ● / √
6 / ● / P
7 / ● / X
8 / ● / √
9 / ● / P

* Requirement type: E: essential; N: non-essential; O: optional feature;

** Compliance level: Y: fully-comply; N: not-comply; P: partially comply;

2.3.7Proposal submission and presentation

2.3.7.1 Submission

(a) Proposals shall be submitted in accordance with the requirements of this document and the instructions of the Call for Proposals. Proposals shall be submitted to the working group Cchair and to the pProcedural viceVice-chair who, in turn, shall post the proposals m on the IEEE 802.20 working group website, within the next 3 business days. The working group shall be alerted to the posting by an email notice.

(b) Proposals shall be presented,in either in interim or plenary sessions, no earlier than 30 calendar days from date of website posting.

2.3.7.2 Presentation

(a) Presentation material shall be fully consistent with the submitted proposal. In case of any inconsistencyies or discrepancyies, either verbal or written, between the proposal and the presentation, the proposal shall prevail and the inconsistency/discrepancy shall be corrected removed fromin the presentation material.

(b) Revised presentation material shall be submitted, if possible, in the course of the same working group session.

(d) Presenters shall be allotted adequate time for presentation, discussion and Q&A. If necessary, presenters may ask for, and be granted, additional time – preferably on in the same session, but, no later than the next session. if possible, or the next.

(e)Meeting minutes shall record document the main points arising in the discussion and Q&A following the presentation. discussion and the associated Q&A. The minutes shall be published on the 802.20 website made available to the working group members no later than 5 business days from the session adjournment date.

2.3.8Proposal amendment Revision and consolidationConsolidation

(a) (a) Amended proposals maProposals may be revised after they were presented. Revisions that include technical changes that y not include significantlyaffect the technology’s changes performance may require a rerun of the simulations and submission of a revised evaluation report. to the original proposal. Significant changes affect the system performance.

(b) Amended Revised proposals shall be submitted to the working group and posted on the 802.20 website at least 14 days before the session they are intended to be presented in. The presentation shall be limited for to the changes made in the proposal and their impact on the technology’s performance. discussion in subsequent session and shall be posted to the working group website at least 14 calendar days prior to the meeting. The working group shall be notified by email on the same day of the posting.

(c) Consolidated proposals shall be considered new proposals and shall be submitted and processed in accordance with these rules.. Proponents shall inform the working group chair of their intention to consolidate their respective proposals. Consolidated proposals shall be considered new proposals.

2.3.9Selection rounds

(a) The final selection process shall be conducted in a plenary session.

(ab) The objective of The the selection process is to assure of the winning proposal shall assure that the most meritoriousproposal is chosen by the working group. The selection process shall be carried out in a “beauty contest” fashion. In the first round, proposals that are not fully compliant with the SRD shall be eliminated from the contest by a working group decision. An elimination motion shall require three-fourths majority (75%) to pass.consist of proposal reviews, performance evaluation, presentations, debate, scoring system and votes.

(bc) In the second round, the performance of the remaining proposals shall be compared using side-by-side comparative data extracted from the respective technical specifications and the evaluation criteria simulation results. The working group chair shall, with the active participation of the working group, rank the proposals based on the comparison data. The ranking of proposals shall be in several key categories (to be defined). Before the voting takes place, each proponent shall be allowed to give a final 15 minute presentation soliciting the working group’s support.. If three or more proposals are submitted, the selection shall be carried out in two rounds. In round 1- the elimination round – the number of candidate proposals shall be reduced to two. Proposal elimination shall be based on the following criteria:

(1) Compliance table. Non compliance with essential requirements shall be the primary consideration.

(2) Member and group scores.

(3) An elimination vote.

(c) In the selection round (round 2), the remaining two proposals shall be compared, side by side, with the main attributes and performance results summarized in a comparison chart. The chart shall include data extracted from the performance evaluation results, the compliance tables, presentation material and score results. A final proposal summary presentation shall conclude the review part of round 2. The final decision shall be made by a vote.

(dd) The third round shall be the final selection vote that will shall decide the winning proposal. Consistent with the working group rules, a 75% majority shall be required to win the 802.20 selection process. This final is selection vote shall also be by roll call.

2.3.10Appeal

TBD

2.3.11Process Finalization

If no appeal is pending, the working group shall proceed with the finalization steps as follows:

Adopt the winning proposal.
Post the selection process meeting minutes on the working group website.
Post the final version of the winning proposal on the working group website..
Create the 802.20 standard drafting task groups.
Update the 802.20 work plan

2.4Selection Process Stages

Overall, the 802.20 technology selection process may take several sessions. For easier management, the process can be organized in distinct stages as follows:

Stage 1: Proposal submissions

Stage 2: Presentations and review

Stage 3: Revisions and consolidation

Stage 4: Elimination of non-compliant proposals

Stage 5: Comparison and ranking of remaining proposals

Stage 6: Final selection of the winner

Stage 7: Process finalization.

(e) Normally, the proposal that scores the highest should be voted the winning proposal. In rare cases when there is a small margin between the scores of two proposals, the working group should consider other, non quantifiable, proposal attributes and may decide to select the runner-up as the winning proposal.

(f) The outcome of all selection votes must pass a “sanity check” that should verify consistency with the proposal scores, evaluation results and compliance tables. Failure to meet the “sanity check” should require the working group to reconsider its decision. A vote to reconsider shall be decided by a simple majority.

In case of repeated inconsistency, the group may decide (by a 75% majority) to suspend the entire process until the cause for such inconsistency is identified and corrected.

2.3.10Appeal

The final decision of the working group may be appealed in accordance with applicable IEEE SA rules.

2.3.11Process Finalization

If no appeal is pending, the working group chair shall initiate the finalization of the 802.20 technology process. The TSP finalization shall include the following:

1.Issue an official notice that declares the winning proposal.

2.Seal all TSP scoring forms and submit to the IEEE 802 EC secretary.

3.Post the TSP meeting minutes on the working group website.

4.Post the final version of the winning proposal. This version shall be the input document for the 802.20 draft standard development.

5.Establish the 802.20 task groups.

6.Update the 802.20 work plan.

2.42.4Selection Process Stages

The selection process shall be conducted in 802.20 working group plenary sessions, in the following stages:

Phase 1:

Stage 1: Proposal submissions.

Stage 2: Working-group Presentations

Stage 3: Review proposal compliance tables and evaluation results

Stage 4: Poll and document the member-scores

Stage 5: Proposal elimination (round 1) – select the top-two proposals.

Stage 6: If proposal consolidation is requested, skip to “Phase 2” below, otherwise, move to round-2 and select the winning proposal.

Stage 7: Finalize the technology selection process.

Phase 2: (optional, follow from stage 6 of “Phase 1” above)

Stage 1: Consolidated proposals submission

Stage 2: Working-group Presentations

Stage 3: Review proposal compliance tables and evaluation results

Stage 4: Poll and document new member-scores

Stage 5: Present the proposal-scores and vote the top-two proposals.

Stage 6: Select the winning-proposal.

Stage 7: Finalize the technology selection process.

At any stage, the working group may decide (by simple majority) to abort it and cycle back to the previous stage.

3.0References

IEEE P802.11 - Task Group N - Selection Procedure. September 17, 2003

Doc #: IEEE 802.11-03/665r8: