IEEE.org     |     IEEE Xplore Digital Library     |     IEEE Standards     |     IEEE Spectrum     |     More Sites

From 1500 (3:00 PM) EST to 1600 (4:00 PM) EST we will be performing a major upgrade of the SA Open platform. This will affect all related services (opensource.ieee.org, opensource-connect.ieee.org, and *.ieee-saopen.org). During this period the listed URLs will be unavailable. For questions, email opensource@ieee.org.

Commit 39ef4c47 authored by Alfredo Herrera's avatar Alfredo Herrera
Browse files

Update design_peer-review.md

parent 193b7ed9
# Design peer-review of official IEEE open source projects and open source projects included in IEEE standards.
##Table of contents
## Table of contents
## Abstarct (Value proposition)
## 1.0 Abstarct (Value proposition)
Comments and changes tracked in [Issue #22](https://opensource.ieee.org/community/peer-review/admin/-/issues/22):
## Definition of peer-review od open source designs
## 2.0 Definition of peer-review of open source designs
Comments and changes tracked in [Issue #15](https://opensource.ieee.org/community/peer-review/admin/-/issues/15):
## Scope of peer-review of open source designs
- IEEE Open Source projects can request *peer review* of some of their aspects
from experts in the relevant fields
## IEEE context of open source design within IEEE
- for the purpose of peer review IEEE Open Source projects might be of
different kinds: pure software projects, hardware designs, documentation, or
any other material --- as long as their development happens on the IEEE open
source platform and their license is open source (**TODO**: add ref to
IEEE-approved open source licenses here)
## Peer-reviewers: joining, vetting, review responsibilities, acknowledgment and recognition of reviewers)
- peer review should be explicitly requested by one of the project developers
(**TODO:** do we have a name for them?) by submitting an issue in the
dedicated `oscom-peer-review` repository (**TODO:** this doesn't exist yet,
it is a proposal at this stage), tagging the issue with the
`peer-review-request` tag. The request should indicate:
- the *repository* hosting the project to be peer reviewed
- the *scope* of the review (see below for additional information on
available scopes)
- the *part* of the project that should undergo review, e.g., the entire
project v. specific parts of it identified by specific files/directories
within the project structure on the platform. The part of the project that
should undergo review can be either code that is already committed in a
specific branch of the project repository; or be part of a proposed change
to the project (e.g., via a pull/merge request). These details should be
specified in the review request
- (optionally) the *suggested reviewers*, if the project developers wish to
receive review from specific people
## Definition of peer-review services offered by IEEE
After the initial request the review enters state `REQUESTED`. (**TODO:** I
haven't explicitly depicted it that way, but we can add a state chart for the
review process, if desired.)
## Procedures for requesting peer-review services
- *review scopes* determine which part of a project should undergo review.
Predetermines/standard review scopes exist, but each review request can ask
for review of other/non-standard aspects of a project. The list of standard
scopes will evolve overtime based on the review scopes that are actually
use. The current list of standard review scopes is (in alphabetical order,
except for "other" at the end):
- architecture (e.g., modularity)
- conformance to specific standards (not necessarily IEEE standards)
- performance
- privacy
- quality (e.g., adoption of state-of-the-art coding best practices)
- security
- usability
- other --- for non-standard review scopes
## IEEE SA (internal) processes to implement open source peer-review services
- OSCom maintains a list of *available reviewers*. They are natural candidates
to perform reviews, but there is no obligation that all reviews will be
performed solely by reviews on this list: it is always possible to assign
reviewers not on the list, and the list will naturally evolve over time. The
names of the chosen reviewers is public and documented in the issue tracker
(**TODO:** this is my judgment call, to be discussed)
- after the initial review request, OSCom (**TODO**: or who else? we need the
equivalent of an "editorial board" of sorts here) assigns from 1 to 3
reviewers to perform the requested review. The review transitions to state
`ONGOING`.
- reviewers have 30 days to perform their reviews. As soon as a review is
ready, the reviewer posts it directly to the issue corresponding to the
review request; or, alternatively, to the merge request if the part of the
project undergoing review is a change proposed via the open source platform
- once all the reviews are in, OSCom will summarize the review result in the
issue tracker. Note that the final result of a review is not a score, but a
qualitative assessment. This final step changes the status of the review to
`COMPLETED`.
## 3.0 Scope of peer-review of open source designs
Comments and changes tracked in [Issue #27](https://opensource.ieee.org/community/peer-review/admin/-/issues/27)
## 4.0 IEEE context of open source design within IEEE
Comments and changes tracked in [Issue #21](https://opensource.ieee.org/community/peer-review/admin/-/issues/21)
## 5.0 Peer-reviewers: joining, vetting, review responsibilities, acknowledgment and recognition of reviewers)
Comments and changes tracked in [Issue #17](https://opensource.ieee.org/community/peer-review/admin/-/issues/17)
## 6.0 Definition of peer-review services offered by IEEE
Comments and changes tracked in [Issue #15](https://opensource.ieee.org/community/peer-review/admin/-/issues/15)
## 7.0 Procedures for requesting peer-review services
Comments and changes tracked in [Issue #16](https://opensource.ieee.org/community/peer-review/admin/-/issues/16)
### 7.1 INTERNAL
1. Scheduling of upcoming calls for participation (e.g. a process of three calls per year).
2. Defining reviewers/editors and reviewing process training for new reviewers.
### 7.2 EXTERNAL
3. Publication of call for participation with subject/documentation requirements.
4. Submissions are given in form of a paper alongside open source documentation.
### 7.3 INTERNAL
5. Initial project screening based only on project documentation.
6. Feedback is given to participants if minor changes are needed.
7. Project evaluation is done by commitee.
###7.4 EXTERNAL
8. Results of revision, alongside technical papers are published in IEEE OSCom page.
## 8.0 IEEE SA (internal) processes to implement open source peer-review services
Comments and changes tracked in [Issue #18](https://opensource.ieee.org/community/peer-review/admin/-/issues/18)
## Annex.1 Language for the Maintainers Manual concerning open source peer-review
## Language for the Maintainers Manual concerning open source peer-review
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment