NIST publication identifiers ("NIST PubID")
Purpose
NIST publications are numbered according to an identification scheme.
This gem implements a mechanism to parse and utilize NIST publication identifiers provided in the NIST-Tech-Pubs repository.
This gem is introduced in the Metanorma blog post "Introducing NIST PubID, and the migration of publication identifiers" on 2022-01-09.
For further details on the Metanorma work on the NIST PubID, refer to the "Ribose comments to Publication Identifier Syntax for NIST Technical Series Publications" document.
Historic identifier patterns
There are at least three major "pattern series" of identifiers due to historical reasons:
-
NIST publications produced by the PubID scheme (202X-)
-
NIST publications produced prior to the PubID scheme (1988-)
-
NBS publications, produced under the National Bureau of Standards (the previous name of NIST, 1901 to 1988)
This implementation attempts to support all three types of publication identifier patterns.
NIST PubID
NIST is currently developing a unified publication identifier scheme ("PubID") for their technical series of publications, which allows machine-readable identifier parts.
This scheme is described in “Publication Identifier Syntax for NIST Technical Series Publications” authored by Jim Foti of the CSD, ITL, and Kathryn Miller and Kate Bucher of the ISO, Management Resources.
Note
|
This scheme is still at proposal stage. |
This gem implements the NIST PubID scheme and allows the conversion to and from historic patterns.
Use cases to support
PubID:
-
Given a PubID (full-form title, abbreviated-form title, short-form title), generate the corresponding machine-readable (MR) PubID format
-
Given a MR PubID:
-
generate the full-form title
-
generate the abbreviated-form title
-
generate short-form title
-
generate the DOI (which is just DOI ID + MR)
-
Historic compatibility:
-
Parse a historic publication identifier into its components:
-
output the original publication identifier (i.e. store the contents in the object)
-
output a MR PubID
-
-
Parse a historic DOI into a PubID:
-
output the original DOI (i.e. store the contents in the object)
-
output a MR PubID
-
CLI conversion tool
We implemented an open-source conversion tool that extracts the required PubID data elements from existing NIST Tech Pubs metadata, such as the legacy identifier, DOI, edition and publication date information, to generate the new PubID.
pubid-nist
provides a CLI (Command-Line Interface) and a Ruby library that can be used to create and manipulate PubID objects.
Installation
The only prerequisite is to have Ruby installed. Please refer to the official Ruby installation guide.
The pubid-nist tool can be installed as follows.
$ gem install pubid-nist
Now you should be able to use the pubid-nist command.
Usage
Installation
The only prerequisite is to have Ruby installed. Please refer to the official Ruby installation guide.
The pubid-nist
tool can be installed as follows.
$ gem install pubid-nist
Now you should be able to use the pubid-nist
command.
When called without arguments (or as pubid-nist help
) the help screen will
be shown.
$ pubid-nist
Commands:
pubid-nist convert # Convert legacy NIST Tech Pubs ID to NIST PubID
pubid-nist help [COMMAND] # Describe available commands or one specific command
pubid-nist report # Create report for NIST Tech Pubs database (fetches from GitHub)
Converting a legacy identifier to NIST PubID
The command pubid-nist
provides a convert
subcommand that converts a legacy
Nist Tech Pubs identifier into the NIST PubID format.
Here’s how it can be used:
$ pubid-nist help convert
Usage:
pubid-nist convert
Options:
-s, [--style=STYLE] # Convert to PubID style (short|long|mr|abbrev)
# Default: short
-f, [--format=FORMAT] # Render in format (JSON, string)
# Default: string
Convert legacy NIST Tech Pubs ID to NIST PubID
$ pubid-nist convert "NIST SP 800-53a"
NIST SP 800-53A
$ pubid-nist convert "NIST SP 800-57p1r3"
NIST SP 800-57pt1r3
The convert
command also supports DOI conversion.
$ pubid-nist convert "NIST.SP.800-57p1r3"
NIST SP 800-57pt1r3
In addition to outputting PubID short style, we can also output other styles and formats of the resulting PubID.
$ pubid-nist convert -s mr "NIST SP 800-53a"
NIST.SP.800-53A
$ pubid-nist convert -s long -f json "NIST SP 800-53a" | jq
{
"styles": {
"short": "NIST SP 800-53A",
"abbrev": "Natl. Inst. Stand. Technol. Spec. Publ. 800-53A",
"long": "National Institute of Standards and Technology Special Publication 800-53A",
"mr": "NIST.SP.800-53A"
},
"publisher": "NIST",
"serie": "NIST SP",
"code": "800-53A"
}
Generating the bulk NIST Tech Pubs migration report
This is the nice part — a single command that generates the full table of converted PubIDs from the NIST Tech Pubs database, comprising of 19,333 entries.
The report
command can be used as follows:
$ pubid-nist help report
Usage:
pubid-nist report
Options:
[--csv], [--no-csv] # Export to CSV format
Create report for NIST Tech Pubs database (fetches from GitHub)
The purpose of this command is to aid the NIST PubID team in assessing the impact and type of changes to be made in enacting this new scheme.
By default, the report
command generates a table to indicate which migrated
identifiers have changed, focusing on changes of two styles:
-
PubID in short style vs legacy publication ID
-
PubID in machine-readable style vs legacy DOI
As seen in the following output, a ✅
or a -
will be shown in the appropriate
column of change.
$ pubid-nist report
ID changed? | New PubID | Document ID | DOI changed? | New PubID-MR | DOI | Title
- | NBS BH 1 | NBS BH 1 | - | NBS.BH.1 | NBS.BH.1 | Recommended minimum requirements for small dwelling construction : report of Building Code Committee July 20, 1922
- | NBS BH 10 | NBS BH 10 | - | NBS.BH.10 | NBS.BH.10 | A city planning primer by the advisory committee on zoning appointed by Secretary Hoover
...
✅ | NBS BH 3A | NBS BH 3a | ✅ | NBS.BH.3A | NBS.BH.3a | A zoning primer by the advisory committee on zoning appointed by Secretary Hoover (Revised)
- | NBS BH 4 | NBS BH 4 | - | NBS.BH.4 | NBS.BH.4 | How to own your home : a handbook for prospective home owners
✅ | NBS BH 5A | NBS BH 5a | ✅ | NBS.BH.5A | NBS.BH.5a | A standard state zoning enabling act under which municipalities may adopt zoning regulations by the advisory committee on zoning appointed by Secretary Hoover (revised edition 1926)
...
✅ | NBS RPT 2751 | NBS report ; 2751 | - | NBS.RPT.2751 | NBS.RPT.2751 | Stochastic search for the maximum of a function
...
✅ | NBS RPT 2831 | NBS report ; 2831 | - | NBS.RPT.2831 | NBS.RPT.2831 | Error bounds for eigenvalues of symmetric integral equations
Better yet, the report
command supports CSV output. The "changes" fields
will display true
or false
accordingly.
$ pubid-nist report --csv
ID changed?,New PubID,Document ID,DOI changed?,New PubID-MR,DOI,Title
false,NBS BH 1,NBS BH 1,false,NBS.BH.1,NBS.BH.1,"Recommended minimum requirements for small dwelling construction : report of Building Code Committee July 20, 1922"
false,NBS BH 10,NBS BH 10,false,NBS.BH.10,NBS.BH.10,A city planning primer by the advisory committee on zoning appointed by Secretary Hoover
false,NBS BH 11,NBS BH 11,false,NBS.BH.11,NBS.BH.11,A standard city planning enabling act by the advisory committee on city planning and zoning appointed by secretary Hoover
...
true,NIST SP 260-214,NIST SP 260-14,false,NIST.SP.260-214,NIST.SP.260-214,"Analysis of Seafood Reference Materials: RM 8256, RM 8257, RM 8258 and RM 8259, Wild-Caught Coho Salmon (RM 8256), Aquacultured Coho Salmon (RM 8257), Wild-Caught Shrimp (RM 8258), Aquacultured Shrimp (RM 8259)"
false,NIST SP 260-14,NIST SP 260-14,false,NIST.SP.260-14,NIST.SP.260-14,"Analysis of Seafood Reference Materials: RM 8256, RM 8257, RM 8258 and RM 8259, Wild-Caught Coho Salmon (RM 8256), Aquacultured Coho Salmon (RM 8257), Wild-Caught Shrimp (RM 8258), Aquacultured Shrimp (RM 8259)"
true,NIST IR 8379,NISTIR 8379,false,NIST.IR.8379,NIST.IR.8379,Summary Report for the Virtual Workshop Addressing Public Comment on NIST Cybersecurity for IoT Guidance
The best part is that this CSV will work properly with spreadsheet editors like Excel and Pages. All you need is to export the CSV values to a CSV file, and open it in your favorite program.
$ pubid-nist report --csv > myreport.csv
It is easy to filter these columns in Microsoft Excel with the following steps:
-
Open the CSV file in Excel
-
Convert the header row into a filter row: first highlight the header row, then click on "Data > Filter"
-
Filter the columns accordingly
Now it’s easy to screen through the anomalies and surface the data issues!
Elements of the PubID
Publisher
Name | Abbrev | Short |
---|---|---|
National Institute of Standards and Technology |
Natl. Inst. Stand. Technol. |
NIST |
National Bureau of Standards |
Natl. Bur. Stand. |
NBS |
Series
Publisher | Prefix | Name | Abbrev | MR (with Publisher) | Example |
---|---|---|---|---|---|
NIST |
NIST AMS |
Advanced Manufacturing Standard |
Adv. Man. Ser |
|
|
NIST |
NIST BSS |
Building Science Series |
Bldg. Sci. Ser. |
|
|
NBS |
NBS BSS |
Building Science Series |
Bldg. Sci. Ser. |
|
|
NBS |
NBS BMS |
Building Material Structures Report |
N/A |
|
|
NBS |
NBS BRPD-CRPL-D |
Basic Radio Propagation Predictions Series |
N/A |
|
|
NBS |
NBS BH |
Building and Housing Reports |
N/A |
|
|
NBS |
NBS CRPL |
Central Radio Propagation Laboratory Reports |
N/A |
|
|
NBS |
NBS CRPL-F-A |
CRPL Ionospheric Data |
N/A |
|
|
NBS |
NBS CRPL-F-B |
CRPL Solar-Geophysical Data |
N/A |
|
|
NBS |
NBS IP |
CRPL Ionospheric Predictions |
N/A |
|
|
NBS |
NBS CIRC |
Circulars |
N/A |
|
|
NBS |
NBS CIS |
Consumer Information Series |
N/A |
|
|
NBS |
NBS CS |
Commercial Standards |
N/A |
|
|
NBS |
NBS CSM |
Commercial Standards Monthly |
N/A |
|
|
NIST |
FIPS PUB |
Federal Information Processing Standards Publication |
Federal Inf. Process. Stds. |
|
|
NIST |
NISTGCR |
Grant/Contract Reports |
N/A |
|
|
NBS |
NBS GCR |
Grant/Contract Reports |
N/A |
|
|
NIST |
NIST HB |
Handbook |
Handb. |
|
|
NBS |
NBS HB |
Handbook |
Handb. |
|
|
NBS |
NBS HR |
Hydraulic Research in the United States |
N/A |
|
|
NBS |
NBS IRPL |
Interservice Radio Propagation Laboratory |
N/A |
|
|
NIST |
ITL Bulletin |
ITL Bulletin |
N/A |
|
|
NIST |
NIST LC |
Letter Circular |
N/A |
|
|
NBS |
NBS LC |
Letter Circular |
N/A |
|
|
NIST |
NIST MN |
Monograph |
Monogr. |
|
|
NBS |
NBS MN |
Monograph |
Monogr. |
|
|
NBS |
NBS MP |
Miscellaneous Publications |
N/A |
|
|
NIST |
NIST NCSTAR |
National Construction Safety Team Report |
Natl. Constr. Tm. Act Rpt. |
|
|
NIST |
NIST NSRDS |
National Standard Reference Data Series |
Natl. Stand. Ret. Data Ser. |
|
|
NBS |
NSRDS-NBS |
National Standard Reference Data Series |
Natl. Stand. Ret. Data Ser. |
|
|
NIST |
NIST IR |
Interagency or Internal Report |
N/A |
|
|
NBS |
NBS IR |
Interagency or Internal Report |
N/A |
|
|
NIST |
NIST OWMWP |
Office of Weights and Measures White Papers |
N/A |
|
|
NBS |
NBS PC |
Photographic Circulars |
N/A |
|
|
NBS |
NBS RPT |
Reports |
N/A |
|
|
NIST |
NIST PS |
Voluntary Product Standards |
Prod. Stand. |
|
|
NBS |
NBS SIBS |
Special Interior Ballistics Studies |
N/A |
|
|
NBS |
NBS PS |
Voluntary Product Standards |
Prod. Stand. |
|
|
NIST |
NIST SP |
Special Publication |
Spec. Publ. |
|
|
NBS |
NBS SP |
Special Publication |
Spec. Publ. |
|
|
NIST |
NIST TN |
Technical Note |
Tech. Note |
|
|
NBS |
NBS TN |
Technical Note |
Tech. Note |
|
|
NBS |
NBS TIBM |
Technical Information on Building Materials |
N/A |
|
|
NIST |
NIST TTB |
Technology Transfer Brief |
N/A |
|
|
NIST |
NIST DCI |
Data Collection Instruments |
Data Collect. Instr. |
|
|
NIST |
NIST EAB |
Economic Analysis Brief |
N/A |
|
|
NIST |
NIST Other |
Other |
Other |
|
|
NIST |
CSRC White Paper |
Cybersecurity Resource Center White Paper |
CSWP |
|
|
NIST |
CSRC Book |
Cybersecurity Resource Center Book |
CSRC Book |
|
|
NIST |
CSRC Use Case |
Cybersecurity Resource Center Use Case |
CSRC Use Case |
|
|
NIST |
CSRC Building Block |
Cybersecurity Resource Center Building Block |
CSRC Building Block |
|
|
NIST |
JPCRD |
Journal of Physical and Chemical Reference Data |
J. Phys. & Chem. Ref. Data |
|
(excluded from PubID scheme) |
NIST |
JRES |
Journal of Research of NIST |
J. Res. Natl. Inst. Stan. |
|
(excluded from PubID scheme) |
Stage
The stage code element only applies to non-final publications.
In most series, documents are only released as final publications, and therefore their PubIDs will not contain a stage code.
Only some series support stage codes, e.g. SP 800 and SP 1800.
Name | Value |
---|---|
Initial Public Draft |
IPD |
Second Public Draft (to the Nth Public Draft) |
2PD (… nPD) |
Final Public Draft |
FPD |
Work-in-Progress Draft |
WD |
Preliminary Draft |
PreD |
Report number
The contents and pattern of the report number are dependent on the series.
Possible values:
-
{sequence number}
-
{subseries}-{sequence number}
-
{sequence number}-{volume}
-
{sequence number}-{edition}
-
{subseries}-{sequence number}-{edition}
-
etc.
A "Part" can also be indicated by an appended alphabetic character to the end.
Part
Name | Abbrev and Short | MR |
---|---|---|
Part |
Pt. |
|
Volume |
Vol. |
|
Section |
Sec. |
|
Supplement |
Suppl. |
|
Index |
Index |
|
Addendum |
Add. |
|
Insert |
Ins. |
|
Errata |
Err. |
|
Appendix |
App. |
|
Edition
Name | Abbrev and Short | MR |
---|---|---|
Revision |
Rev. |
|
Edition |
Ed. |
|
Version |
Ver. |
|
Translation
An ISO 639-2 code that represents a translated document from English.
If a document is translated from English, suffix the document with a 3-letter ISO 639-2 code within parentheses.
Raw values seen in legacy DOIs include:
Name | Correct value | MR | Legacy values seen in DOI |
---|---|---|---|
Spanish |
(ESP) |
|
|
Vietnamese |
(VIE) |
|
|
Portuguese |
(POR) |
|
|
Chinese |
(ZHO) |
|
|
Indonesian |
(IND) |
|
|
Update
General
When a document is updated with an errata, the original edition may be reissued to include the errata.
These documents will display the text "includes updates as of…".
In this case the document identifier will include the element "Update".
Name | Abbrev and Short | MR |
---|---|---|
Update |
Upd. |
|
Note
|
This is similar to the ISO/IEC "Amd." and "Cor." except that the NIST "Upd." is applied to the original document and (typically) does not exist separately. |
Update number
A sequential integer numbering of the update counting from the original document.
The first update is numbered 1, and so forth.
Update year
The year last updated, shown as a suffix to the identifier.
-
"{identifier}:{update-year}"
PubID patterns
Presentation
Generally in this order:
-
No update:
{series} {stage} {report number}{part}{edition}({translation})
-
With update:
{series} {stage} {report number}{part}{edition}({translation})/{update} {update number}:{update year}
Full PubID
Samples:
-
National Institute of Standards and Technology Federal Information Processing Standards Publication 199
-
National Institute of Standards and Technology Special Publication 800-27, Revision A
-
National Institute of Standards and Technology Special Publication 800-39 (Second Public Draft)
(originally described in https://github.com/metanorma/metanorma-nist/issues/98)
Abbreviated PubID
This form is used in the Authority section.
Natl. Inst. Stand. Technol. {abbrev(series)} {reportnumber} {part | abbrev(volume)}, {abbrev(revision)}
-
abbrev(series)
represent the abbreviation of the Series title -
(Note: there is no
{update-date}
)
Examples:
-
"Natl. Inst. Stand. Technol. Spec. Publ. 800-78-4"
-
"Natl. Inst. Stand. Technol. Spec. Publ. 800-116"
-
"Natl. Inst. Stand. Technol. Spec. Publ. 800-57 Part 1, Revision 4"
(originally from https://github.com/metanorma/metanorma-nist/issues/88)
Short PubID
The "short form" is used to cite the documents within text.
It is used in these situations:
-
Locality references. "In Section 3.2 of SP 800-187…" (the "SP 800-187" is a link).
NoteNIST pubs are composed of "Sections" not "Clauses" -
A generic document reference. "SP 800-53 describes…". This form does not specify a revision or update date.
-
"All parts". "The SP 800-57 subseries describes key management…".
("NISTIR 6885 2003 Edition (February 2003)" is a poor indicator.)
The correct rule should be:
{abbrev(series)} {reportnumber} {abbrev(volume)} {abbrev(revision)} {edition, optional}
Note
|
For FIPS, reportnumber is the full FIPS number, including revision, e.g.,
140-2 .
|
(originally from https://github.com/metanorma/metanorma-nist/issues/88)
Short form date:
-
Month YYYY
Examples:
-
NIST SP 800-53r4 (20152201) supersedes NIST SP 800-53r4 (20140115)
-
NIST SP 800 63A (December 2017) supersedes NIST SP 800-63A
-
NIST SP 800 57 Part 1 Revision 4 supersedes NIST SP 800-57 Part 1 Revision 3 ("Rev." is also accepted, and converted to "Revision")
-
NIST SP 800 160 Volume 1 supersedes NIST SP 800-160 (20180103) ("Vol." is also accepted, and converted to "Volume")
-
Undated form "NIST SP 800 53r4"
Strip Revision and Date from title, only if the Revision and Date are unique for each document number. These are identified as "Rev. …", "Revision …." and " (Month YYYY)", whichever comes first.
Machine-readable PubID
Machine-readable syntax:
{publisher}.{series}.[{stage}.]{reportnumber}.{part}.{revision}.[{lang}.]{update-date}
{publisher}.{series}.[{stage}.]{reportnumber}.{part}.{revision}.[{lang}.][{update}]{update-date}
Generally, this rule should be able to uniquely identify an edition of a document.
-
{part}
-
Part
-
A "Part 1" document is encoded as "pt1";
-
When a letter part is indicated, e.g. "800-63A", we should keep it as part of the reportnumber (i.e., don’t indicate "pt").
-
-
Volumes
-
"Volume 1" is encoded as "v1";
-
-
-
{revision}
-
"Revision 1" is encoded as "r1"
-
If a superseding edition is a full revision, it will get the next Rev. #.
-
If a superseding edition is just an errata update, we use the update date from the title page ("includes updates as of …") to uniquely identify this edition. Preferably in the
-yyyymmdd
format.
-
-
{update}
-
"Update 1" is encoded as "upd1"
-
Some examples:
-
NIST.SP.800-53r4-20152201
supersedesNIST.SP.800-53r4-20140115
-
NIST.SP.800-63A-20171201
supersedesNIST.SP.800-63A
-
NIST.SP.800-57pt1r4
supersedesNIST.SP.800-57pt1r3
-
NIST.SP.800-160v1
supersedesNIST.SP.800-160-20180103
-
NIST.IR.8204.upd1-2019
supersedesNIST.IR.8204
-
The undated form is
NIST.SP.800-53r4
(originally from https://github.com/metanorma/metanorma-nist/issues/88)
Examples of PubID
-
SP 800 series
-
NIST SP 800-40r3
-
NIST SP 800-45ver2
-
NIST SP 800-53r4/Upd 3:2015
-
NIST SP IPD 800-53r5
-
NIST SP 800-53Ar4/Upd 1:2014
-
NIST SP 800-60v1r1
-
NIST SP 800-57pt1r4
-
NIST SP 800-73-4/Upd 1:2016
-
NIST SP 2PD 800-188
-
-
SP 1800 series
-
NIST SP 2PD 1800-13B
-
NIST SP PreD 1800-19B
-
-
NIST IR series
-
NIST IR 8011v3
-
NIST IR 8204/Upd 1:2019
-
NIST IR 8115(spa)
-
-
Other NIST Series
-
NIST HB 130e2019
-
NIST SP 1041r1/Upd 1:2012
-
NIST NCSTAR 1-1Cv1
-