Relaton for NIST: bibliographic retrieval of NIST publications

Purpose

Relaton for NIST provides bibliographic information of NIST publications using the NistBibliographicItem model.

Relaton for NIST has been developed in cooperation with the NIST Cybersecurity Resource Center (CSRC) and the Computer Security Division (ITL/CSD).

Data sources

General

Relaton for NIST retrieves bibliographic information from two sources, in the following priority:

NIST Cybersecurity Resource Center (CSRC)
NIST Library of the NIST Information Services Office (ISO)

NIST Cybersecurity Resource Center (CSRC)

The NIST CSRC bibliographic feed is used as the primary source for all CSRC publications.

Bibliographic information offered through CSRC is provided with enhanced metadata unavailable from the NIST Library dataset, including:

versioned drafts (the NIST Library dataset only contains the latest publication: latest draft or final publication)
revision information: revision number, iteration
document stage information: retired, withdrawn, etc.
bibliographic dates, including issued date, updated date, published date, obsolete date, commenting period
document relationships: supersession and replacements
contacts: enhanced name parts and affiliation information

The NIST CSRC provides the following documents:

NIST SP 800-*
NIST SP 500-*
NIST SP 1800-*
NIST FIPS {31, 39, 41, 46, 46-*, 48, 65, 73, 74, 81, 83, 87, 102, 112, 113, 139, 140-*, 141, 171, 180, 180-*, 181, 186, 186-*, 188, 190, 191, 196, 197, 198, 198-1, 199, 200, 201, 201-*, 202}

As this feed is provided by CSRC in the native Relaton JSON format, no data transform is necessary.

NIST Library

The NIST Library offers the full NIST Technical Publications dataset at GitHub.

The library data is managed internally using MARC21, and the GitHub repository offers this information via MARC21 XML and MODS (in XML).

Relaton for NIST uses the MODS dataset obtained from the following location:

NIST-Tech-Pubs/releases, the allrecords-MODS.xml file

The MODS XML file is parsed using the loc_mods library and its data mapped to the Relaton model.

The NIST Library dataset provides documents listed in the index.

Installation

Add this line to your application’s Gemfile:

gem 'relaton-nist'

And then execute:

$ bundle

Or install it yourself as:

$ gem install relaton_nist

Usage

Retrieving an entry

By NIST PubID

Relaton supports using the NIST PubID as the retrieval key for bibliographic entries.

Note	Relaton for NIST is the first public implementation of the NIST PubID.

Example 1. Retrieving a bibliographic entry using NIST PubID

> require 'relaton_nist'
=>
true

> hit_collection = RelatonNist::NistBibliography.search("NISTIR 8200")
[relaton-nist] (NIST IR 8200) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST IR 8200) Fetching from Relaton repository ...
=>
<RelatonNist::HitCollection:0x00000000004b28 @ref=NIST IR 8200 @fetched=false>

> item = hit_collection[0].fetch
=>
#<RelatonNist::NistBibliographicItem:0x007fc049aa6778
# ...

The publication year can also be given if the publication has seen multiple revisions or editions.

Example 2. Retrieving a bibliographic entry using NIST PubID with year

> RelatonNist::NistBibliography.get("NIST IR 8200", "2018")
[relaton-nist] (NIST SP 8200:2018) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST IR 8200:2018) Fetching from Relaton repository...
[relaton-nist] (NIST IR 8200:2018) Found: `NIST IR 8200`
=>
#<RelatonNist::NistBibliographicItem:0x00007fab74a572c0
# ...

A NIST PubID can contain these optional parameters {ptN}{vN}{verN}{rN}{/Add}:

Part is specified as ptN (SP 800-57pt1)
Volume is specified as vN (SP 800-60v1)
Version is specified as verN (SP 800-45ver2)
Revision is specified as rN (SP 800-40r3) or -N
Addendum is specified as /Add (SP 800-38A/Add)

Example 3. Retrieving a bibliographic entry with a PubID that contains revision

item = RelatonNist::NistBibliography.get 'NIST SP 800-67r1'
[relaton-nist] (NIST SP 800-67r1) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST SP 800-67r1) Found: `NIST SP 800-67 Rev. 1`
=> #<RelatonNist::NistBibliographicItem:0x00000001105acd08
...

item.docidentifier.first.id
=> "NIST SP 800-67 Rev. 1"

Example 4. Retrieving a bibliographic entry with a PubID that specifies an addendum

item = RelatonNist::NistBibliography.get 'NIST SP 800-38A/Add'
[relaton-nist] (NIST SP 800-38A/Add) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST SP 800-38A/Add) Found: `NIST SP 800-38A-Add`
=> #<RelatonNist::NistBibliographicItem:0x00000001105abf48
...

item.docidentifier.first.id
=> "NIST SP 800-38A-Add"

By NIST PubID abbreviated form

The NIST PubID "abbreviated form" is the form of PubID printed on the inner cover of a NIST publication.

The NIST PubID abbreviated format has the following syntax:

NIST {abbrev(series)} {docnumber} {(edition), optional} {(stage), optional}, or
{abbrev(series)} {docnumber} {(edition), optional} {(stage), optional}

Where,

(stage)

empty if the state is "final" (published). In case the state is "draft" it should be:

PD: for the public draft
IPD: for the initial public draft
{n}PD: for subsequent public drafts, such as 2PD, 3PD and so on.
FPD: for the final public draft

(edition)

the date of publication or update

docnumber

the full NIST document number, e.g., 800-52.

Example 5. Retrieving an entry using the NIST PubID abbreviated form

> RelatonNist::NistBibliography.get("NIST SP 800-205 (February 2019) (IPD)")
[relaton-nist] (NIST SP 800-205) Fetching from csrc.nist.gov ...
[relaton-nist] (NIST SP 800-205) Found: `NIST SP 800-205 (Draft)`
=>
#<RelatonNist::NistBibliographicItem:0x00000001105afdc8
# ...

By FIPS PubID

The format for FIPS publications is:

FIPS {docnumber}, or
NIST FIPS {docnumber}

Example 6. Retrieving an entry using the FIPS PubID

> RelatonNist::NistBibliography.get("NIST FIPS 140-3")
[relaton-nist] INFO: (NIST FIPS 140-3) Fetching from csrc.nist.gov ...
[relaton-nist] INFO: (NIST FIPS 140-3) Found: `NIST FIPS 140-3`
=>
# #<RelatonNist::NistBibliographicItem:0x000000013b68fb50
# ...

Serializing an entry

To Relaton XML

Example 7. Serializing an entry to Relaton XML

> item.to_xml
#=> "<bibitem id="NISTIR8200" type="standard" schema-version="v1.2.9">
#      <fetched>2023-10-16</fetched>
#      <title format="text/plain" language="en" script="Latn">Interagency report on the status of international cybersecurity standardization for the internet of things (IoT)</title>
#      ...
#    <bibitem>"

When the option bibdata: true is given, the method outputs XML wrapped by the bibdata element and adds a flavor-specific ext element.

Example 8. Serializing an entry to Relaton XML with bibdata: true

> item.to_xml bibdata: true
=>
"<bibdata type="standard" schema-version="v1.2.9">
#      <fetched>2023-10-16</fetched>
#      <title format="text/plain" language="en" script="Latn">Interagency report on the status of international cybersecurity standardization for the internet of things (IoT)</title>
#      ...
#      <ext schema-version="v1.0.0">
#        <doctype>standard</doctype>
#      </ext>
#  </bibdata>"

Accessing attributes

Identifier components

A NIST PubID can contain these optional parameters {ptN}{vN}{verN}{rN}{/Add}.

They can be accessed through the following methods:

part
volume
version
revision
addendum

Typed links

NIST publications may have src and doi links, obtained using the #link method.

Example 9. Accessing links of a NIST bibliographic item

item.link
=> [#<RelatonBib::TypedUri:0x0000000111087a98
  @content=#<Addressable::URI:0x8c0 URI:https://csrc.nist.gov/pubs/sp/800/38/a/sup/final>,
  @language=nil,
  @script=nil,
  @type="src">,
 #<RelatonBib::TypedUri:0x00000001110879a8 @content=#<Addressable::URI:0x8d4 URI:https://doi.org/10.6028/NIST.SP.800-38A-Add>, @language=nil, @script=nil, @type="doi">]

Loading entries

From YAML

Offline Relaton YAML files can be loaded directly as bibliographic items.

Example 10. Loading a Relaton YAML file

hash = YAML.load_file 'spec/examples/nist_bib_item.yml'
=> {"id"=>"NISTIR 8011 Vol. 3",
...

RelatonNist::NistBibliographicItem.from_hash hash
=> #<RelatonNist::NistBibliographicItem:0x007f8b708505b8
...

Fetching data

On a search, the CSRC data sources are fetched and stored in the user’s cache.

This gem uses the https://github.com/usnistgov/NIST-Tech-Pubs/releases/download/May2024/allrecords-MODS.xml dataset as NIST-Tech-Pubs data source.

The following method fetches all the documents from the NIST-Tech-Pubs dataset, then saves them to the ./data folder in YAML format.

Example 11. Fetching all data with default configuration

> RelatonNist::DataFetcher.fetch
Started at: 2021-09-01 18:01:01 +0200
Stopped at: 2021-09-01 18:01:43 +0200
Done in: 42 sec.
=> nil

Example 12. Fetching all data to the data folder

> RelatonNist::DataFetcher.fetch(output: "data", format: "yaml")

Options:

output

folder to save documents (default ./data).

format

format in which the documents are saved. Possible formats are:

yaml: (default) save in Relaton YAML format
xml: save in Relaton XML format
bibxml: save in the IETF BibXML format

Logging

Relaton for NIST uses relaton-logger for logging.

By default, it logs to $STDOUT. To change log levels and add other loggers, refer to the relaton-logger documentation.

Contributing

Bug reports and pull requests are welcome.

License

The gem is available as open source under the terms of the MIT License.

nistbib

Development

Runtime