Release Description Document (build 11.1)
This release of the PDS4 System is intended as an operational release of the system components to date. The original plan for this release can be found here: plan B11.1
The following sections can be found in this document:
PDS4 Standards and Information Model Changes
This section details the changes to the PDS4 Standards and Information Model approved by the PDS4 Change Control Board and implemented by the PDS within the latest build period.
Ref |
Title |
|---|---|
CCB-204: Define and enforce best practices for discipline and project dictionaries. - Part 1 |
|
CCB-268 Add optional attribute to class Terminological_Entry |
|
CCB-204: Validate that no attribute is named “unit” - Part 2 |
|
CCB-138 Fix mismatch between context object types and values of <type> in <Observing_System_Component> class |
|
Clean up IMTool/LDDTool UML/XMI file writer for MagicDraw UML Class Diagrams |
|
LDDTool aborts on short filename |
|
CCB-204: Define and enforce best practices for discipline and project dictionaries. Part-3 |
|
CCB-220: Add ability to specify many source products via table |
|
CCB-271: Add appropriate reference_type values for Product_Ancillary |
|
CCB-204: Define and enforce best practices for discipline and project dictionaries. Part-4 |
|
Nillable attributes are not declared nillable in class definitions. |
|
Sync up LDDTool version with Maven build version |
|
CCB-256: Need method for providing permissible value definitions for external namespaces in Ingest_LDD |
|
CCB-271: Add reference_types for Product_Ancillary |
|
CCB-220: Add ability to specify many source products via table. |
|
CCB-274 - Add attribute dictionary_type to Ingest_LDD |
|
CCB-278: Fix errors in logical_identifier, ASCII_LID, ASCIIVID and ASCII_LIDVID_LID |
|
CCB-274 - Add attribute dictionary_type to Ingest_LDD - Update |
|
CCB-272: Reinstate Array_1D in the Information Model |
|
CCB-279: Mis-Matched <axes> and Axis_Array Specifications |
|
CCB-284: Streamline process for adding or removing standard values. |
|
CCB-285: GeoTIFF format as operational PDS4 image |
|
CCB-323: Fix schema formation rule for lidvid_reference (Part II) |
|
CCB-315: “PDS3” is an allowed parsing standard for Bundle documentation file |
|
CCB-304: Cleanup unused Vector classes in IM before 2.0.0.0 |
|
CCB-313: Definition of <external_source_product_identifier> refers to non-existent documentation |
|
CCB-305: Missing validation constraint on <specified_unit_id> |
|
CCB-300: Apparently deprecated units of measure are not actually deprecated |
|
CCB-312: <ldd_version_id> does not appear to be constrained the way LDDTool expects |
|
CCB-302: No <reference_type> values defined in DD_Attribute or DD_Class contexts |
|
CCB-317: Add FITS 4.0 to parsing_standard_id enumerated values for Header object |
|
CCB-313: Definition of <external_source_product_identifier> refers to non-existent documentation. |
|
CCB-321: Add MPEG-4 as an encoding_standard_id for Product_Native |
|
CCB-328 : Inconsistency in <title> type definition |
Software changes
harvest
Provides software provides functionality for capturing and indexing product metadata in PDS Registry. A sub-component of the PDS Registry App (https://github.com/NASA-PDS/pds-registry-app)
requirement
Issue |
Priority / Bug Severity |
|---|---|
harvest#37 As a user, I want a default configuration for harvest included in the tool package |
p.must-have |
harvest#45 As a user, I want to be able to ingest a directory of data that is not part of a bundle |
unknown |
pds-api
PDS API Application with client and server integrated into one package
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-api#31 Streamline testing of API server implementation |
unknown |
pds-api#12 Initial Query Syntax Lexer Implementation |
unknown |
pds-api#14 Define initial set of intra-discipline (product-level) search scope |
unknown |
pds-api#17 Define initial structure for response format conventions and parameter definition |
unknown |
pds-api#34 Deploy PDS API v0 (alpha) for beta testing |
p.must-have |
pds-api#35 Initial Federated API implementation |
unknown |
pds-api#40 add lexer to registry api |
unknown |
pds-api#41 Manage field preselection in queries |
unknown |
pds-api#43 Implement content negotiation |
unknown |
pds-api#47 develop a jupyter notebook demo where a user can browse PDS archive from bundle to product data file |
unknown |
pds-api#52 Get investigation area/targets/instruments from external ids |
unknown |
pds-api-javalib
base repository for PDS API definition and libraries
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-api-javalib#1 Add time out specification in swaggerHub definition of the API |
unknown |
pds-api-javalib#2 Add list of available fields in response format |
unknown |
pds-deep-archive
PDS Open Archival Information System (OAIS) utilities, including Submission Information Package (SIP) and Archive Information Package (AIP) generators
bug
Issue |
Priority / Bug Severity |
|---|---|
pds-deep-archive#99 aip_label_checksum is not for the correct file |
unknown |
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-deep-archive#80 NSSDCA Delivery Onboarding |
unknown |
pds-deep-archive#93 add year to directory path in URL |
unknown |
pds-doi-service
Service and tools for generating DOIs for PDS bundles, collections, and data sets
bug
Issue |
Priority / Bug Severity |
|---|---|
pds-doi-service#119 Raise a specific exception when the OSTI server is not reachable |
s.medium |
pds-doi-service#141 api does not ignore ‘/’ at the end of url |
s.low |
pds-doi-service#143 when release command keywords are broken with encoded characters |
s.high |
pds-doi-service#150 when doing draft with warnings (e.g. duplicated title) the -f option does not help |
s.high |
pds-doi-service#154 draft OSTI label |
s.high |
pds-doi-service#159 The url /dois/{lidvid} should still return XML in the record attribute |
s.critical |
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-doi-service#52 API Implementation for DOI Service |
p.must-have |
pds-doi-service#91 Develop User Access / Management Strategy |
unknown |
pds-doi-service#114 Draft action: read the doi from the pds4 label |
p.must-have |
pds-doi-service#125 Update DOI UI and Service with new workflow for operational deployment |
unknown |
pds-doi-service#140 Update submission to OSTI to handle the removal of a field from the OSTI metadata |
p.should-have |
pds-doi-service#144 enable filter by status in sub-action ‘pds-doi-cmd list’ |
unknown |
pds-doi-service#148 API POST /dois should accept DOI OSTI format in payload |
p.must-have |
pds-doi-service#157 When a pds4 label or osti can not be parsed generate error 400 in API |
p.should-have |
pds-doi-ui
web UI for pds-doi-service
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-doi-ui#14 Prevent the discipline node for doing DOI release without a ENG node review |
p.must-have |
pds-doi-ui#17 PDS Label url should take a public URL |
p.must-have |
pds-doi-ui#19 Release step needs a submitter/node |
p.must-have |
pds-doi-ui#27 Error need to be catched when one draft a pds4 label from the url |
p.must-have |
pds-doi-ui#28 Enable force warning |
p.must-have |
pds-registry-app
Registry application enabling a PDS node to register all its data products for long term preservation and sharing with the rest of the PDS system.
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-registry-app#13 Beta test operational deployment |
unknown |
pds-registry-app#27 Manage PDS4 product relationships |
unknown |
pds-registry-app#92 Update Registry API per PDS API v0-beta |
unknown |
pds-registry-app#102 Add the API to the pds-registry-app package, with documentation |
p.must-have |
pds-registry-app#107 Prep Registry Demo for Discipline Nodes |
unknown |
pds-registry-app#114 Manage product relationships v2 - collection inventories |
unknown |
pds-registry-app#120 Develop kibana configuration for EN registry UI |
unknown |
pds-registry-app#123 Update registry-mgr documentation as stated in #86 |
p.could-have |
pds-registry-app#125 analyze need, propose architecture |
unknown |
pds-registry-app#127 load data for kibana test |
unknown |
pds-registry-app#133 Verify pds-registry-app doc is up to date, have registry configuration in the harvest conf examples |
unknown |
requirement
Issue |
Priority / Bug Severity |
|---|---|
pds-registry-app#141 As a manager, I want a cost model for deploying a registry + API in AWS |
p.must-have |
pds-registry-app#146 As a node operator, I want the the registry schema to update autonomously when new data is ingested. |
p.should-have |
pds-registry-common
Common code used by both Harvest and Registry Manager
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-registry-common#1 verify artifact is pushed to Maven central and setup the ci |
unknown |
pds-wds-web
PDS Web Design System - Basic web implementation
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds-wds-web#17 App Bar Node List Reordering |
p.could-have |
PDS.nasa.gov-Search
Front-end interface for PDS.nasa.gov data search capability
enhancement
Issue |
Priority / Bug Severity |
|---|---|
PDS.nasa.gov-Search#29 Iterate with DOI Working Group for improvements to landing pages |
p.must-have |
PDS.nasa.gov-UX
PDS.nasa.gov User Experience Task Issue and Prototype repository
enhancement
Issue |
Priority / Bug Severity |
|---|---|
PDS.nasa.gov-UX#3 Formulate user stories for prototype |
unknown |
PDS.nasa.gov-UX#6 Model workflows and initial design directions |
unknown |
PDS.nasa.gov-UX#8 Design mockups and develop Figma prototype |
unknown |
PDS.nasa.gov-UX#61 Refine user interview and survey analysis per MC comments |
unknown |
PDS.nasa.gov-UX#70 Refine PDS Web Modernization working group plan |
unknown |
PDS.nasa.gov-UX#71 Upload all data from airtable / survey to GDrive |
unknown |
pds4-information-model
The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.
bug
Issue |
Priority / Bug Severity |
|---|---|
pds4-information-model#266 Throw WARNING message when enumeration_flag = false but enumerations are specified |
s.low |
pds4-information-model#280 CSV files fail to escape double quotes. |
s.low |
pds4-information-model#302 LDDTool does not allow the bundle to be specified for generated dictionaries |
s.medium |
pds4-information-model#331 Invalid output schema when trying to set an Internal_Reference reference_type value set |
s.high |
pds4-information-model#339 CCB-328 : Inconsistency in <title> type definition |
s.low |
enhancement
Issue |
Priority / Bug Severity |
|---|---|
pds4-information-model#238 Continuing refactoring of IMTool / LDDTool |
p.should-have |
pds4-information-model#242 DocBook HTML/WebHelp generation and conversion processes |
unknown |
pds4-jparser
Java Library providing APIs for parsing and exporting information on PDS4 products, including table and image objects to various formats including CSV, PNG, VICAR, FITs, etc.
bug
Issue |
Priority / Bug Severity |
|---|---|
pds4-jparser#36 NoSuchMethodError: java.nio.ByteBuffer with Java9 Builds |
s.medium |
PLAID
APPS PDS Label Assistant for Interactive Design (PLAID). See an overview of the software on YouTube. https://www.youtube.com/watch?v=WCo8erW_rL8
enhancement
Issue |
Priority / Bug Severity |
|---|---|
PLAID#10 Develop procedure for how to update PLAID with latest IM |
unknown |
registry-api-service
PDS Registry API service. Complies with PDS API specification
enhancement
Issue |
Priority / Bug Severity |
|---|---|
registry-api-service#2 Manage relationships bundle-collection-product in API |
unknown |
registry-api-service#4 Implement a lid resolver |
unknown |
tracking-service
Provides functionality for tracking status and other aspects pertaining to PDS products that are not captured in the Registry Service.
bug
Issue |
Priority / Bug Severity |
|---|---|
tracking-service#18 tracking front page does not keep tracking in relative anchor href |
unknown |
enhancement
Issue |
Priority / Bug Severity |
|---|---|
tracking-service#10 Dockerize Tracking Service |
unknown |
tracking-service#14 Add mysql database to docker deployment |
unknown |
validate
Validates PDS4 product labels, data and PDS3 Volumes
bug
Issue |
Priority / Bug Severity |
|---|---|
validate#5 Improve file base name check according to Standards Reference |
s.low |
validate#6 Improve pds4.bundle unlabeled files check to handle files without a file suffix |
s.medium |
validate#11 Update allowable field_format values per Standards Reference definition regarding [+-] characters |
s.low |
validate#153 Update validate to throw error when a file has a space in the filename |
s.medium |
validate#240 Unexpected error for data collection in a sub-directory |
s.medium |
validate#271 validate 1.25.0-SNAPSHOT raises an exception when validating a product |
s.low |
validate#273 Bug performing bundle validation with nested directories |
s.medium |
validate#278 Registered context products file does not retain older versions of context products |
s.medium |
validate#281 Validate fails to report error in File.file_size |
s.medium |
validate#291 When validating a product with a bad schematron definition, bundle validation also fails indicating the associated product does not exist |
s.low |
validate#294 Content validation incorrectly reports error for floating-point values out of specified min/max range |
s.medium |
validate#298 validate misses double quotes within a delimited table |
s.medium |
validate#301 unclear error message for field count matching |
s.medium |
enhancement
Issue |
Priority / Bug Severity |
|---|---|
validate#51 Provide the capability to specify multiple locations for pds4.bundle validation |
p.could-have |
validate#81 Validate and throw error when duplicate LIDs are found in Bundle |
p.should-have |
validate#230 Update validate per SR requirements for collection inventories |
p.should-have |
validate#249 Improvements for validating accumulating bundles / collections |
unknown |
validate#252 Implement initial behavioral testing framework with cucumber |
p.should-have |
validate#290 Migrate subset of existing regression tests to cucumber behavioral testing |
p.should-have |
validate#322 Update installation documentation to require Java 1.9+ |
p.could-have |
validate#323 Upgrade to Java 9+ |
p.should-have |
requirement
Issue |
Priority / Bug Severity |
|---|---|
validate#188 As a user, I want to validate a bundle that uses multiple versions of the Information Model / Discipline LDDs |
p.should-have |
validate#210 As a user, I want validate to raise a WARNING when differing versions of IM are used within a bundle |
p.could-have |
validate#292 CCB-264: Make the Line Feed (LF) character an allowed record delimiter |
p.must-have |
theme
Issue |
Priority / Bug Severity |
|---|---|
validate#250 Improvements to meet updated Standards Reference since initial requirements implementation |
unknown |
Liens
Issue |
Title |
Rationale |
|---|---|---|
pds-swg_7 [CR] Defer PDS UX Tasks to B12.0 |
[CR] Defer PDS UX Tasks to B12.0 |
Deferring these tasks to B12.0 in order to develop a comprehensive, detailed plan, milestones, and activities for a PDS Web Modernization Team. https://github.com/nasa-pds/pds.nasa.gov-ux/issues/70 |
pds-swg_6 [CR] Defer PDS API Tasks to B12.0 |
[CR] Defer PDS API Tasks to B12.0 |
Inaccurate estimates for time to complete other designs and implementations with PDS API WG. |
pds-swg_5 [CR] Defer Tracking Service Tasks to B12.0 |
[CR] Defer Tracking Service Tasks to B12.0 |
Tracking Service design and partial implementation was completed in 2018. Misunderstanding during handoff from previous developers that the software design did not include requirements definition, and design did not form to common API standards. Caused significant increase in scope for [requirements definition task](https://github.com/nasa-pds/tracking-service/issues/2) |
pds-swg_4 [CR] Add additional PDS4 SCRs to Release Plan |
[CR] Add additional PDS4 SCRs to Release Plan |
Several additional SCRs passed by the CCB. |
Engineering Node Software Catalog
The Engineering Node Software resources are listed in the software release summary (B11.1)
Installation and operation
PDS Engineering node software are meant to be deployed in 3 contexts: standalone, discipline nodes or engineering node For the installation and operation manual see the users manuals in the software summary sections below:
Reference documents
This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:
PDS Level 1, 2 and 3 Requirements, April 20, 2017.
PDS4 Project Plan, July 17, 2013.
PDS4 System Architecture Specification, Version 1.3, September 1, 2013.
PDS4 Operations Concept, Version 1.0, September 1, 2013.
PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.
PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.
PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.
PDS Search Scenarios, Version 1.0, September 1, 2013.
PDS Search Protocol, Version 1.2, March 21, 2014.
PDAP Search Protocol, Version 1.0, March 21, 2014.
PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Deep Archive Sotware Requirements and Design Document (SRD/SDD)