Build 12.0 Plan

The Planetary Data System (PDS) is a long-term archive of digital data products returned from NASA’s planetary missions, and from other kinds of flight and ground-based data acquisitions, including laboratory experiments. But it is more than just a facility - the archive is actively managed by planetary scientists to help ensure its usefulness and usability by the world wide planetary science community.

The following page outlines the planned PDS software enhancements, bug fixes, and impacted requirements for the upcoming Build .

A more detailed breakdown of these tasks by date and priority can be found in our Zenhub Roadmap (login required).

Key Dates

Key Dates for Discipline Nodes
Date	Name	Responsible Party	Description
08/24/2021	SCR Freeze Date	DDWG	Freeze date for identifying SCRs to be implemented in this release.
09/29/2021	Standards Documents Updates Due	Document Authoring Team	Updates to PDS4 standards documents due to EN for review.
10/15/2021 - 11/12/2021	Beta Testing Period	DNs, IPDA	Changes to PDS4 IM and system components available for beta testing.
10/15/2021 - 11/12/2021	dLDD Integration and Test	dLDD Stewards	Auto-generated dLDDs should be reviews by dLDD Stewards, and final updates to dLDDs should be completed during this time.
11/12/2021	Standards Documents Review Due	Document Review Team	Review updates to PDS4 standards documents.
12/03/2021	System Release	EN	PDS4 system is operationally deployed and website is updated accordingly.

Key Dates for Engineering Node
Date	Name	Description
09/17/2021	Delivery to I&T	The PDS4 IM and all software updated during this build cycle are delivered to EN I&T.
09/30/2021	Test Readiness Review (TRR)	PDS EN internal review to verify readiness to enter I&T phase.
11/17/2021	Delivery and Deployment Review (DRR)	PDS internal review with PDS Software Working Group to verify readiness to deploy build system.
12/03/2021	Operational Deployment	PDS4 system is operationally deployed and website is updated accordingly.

For a more detailed schedule, see https://pds-engineering.jpl.nasa.gov/content/schedules.

Planned PDS4 Information Model Changes

This section details the planned changes to the PDS4 Information Model for this build. These changes are deliverables by the PDS Data Design Working Group (DDWG) to the Engineering Node for implementation.

validate#310 Validate missing collections in bundle after CCB-282 updates

Planned Software Changes

This section details the planned software changes for this build.

Planned tasks for the next release cycle are grouped by software component, and sub-tasks are included where known at the time of this plan generation. Each section also includes additional information, such as:

Software description
Link to 🔍 User Guide
Link to 💻 Github Repo
Link to 🚑 Issue Tracking
Link to 📒 Backlog - future work not planned for this release cycle

Don’t see the tool you are looking for? This means there is no active development planned for this build cycle. Head to the PDS Tool Registry to see all possible tools available for PDS data.

Have a new feature or tool you think we should be working on? Create a ticket in the applicable Github repo or in our PDS EN Operations repo and we will see how that can fit into our plans moving forward.

validate

Validates PDS4 product labels, data and PDS3 Volumes

validate#319 B12.0 Configuration Improvements
- validate#150 Update software to handle catalog files the same fashion when specified via command-line versus config file
validate#318 B12.0 Content Validation Improvements
- validate#57 As a user, I want to be warned when there are alphanumeric characters between fields in Table_Character
- validate#164 As a user, I want to validate PDF files are PDF/A
- validate#303 As a user, I want to the raise a WARNING if the object-defined size in the label does not match the file_size value
- validate#314 Revert #9: Do not throw ERROR message when ASCII Numeric field consists of only white space
validate#317 B12.0 Referential Integrity Improvements
- validate#69 As a user, I want to validate that all context objects specified in observational products are referenced in the parent bundle/collection Reference_List
- validate#307 As a user, I want to validate that all XML Schema collections in NASA PDS archives are secondary collections
- validate#308 As a user, I want to check that all Internal References are valid references to other PDS4 products within the current validating bundle

pds4-information-model

The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.

pds4-information-model#350 B12.0 PDS4 Information Model SCR Implementation
pds4-information-model#349 Establish LDD Management Teams and Processes
- pds4-information-model#347 Create dLDD CCB
- pds4-information-model#348 Create dLDD Stewardship Teams
pds4-information-model#326 B12.0 LDDTool Improvements
- pds4-information-model#240 Improve argument handling using argument parsing library
- pds4-information-model#269 The PDS4 IM Specification Document is not consistent in the ordering of Permissible Values
- pds4-information-model#330 As a user I want to know the output of the tool after it completes execution.
- pds4-information-model#338 As a developer, I want to know the dLDD version from the output JSON data
- pds4-information-model#341 As a user, I want to specify an IngestLDD using a relative path
- pds4-information-model#346 lddtool’s doc directory is not in tgz file
pds4-information-model#306 B12.0 Refactoring of IMTool
- pds4-information-model#239 Refactor Class information from config to Data Dictionary Protege ontology
- pds4-information-model#281 Tag ops classes/attributes in IM so they are easily identifiable by users

pds-deep-archive

PDS Open Archival Information System (OAIS) utilities, including Submission Information Package (SIP) and Archive Information Package (AIP) generators

pds-deep-archive#105 Integrate PDS Deep Archive with Registry
- pds-deep-archive#7 As a user, I want to generate AIPs and SIPs using Registry
- pds-deep-archive#106 Improvements per API updates to remove workarounds from code
- pds-deep-archive#107 Improvements in API for “performance” workarounds in PDS Deep “Registry” Archive

pds-doi-service

Service and tools for generating DOIs for PDS bundles, collections, and data sets

pds-doi-service#179 Develop DataCite Interface
- pds-doi-service#103 As the PDS, I want to mint DOIs through DataCite
pds-doi-service#178 B12.0 Refinements to DOI Service and UI for Initial Release
- pds-doi-ui#13 Enable to remind previously used value in submitter and node for reserve
- pds-doi-ui#20 Autocompletion for doi or lidvid selection in release form
- pds-doi-ui#22 After reserve one file successfully, resubmit a new file
- pds-doi-service#153 Prepare API and UI for production deployment
- pds-doi-service#163 Dockerize API Service
- pds-doi-ui#34 As a user, I want to see the error/warnings messages on the same page from where they were raised
- pds-doi-service#176 As an API user, I want to have pagination that is consistent with the PDS API.
- pds-doi-service#177 As an API user I want to filter on lidvids with wildcards
- pds-doi-service#180 As an API user I want to filter on PDS3 Data Set IDs with wildcards
- pds-doi-service#188 As a user, I want to make sure I can not override existing DOI with new LIDVID
- pds-doi-service#192 As a user, I want the application to support the history of PDS’s DOIs, especially the one created for PDS3 products
pds-doi-service#86 Deploy DOI API and UI
- pds-doi-service#87 Dev beta testing with API
- pds-doi-service#88 Operations team test interface with DOI Service
- pds-doi-service#89 Update procedures to use DOI web interface submission
- pds-doi-service#90 Deploy point build of DOI service and UI
- pds-doi-service#145 review the full doi workflow and make it smooth for eng operators/users
- pds-doi-ui#35 As a SA, I don’t want security vulnerabilities in the public UI
- pds-doi-service#187 As a SA, I want the application to be deployed without security risks

pds-registry-app

Registry application enabling a PDS node to register all its data products for long term preservation and sharing with the rest of the PDS system.

pds-registry-app#155 [registry] Registry Regression Test and Documentation
- pds-registry-app#153 As a developer, I want to extend the registry-mgr and harvest using the Java API
- pds-registry-app#154 As a manager, I want the registry to have regression tests included in CI
- pds-registry-app#156 As a user, I want to uninstall the Registry and all its components
- pds-registry-app#157 As a user, I want to upgrade the Registry and all its components
- pds-registry-app#158 Update Registry App landing page to divert data users to separate landing page for using the PDS API
pds-registry-app#152 Pilot DN Registry + API AWS Deployments
- pds-registry-app#90 Develop cost model and reporting for Registry deployments

pds-api

PDS API Application with client and server integrated into one package

pds-api#24 [registry] Handle PDS Supplemental Metadata
- pds-registry-app#12 As a user, I want to ingest supplemental metadata for PDS products.
- pds-api#67 As an API user, I want to access supplemental metadata for PDS products.
pds-api#81 [pds-api] B12.0 Improve API query handling
- pds-api#71 As a user, I want to do a complex query to the API without being limited by the length of the url request
- pds-api#72 As an API user, I want to search by a temporal range as an ISO-8601 time interval.
- pds-api#83 As an API user, I want to search using URL parameters
pds-api#84 [pds-api] Initial Google-like Search
- pds-registry-app#49 Enhance weighting and expected search results
pds-api#75 [pds-api] B12.0 API Response Improvements
- pds-api#65 As an API user, I want to get only the properties I explicitly requested.
- pds-api#66 As an API user, I want to get ‘application/xml’ content which matches the json structure
- pds-api#68 As an API user, I want to know in the response how many hits are returned for an API query.
- pds-api#70 As an API user, I want to know the unique values for a specific API field.
- registry-api-service#26 As a user, I want to know why my query syntax is invalid
pds-api#76 [pds-api] Improve API Performance
- registry-api-service#13 As a developer, I want to utilize ElasticSearch performance robustness for API response time requirements.
- registry-api-service#18 As an API user, I want an average query response time of 1 second.
- registry-api-service#19 As an API user, I want to handle long-running queries that take >10 seconds.
- pds-api#80 As a developer, I never want the label blob to be returned
pds-api#77 [pds-api] PDS4 Product Relationships
- pds-api#56 As an API user, I want to know the children and ancestors of bundle, collections, and products
- pds-api#59 As an API user, I want to know the Product(s) that belong to a given Bundle.
- pds-api#60 As an API user, I want to know the Bundle for a given Product.
- pds-api#61 As an API user, I want to know the Collection(s) for a given Product.
- pds-api#62 As an API user, I want to know the Bundle for a given Collection.
pds-api#79 [pds-api] PDS Core Registry
- pds-registry-app#150 Develop plan for instituting AWS ES Cross Cluster Search

pdsen-corral

*umbrella project to manage the build cycle (continuous integration and stable release) for the PDS EN tools. *

pdsen-corral#28 B12.0 Release Doc Improvements
- pdsen-corral#19 Propose a good management of requirements for parent/child components
- pds-github-util#7 As a PDS software user, I want to have a hierarchical view of the PDS EN components that apply to my user interests

PDS.nasa.gov-UX

PDS.nasa.gov User Experience Task Issue and Prototype repository

PDS.nasa.gov-UX#78 Initial Design Validation and Usability Testing
PDS.nasa.gov-UX#77 Initial Information Architecture
PDS.nasa.gov-UX#79 Initial Visual Design
PDS.nasa.gov-UX#76 User Personas and Stories
PDS.nasa.gov-UX#75 User Testing for NASA CMS Trade Study
PDS.nasa.gov-UX#74 Plan and Kick-off PDS WMWG
- PDS.nasa.gov-UX#73 Rev 3 of milestones to breakdown of notional goals and deliverables

pdsen-operations

Tickets for the PDSEN Operations Team

pdsen-operations#94 Develop and implement PDS CD Strategy
- pds-api#51 As a developer, I want a continuous deployment of the API available for testing
- pdsen-operations#89 As an SA, I want a monitoring capability to check that PDS search and access services are up and running 99.9% of the time
- pdsen-operations#95 As a developer, I want a system for continuous deployment of PDS EN services in development.
pdsen-operations#93 Refine Continuous Integration processes
- roundup-action#31 Improve roundup action for tagging and releasing software packages
- roundup-action#36 Java documentation is empty when attached to repo
- pdsen-operations#90 As a developer, I want to make sure I have access to all our Python PyPi package repos.
- pdsen-operations#91 As an EN developer, I want to make sure I have access to all PDS repos.
- roundup-action#37 Document stable release CI/CD
- roundup-action#42 Roundup environment doesn’t appear to be taking packages seriously

cloud-initiative

PDS Cloud Migration documentation, issue, tracking and simple tools for assisting in the PDS hybrid cloud study and migration efforts.

cloud-initiative#2 B12.0 Refine AWS Registry Deployment Strategy
- pds-registry-app#89 Document AWS deployment for Registry
- pds-registry-app#122 Deploy the API on AWS as test
- cloud-initiative#1 As a manager, I want a cost monitoring capability for AWS deployments
- pds-registry-app#151 Employ budget alerts and if possible, rate/egress limits on AWS services

Other Information

Reference Documents

This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:

PDS Level 1, 2 and 3 Requirements, April 20, 2017.
PDS4 Project Plan, July 17, 2013.
PDS4 System Architecture Specification, Version 1.3, September 1, 2013.
PDS4 Operations Concept, Version 1.0, September 1, 2013.
PDS General System Software Requirements Document (SRD), Version 1.1, September 1, 2013.
PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.
PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.
PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.
- PDS Search Scenarios, Version 1.0, September 1, 2013.
- PDS Search Protocol, Version 1.2, March 21, 2014.
- PDAP Search Protocol, Version 1.0, March 21, 2014.
PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS NSSDC Delivery Software Requirements and Design Document (SRD/SDD), Version 0.1, October 29, 2019.

System Requirements, Dependencies, and Assumptions

See individual applications for specific dependencies and system requirements.