Build 12.1 Plan
The Planetary Data System (PDS) is a long-term archive of digital data products returned from NASA’s planetary missions, and from other kinds of flight and ground-based data acquisitions, including laboratory experiments. But it is more than just a facility - the archive is actively managed by planetary scientists to help ensure its usefulness and usability by the world wide planetary science community.
The following page outlines the planned PDS software enhancements, bug fixes, and impacted requirements for the upcoming Build .
A more detailed breakdown of these tasks by date and priority can be found in our Zenhub Roadmap (login required).
Key Dates
Date |
Name |
Responsible Party |
Description |
|---|---|---|---|
02/25/2022 |
SCR Freeze Date |
DDWG |
Freeze date for identifying SCRs to be implemented in this release. |
04/21/2022 |
Standards Documents Updates Due |
Document Authoring Team |
Updates to PDS4 standards documents due to EN for review. |
05/02/2022 - 05/20/2022 |
Beta Testing Period |
DNs, IPDA |
Changes to PDS4 IM and system components available for beta testing. |
04/22/2022 - 05/20/2022 |
dLDD Integration and Test |
dLDD Stewards |
Auto-generated dLDDs should be reviews by dLDD Stewards, and final updates to dLDDs should be completed during this time. |
05/10/2022 |
Standards Documents Review Due |
Document Review Team |
Review updates to PDS4 standards documents. |
06/03/2022 |
System Release |
EN |
PDS4 system is operationally deployed and website is updated accordingly. |
Date |
Name |
Description |
|---|---|---|
03/25/2022 |
Delivery to I&T |
The PDS4 IM and all software updated during this build cycle are delivered to EN I&T. |
04/07/2022 |
Test Readiness Review (TRR) |
PDS EN internal review to verify readiness to enter I&T phase. |
05/23/2022 |
Delivery and Deployment Review (DRR) |
PDS internal review with PDS Software Working Group to verify readiness to deploy build system. |
06/03/2022 |
Operational Deployment |
PDS4 system is operationally deployed and website is updated accordingly. |
For a more detailed schedule, see https://pds-engineering.jpl.nasa.gov/content/schedules.
Planned PDS4 Information Model Changes
This section details the planned changes to the PDS4 Information Model for this build. These changes are deliverables by the PDS Data Design Working Group (DDWG) to the Engineering Node for implementation.
Planned Software Changes
This section details the planned software changes for this build.
Planned tasks for the next release cycle are grouped by software component, and sub-tasks are included where known at the time of this plan generation. Each section also includes additional information, such as:
Software description
Link to 🔍 User Guide
Link to 💻 Github Repo
Link to 🚑 Issue Tracking
Link to 📒 Backlog - future work not planned for this release cycle
Don’t see the tool you are looking for? This means there is no active development planned for this build cycle. Head to the PDS Tool Registry to see all possible tools available for PDS data.
Have a new feature or tool you think we should be working on? Create a ticket in the applicable Github repo or in our PDS EN Operations repo and we will see how that can fit into our plans moving forward.
Data Stewardship Tools
Validate
Validates PDS4 product labels, data and PDS3 Volumes
- validate#407 B12.1 Referential Integrity Improvements
validate#307 As a user, I want to validate that all XML Schema collections in NASA PDS archives are secondary collections
- validate#409 B12.1 PDF/A Handling Improvements
validate#388 Improve PDF/A validation to include more robust reporting on failures
validate#412 Update PDF validation to check against flavour specified in PDF metadata
- validate#414 Referential Integrity Checking with the Registry
validate#415 As a user, I want to perform referential integrity checking against a registry
validate#422 B12.1 Re-imagine Output Logging for Validity vs Metadata Quality
- validate#426 B12.1 Content Validation Improvements
validate#423 Validate does not allow a single-character subdirectory
validate#424 Validate does not allow SIP tab file to have lines of differing lengths
validate#425 Refactor content validation to more robustly handle intermingled Headers
LDDTool and PDS4 Information Model
The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.
- pds4-information-model#306 Refactoring of IMTool
pds4-information-model#239 Refactor Class information from config to Data Dictionary Protege ontology
pds4-information-model#281 Tag ops classes/attributes in IM so they are easily identifiable by users
- pds4-information-model#356 Kick-off PDS4 Information Model in GraphDB
pds4-information-model#357 Ingest PDS4 IM into BlazeGraph
pds4-information-model#358 Replicate diagram from PDS4 IM Spec
pds4-information-model#359 Deploy new Amazon Neptune instance for PDS4 IM prototype
pds4-information-model#387 As an operator, I want to be able to convert from XML to RDF using the PDS4 Information Model
pds4-information-model#410 B12.1 LDDTool / PDS4 IM Documentation Updates
PDS Deep Archive
PDS Open Archival Information System (OAIS) utilities, including Submission Information Package (SIP) and Archive Information Package (AIP) generators
- pds-deep-archive#115 Revisit PDS Deep Archive Integration with Registry
pds-deep-archive#107 Update PDS Deep “Registry” Archive and remove workaround + implement latest-only-feature per API updates
DOI Services
PDS DOI Service
Service and tools for generating DOIs for PDS bundles, collections, and data sets
- pds-doi-service#251 B12.1 Enhancements to PDS DOI Metadata
pds-doi-service#102 Add new alternateIdentifier to match SBN schema
pds-doi-service#224 As a user, I would like to include licensing information with all PDS DOIs
pds-doi-service#282 Improve metadata based upon parameters ADS keys off of
pds-doi-service#283 As a publisher, I want to be notified when a new DOI has been minted or significant update to the metadata
- pds-doi-service#252 B12.1 Enhancements to DOI Search Component
pds-doi-ui#61 As a PDS.nasa.gov user, I want to search for the DOI for a PDS4 data product from a public facing website (pds.nasa.gov)
pds-doi-ui#83 Combine NPM Package And Embeddable JS Script Toolchains
pds-wds-react#4 Add Search DOI Component
pds-doi-ui#88 Pre-existing keyword show i a weird way in the UI
- pds-doi-service#270 B12.1 DOI Metadata and Update Functionality Improvements
pds-doi-service#257 As a PDS Operator, I want to perform a bulk update of a specific field across many DOI records
pds-doi-service#278 As a user, I want to update the LIDVID associated with a DOI
PDS DOI UI
web UI for pds-doi-service
pds-doi-ui#60 DOI Search Component
Registry + API Services
PDS Registry App
Registry application enabling a PDS node to register all its data products for long term preservation and sharing with the rest of the PDS system.
- pds-registry-app#184 B12.1 Registry Tools Performance Improvements
pds-registry-app#179 As a user, I want the registry app components to be able to ingest data sets containing 1+ million products
pds-registry-mgr-elastic#31 Update registy-mgr to use schema from schemaLocation in file
pds-registry-app#195 Big Data Harvest analysis / POC
pds-registry-app#200 As a user, I want big data ingestion tools packaged and dockerized together
- pds-registry-app#190 Support Staging vs. Operational Data
pds-api#106 As a API manager, I want to restrict access to registered products that should not be publicly accessible
pds-registry-app#201 As a user, I want to be able to ingest a set of files specified in a file listing
- pds-registry-app#191 B12.1 Initial external source integration
pds-registry-app#176 As a registry user, I want to ingest supplemental metadata from external data sources
pds-api#90 As an API user, I want to search for registered supplemental metadata ingested from external data sources
- pds-registry-app#194 Populate EN Registry
pds-registry-app#192 As a user, I want to ingest legacy PDS3 data sets
pds-registry-app#193 Deploy harvest / registry-mgr as cron to regularly upload PDS EN data to registry
pds-registry-app#199 Dockerize Registry Components for Dev and Ops Deployments
PDS API
PDS API Application with client and server integrated into one package
- pds-api#114 Improve PDS API Development Workflow and Versioning
pds-api#122 analysis of current workflow, ideas for improvment
pds-api#123 create a super registry+api repository
- pds-api#117 [pds-api] B12.1 API Response Improvements
pds-api#45 As a user, I want the API to be schema.org compliant
pds-api#65 As an API user, I want to get only the fields I explicitly requested
pds-api#66 As an API user, I want to get an XML response
pds-api#68 As an API user, I want to know in the response how many hits are returned for an API query.
registry-api-service#33 As a user, I want to get a 404 error when I attempt to request a resource which does not exists
pds-api#88 As a user,I want to get the list of properties available in summary sorted in alphabetical order
pds-api#89 As a user, I want to know the DOI associated with a product
pds-api#110 Extend application/pds4+json support to all endpoints
pds-api#118 As an API user, I want to receive a HTML response
pds-api#120 As an API user, I want a CSV response format option
pds-api#125 As a user, I want to get the PDS4 original XML label
pds-api#127 As a user, I want to clearly see which formats are accepted by the API
PDS Web Modernization and Unification
PDS Web Design
PDS.nasa.gov User Experience Task Issue and Prototype repository
- PDS.nasa.gov-UX#100 B12.1 Continuation of User Persona and User Story Development
PDS.nasa.gov-UX#99 Document initial User Stories using WMWG feedback
PDS.nasa.gov-UX#103 Iterate on WMWG feedback
PDS.nasa.gov-UX#107 Tag user stories to modules and templates
PDS.nasa.gov-UX#101 Deploy new Citing PDS Data Web Pages
PDS Web Implementation
PDS Web Implementation Team assessing and prioritizing the unification of the PDS web presence.
pds-web-mgmt#1 Initial Assessment of Today’s Web Management Landscape
pds-web-mgmt#2 Preliminary Design of Migration
pds-web-mgmt#3 High level Implementation Plan
Cloud Strategy and Services
PDS Cloud Migration documentation, issue, tracking and simple tools for assisting in the PDS hybrid cloud study and migration efforts.
- cloud-strategy#3 B12.1 pds.nasa.gov Website Lift-and-Shift Pilot Project
cloud-strategy#9 AWS testing for PDS and PDS Engineering sites
- cloud-strategy#4 B12.1 pds-engineering.nasa.gov Website Lift-and-Shift Pilot Project
cloud-strategy#9 AWS testing for PDS and PDS Engineering sites
cloud-strategy#5 B12.1 NAIF.nasa.gov Website Lift-and-Shift Pilot Project
cloud-strategy#6 B12.1 pds-imaging.jpl.nasa.gov Website Lift-and-Shift Pilot Project
- cloud-strategy#7 B12.1 Refine AWS Registry Deployment Strategy
pds-registry-app#87 Deploy GEO Registry in AWS
pds-registry-app#88 Deploy IMG Registry in AWS
pds-registry-app#89 Document AWS deployment for Registry
pds-registry-app#90 Develop cost model and reporting for Registry deployments
pds-doi-service#164 Develop AWS deployment architecture and test deployment for DOI Service
cloud-strategy#1 As a manager, I want a cost monitoring capability for AWS deployments
registry-api-service#22 Work w/ SAs to define production deployment strategy in AWS
registry-api-service#78 Add aws cost tagging and secret/parameter creation to ECS terraform script
cloud-strategy#8 Develop Roadmap and Cost Model for PDS Cloud
cloud-strategy#10 PDS-ESDIS NGAP Pilot Project
Nucleus
Pipeline for automated, streamlined deployment and execution of PDS validation, ingestion, and transformation services.
Other EN Internal Plans
PDS Github Util
github utility functions to enforce the PDS engineering node software life cycle.
- pds-github-util#47 B12.1 Software Release Documentation Updates
pds-github-util#14 As a user, I want to see the status of the tickets shown in the current build plan
Systems Engineering
Issue tracking repository as a centralized entry point for general PDS software bugs and feature requests.
- PDS-Software-Issues-Repo#14 B12.0 Prep for I&T
PDS-Software-Issues-Repo#15 Generate Draft B12.0 RDD
PDS-Software-Issues-Repo#16 Final testing and delivery to I&T
PDS-Software-Issues-Repo#17 B12.1 Generate Release Plan
DevOps
Parent repo for PDS DevOps activities
- devops#13 Continuous improvements to Continuous Integration
pds-registry-app#177 stable continuous integration don’t generate the packages tar.gz and .zip properly.
devops#10 Rollout template updates to existing repos
roundup-action#67 Roundup’s Python Environment Should Be Separate from the Packages It Builds
devops#12 Remove Versioneer
devops#14 Update Stable Major Releases of PDS software with DOIs
devops#15 Update stable releases to handle Java repos
roundup-action#69 Roundup handling VERSION.txt
pds-github-util#48 Versions seem to be not incrementing correctly for python unstable releases
Schedule
Other Information
This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:
PDS Level 1, 2 and 3 Requirements, April 20, 2017.
PDS4 Project Plan, July 17, 2013.
PDS4 System Architecture Specification, Version 1.3, September 1, 2013.
PDS4 Operations Concept, Version 1.0, September 1, 2013.
PDS General System Software Requirements Document (SRD), Version 1.1, September 1, 2013.
PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.
PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.
PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
- PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.
PDS Search Scenarios, Version 1.0, September 1, 2013.
PDS Search Protocol, Version 1.2, March 21, 2014.
PDAP Search Protocol, Version 1.0, March 21, 2014.
PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS NSSDC Delivery Software Requirements and Design Document (SRD/SDD), Version 0.1, October 29, 2019.
See individual applications for specific dependencies and system requirements.