Build 13.1 Plan

The Planetary Data System (PDS) is a long-term archive of digital data products returned from NASA’s planetary missions, and from other kinds of flight and ground-based data acquisitions, including laboratory experiments. But it is more than just a facility - the archive is actively managed by planetary scientists to help ensure its usefulness and usability by the world wide planetary science community.

The following page outlines the planned PDS software enhancements, bug fixes, and impacted requirements for the upcoming Build .

A more detailed breakdown of these tasks by date and priority can be found in our Zenhub Roadmap (login required).

Key Dates

Key Dates for Discipline Nodes

Date

Name

Responsible Party

Description

03/02/2022

SCR Freeze Date

DDWG

Freeze date for identifying SCRs to be implemented in this release.

04/06/2023

Standards Documents Updates Due

Document Authoring Team

Updates to PDS4 standards documents due to EN for review.

04/18/2023 - 05/16/2023

Beta Testing Period

DNs, IPDA

Changes to PDS4 IM and system components available for beta testing.

04/18/2023 - 05/16/2023

dLDD Integration and Test

dLDD Stewards

Auto-generated dLDDs should be reviews by dLDD Stewards, and final updates to dLDDs should be completed during this time.

05/08/2023

Standards Documents Review Due

Document Review Team

Review updates to PDS4 standards documents.

06/02/2023

System Release

EN

PDS4 system is operationally deployed and website is updated accordingly.
Key Dates for Engineering Node

Date

Name

Description

03/21/2023

Delivery to I&T

The PDS4 IM and all software updated during this build cycle are delivered to EN I&T.

04/03/2023

Test Readiness Review (TRR)

PDS EN internal review to verify readiness to enter I&T phase.

05/22/2023

Delivery and Deployment Review (DRR)

PDS internal review with PDS Software Working Group to verify readiness to deploy build system.

06/02/2023

Operational Deployment

PDS4 system is operationally deployed and website is updated accordingly.

For a more detailed schedule, see https://pds-engineering.jpl.nasa.gov/content/schedules.

Planned PDS4 Information Model Changes

This section details the planned changes to the PDS4 Information Model for this build. These changes are deliverables by the PDS Data Design Working Group (DDWG) to the Engineering Node for implementation.

Planned Software Changes

This section details the planned software changes for this build.

Planned tasks for the next release cycle are grouped by software component, and sub-tasks are included where known at the time of this plan generation. Each section also includes additional information, such as:

  • Software description

  • Link to 🔍 User Guide

  • Link to 💻 Github Repo

  • Link to 🚑 Issue Tracking

  • Link to 📒 Backlog - future work not planned for this release cycle

Don’t see the tool you are looking for? This means there is no active development planned for this build cycle. Head to the PDS Tool Registry to see all possible tools available for PDS data.

Have a new feature or tool you think we should be working on? Create a ticket in the applicable Github repo or in our PDS EN Operations repo and we will see how that can fit into our plans moving forward.

validate

Validates PDS4 product labels, data and PDS3 Volumes

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release
  • validate#414 Referential Integrity Checking with the Registry

    • validate#415 As a user, I want to perform referential integrity checking against a registry

    • validate#462 As a user, I want validate to throw an error when the LID specified for secondary context product in a collection inventory is not valid

  • validate#498 Improve Content Validation Performance through Spot Checking

    • validate#1 As a user, I want to execute content validation against every nth file

  • validate#534 B13.1 Content Validation Improvements: Additional Table Types, Additional File Areas

    • validate#217 As a user, I want to validate content for all possible PDS4 table types

    • validate#436 Improve error messages for overlapping objects in a label

    • validate#480 Validate does not calculate overlaps correctly when Header is not first object in file

  • validate#557 Dockerize Validate

    • validate#556 As a user, I want to be able to use validate from a docker container

  • validate#578 B13.1 Fix Must-Have Priority Bugs

    • validate#444 pds4.bundle option seems to not travel through enough subdirectories

    • validate#453 Validate should not check PDF/A validity if content validation is disabled

    • validate#474 Validate can’t find files in directory specified by <directory_path_name>

    • validate#499 validate doesn’t flag a data file with only LF

    • validate#503 validate passes confusing message to the command window

    • validate#511 Table_Character not accurately checking field formats

    • validate#519 Validate should throw record length error when record delimiter does not occur in correct location

    • validate#529 ERROR [error.array.value_out_of_min_max_range] evaluation is not correct

    • validate#531 ERROR [error.table.bad_file_read] incorrectly reports that GroupFieldBinary group_length is larger than size of contained fields

    • validate#544 validate gives a error.table.bad_field_read error

    • validate#554 –spot-check-data flag throws IOException

    • validate#564 Array object validation regression in v3.0.3

    • validate#576 validate does not correctly handle field format checks for hex values

pds4-information-model

The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

pds-api

PDS web APIs specifications and user’s manual

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release
  • pds-api#181 Search Improvements to Support Web Modernization

    • pds-api#104 Improvements to Keyword Search for Weighting and Suggested Searches

    • pds-api#165 As a user, I want to search for PDS software tools and services

  • pds-api#182 Implement Faceting

    • pds-api#70 As an API user, I want to know the unique values for a specific API field.

    • pds-api#103 As a client developer, I want to facet on 1 or more fields in the registry

  • pds-api#230 Support latest product search from API

    • pds-api#220 Past versions are being returned by API, by default

    • pds-api#221 As a user, I want to query only the latest versions of products unless explicitly requested

    • pds-api#236 Make the registry-api use the new latest index

    • registry#140 Integrate new provence.py script into integration tests to tag products with Provenance information

    • registry#141 Integrate new provence.py script into operational deployment to tag products with Provenance information

  • pds-api#235 Enable Swagger Interface on pds.nasa.gov

    • pds-api#193 As a user, I want to be able to access the Search API Swagger interface from pds.nasa.gov

    • pds-api#239 As a user, I want to get application/json response format by default if I request an API url in my browser

  • pds-api#237 Support Searching for Past Versions of Products with Differing LIDs

    • pds-api#148 As a user, I want to search for past versions of a product where the LID changed during the product history

wds-implementation

PDS Web Design System

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

design-team

PDS.nasa.gov User Experience Task Issue and Prototype repository

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

cloud-tasks

PDS Cloud Migration documentation, issue, tracking and simple tools for assisting in the PDS hybrid cloud study and migration efforts.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

planetarydata.org

Website and related services for the International Planetary Data Alliance, nominally run at https://planetarydata.org/

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

nucleus

None

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • nucleus#2 Initial Implementation and Operationalize Nucleus

  • nucleus#14 B13.1 Technology Selection and Internal Review

    • nucleus#11 Proof-of-concept with Apache Airflow

    • nucleus#13 Proof of concept with Cumulus on NGAP

    • nucleus#15 Proof of concept for Nucleus in CWS

    • nucleus#19 Complete JPL Internal Peer Review of Trade Study and Technology Selection

  • nucleus#16 Develop Cost Model

  • nucleus#17 Pilot Nucleus with SBN-PSI Catalina Sky Survey Pipeline

  • nucleus#18 Develop Detailed Technical Architecture

registry

PDS Registry provides service and software application necessary for tracking, searching, auditing, locating, and maintaining artifacts within the system. These artifacts can range from data files and label files, schemas, dictionary definitions for objects and elements, services, etc.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release
  • registry#85 Enhance Registry Delete Functionality
  • registry#99 Populate EN Registry with PDS3 Data Sets

    • registry#62 As a user, I want to ingest legacy PDS3 data sets from EN legacy registry

    • registry#63 Deploy harvest / registry-mgr as cron to regularly upload PDS EN data to registry

    • operations#270 Deployment of Registry Loader Tools and Initial Ingestion of Engineering Node Registry on Latest AWS Deployment

  • registry#134 Enhanced Query Support through Registry Tools

    • registry#100 Update documentation to include explicit example of how to query staged data

    • registry-mgr#59 As a user, I want to query OpenSearch for staged data via a CLI command

Other Information

Reference Documents

This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:

  • PDS Level 1, 2 and 3 Requirements, April 20, 2017.

  • PDS4 Project Plan, July 17, 2013.

  • PDS4 System Architecture Specification, Version 1.3, September 1, 2013.

  • PDS4 Operations Concept, Version 1.0, September 1, 2013.

  • PDS General System Software Requirements Document (SRD), Version 1.1, September 1, 2013.

  • PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.

  • PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.

  • PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.

    • PDS Search Scenarios, Version 1.0, September 1, 2013.

    • PDS Search Protocol, Version 1.2, March 21, 2014.

    • PDAP Search Protocol, Version 1.0, March 21, 2014.

  • PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS NSSDC Delivery Software Requirements and Design Document (SRD/SDD), Version 0.1, October 29, 2019.

System Requirements, Dependencies, and Assumptions

See individual applications for specific dependencies and system requirements.