Build 14.1 Plan

The Planetary Data System (PDS) is a long-term archive of digital data products returned from NASA’s planetary missions, and from other kinds of flight and ground-based data acquisitions, including laboratory experiments. But it is more than just a facility - the archive is actively managed by planetary scientists to help ensure its usefulness and usability by the world wide planetary science community.

The following page outlines the planned PDS software enhancements, bug fixes, and impacted requirements for the upcoming Build .

A more detailed breakdown of these tasks by date and priority can be found in our Zenhub Roadmap (login required).

Key Dates

Key Dates for Discipline Nodes

Date

Name

Responsible Party

Description

02/12/24

SCR Freeze Date

DDWG

Freeze date for identifying SCRs to be implemented in this release.

02/13/24 - 04/12/24

Standards Documents Updates Due

Document Authoring Team

Updates to PDS4 standards documents due to EN for review.

04/23/24 - 05/21/24

Beta Testing Period

DNs, IPDA

Changes to PDS4 IM and system components available for beta testing.

04/23/24 - 05/21/24

dLDD Integration and Test

dLDD Stewards

Auto-generated dLDDs should be reviews by dLDD Stewards, and final updates to dLDDs should be completed during this time.

04/12/24 - 05/22/24

Standards Documents Review Due

Document Review Team

Review updates to PDS4 standards documents.

06/07/24

System Release

EN

PDS4 system is operationally deployed and website is updated accordingly.

Key Dates for Engineering Node

Date

Name

Description

03/21/24

Delivery to I&T

The PDS4 IM and all software updated during this build cycle are delivered to EN I&T.

04/8/24

Test Readiness Review (TRR)

PDS EN internal review to verify readiness to enter I&T phase.

05/30/24

Delivery and Deployment Review (DRR)

PDS internal review with PDS Software Working Group to verify readiness to deploy build system.

06/07/24

Operational Deployment

PDS4 system is operationally deployed and website is updated accordingly.

For a more detailed schedule, see https://pds-engineering.jpl.nasa.gov/content/schedules.

Planned PDS4 Information Model Changes

This section details the planned changes to the PDS4 Information Model for this build. These changes are deliverables by the PDS Data Design Working Group (DDWG) to the Engineering Node for implementation.

Planned Software Changes

This section details the planned software changes for this build.

Planned tasks for the next release cycle are grouped by software component, and sub-tasks are included where known at the time of this plan generation. Each section also includes additional information, such as:

  • Software description

  • Link to 🔍 User Guide

  • Link to 💻 Github Repo

  • Link to 🚑 Issue Tracking

  • Link to 📒 Backlog - future work not planned for this release cycle

Don’t see the tool you are looking for? This means there is no active development planned for this build cycle. Head to the PDS Tool Registry to see all possible tools available for PDS data.

Have a new feature or tool you think we should be working on? Create a ticket in the applicable Github repo or in our PDS EN Operations repo and we will see how that can fit into our plans moving forward.


validate

Validates PDS4 product labels, data and PDS3 Volumes

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • validate#695 Update Validate Test Suite to Support Larger Data Sets
  • validate#696 Use New Registry API for Context Product Validation
    • validate#592 As a user, I want to when a context product has been deprecated

    • validate#675 Cutover to using new Registry API for generating context products json


pds4-information-model

The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


harvest

Standalone Harvest client application providing the functionality for capturing and indexing product metadata into the PDS Registry system (https://github.com/nasa-pds/registry).

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • harvest#129 Add Support for LBLX File Extension
    • harvest#130 As a user, I want to ingest data products with labels having .lblx file extension

  • harvest#131 Enhance support for searching lid/lidvid references
    • registry#215 Verify fields types are changed when LDD fields are reloaded


software-issues-repo

Issue tracking repository as a centralized entry point for general PDS software bugs and feature requests.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


portal-wp-tasks

PDS Web Design System

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


operations

Tickets for the PDSEN Operations Team

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


roundup-action

Do a “roundup”, a/k/a PDS-style continuous integration and delivery

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


doi-ui

The web interface for the PDS DOI Service providing the ability management PDS archive DOIs. See the DOI Service for more details on the available capabilities. https://nasa-pds.github.io/doi-service/

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • doi-ui#184 Develop DOI Service User Guide and Training


search-api-notebook

Jupyter notebooks for demonstrating and utilizing the Planetary Data System (PDS) Search API

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


planetary-data-cloud

PDS Cloud Migration documentation, issue, tracking and simple tools for assisting in the PDS hybrid cloud study and migration efforts.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


devops

Parent repo for PDS DevOps activities

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • devops#37 Automatically deploy an up to date I&T platform where all software can be tested (Part 2)
    • registry-api#269 Add registry/docker compose integration tests to github action on dev branches

    • devops#36 For registry components, add a github action which runs the integration test when something is pushed on a dev branch

    • registry-sweepers#6 Create the docker image and push it to docker hub as part of the CICD

    • devops#41 Push docker image on AWS ECR in github-actions, in addition to docker hub

    • devops#44 Ask new Application developer profile on JPL AWS

    • devops#45 Use Github Actions Github-Hosted Runner to deploy to NGAP

    • devops#46 Fully terraform the registry application

    • devops#47 Other CICD improvements

    • devops#48 Use JPL AWS jenkins to launch terraform scripts


nucleus

Nucleus is a software platform used to create workflows for the Planetary Data (PDS).

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • nucleus#17 CSS MVP: Deploy Baseline Nucleus for Catalina Sky Survey

  • nucleus#66 MCP Pilot: Pilot Nucleus Deployment for CSS and MESSENGER MDIS Data
    • nucleus#52 Work with IMG to connect to S3 Bucket with MESSENGER data

    • nucleus#53 Deploy and Test Nucleus DAG for MESSENGER processing

    • nucleus#54 Validate and Load all PDS4 MESSENGER data products with Nucleus

    • nucleus#61 Manually validate and load MESSENGER MSGRMDS_4001 and MESSDEM_1001 data

    • nucleus#68 Setting-up MWAA Environment on MCP

    • nucleus#74 Support processing of PDS products with fitz file format in Nucleus

  • nucleus#62 Develop Support For Off-Nominal Data Loads


registry

PDS Registry provides service and software application necessary for tracking, searching, auditing, locating, and maintaining artifacts within the system. These artifacts can range from data files and label files, schemas, dictionary definitions for objects and elements, services, etc.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • registry#155 Registry Multi-tenancy Design and implementation with Cognito
    • registry#179 Implement authorization handshake with API Gateway and Lambda for multi-tenancy approach

  • registry#178 Improved Fault Tolerance for Registry and Registry API
    • registry-common#26 [SECURITY] Fix code scanning alert - TrustManager that accepts all certificates

    • harvest#101 Unexpected WARNINGs related to http requests

    • registry-api#292 Queries for data products with lots of metadata attributes crash in browser

    • registry-api#300 Update Java params and Docker image to expand JVM memory beyond 50%

    • harvest#119 As a developer, I want to know what version of Harvest was used to load a product

    • registry-api#337 Enhance robustness of API Monitoring

    • registry-sweepers#25 Enhance provenance sweeper with more graceful failure when registry contains zero documents

    • harvest#125 Improve Fault Tolerance of Harvest for Forbidden Access error and Timeout

    • registry#211 As an operator, I want to be notified of when Registry storage capacity exceeds 75% capacity.

    • registry-api#361 As a user, I want my API request to execute successfully even when the registry contains corrupted documents

    • registry#224 Write documentation and Training for Registry monitoring

    • registry#225 Registry-Sweepers Multiple Updates Detected

    • registry-sweepers#69 Registry-Sweepers Error: contained no hits when hits were expected

    • registry-api#378 Investigate and fix intermittent Registry-API Errors

    • harvest#133 Data too large error from very large data products

    • harvest#134 Too many requests error to OpenSearch

  • registry#181 Support Registration and Search of Products Replicated Across Archives
    • registry-api#306 If >1 products with the same LIDVID are registered by different nodes, we should respond with one of them

    • registry#245 As a user, I want to register data product(s) that is mirrored across 2 or more organizations

  • registry#185 Implement Registry Multi-tenancy without Cognito in the loop
  • registry#230 Enable Support for Changing Field Types

  • registry#231 Design and Implement Support for Alternate Data File Paths
    • registry#96 As a user I want to change the file paths of products in the registry

    • harvest#123 As a Node Operator, I want to specify an alternate file paths for 1 or more archive products

    • registry#209 As a user, I want to manually update a product’s file location (file_ref)


registry-api

Web API service for the PDS Registry, providing the implementation of the PDS Search API (https://github.com/nasa-pds/pds-api) for the PDS Registry.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

  • registry-api#285 Implement Faceting
    • registry-api#283 As a client developer, I want to facet on 1 or more fields in the registry

    • registry-api#284 As an API user, I want to know the unique values for a specific API field.

    • registry-api#323 As a user, I want the /properties endpoint to display counts for products having each property

    • registry-api#324 As a user, I want the /properties endpoint to display range/enum type and values for each property

    • registry-api#325 As a user, I want the /properties endpoint to include a link to explanatory documentation (i.e. DD or similar)


planetary-data-engine

Free-text search capability for planetary data, services, tools, and information

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release


data-upload-manager

Data Upload Manager (DUM) component for managing the interface for data uploads to the Planetary Data Cloud from Data Providers and PDS Nodes.

User Guide

Github Repo

Issue Tracking

Backlog

Stable Release

Dev Release

Other Information

Reference Documents

This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:

  • PDS Level 1, 2 and 3 Requirements, April 20, 2017.

  • PDS4 Project Plan, July 17, 2013.

  • PDS4 System Architecture Specification, Version 1.3, September 1, 2013.

  • PDS4 Operations Concept, Version 1.0, September 1, 2013.

  • PDS General System Software Requirements Document (SRD), Version 1.1, September 1, 2013.

  • PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.

  • PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.

  • PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.
    • PDS Search Scenarios, Version 1.0, September 1, 2013.

    • PDS Search Protocol, Version 1.2, March 21, 2014.

    • PDAP Search Protocol, Version 1.0, March 21, 2014.

  • PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS NSSDC Delivery Software Requirements and Design Document (SRD/SDD), Version 0.1, October 29, 2019.

System Requirements, Dependencies, and Assumptions

See individual applications for specific dependencies and system requirements.