Release Description Document (Build B18)

This release of the PDS4 System is intended as an operational release of the system components to date. The original plan for this release can be found here: plan B18

The following sections can be found in this document:

PDS4 Standards and Information Model Changes

This section details the changes to the PDS4 Standards and Information Model approved by the PDS4 Change Control Board and implemented by the PDS within the latest build period.

Ref

Title

pds4-information-model#1002

CCB-75: Add new value for object type - “Interstellar Object”

pds4-information-model#1003

CCB-58: New Product_Native values for MSL/Mastcam

pds4-information-model#1004

CCB-79: Add enumerated value day**-1 to Units_of_Rates

pds4-information-model#1005

CCB-80: Support NEF as a Native format for Artemis

pds4-information-model#1006

CCB-85: Expand definition of DataCurator to account for non-PDS archivists and for cloud storage

pds4-information-model#1007

CCB-86: List_Author and similar list classes should have cardinality 0:1 instead of 0:M

pds4-information-model#1008

CCB-88: Broken link to the PDS Policy on Packed Data in the Standards Reference

pds4-information-model#1032

[documentation] CCB-68: Update ASCII_File_Name and ASCII_Directory_Path_Name

pds4-information-model#1033

[documentation] CCB-58: New Product_Native values for MSL/Mastcam

pds4-information-model#1034

[documentation] CCB-79 : Add enumerated value day**-1 to Units_of_Rates

pds4-information-model#1035

[documentation] CCB-80: Support NEF as a Native format for Artemis

Software Changes

For each software repository, the changes are listed in 2 categories:

  • Planned Updates

  • Other Updates

The ‘Planned Updates’ are organized by ‘Themes’ (or ‘Release Themes’), which are defined in advance and approved by the PDS Software Working Group (see Plan B18).

The ‘Other Updates’ occurs during the build cycle without being planned or attached to a theme. They are organized by types (bug, enhancements, requirements, tasks). Any updates that require a de-scope of planned tasks are reviewed by the PDS Software Working Group.

The deliveries are validated by the development team and go through an additional Integration & Test process, as applicable, as indicated by the `Testing Status` column in the tables below. There are 3 possible statuses for testing:

  • 🔵 Skip Testing - Testing is not needed for this ticket. These are determined at the discretion of the team based upon the technical or operational nature of the closed task.

  • 🟡 Testing Needed

  • 🟢 Testing Complete - Initial testing complete, and test cases/results documented.

Component: Cloud Operations

Cloud operations, utilities, and authentication

Component: Cloud Platform Engineering

Planetary Data Cloud platform infrastructure and engineering

Component: Data Upload Manager

Data Upload Manager (DUM) for managing data uploads to PDC

Data-upload-manager

Data Upload Manager (DUM) component for managing the interface for data uploads to the Planetary Data Cloud from Data Providers and PDS Nodes.

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
data-upload-manager#349 B18 DUM Support: High Priority Enhancements / Bug Fixes / Tasks

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

data-upload-manager#352 DUM v2.4.2 CloudWatch Logs SerializationException Errors

🟡

s.medium

Component: Deep Archive

PDS Open Archival Information System (OAIS) utilities for deep archive operations

Component: Devops

DevOps tools, CI/CD pipelines, and automation infrastructure

Roundup-action

Do a “roundup”, a/k/a PDS-style continuous integration and delivery

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates

No planned updates realized for this build in this repository.

Other Updates
Requirements

Issue

I&T Status

Priority / Bug Severity

roundup-action#170 As a developer, I want Maven builds to always force re-resolution of previously failed artifact downloads

🟡

p.should-have

Component: DOI Service

Digital Object Identifier (DOI) service for PDS data products

Component: Integration And Testing

Integration and testing infrastructure

Component: Nucleus

Nucleus workflow platform for planetary data processing

Component: Operations

PDS Engineering Node operations and issue tracking

En-ops-utils

None

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates

No planned updates realized for this build in this repository.

Other Updates
Requirements

Issue

I&T Status

Priority / Bug Severity

en-ops-utils#18 As a user, I want to verify context product version_id matches filename version suffix

🟡

unknown

en-ops-utils#19 As a user, I want to verify deprecated context product LIDs have been removed from the PDS Search API

🟡

unknown

en-ops-utils#16 As a steward, I want auto-generated LDD release PRs to include a reviewer checklist

🟡

p.should-have

Component: PDS4 Information Model

PDS4 Information Model, ontology, and data dictionary tools

Pds4-context-products

Repository for managing PDS4 Context Products

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
pds4-context-products#85 Cleanup Deprecated Context Products from API and Web Portal

Issue

I&T Status

Level

Priority / Bug Severity

pds4-context-products#84 Chandrayaan-1 NASA PDS investigation (and maybe instrument host, and instruments?) are not accurately deprecated

🔵

bug

s.medium
Other Updates
Requirements

Issue

I&T Status

Priority / Bug Severity

pds4-context-products#94 As a user, I want CI to verify deprecated LIDs are removed from the PDS Search API when lids_deprecated.csv is updated

🟡

p.should-have

Pds4-information-model

The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
pds4-information-model#984 B17 Information Model Delivery to I&T

Issue

I&T Status

Priority / Bug Severity

pds4-information-model#984 B17 Information Model Delivery to I&T

🔵

unknown
pds4-information-model#1014 B18 Information Model SCR Implementation and LDDTool Updates

Issue

I&T Status

Level

Priority / Bug Severity

pds4-information-model#1047 ldd-prov test cases are failing

🟡

bug

s.high
pds4-information-model#1015 B18 SCR Freeze

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

pds4-information-model#1016 B18 Information Model Delivery to I&T

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

pds4-information-model#1017 B18 Standards Documents Updates

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

pds4-information-model#1026 Complete v1.0 of OPS LDD

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

pds4-information-model#1042 Radar namespace is not present in namespace registry

🟡

s.medium

pds4-information-model#1049 lddtool generates confusing error message for Local_Internal_Reference

🟡

s.low
Enhancements

Issue

I&T Status

Priority / Bug Severity

pds4-information-model#1038 [namespace-registry] add new namespace AV

🔵

p.must-have

pds4-information-model#1040 [namespace-registry] add new namespace BOPPS

🔵

p.must-have

Component: PDS4 Viewer

Tools for viewing and visualizing PDS4 data products

Pds4-jparser

Java Library providing APIs for parsing and exporting information on PDS4 products, including table and image objects to various formats including CSV, PNG, VICAR, FITs, etc.

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates

No planned updates realized for this build in this repository.

Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

pds4-jparser#200 RawTableReader readNextLine() treats 0xFF data bytes as EOF due to byte-to-int sign-extension

🟡

s.medium
Requirements

Issue

I&T Status

Priority / Bug Severity

pds4-jparser#197 As a PDS data engineer, I want RawTableReader to use buffered bulk I/O so that large table products with millions of records can be validated without extreme performance degradation

🟡

p.should-have
Enhancements

Issue

I&T Status

Priority / Bug Severity

pds4-jparser#197 As a PDS data engineer, I want RawTableReader to use buffered bulk I/O so that large table products with millions of records can be validated without extreme performance degradation

🟡

p.should-have

Component: Peppi

Planetary Data Explorer Python (PEPPi) client library

Peppi

Planetary Data Explorer: Python (PEPPi) client library (pds.peppi) to access Planetary Data from the NASA Planetary Data System

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

peppi#168 count() does not return the total hit count

🟡

s.medium

Component: Registry Tools

PDS Registry backend services and data loading tools

Harvest

Version<=4 of the standalone Harvest client application providing the functionality for capturing and indexing product metadata into the PDS Registry system. Version 5+ are now managed through https://github.com/NASA-PDS/registry-loader/

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates

No planned updates realized for this build in this repository.

Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

harvest#316 ops:Harvest_Meta fields missing data type definitions in registry data dictionary

🟡

s.critical

Registry

PDS Registry provides service and software application necessary for tracking, searching, auditing, locating, and maintaining artifacts within the system. These artifacts can range from data files and label files, schemas, dictionary definitions for objects and elements, services, etc.

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
registry#266 Implement design for alternate data file paths / file path updates

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry#371 Adapt Harvest to Support Using Inventories for Data File Metadata

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry#432 Onboard IPDA Partner: ISRO

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry#498 B18 Registry Support: High Priority Enhancements / Bug Fixes / Tasks

Issue

I&T Status

Level

Priority / Bug Severity

registry#497 As a developer, I want human-readable Postman collection docs auto-generated on update

🟡

requirement

p.should-have

registry#509 set-archive-status does not apply to secondary products in a collection

🟡

bug

s.medium
registry#517 Integrate Registry sub-components with Managed Opensearch

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

Other Updates
Requirements

Issue

I&T Status

Priority / Bug Severity

registry#510 As a registry operator, I want burnup charts and per-node tracking in status reports to monitor data loading progress over time

🟡

p.should-have
Enhancements

Issue

I&T Status

Priority / Bug Severity

registry#510 As a registry operator, I want burnup charts and per-node tracking in status reports to monitor data loading progress over time

🟡

p.should-have

Registry-api

Web API service for the PDS Registry, providing the implementation of the PDS Search API (https://github.com/nasa-pds/pds-api) for the PDS Registry.

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
registry-api#715 Complete Design for Supporting Registration and Search of Mirrored Archives

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#617 Develop Test Suite for Testing Production Registry API

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#682 Enable Free-Text Search for the PDS Registry

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#698 Refactor Registry API Functionality for /products/{identifier}/members/members

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#708 Develop a Throttling and Scaling Strategy for Registry and API

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#729 Update Functionality To Find When Class / Attribute Has Been Used / Exists

Issue

I&T Status

Level

Priority / Bug Severity

registry-api#727 As a user, I want the exists operator to be prepended to the query

🔵

enhancement

p.should-have
registry-api#749 B18 Registry API Support: High Priority Enhancements / Bug Fixes / Tasks

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#751 Finalize Release of`/members` Enhancements to Production

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#760 Automate Integration Testing in GitHub Actions CI Pipeline

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-api#768 Complete Transition to Full Structured XML Metadata in Registry

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

registry-api#742 A query to pds.nasa.gov does not respond the same as a query to pds.mcp.nasa.gov

🟡

s.medium

registry-api#745 Integration tests in unstable build suite do not pass when run locally

🟡

s.medium

registry-api#748 Investigate and fix skipped product/{id}/member* integration tests

🔵

s.medium

registry-api#744 Unstable build does not complete on develop branch due to GitHub Actions runner timeout

🟡

s.medium
Requirements

Issue

I&T Status

Priority / Bug Severity

registry-api#611 As a user, I want to search by a full/unique hierarchical path for a specific attribute

🟡

p.must-have

Registry-loader

*Tools used to load and update data in the registry, currently harvest and registry manager package in a docker image. *

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
registry-loader#67 Enforce strict schema and data type resolution during product loading

Issue

I&T Status

Level

Priority / Bug Severity

registry-loader#65 As a node operator, I want harvest to fail product loading when a namespace schema or attribute data type cannot be found

🟡

requirement

p.must-have

registry-loader#66 As a node operator, I want to force-load products when namespace schema or attribute type is unresolvable, knowing affected fields will not be fully indexed

🟡

requirement

p.must-have
Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

registry-loader#77 Harvest does not download LDD on Windows: temp file creation fails with ‘The system cannot find the path specified’

🟡

s.high

Registry-mgr

Version<=5 of the standalone Registry Manager application responsible for managing the PDS Registry (https://github.com/NASA-PDS/registry) schemas and indexes. Version 6+ is now managed through https://github.com/NASA-PDS/registry-loader/ .

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates

No planned updates realized for this build in this repository.

Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

registry-mgr#72 When delete-data command is used, log file specified is empty

🔵

s.medium

Registry-sweepers

*Scripts that run regularly on the registry database, to clean and consolidate information *

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
registry-sweepers#201 Implement Handling of Partial Collections Uploads and Orphan Products

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-sweepers#205 Implement Support for Superseded Data using PROV LDD

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

registry-sweepers#210 Implement Sweepers Execution in Airflow

Issue

I&T Status

Level

Priority / Bug Severity

registry-sweepers#208 legacy-sync sweeper does not run in DEV venue on EN node via airflow.

🟡

bug

s.medium
Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

registry-sweepers#212 Legacy registry sync node assignment does not use registry API metadata, causing incorrect node assignments

🟡

s.medium

registry-sweepers#220 Race condition exists when sweepers ensures index mappings and writes document updates very soon after

🟡

s.medium

Component: Search API

PDS Search API and client libraries

Component: System Engineering

System-level engineering, requirements, and documentation

Component: System Tools

System-level utilities and libraries

Pds-agent-skills

Claude Code skills marketplace for NASA PDS workflows - automated release notes, program status reports, and more

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates

No planned updates realized for this build in this repository.

Other Updates
Requirements

Issue

I&T Status

Priority / Bug Severity

pds-agent-skills#6 As a PDS developer, I want to export and triage GitHub Dependabot alerts so that I can track and remediate dependency vulnerabilities across NASA-PDS repositories

🟡

p.should-have

Component: Validate

PDS4 data product validation tools

Validate

Validates PDS4 product labels, data and PDS3 Volumes

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
validate#1549 B18 Validate Support: High Priority Enhancements / Bug Fixes / Tasks

No closed sub-issues identified for this theme in the current build yet. Click on the link in this section title for details.

validate#1595 Performance Improvements for Large Bundle Validation

Issue

I&T Status

Level

Priority / Bug Severity

validate#1570 As a PDS developer, I want inline regex patterns in FieldValueValidator pre-compiled as static constants so that per-field Pattern compilation overhead is eliminated

🔵

requirement

p.should-have

validate#1571 validate does not complete validation of large bundles (23000+ products) due to OutOfMemoryError in InMemoryRegistrar

🟡

bug

s.medium

validate#1565 As a PDS data engineer, I want schematron transformers to be cached rather than recompiled for every label so that bundle validation is significantly faster

🔵

requirement

p.should-have

validate#1568 As a PDS data engineer, I want parsed DOM trees cached and reused during referential integrity checks so that labels are not re-parsed from disk a second time

🟡

enhancement

p.should-have
Other Updates
Bugs

Issue

I&T Status

Priority / Bug Severity

validate#1548 validate does not fail or report errors when an explicitly-specified target file does not exist

🟡

s.medium

validate#1601 warning.integrity.member_not_found should be an ERROR by default, with new flag to downgrade to warning

🟡

s.medium

Component: Web Analytics

Web analytics tools and monitoring

Component: Web Content

PDS website content management and legacy systems

Portal

Parent repository for the NASA Planetary Data System (PDS) portal ecosystem. It serves as the central location for tracking update requests, tasks, and issues related to the pds.nasa.gov website.

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
portal#143 B18 Website Support: High Priority Enhancements / Bug Fixes / Tasks

Issue

I&T Status

Level

Priority / Bug Severity

portal#144 Citing page does not link to correct PDS DOI policy document

🟡

bug

s.medium

portal#145 Feedback widget does not direct users to pds-operator@jpl.nasa.gov

🟡

bug

s.medium

Component: Web Design

PDS Web Design System (WDS) components and implementations

Component: Web Dev

Web development tools and demonstration applications

S3-browser-cloudfront

Web view for files in S3 buckets

User Guide

Github Repo

Issue Tracking

Requirements

Stable Release

Dev Release
Planned Updates
s3-browser-cloudfront#138 S3 Browser Testing and Enhancements to Support LROC Launch

Issue

I&T Status

Level

Priority / Bug Severity

s3-browser-cloudfront#144 open-data-registry configuration broken by CloudFront PR changes

🟡

bug

p.must-have
Other Updates
Requirements

Issue

I&T Status

Priority / Bug Severity

s3-browser-cloudfront#143 As a developer, I want the S3 browser to work with CloudFront as the S3 passthrough vs. direct S3 bucket access

🟡

p.must-have

s3-browser-cloudfront#158 As a developer, I need the ability to deep link into a bucket that is served via CloudFront

🟡

p.must-have

s3-browser-cloudfront#174 As a PDS Engineer, I want the domain of data paths for buckets we control to be automatically configured

🟡

unknown

s3-browser-cloudfront#159 As a user, I need s3-browser to properly display breadcrumbs that include the bucket I chose

🟡

p.must-have

Release Summary Metrics

This section provides a summary of the issues addressed in this release, organized by issue type.

Component/Repo

Bugs

Enhancements

Requirements

Tasks

Themes

Total

Data Upload Manager

1

0

0

0

1

2

Devops

0

0

1

0

0

1

Operations

0

0

3

0

0

3

PDS4 Information Model

4

2

1

0

7

14

PDS4 Viewer

1

1

1

0

0

3

Peppi

1

0

1

0

1

3

Registry Tools

11

2

5

0

19

37

System Tools

0

0

1

0

0

1

Validate

3

1

2

0

2

8

Web Content

2

0

0

0

1

3

Web Dev

1

0

4

0

1

6

TOTAL

24

6

19

0

32

81

Liens

Issue

Title

Rationale

Engineering Node Software Catalog

The Engineering Node Software resources are listed in the Software Release Summary (B18)

Installation and Operation

PDS Engineering Node Software have 3 different venues/purposes for execution: Standalone, Discipline Node Deployment or Engineering Node-only Deployment For the Installation and Operation manual see the users manuals in the software summary sections below:

Reference documents

This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:

  • PDS Level 1, 2 and 3 Requirements, April 20, 2017.

  • PDS4 Project Plan, July 17, 2013.

  • PDS4 System Architecture Specification, Version 1.3, September 1, 2013.

  • PDS4 Operations Concept, Version 1.0, September 1, 2013.

  • PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.

  • PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.

  • PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.

  • PDS Search Scenarios, Version 1.0, September 1, 2013.

  • PDS Search Protocol, Version 1.2, March 21, 2014.

  • PDAP Search Protocol, Version 1.0, March 21, 2014.

  • PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.

  • PDS Deep Archive Software Requirements and Design Document (SRD/SDD)

  • PDS DOI Service Requirements and Design Document (SRD/SDD)