Release Description Document (Build B16)

This release of the PDS4 System is intended as an operational release of the system components to date. The original plan for this release can be found here: plan B16

The following sections can be found in this document:

PDS4 Standards and Information Model Changes

This section details the changes to the PDS4 Standards and Information Model approved by the PDS4 Change Control Board and implemented by the PDS within the latest build period.

Ref	Title
pds4-information-model#945	CCB-60: New permissible value for File_Area_Observational.Header.parsing_standard_id: “CDF 3.9 ISTP/IACG”
pds4-information-model#946	CCB-59: funding_year only allows the value “9999”
pds4-information-model#949	CCB-62: Transfer_Manifest class should enforce data types for its two mandatory columns
pds4-information-model#967	CCB-28: As a user, I want validate to allow delimited tables with whitespace-only numeric fields

Software Changes

For each software repository, the changes are listed in 2 categories:

Planned Updates
Other Updates

The ‘Planned Updates’ are organized by ‘Themes’ (or ‘Release Themes’), which are defined in advance and approved by the PDS Software Working Group (see Plan B16).

The ‘Other Updates’ occurs during the build cycle without being planned or attached to a theme. They are organized by types (bug, enhancements, requirements, tasks). Any updates that require a de-scope of planned tasks are reviewed by the PDS Software Working Group.

The deliveries are validated by the development team and go through an additional Integration & Test process, as applicable, as indicated by the `Testing Status` column in the tables below. There are 3 possible statuses for testing:

🔵 Skip Testing - Testing is not needed for this ticket. These are determined at the discretion of the team based upon the technical or operational nature of the closed task.
🟡 Testing Needed
🟢 Testing Complete - Initial testing complete, and test cases/results documented.

Data-upload-manager

Data Upload Manager (DUM) component for managing the interface for data uploads to the Planetary Data Cloud from Data Providers and PDS Nodes.

Planned Updates

data-upload-manager#207 Improvements to Support LRO Uploads

data-upload-manager#224 Verify scalability of manifest file usage for archives >5 million files

data-upload-manager#231 Support Upload of Web Logs

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
data-upload-manager#241 When DUM crashes, the report it generates falsely indicates success	🟡	s.medium
data-upload-manager#282 During DUM load, undocumented message about “Backing off”	🟡	s.medium
data-upload-manager#271 DUM output has typo	🟡	s.low

Requirements

Issue	I&T Status	Priority / Bug Severity
data-upload-manager#209 As a user, I can read rclone-generated checksums when checking for existing files in the staging bucket	🟡	p.must-have
data-upload-manager#221 As a user, I want to support upload of files >5GB	🟡	p.must-have
data-upload-manager#232 As a user, I want to load web logs to the web analytics bucket	🟡	p.should-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
data-upload-manager#289 Enable Python 3.12 compatibility	🔵	p.must-have

Deep-archive

PDS Open Archival Information System (OAIS) utilities, including Submission Information Package (SIP) and Archive Information Package (AIP) generators

Planned Updates

deep-archive#204 Add Support for Labels with LBLX file suffix

deep-archive#210 B16 Deep Archive Maintenance Tasks

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
deep-archive#213 pds-deep-registry-archive does not include all collections within a bundle in SIP/AIP files	🟡	s.high
deep-archive#208 Output manifest .tab files do not use forward slashes on Windows	🟡	s.medium

Requirements

Issue	I&T Status	Priority / Bug Severity
deep-archive#157 As a user, I want deep archive to support LBLX label extensions	🟡	p.should-have
deep-archive#136 As a user, I want to know that the software is still executing on long running jobs.	🔵	p.could-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
deep-archive#218 Enable Python 3.12 compatibility	🔵	p.must-have

Devops

Parent repo for PDS DevOps activities

Planned Updates

devops#81 Enable Continuous Deployment for Portal

Doi-service

Service and tools for generating DOIs for PDS bundles, collections, and data sets

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
doi-service#456 Large majority of units tests failing	🔵	s.high
doi-service#476 Tests do not run successfully on windows	🔵	s.medium
doi-service#466 DOI service does not properly process UTF-16 in a JSON label	🔵	s.medium
doi-service#455 SBN DOIs contain unexpected metadata that is breaking our database load script	🔵	s.medium

Requirements

Issue	I&T Status	Priority / Bug Severity
doi-service#432 As a user, I want to submit a label utilizing the updated author/contributor metadata (List_Author, List_Contributor, List_Editor)	🟡	p.must-have
doi-service#478 As a user, I want to support labels with lblx suffix	🔵	p.must-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
doi-service#485 Enable Python 3.12 compatibility	🔵	p.must-have

Ds-view

Data Set View application (pds.nasa.gov/ds-view)

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
ds-view#56 Fix potential XSS security vulnerability on ds-view	🟡	s.high
ds-view#53 DOIs are not properly resolving on some landing pages	🔵	s.medium

Edwg

PDS Editorial Working Group Working Area

Planned Updates

EdWG#85 Gather Investigation Imagery

EdWG#86 Context Updates Phase 1: Facilities and Telescopes

Harvest

Version<=4 of the standalone Harvest client application providing the functionality for capturing and indexing product metadata into the PDS Registry system. Version 5+ are now managed through https://github.com/NASA-PDS/registry-loader/

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
harvest#239 Harvest failing due to AOSS OCU limits without retrying	🟡	s.high
harvest#283 Harvest cannot parse Date without time	🟡	s.high

Requirements

Issue	I&T Status	Priority / Bug Severity
harvest#262 As a user, I want to ingest all XML attribute values (not PDS4 attributes) as string, by default	🟡	p.must-have
harvest#261 As a user, I want harvest to raise an error if an attribute is not found in a local data dictionary (LDD)	🟡	p.must-have

Lasso-releasers

Automatic “release” of software to artifact repositories, publishing packages for general availability

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
lasso-releasers#5 Stable releases of ds-view are not publishing releases or their assets	🟡	s.high

Nucleus

Nucleus is a software platform used to create workflows for the Planetary Data (PDS).

Planned Updates

nucleus#131 Update Architecture and Implement Separate EFS Per Node Baseline DAG

nucleus#142 Support Data Backlog Use Case

nucleus#147 Design and Prototype New Tracking Service

nucleus#149 Upgrade Nucleus to the latest 2.x version in MWAA

nucleus#156 Develop a DAG Task Library

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
nucleus#159 Invalid creation of archive buckets with terraform	🔵	s.medium

Pds4-information-model

The software tools and data necessary for generating the Information Model including PDS4 ontology, data, and information model.

Planned Updates

pds4-information-model#917 B16 Information Model SCR Implementation and LDDTool Updates

pds4-information-model#920 B16 Information Model Delivery to I&T

pds4-information-model#921 B16 SCR Freeze

pds4-information-model#922 B16 Standards Documents Updates

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
pds4-information-model#938 Current WebHelp documentation (1O00/1.24.0.0) does not reflect latest Rings LDD	🟡	s.medium
pds4-information-model#951 PDS4 Instrument class (Product_Context) lists attributes that cannot be used	🟡	s.medium
pds4-information-model#947 LDD generator is not generating valid LDDs	🔵	s.high

Requirements

Issue	I&T Status	Priority / Bug Severity
pds4-information-model#948 As a user, I want to override the default namespace version used by LDDTool	🟡	p.must-have
pds4-information-model#949 CCB-62: Transfer_Manifest class should enforce data types for its two mandatory columns	🟡	p.must-have
pds4-information-model#939 As a user, I want to easily identify which IM and LDD version the WebHelp documentation is for	🟡	p.should-have
pds4-information-model#945 CCB-60: New permissible value for File_Area_Observational.Header.parsing_standard_id: “CDF 3.9 ISTP/IACG”	🟡	p.must-have
pds4-information-model#946 CCB-59: funding_year only allows the value “9999”	🟡	p.must-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
pds4-information-model#968 Upgrade to Java 17 for compatibility with validate dependency	🔵	p.must-have
pds4-information-model#958 [namespace-registry] add new namespace “prov”	🟡	p.must-have
pds4-information-model#953 [namespace-registry] add new namespace “galileo”	🔵	p.must-have

Pdsen-maven-parent

PDS Engineering Node Maven Parent POM

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Enhancements

Issue	I&T Status	Priority / Bug Severity
pdsen-maven-parent#70 Manually push 1.20.0 to maven central to enable update of registry tools	🔵	unknown

Peppi

Planetary Data Explorer: Python (PEPPi) client library (pds.peppi) to access Planetary Data from the NASA Planetary Data System

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
peppi#104 Simple Peppi query has long runtime	🟡	s.medium
peppi#102 as_dataframe method fails	🟡	s.high

Planetary-data-cloud

PDS Cloud Migration documentation, issue, tracking and simple tools for assisting in the PDS hybrid cloud study and migration efforts.

Planned Updates

planetary-data-cloud#87 Develop PDC User Engagement Strategy

planetary-data-cloud#100 Phase 2 Migration of Existing Production JPL AWS Infra to MCP

planetary-data-cloud#159 [IMG] LROC MVP: Setup IMG Prod Env, Replicate Data, Deploy Nucleus, and Kickoff Processing

planetary-data-cloud#139 Define Tagging Strategy for Planetary Data Cloud

planetary-data-cloud#144 Develop Egress Management Strategy

planetary-data-cloud#145 Develop Data Management and Storage Strategy

planetary-data-cloud#146 B16 Cost Model Iteration with SMD Team

planetary-data-cloud#165 Define PDC Assumed User Roles

planetary-data-cloud#188 Document Archive Management Guidelines and Constraints

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
planetary-data-cloud#171 Cloudfront caching of API requests causing confusion with user expectations	🟡	s.high

Requirements

Issue	I&T Status	Priority / Bug Severity
planetary-data-cloud#153 As a user, I need a lambda function to tag S3 objects based upon file suffix	🔵	p.must-have

Portal-tasks

PDS Portal tasks repo used to track update requests for the website. Actual code and website are managed in separate private repo

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
portal-tasks#123 Data release item details (data_product_information.cfm?) does not display on testing machine	🔵	s.medium
portal-tasks#129 ColdFusion applications do not fully function with MCP	🟡	s.low

Registry

PDS Registry provides service and software application necessary for tracking, searching, auditing, locating, and maintaining artifacts within the system. These artifacts can range from data files and label files, schemas, dictionary definitions for objects and elements, services, etc.

Planned Updates

registry#370 Resurrect Legacy Registry Dashboards

registry#381 B16 Registry Maintenance: High Priority Enhancements / Bug Fixes / Tasks

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
registry#422 Some of product urn:nasa:pds:lro_diviner_derived2::1.0’s fields are not indexed	🟡	s.high
registry#403 members of GEO bundle/collection collection not found	🟡	s.high
registry#380 CSV response not returning the values associated with fields argument	🟡	s.medium
registry#375 product has date for sweeper index but is listed as not indexed	🟡	s.medium
registry#440 Error FIPS mode: only SunJSSE TrustManagers may be used	🟡	s.high
registry#404 Registry is not accepting date time with leap second, e.g. 2015-06-30T23:59:60.862Z	🟡	s.high
registry#423 lid and pds:Identification_Area.pds:logical_identifier do not behave the same in the API	🟡	s.medium

Requirements

Issue	I&T Status	Priority / Bug Severity
registry#113 As a node operator, I want the the registry schema to update autonomously / dynamically when new data is ingested.	🟡	p.should-have
registry#424 As a user, I want to receive an alert when my registry credentials are readable by someone else than myself	🟡	p.should-have
registry#396 As a node user, I want to have multiple cognito groups and still have the main cognito node group to be effective	🟡	p.must-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
registry#360 Fix GEO node products which ops:Harvest_Info/ops:node_name: geo	🔵	p.must-have

Registry-api

Web API service for the PDS Registry, providing the implementation of the PDS Search API (https://github.com/nasa-pds/pds-api) for the PDS Registry.

Planned Updates

registry-api#623 Implement Registry API Performance Improvements

registry-api#633 Prototype Semantic Search Using AI / LLMs

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
registry-api#677 API search results using “search-after” returns empty [data] block even though I can find the product by lidvid	🟡	s.high
registry-api#638 The API fails when no Accept header is provided	🟡	s.high

Registry-client

A simple PDS Registry Client which authenticates users with PDS SSO and signs requests to the serverless OpenSearch (AOSS) hosting the Registry database.

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Enhancements

Issue	I&T Status	Priority / Bug Severity
registry-client#34 Enable Python 3.12 compatibility	🟡	p.must-have

Registry-common

Version<=2 of the library utilized by tools that manage and load data into the PDS Registry, including Harvest, Registry Manager, and Supplementer. Version 3+ is now managed through https://github.com/NASA-PDS/registry-loader/.

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Enhancements

Issue	I&T Status	Priority / Bug Severity
registry-common#149 Default to throwing an error when an attribute is missing from a LDD	🟡	p.must-have

Registry-legacy-solr

Legacy Registry Software components leveraging Apache Solr. Includes Legacy Harvest Tool, Registry Manager, PDS3 Catalog Tool, and Search Core library. These components provide the capabilities for loading PDS3 and PDS4 data into the Legacy Solr Registry, driving the PDS keyword search.

Planned Updates

registry-legacy-solr#200 B16 Registry Legacy Maintenance

Other Updates

Enhancements

Issue	I&T Status	Priority / Bug Severity
registry-legacy-solr#217 Auto-restart the registry legacy docker container unless stopped	🔵	unknown

Registry-mgr

Version<=5 of the standalone Registry Manager application responsible for managing the PDS Registry (https://github.com/NASA-PDS/registry) schemas and indexes. Version 6+ is now managed through https://github.com/NASA-PDS/registry-loader/ .

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
registry-mgr#134 registry-manager is incorrectly changing the archive_status of collection versions not in the bundle version	🟡	s.high
registry-mgr#137 set-archive-status is changing the status of more products than are in the bundle	🟡	s.high

Registry-sweepers

*Scripts that run regularly on the registry database, to clean and consolidate information *

Planned Updates

registry-sweepers#166 Optimize Sweepers Performance - Part 1

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
registry-sweepers#164 ATM Registry-Sweeper ECS task is failing	🔵	s.critical

Enhancements

Issue	I&T Status	Priority / Bug Severity
registry-sweepers#181 Enable Python 3.12 compatibility	🟡	p.must-have

Roundup-action

Do a “roundup”, a/k/a PDS-style continuous integration and delivery

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
roundup-action#157 Doc generation conflicting if package uses Sphinx extensions not in the GitHub actions base and tox generates conflicting files	🟡	s.medium
roundup-action#154 All stable builds are not tagging Releases in GitHub	🟡	s.high

Search-ui-legacy

Legacy Keyword Search UI querying the Legacy Registry through Apache Solr.

Planned Updates

No planned updates realized for this build in this repository.

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
search-ui-legacy#59 Bug with XSS vulnerability	🟡	s.high

Software-issues-repo

Issue tracking repository as a centralized entry point for general PDS software bugs and feature requests.

Planned Updates

software-issues-repo#118 2025 Annual A&A Assessment Prep and Review

software-issues-repo#122 B16 Code Freeze

software-issues-repo#124 Re-imagine Data Release and DOI Processes in GitHub

software-issues-repo#125 Complete ISA with NSSDCA

software-issues-repo#128 Upgrade Java Repos to use Maven Central Portal from OSSRH

software-issues-repo#139 Add Google Analytics to GitHub Sites

Other Updates

Enhancements

Issue	I&T Status	Priority / Bug Severity
software-issues-repo#113 Create tool to generate SLOC reports for software deliveries	🔵	unknown
software-issues-repo#135 Update LDD off-nominal release process to include regeneration of WebHelp	🔵	unknown

System-i-n-t

Repo for PDS EN System I&T scripts and issue tracking

Planned Updates

system-i-n-t#69 B16 Test Readiness Review (TRR)

system-i-n-t#73 B16 Information Model I&T

Template-repo-python

*template repository for PDS python developments *

Planned Updates

template-repo-python#104 Enable Code Coverage for Python repos

Validate

Validates PDS4 product labels, data and PDS3 Volumes

Planned Updates

validate#1184 Update Context Metadata Quality Checks to Support Aliases

validate#1203 B16 High Priority Bug Fixes and Updated Standards Implementations

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
validate#1316 Encoded_Audio encoding_standard_id = WAV gives MP4 error in Validate 3.7.1	🟡	s.medium
validate#1276 –strict-field-checks not working as expected	🟡	s.medium
validate#1408 JSON reports duplicates messages per product specified	🟡	s.medium
validate#1357 Encoded_External/encoding_standard_id does not allow NetCDF	🟡	s.medium
validate#1379 missing_constant treated differently to valid_minimim/valid_maximum	🟡	s.medium
validate#1149 validate-bundle has no default value for –report-dir	🟡	s.medium
validate#1359 Validate does not allow LIDVID for external products	🟡	s.medium
validate#1331 validate-bundle does not find same files as validate (uppercase label filenames)	🟡	s.medium
validate#1332 Validate Incorrectly Reports Overlap Error If Initial Bits Are Not Specified in Packed_Data_Fields	🟡	s.medium
validate#1358 Validate doesn’t correctly handle an integer value of valid_maximum associated with floating point data array	🟡	s.medium
validate#967 validate configuration includes context product names that do not match the products	🟡	s.medium
validate#1234 validate does not raise a warning when a table has more records than label says it has	🟡	s.medium

Requirements

Issue	I&T Status	Priority / Bug Severity
validate#970 As a user, I want to includes titles and aliases in context product name validation	🟡	p.could-have
validate#1201 As a user, I want validate to report empty (blank) PDS4 labels	🟡	p.should-have
validate#1370 As a Validate user, I want to include the lidvid for each label in all reports	🟡	p.should-have
validate#1391 As a validate user, I want validate to allow delimited tables with whitespace-only numeric fields (CCB-28)	🟡	p.must-have
validate#1137 As a user, I want to know how much progress is being made during a validate run	🟡	p.should-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
validate#1424 Upgrade to Java 17	🔵	unknown

Web-analytics

None

Planned Updates

web-analytics#50 Deploy Web Analytics System in Production

web-analytics#57 Complete Logstash Wrapper Tools for Prod Deployment

Other Updates

Bugs

Issue	I&T Status	Priority / Bug Severity
web-analytics#51 en-web-analytics is locked up and unable to use	🟡	s.high

Requirements

Issue	I&T Status	Priority / Bug Severity
web-analytics#53 As a user, I want to automatically deploy all web analytics components	🔵	p.must-have

Enhancements

Issue	I&T Status	Priority / Bug Severity
web-analytics#61 Enable Python 3.12 compatibility	🟡	p.must-have

Web-modernization

PDS.nasa.gov Web Modernization repo for managing the overall design process and user testing for modernizing the PDS web experience.

Planned Updates

web-modernization#257 Design Instrument Package Portal Pages

web-modernization#258 Design Learning Hub Page

web-modernization#259 Design-System Housekeeping

Liens

Issue	Title	Rationale

Engineering Node Software Catalog

The Engineering Node Software resources are listed in the Software Release Summary (B16)

Installation and Operation

PDS Engineering Node Software have 3 different venues/purposes for execution: Standalone, Discipline Node Deployment or Engineering Node-only Deployment For the Installation and Operation manual see the users manuals in the software summary sections below:

Reference documents

This section details the controlling and applicable documents referenced for this release. The controlling documents are as follows:

PDS Level 1, 2 and 3 Requirements, April 20, 2017.
PDS4 Project Plan, July 17, 2013.
PDS4 System Architecture Specification, Version 1.3, September 1, 2013.
PDS4 Operations Concept, Version 1.0, September 1, 2013.
PDS Harvest Tool Software Requirements and Design Document (SRD/SDD), Version 1.2, September 1, 2013.
PDS Preparation Tools Software Requirements and Design Document (SRD/SDD), Version 0.3, September 1, 2013.
PDS Registry Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Report Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Search Service Software Requirements and Design Document (SRD/SDD), Version 1.0, September 1, 2013.
PDS Search Scenarios, Version 1.0, September 1, 2013.
PDS Search Protocol, Version 1.2, March 21, 2014.
PDAP Search Protocol, Version 1.0, March 21, 2014.
PDS Security Service Software Requirements and Design Document (SRD/SDD), Version 1.1, September 1, 2013.
PDS Deep Archive Software Requirements and Design Document (SRD/SDD)
PDS DOI Service Requirements and Design Document (SRD/SDD)