Skip to content

Derek furst/remove image file #1030

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuanzhou merged 3 commits into from
Sep 1, 2026
Merged

Derek furst/remove image file #1030

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
758ab49
Merge pull request #1026 from hubmapconsortium/Derek-Furst/remove-sam…
yuanzhou Aug 14, 2026
e7e1dfb
removed metadata_files, and image_files for donors and samples wherev…
DerekFurstPitt Aug 28, 2026
d4b9eff
removed usages of metadata_files, added a comment block explaining th…
DerekFurstPitt Aug 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 0 additions & 82 deletions entity-api-spec.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -300,25 +300,6 @@ components:
- consortium
- public
description: 'One of the values: public, consortium'
image_files:
readOnly: true
type: array
items:
$ref: '#/components/schemas/File'
description: 'List of uploaded image files and descriptions of the files. Stored in db as a stringfied json array.'
image_files_to_add:
writeOnly: true
type: array
items:
type: string
description: 'List of temporary file ids with an optional description. Provide as a json array with an temp_file_id and description attribute for each element like {"files": [{"temp_file_id":"dzevgd6xjs4d5grmcp4n","description":"This is image file one"},{"temp_file_id":"yrahjadfhadf","description":"This is image file two"}]}'
image_files_to_remove:
writeOnly: true
type: array
items:
type: string
format: uuid
description: 'List of image files previously uploaded to delete. Provide as a json array of the file_uuids of the file like: ["232934234234234234234270c0ea6c51d604a850558ef2247d0b4", "230948203482234234234a57bfe9c056d08a0f8e6cd612baa3bfa"]'
metadata:
type: object
properties:
Expand Down Expand Up @@ -503,42 +484,6 @@ components:
visit:
type: string
description: 'The visit id for the donor/patient when the sample was obtained.'
image_files:
readOnly: true
type: array
items:
$ref: '#/components/schemas/File'
description: 'List of uploaded image files and descriptions of the files. Stored in db as a stringfied json array.'
image_files_to_add:
writeOnly: true
type: array
items:
type: string
description: 'List of temporary file ids with an optional description. Provide as a json array with an temp_file_id and description attribute for each element like {"files": [{"temp_file_id":"dzevgd6xjs4d5grmcp4n","description":"This is image file one"},{"temp_file_id":"yrahjadfhadf","description":"This is image file two"}]}'
image_files_to_remove:
writeOnly: true
type: array
items:
type: string
format: uuid
description: 'List of image files previously uploaded to delete. Provide as a json array of the file_uuids of the file like: ["232934234234234234234270c0ea6c51d604a850558ef2247d0b4", "230948203482234234234a57bfe9c056d08a0f8e6cd612baa3bfa"]'
metadata_files:
readOnly: true
type: array
items:
$ref: '#/components/schemas/File'
description: 'List of uploaded image files and descriptions of the files. Stored in db as a stringfied json array.'
metadata_files_to_add:
type: array
items:
type: string
description: 'List of temporary file ids with an optional description. Provide as a json array with an temp_file_id and description attribute for each element like {"files": [{"temp_file_id":"dzevgd6xjs4d5grmcp4n","description":"This is image file one"},{"temp_file_id":"yrahjadfhadf","description":"This is image file two"}]}'
metadata_files_to_remove:
type: array
items:
type: string
format: uuid
description: 'List of image files previously uploaded to delete. Provide as a json array of the file_uuids of the file like: ["232934234234234234234270c0ea6c51d604a850558ef2247d0b4", "230948203482234234234a57bfe9c056d08a0f8e6cd612baa3bfa"]'
rui_exemption:
type: boolean
description: 'Identifies Sample Blocks that are exempt from spatial registration via RUI.'
Expand Down Expand Up @@ -1372,9 +1317,6 @@ components:
contacts: [contact objects]
description: 'Description of the property'
data_access_level: 'public'
image_files: [image_files objects]
image_files_to_add: [strings]
image_files_to_remove: [strings]
metadata: object
protocol_url: '10.35079/protocols.io.abcdefghijkl'
lab_donor_id: 'AB123'
Expand Down Expand Up @@ -1404,12 +1346,6 @@ components:
data_access_level: 'public'
sample_category: 'organ'
organ: 'AO'
image_files: [image_files objects]
image_files_to_add: [strings]
image_files_to_remove: [strings]
metadata_files: [file object]
metadata_files_to_add: [string]
metadata_files_to_remove: [string]
protocol_url: '10.35079/protocols.io.abcdefghijkl'
lab_tissue_sample_id: '12-345 Aorta AA-1'
submission_id: 'AB1234'
Expand Down Expand Up @@ -1725,12 +1661,6 @@ components:
data_access_level: 'public'
sample_category: 'organ'
organ: 'AO'
image_files: [image_files objects]
image_files_to_add: [strings]
image_files_to_remove: [strings]
metadata_files: [file object]
metadata_files_to_add: [string]
metadata_files_to_remove: [string]
protocol_url: '10.35079/protocols.io.abcdefghijkl'
lab_tissue_sample_id: '12-345 Aorta AA-1'
submission_id: 'AB1234'
Expand Down Expand Up @@ -1758,12 +1688,6 @@ components:
data_access_level: 'public'
sample_category: 'organ'
organ: 'AO'
image_files: [image_files objects]
image_files_to_add: [strings]
image_files_to_remove: [strings]
metadata_files: [file object]
metadata_files_to_add: [string]
metadata_files_to_remove: [string]
protocol_url: '10.35079/protocols.io.abcdefghijkl'
lab_tissue_sample_id: '12-345 Aorta AA-1'
submission_id: 'AB1234'
Expand Down Expand Up @@ -1934,9 +1858,6 @@ components:
contacts: [contact objects]
description: 'Description of the property'
data_access_level: 'public'
image_files: [image_files objects]
image_files_to_add: [strings]
image_files_to_remove: [strings]
metadata: object
protocol_url: '10.35079/protocols.io.abcdefghijkl'
lab_donor_id: 'AB123'
Expand All @@ -1961,9 +1882,6 @@ components:
contacts: [contact objects]
description: 'Description of the property'
data_access_level: 'public'
image_files: [image_files objects]
image_files_to_add: [strings]
image_files_to_remove: [strings]
metadata: object
protocol_url: '10.35079/protocols.io.abcdefghijkl'
lab_donor_id: 'AB123'
Expand Down
10 changes: 5 additions & 5 deletions src/app.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5049,7 +5049,7 @@ def create_entity_details(request, normalized_entity_type, user_token, json_data
# Filter out the merged_dict by getting rid of the transitent properties (not to be stored)
# and properties with None value
# Meaning the returned target property key is different from the original key
# in the trigger method, e.g., Donor.image_files_to_add
# in the trigger method
filtered_merged_dict = schema_manager.remove_transient_and_none_values(merged_dict, normalized_entity_type)
# Create new entity
try:
Expand All @@ -5069,7 +5069,7 @@ def create_entity_details(request, normalized_entity_type, user_token, json_data


# Important: use `entity_dict` instead of `filtered_merged_dict` to keep consistent with the stored
# string expression literals of Python list/dict being used with entity update, e.g., `image_files`
# string expression literals of Python list/dict being used with entity update
# Important: the same property keys in entity_dict will overwrite the same key in json_data_dict
# and this is what we wanted. Adding json_data_dict back is to include those `transient` properties
# provided in the JSON input but not stored in neo4j, and will be needed for after_create_trigger/after_update_trigger,
Expand Down Expand Up @@ -5204,7 +5204,7 @@ def create_multiple_samples_details(request, normalized_entity_type, user_token,
# Filter out the merged_dict by getting rid of the transitent properties (not to be stored)
# and properties with None value
# Meaning the returned target property key is different from the original key
# in the trigger method, e.g., Donor.image_files_to_add
# in the trigger method
filtered_merged_dict = schema_manager.remove_transient_and_none_values(merged_dict, normalized_entity_type)

samples_dict_list = []
Expand Down Expand Up @@ -5332,7 +5332,7 @@ def create_multiple_component_details(request, normalized_entity_type, user_toke
# Filter out the merged_dict by getting rid of the transitent properties (not to be stored)
# and properties with None value
# Meaning the returned target property key is different from the original key
# in the trigger method, e.g., Donor.image_files_to_add
# in the trigger method
filtered_merged_dict = schema_manager.remove_transient_and_none_values(merged_dict, normalized_entity_type)
dataset_dict = {**filtered_merged_dict, **new_ids_dict_list[i]}
dataset_dict['dataset_link_abs_dir'] = dataset_link_abs_dir
Expand Down Expand Up @@ -5445,7 +5445,7 @@ def update_entity_details(request, normalized_entity_type, user_token, json_data
# Filter out the merged_dict by getting rid of the transitent properties (not to be stored)
# and properties with None value
# Meaning the returned target property key is different from the original key
# in the trigger method, e.g., Donor.image_files_to_add
# in the trigger method
filtered_merged_dict = schema_manager.remove_transient_and_none_values(merged_dict, normalized_entity_type)

# By now the filtered_merged_dict contains all user updates and all triggered data to be added to the entity node
Expand Down
108 changes: 0 additions & 108 deletions src/schema/provenance_schema.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -838,40 +838,6 @@ ENTITIES:
indexed: true
description: "One of the values: public, consortium"
before_create_trigger: set_data_access_level
image_files:
# This property has no before_create_trigger method, but will be created via `image_files_to_add` and updated via `image_files_to_delete`
type: list
generated: true # Disallow direct user input from create via POST, but can be used for update via PUT
indexed: false
description: "List of uploaded image files and descriptions of the files. Stored in db as a stringfied json array."
# Only file descriptions are allowed to be updated directly using this property
# Adding or deleting files must via the other two properties
before_update_trigger: update_file_descriptions
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
image_files_to_add:
type: list
transient: true
exposed: false
indexed: false
description: 'List of temporary file ids with an optional description. Provide as a json array with an temp_file_id and description attribute for each element like {"files": [{"temp_file_id":"dzevgd6xjs4d5grmcp4n","description":"This is image file one"},{"temp_file_id":"yrahjadfhadf","description":"This is image file two"}]}'
before_create_trigger: commit_image_files
before_update_trigger: commit_image_files
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
image_files_to_remove:
# This is only valid on update via a PUT request
type: list
transient: true
exposed: false
indexed: false
description: 'List of image files previously uploaded to delete. Provide as a json array of the file_uuids of the file like: ["232934234234234234234270c0ea6c51d604a850558ef2247d0b4", "230948203482234234234a57bfe9c056d08a0f8e6cd612baa3bfa"]'
before_update_trigger: delete_image_files
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
metadata:
type: json_string # dict
indexed: true
Expand Down Expand Up @@ -994,12 +960,6 @@ ENTITIES:
indexed: true
required_on_create: true
description: "The protocols.io doi url pointing the protocol under wich the sample was obtained and/or prepared."
image_file_metadata:
#todo: migrate to new attribute set as above portal_metadata_upload files
type: json_string
indexed: true
description: "A list of uploaded image files and descriptions of the files."
# A user who is a member of multiple groups HAS to send in the group_uuid
group_uuid:
type: string
immutable: true
Expand Down Expand Up @@ -1079,74 +1039,6 @@ ENTITIES:
type: string
indexed: true
description: "The visit id for the donor/patient when the sample was obtained."
image_files:
# This property has no before_create_trigger method, but will be created via `image_files_to_add` and updated via `image_files_to_remove`
type: list
generated: true # Disallow direct user input from create via POST, but can be used for update via PUT
indexed: false
description: "List of uploaded image files and descriptions of the files. Stored in db as a stringfied json array."
# Only file descriptions are allowed to be updated directly using this property
# Adding or deleting files must via the other two properties
before_update_trigger: update_file_descriptions
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
image_files_to_add:
type: list
transient: true
exposed: false
indexed: false
description: 'List of temporary file ids with an optional description. Provide as a json array with an temp_file_id and description attribute for each element like {"files": [{"temp_file_id":"dzevgd6xjs4d5grmcp4n","description":"This is image file one"},{"temp_file_id":"yrahjadfhadf","description":"This is image file two"}]}'
before_create_trigger: commit_image_files
before_update_trigger: commit_image_files
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
image_files_to_remove:
# This is only valid on update via a PUT request
type: list
transient: true
exposed: false
indexed: false
description: 'List of image files previously uploaded to delete. Provide as a json array of the file_uuids of the file like: ["232934234234234234234270c0ea6c51d604a850558ef2247d0b4", "230948203482234234234a57bfe9c056d08a0f8e6cd612baa3bfa"]'
before_update_trigger: delete_image_files
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
metadata_files:
# This property has no before_create_trigger method, but will be created via `metadata_files_to_add` and updated via `metadata_files_to_remove`
type: list
generated: true # Disallow direct user input from create via POST, but can be used for update via PUT
indexed: false
description: "List of uploaded image files and descriptions of the files. Stored in db as a stringfied json array."
# Only file descriptions are allowed to be updated directly using this property
# Adding or deleting files must via the other two properties
before_update_trigger: update_file_descriptions
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
metadata_files_to_add:
type: list
transient: true
exposed: false
indexed: false
description: 'List of temporary file ids with an optional description. Provide as a json array with an temp_file_id and description attribute for each element like {"files": [{"temp_file_id":"dzevgd6xjs4d5grmcp4n","description":"This is image file one"},{"temp_file_id":"yrahjadfhadf","description":"This is image file two"}]}'
before_create_trigger: commit_metadata_files
before_update_trigger: commit_metadata_files
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
metadata_files_to_remove:
# This is only valid on update via a PUT request
type: list
transient: true
exposed: false
indexed: false
description: 'List of image files previously uploaded to delete. Provide as a json array of the file_uuids of the file like: ["232934234234234234234270c0ea6c51d604a850558ef2247d0b4", "230948203482234234234a57bfe9c056d08a0f8e6cd612baa3bfa"]'
before_update_trigger: delete_metadata_files
# The updated_peripherally tag is a temporary measure to correctly handle any attributes
# which are potentially updated by multiple triggers
updated_peripherally: true
thumbnail_file:
generated: true
indexed: true
Expand Down
2 changes: 1 addition & 1 deletion src/schema/schema_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -843,7 +843,7 @@ def remove_none_values(merged_dict):
Filter out the merged_dict by getting rid of the transitent properties (not to be stored)
and properties with None value
Meaning the returned target property key is different from the original key
in the trigger method, e.g., Donor.image_files_to_add
in the trigger method

Parameters
----------
Expand Down
Loading
Loading