Collaborative Distributed Science Guide

Welcome to the Collaborative Distributed Science Guide!

Just joining or starting a new project? Check out the Collaborative Distributed Science Guide for guidance on conventions and best practices.

Plant Species Label Workflow

This repository includes scripts for preparing plant species labels and running BioCLIP 2 grid predictions. The workflow is:

1. Create TaxonoPy-passed species lists for NEON plots to get ground-truth labels and for species lists from different sources.
2. Map downstream labels by lookup from that resolved species list.
3. Run BioCLIP 2 over image grid crops using the resolved species list.

Directory Layout

• assets/NEON_plotData.csv: source NEON plot data used to extract plot-level species labels.
• assets/neonSiteSpeciesList.csv: source plant list from NEON used for site-level species lists.
• assets/<name>_BONAPlist.csv: source BONAP plant list used for region-level species lists.
• assets/conus_plant_lists_accepted.csv: source CONUS accepted plant list used for state-level species lists.
• assets/species_list/: TaxonoPy-passed species lists for BioCLIP 2 label sets.
• assets/test_labels/: subplot-level label files used for image benchmarking.

TaxonoPy/GNVerifier Setup

TaxonoPy uses GNVerifier for name resolution. Install taxonopy from requirements.txt, then make sure a gnverifier executable is available.

You can find different versions of gnverifier from the GitHub release. On macOS ARM64, download and extract the GNVerifier release:

mkdir -p outputs/tools/gnverifier/

curl -L \
  -o outputs/tools/gnverifier/gnverifier-v1.3.7-mac-arm64.tar.gz \
  https://github.com/gnames/gnverifier/releases/download/v1.3.7/gnverifier-v1.3.7-mac-arm64.tar.gz

tar -xzf outputs/tools/gnverifier/gnverifier-v1.3.7-mac-arm64.tar.gz \
  -C outputs/tools/gnverifier \
  --strip-components 1

chmod +x outputs/tools/gnverifier/gnverifier

The species-list scripts prepend that directory to PATH automatically and redirect HOME into the run-specific outputs/species_list/<name>/taxonopy/ directory. That keeps GNVerifier config files out of the user home directory and avoids permission issues in sandboxed runs.

Create Resolved Species Lists

Use scripts/create_taxonopy_species_list.py when starting from assets/NEON_plotData.csv. It extracts unique species labels for a plot, runs TaxonoPy, and writes only the final resolved list to assets/species_list/.

Example for one SCBI plot:

python scripts/create_taxonopy_neon_plot_species_list.py \
  --source-csv assets/NEON_plotData.csv \
  --plot-id SCBI_008 \
  --name SCBI_008

To create a merged species list for multiple plots:

python scripts/create_taxonopy_neon_plot_species_list.py \
  --source-csv assets/NEON_plotData.csv \
  --plot-id SCBI_005 --plot-id SCBI_008 --plot-id SCBI_015 --plot-id SCBI_021 \
  --name SCBI_plot

This writes intermediate files to outputs/species_list/SCBI_plot/ and the final label file to assets/species_list/SCBI_plot_labels.csv.

For a single state from the CONUS accepted plant list, use scripts/create_taxonopy_conus_species_list.py:

python scripts/create_taxonopy_conus_species_list.py \
  --source-csv assets/conus_plant_lists_accepted.csv \
  --state Colorado \
  --name CPER_state

This writes the intermediate state species CSV to outputs/species_list/CPER_state/ and the final resolved list to assets/species_list/CPER_state_labels.csv.

For other plant lists without the need to filter states, use scripts/create_taxonopy_other_species_list.py:

python scripts/create_taxonopy_other_species_list.py \
  --source-csv assets/CPER_BONAPlist.csv \
  --column scientificName \
  --name CPER_BONAPlist

This writes the intermediate state species CSV to outputs/species_list/CPER_BONAPlist/ and the final resolved list to assets/species_list/CPER_BONAPlist_labels.csv.

Create Test Label Files

Once a TaxonoPy-passed species list exists, downstream label files should be mapped by lookup instead of resolving the same labels again. Use scripts/map_labels_from_resolved_species_list.py to create one subplot-level test label file for each benchmark plot:

python scripts/map_labels_from_resolved_species_list.py \
  --plot-id SCBI_005

By default this reads assets/NEON_plotData.csv, looks up labels in assets/species_list/<plot_name>_plot_labels.csv, and writes assets/test_labels/<plot_id>_subplot_labels.csv. The output has one row per subplot, with original NEON labels, resolved BioCLIP labels, resolved scientific names, TaxonoPy taxonomy strings, TaxonoPy resolution statuses, and any labels that could not be mapped.

Grid-Based BioCLIP 2 Predictions

After preparing a species list, use scripts/predict_grid_species.py to split an image into a 3x3 or 4x4 grid and run BioCLIP 2 on each crop. The script uses the Python API and saves the full per-grid species probability list; integration across crops is left for a later step.

Example:

python scripts/predict_grid_species.py \
  --data-root data \
  --plot-id SCBI_008 \
  --grid-size 3 \
  --output-csv outputs/grid_predictions/SCBI_008_predictions.csv

When --species-list is omitted, the script infers the site-level label set from the matched images, for example assets/species_list/SCBI_labels.csv for SCBI plots and assets/species_list/CPER_labels.csv for CPER plots. Pass --species-list explicitly only when you want to override that behavior with a plot-specific or custom label set. The script can also be pointed at explicit image files, directories, or globs with --images.

By default, inference labels come from the resolved_taxonomic_labels column. Pass --species-column only when testing a different label form.

The first run for a species list computes BioCLIP text embeddings and caches them under outputs/text_embeddings/. Later runs with the same model, species list, species column, and label order reuse that cache. Use --no-text-embedding-cache to force recomputation.

The prediction CSV records the image path, parsed plot metadata, subplot ID, image date, grid position, crop bounds, and one probability column per species. Probability column names come from the species list's resolved_labels column; if that column is unavailable, the selected inference label column is used. The year remains available as parsed metadata, but it is not a required filtering layer.

Project Predictions Onto Original Images

Use scripts/project_grid_predictions.py to create a PNG for each original image represented by a prediction CSV. The figure keeps the source image's native pixel dimensions, draws the grid boundaries, and annotates each grid cell with every species and its probability when the probability is greater than 0.1.

python scripts/project_grid_predictions.py \
  --predictions temp_results \
  --output-dir temp_results/projected_figures \
  --threshold 0.1

The output directory mirrors the prediction CSV directory structure, preventing files generated with different species lists or grid sizes from overwriting one another. Source images are resolved first from image_path, then from relative_image_path under --data-root (which defaults to data).

About the Guide

This guide started as an Imageomics Institute-internal wiki, focused on providing guidance and best practices for collaborative and interdisciplinary (computer science + biology) work. Recognizing that the topics and suggestions are broadly applicable to anyone working in similar or adjacent fields, we moved the vast majority to this guide. To increase accessibility for those less familiar with GitHub, we generated the website from our Markdown documents (which used to be wiki pages) with Material for MkDocs.

Please feel free to open an issue with any questions regarding the content of this guide.

How to Use the Guide

This Guide is set up as a template repository such that there are two primary means of interacting with it:

1. Building a personalized version of the Guide: select "Use this Template" at the top of the repo to generate your own version. This will create a new repository (generated from the template repo) that does not share the commit history of the template. Updates can still be added from the template upstream through git cherry-pick (instructions). More details are provided below, in Personalizing the Guide.
2. Contributing to the Guide: fork this repo, make changes, and submit a pull request (PR) for review. Some guidance is provided in the Pull Request Guide; please provide a detailed description of your changes and review the contributing guidelines (coming soon).

Personalizing the Guide

Welcome to your new guide repo! The first step in updating your new guide is to give it a name; ideally this should match the name you give to the repo when using the template. You will need to update the name and hardcoded repo links across various files.

Primary pages to personalize are:

• mkdocs.yaml has comments indicating locations to personalize (e.g., updating name, logos, socials). All pages use relative links within the repository, so those connections only need to be updated if adding or removing pages.
• index.md: this is the homepage for the site and should reflect your organization or group's priorities.
• CITATION.cff: please set the Collaborative Distributed Science Guide citation as a reference in your citation file after making your own modifications (see template for guidance on formatting).
• Digital-products-release-licensing-policy.md: at a minimum, links should be redirected to your GitHub and Hugging Face organization pages.
  • Other locations in pages such as the GitHub-Repo-Guide and Hugging-Face-Repo-Guide.md also have links to our organization pages. A search and replace for "Imageomics" can be done to update these.
• Technical-Infrastructure.md: this page is Imageomics-specific and includes internal-only links.
• Glossary for Imageomics: this page should be updated to a topic relevant to your field or focus.
• Templates should be updated for your organization (mainly the Acknowledgements sections, pre-filled URL suggestions, domain-specific suggestions and guidance).

Additionally, you may wish to match the color scheme to your organization or group. This can be done by changing the preset values in docs/stylesheets/extra.css. To check accessibility of color schemes, tanaguru contrast finder has good flexibility in how you provide the color and how they fix it (providing a similar alternate color with sufficient contrast). Everything following color definition can be removed if you prefer to use a predefined MkDocs color scheme.

Contributing

If you'd like to contribute to this guide, please read our Contributing Guidelines for information about our standards, development workflow, and submission process.

Testing

To test this site locally, first clone this repository, then create an environment with requirements.txt

pip install -r requirements.txt

and run mkdocs serve:

mkdocs serve

Then the site will run at http://127.0.0.1:8000/Collaborative-distributed-science-guide/.

History

This guide was developed alongside the Imageomics Guide, which houses the information needed to get started with and use institute resources readily available to all members. However, most of its content is applicable to anyone working more broadly in the field of imageomics or adjacent fields of computer and data science, and it is tailored to help domain scientists bridging that gap. This guide is intended to serve as a template for others wishing to develop a similar organization-specific guide, and this solution was born out of the desire to do so for the AI and Biodiversity Change (ABC) Global Center while limiting duplicative updates between guides (Imageomics and ABC share some team members on this project). The general structure of the website should be broadly applicable, but see Personalizing the Guide for suggestions on tailoring it for the particular organization or group's needs.

Acknowledgments

Special Thanks to...

• @EmersonFras for his help in setting up automated linting for easier contributing!
• @annavik for her help designing and implementing the color scheme!

Funding

This work was supported by both the Imageomics Institute and the AI and Biodiversity Change (ABC) Global Center. The Imageomics Institute is funded by the US National Science Foundation's Harnessing the Data Revolution (HDR) program under Award #2118240 (Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning). The ABC Global Center is funded by the US National Science Foundation under Award No. 2330423 and Natural Sciences and Engineering Research Council of Canada under Award No. 585136. This guide draws on research supported by the Social Sciences and Humanities Research Council. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, Natural Sciences and Engineering Research Council of Canada, or Social Sciences and Humanities Research Council.