Skip to content

Update the minimal transfer scripts examples #1419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sirosen merged 1 commit into from
Aug 13, 2026
Merged

Update the minimal transfer scripts examples #1419

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 22 additions & 35 deletions docs/examples/minimal_transfer_script/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ The following is an extremely minimal script to demonstrate a file transfer
using the :class:`TransferClient <globus_sdk.TransferClient>`.

It uses the tutorial client ID from the :ref:`tutorials <tutorials>`.
For simplicity, the script will prompt for login on each use.

.. note::
You will need to replace the values for ``source_collection_id`` and
Expand All @@ -20,47 +19,35 @@ For simplicity, the script will prompt for login on each use.
:caption: ``transfer_minimal.py`` [:download:`download <transfer_minimal.py>`]
:language: python


Minimal File Transfer Script Handling ConsentRequired
-----------------------------------------------------

The above example works with certain endpoint types, but will fail if either
the source or destination endpoint requires a ``data_access`` scope. This
requirement will cause the Transfer submission to fail with a
``ConsentRequired`` error.

The example below catches the ``ConsentRequired`` error and retries the
submission after a second login.

This kind of "reactive" handling of ``ConsentRequired`` is the simplest
strategy to design and implement.

We'll also enhance the example to take endpoint IDs from the command line.

.. literalinclude:: transfer_consent_required_reactive.py
:caption: ``transfer_consent_required_reactive.py`` [:download:`download <transfer_consent_required_reactive.py>`]
:language: python


Best-Effort Proactive Handling of ConsentRequired
-------------------------------------------------

The above example works in most cases, and especially when there is a low cost
to failing and retrying an activity.
to failing and retrying an activity. The ``auto_redrive_gares`` flag enables a
behavior which will prompt the user for a fresh login if they are missing
consents for access to various collections.

However, in some cases, responding to ``ConsentRequired`` errors when the task
is submitted is not acceptable. For example, for scripts used in batch job
systems, the user cannot respond to the error until the job is already
executing. The user would rather handle such issues when submitting their job.
However, in some cases, responding to missing consents when the task is
submitted is not acceptable. For example, for scripts used in batch job systems,
the user cannot respond to the error until the job is already executing. The
user would rather handle such issues when submitting their job.

``ConsentRequired`` errors in this case can be avoided on a best-effort basis.
Note, however, that the process for consenting ahead of time is more error
prone and complex.
The service still relies on ``ConsentRequired`` errors to indicate that some
additional user consent is needed. But we can intentionally trigger them early
to control when the user is prompted to resolve them.

The example below tries an ``ls`` operation before starting to build the task
data. If the ``ls`` fails with ``ConsentRequired``, the user can be put through
the relevant login flow. Other errors (e.g., bad permissions) are suppressed, as
they probably aren't relevant to the user.

.. note::
The ``UserApp`` object is instantiated a second time, later in the script, to
actually start the transfer. This loads the same tokens from the earlier login
via the default token storage in ``~/.globus/``.

The example below enhances the previous reactive error handling to try an
``ls`` operation before starting to build the task data. If the ``ls`` fails
with ``ConsentRequired``, the user can be put through the relevant login flow.
And if not, we can relatively safely assume that any errors are not relevant.
To manage tokens in another way, please see the documentation on
:ref:`Token Storages <token_storages>`.

.. literalinclude:: transfer_consent_required_proactive.py
:caption: ``transfer_consent_required_proactive.py`` [:download:`download <transfer_consent_required_proactive.py>`]
Expand Down
123 changes: 52 additions & 71 deletions docs/examples/minimal_transfer_script/transfer_consent_required_proactive.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,58 +3,47 @@
import globus_sdk
from globus_sdk.scopes import TransferScopes

# do basic argument parsing
parser = argparse.ArgumentParser()
parser.add_argument("SRC")
parser.add_argument("DST")
args = parser.parse_args()

# tutorial client ID (we recommend replacing this with your own client)
CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
auth_client = globus_sdk.NativeAppAuthClient(CLIENT_ID)
APP_NAME = "proactive-transfer-consent-example"


# we will need to do the login flow potentially twice, so define it as a
# function
# Try an ls on the source and destination to see if ConsentRequired errors are raised --
# if they are, a fresh login flow will *not* be triggered.
#
# we default to using the Transfer "all" scope, but it is settable here
# look at the ConsentRequired handler below for how this is used
def login_and_get_transfer_client(*, scopes=TransferScopes.all):
# note that 'requested_scopes' can be a single scope or a list
# this did not matter in previous examples but will be leveraged in
# this one
auth_client.oauth2_start_flow(requested_scopes=scopes)
authorize_url = auth_client.oauth2_get_authorize_url()
print(f"Please go to this URL and login:\n\n{authorize_url}\n")

auth_code = input("Please enter the code here: ").strip()
tokens = auth_client.oauth2_exchange_code_for_tokens(auth_code)
transfer_tokens = tokens.by_resource_server["transfer.api.globus.org"]

# return the TransferClient object, as the result of doing a login
return globus_sdk.TransferClient(
authorizer=globus_sdk.AccessTokenAuthorizer(transfer_tokens["access_token"])
)


# get an initial client to try with, which requires a login flow
transfer_client = login_and_get_transfer_client()

# now, try an ls on the source and destination to see if ConsentRequired
# errors are raised
consent_required_scopes = []


def check_for_consent_required(target):
try:
transfer_client.operation_ls(target, path="/")
# catch all errors and discard those other than ConsentRequired
# e.g. ignore PermissionDenied errors as not relevant
except globus_sdk.TransferAPIError as err:
if err.info.consent_required:
consent_required_scopes.extend(err.info.consent_required.required_scopes)


check_for_consent_required(args.SRC)
check_for_consent_required(args.DST)
# This is more sophisticated than handling with `redrive_gares=True` and makes
# sure that the user is only prompted to login *one* extra time, even if both
# collections require additional consent.
def probe_for_consent_required(
transfer_client: globus_sdk.TransferClient, targets: list[str]
) -> list[str]:
consent_required_scopes: list[str] = []

for target in targets:
try:
transfer_client.operation_ls(target, path="/")
# catch all errors and discard those other than ConsentRequired
# e.g. ignore PermissionDenied errors as not relevant
except globus_sdk.TransferAPIError as err:
if err.info.consent_required:
consent_required_scopes.extend(
err.info.consent_required.required_scopes
)

return consent_required_scopes


with globus_sdk.UserApp(APP_NAME, client_id=CLIENT_ID) as app:
with globus_sdk.TransferClient(app=app) as transfer_client:
consent_required_scopes = probe_for_consent_required(
transfer_client, [args.SRC, args.DST]
)

# the block above may or may not populate this list
# but if it does, handle ConsentRequired with a new login
Expand All @@ -63,15 +52,21 @@ def check_for_consent_required(target):
"One of your endpoints requires consent in order to be used.\n"
"You must login a second time to grant consents.\n\n"
)
transfer_client = login_and_get_transfer_client(scopes=consent_required_scopes)

# from this point onwards, the example is exactly the same as the reactive
# case, including the behavior to retry on ConsentRequiredErrors. This is
# not obvious, but there are cases in which it is necessary -- for example,
# if a user consents at the start, but the process of building task_data is
# slow, they could revoke their consent before the submission step
#
# in the common case, a single submission with no retry would suffice
with globus_sdk.UserApp(
APP_NAME,
client_id=CLIENT_ID,
scope_requirements={
TransferScopes.resource_server: consent_required_scopes
+ [TransferScopes.all]
},
) as app:
app.login()


# From this point onwards, the example is exactly the same as the previous scripts.
# We will *not* set `redrive_gares=True`, on the grounds that if you want to use this
# in a context like a job submission system, a prompt for login is not helpful if the
# consent was revoked or insufficient.

task_data = globus_sdk.TransferData(
source_endpoint=args.SRC, destination_endpoint=args.DST
Expand All @@ -81,23 +76,9 @@ def check_for_consent_required(target):
"/~/example-transfer-script-destination.txt", # dest
)

with globus_sdk.UserApp(APP_NAME, client_id=CLIENT_ID) as app:
with globus_sdk.TransferClient(app=app) as transfer_client:
task_doc = transfer_client.submit_transfer(task_data)

def do_submit(client):
task_doc = client.submit_transfer(task_data)
task_id = task_doc["task_id"]
print(f"submitted transfer, task_id={task_id}")


try:
do_submit(transfer_client)
except globus_sdk.TransferAPIError as err:
if not err.info.consent_required:
raise
print(
"Encountered a ConsentRequired error.\n"
"You must login a second time to grant consents.\n\n"
)
transfer_client = login_and_get_transfer_client(
scopes=err.info.consent_required.required_scopes
)
do_submit(transfer_client)
task_id = task_doc["task_id"]
print(f"submitted transfer, task_id={task_id}")
77 changes: 0 additions & 77 deletions docs/examples/minimal_transfer_script/transfer_consent_required_reactive.py
View file
Open in desktop

This file was deleted.

41 changes: 16 additions & 25 deletions docs/examples/minimal_transfer_script/transfer_minimal.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,39 +1,30 @@
import globus_sdk
from globus_sdk.scopes import TransferScopes

# tutorial client ID (we recommend replacing this with your own client)
CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
auth_client = globus_sdk.NativeAppAuthClient(CLIENT_ID)

# requested_scopes specifies a list of scopes to request
# instead of the defaults, only request access to the Transfer API
auth_client.oauth2_start_flow(requested_scopes=TransferScopes.all)
authorize_url = auth_client.oauth2_get_authorize_url()
print(f"Please go to this URL and login:\n\n{authorize_url}\n")

auth_code = input("Please enter the code here: ").strip()
tokens = auth_client.oauth2_exchange_code_for_tokens(auth_code)
transfer_tokens = tokens.by_resource_server["transfer.api.globus.org"]

# construct an AccessTokenAuthorizer and use it to construct the
# TransferClient
transfer_client = globus_sdk.TransferClient(
authorizer=globus_sdk.AccessTokenAuthorizer(transfer_tokens["access_token"])
)

# Replace these with your own collection UUIDs
source_collection_id = "..."
dest_collection_id = "..."
SOURCE_COLLECTION_ID = "..."
DEST_COLLECTION_ID = "..."

# create a Transfer task consisting of one or more items
task_data = globus_sdk.TransferData(
source_endpoint=source_collection_id, destination_endpoint=dest_collection_id
)
task_data = globus_sdk.TransferData(SOURCE_COLLECTION_ID, DEST_COLLECTION_ID)
task_data.add_item(
"/share/godata/file1.txt", # source
"/~/minimal-example-transfer-script-destination.txt", # dest
)

# submit, getting back the task ID
task_doc = transfer_client.submit_transfer(task_data)
# create an app to manage login, use it to create a client, and submit,
# getting back the task ID
with globus_sdk.UserApp(
"minimal-transfer-example",
client_id=CLIENT_ID,
# we set the 'auto_redrive_gares' flag, which enables handling for missing
# auth requirements when the script is run against a changing set of collection IDs
config=globus_sdk.GlobusAppConfig(auto_redrive_gares=True),
) as app:
with globus_sdk.TransferClient(app=app) as transfer_client:
task_doc = transfer_client.submit_transfer(task_data)

task_id = task_doc["task_id"]
print(f"submitted transfer, task_id={task_id}")
Loading