Skip to content

Retry network requests on failure #444

Description

@zealws

Describe the bug

Requests to fetch data from external sources can fail as a result of transient networking issues.
These should not cause the overall workflow to fail.

Encountered unexpected error fetching lrg_refseqgene: HTTPSConnectionPool(host='[ftp.ncbi.nlm.nih.gov](http://ftp.ncbi.nlm.nih.gov/)', port=443): Read timed out. (read timeout=30)

Instead, network requests should be retried a small number of times with a delay between calls.

Making 3 calls in total with a 1 second gap between each is usually reasonable for most applications. For our use-cases in Fusor, adding a second or two to startup to tolerate network hiccups is definitely tolerable.

Steps to reproduce

N/A

Expected behavior

Application doesn't fail for transient network issues.

Current behavior

Application fails if it hits a transient network issue while fetching data.

Possible reason(s)

No response

Suggested fix

No response

Branch, commit, and/or version

0.14.5

Screenshots

No response

Environment details

Docker: public.ecr.aws/docker/library/python:3.12.11

Additional details

No response

Contribution

None

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions