Skip to content

Resolve CI timeouts and miscellaneous updates for 2026-W35 - #66

Merged
khaeru merged 11 commits into
mainfrom
ci/2026-w35
Aug 24, 2026
Merged

Resolve CI timeouts and miscellaneous updates for 2026-W35#66
khaeru merged 11 commits into
mainfrom
ci/2026-w35

Conversation

@khaeru

@khaeru khaeru commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Around 2026-05-31 (see here), CI jobs began to fail by timing out at the 6-hour limit. Previously (e.g. here) all jobs ran in 5–10 minutes.

This PR and branch are to diagnose and resolve the issue.

Actual steps:

  • Add pytest-timeout configuration.
    • Limit all tests
  • Identified one case of .test_ckan.TestClient.test_call_action_getattr() as
    timing out. Specifically, the case that calls the tag_list CKAN API endpoint on the TDCI CKAN 'dev' instance times out. I did not investigate whether this happens because of slowness on the server side; a very large response (that never completes?); or some client-side issue in processing a complete response. The test case is skipped.
  • ATO provider:
    • Update hashes for known files.

    • Update the Zenodo record (DOI 10.5281/zenodo.14913729) with newer files.

    • Handle changes in the file format. Previously, the footer of each sheet contained 2 rows/cells like:

      Developed with the support of: Asian Development Bank and Asian Infrastructure Investment Bank
      See terms of use at https://asiantransportoutlook.com/termsofservice
      

      Recent files have instead:

      The Asian Transport Observatory (ATO) is developed with the support of the Asian Development Bank (ADB) and the Asian Infrastructure Investment Bank (AIIB).
      Activities in the Pacific Island Countries are additionally supported by the World Bank.
      See terms of use at https://asiantransportobservatory.org/termsofservice
      

      Now, instead of removing a fixed number of footer rows, a regular expression is added to match and discard these entries. This regex handles both the old and new style.

Some ideas that were not tried in this PR:

  • Ensure data retrieval/caching is not duplicated across pytest-xdist workers.
  • Separate slow-running checks into separate jobs; or only run them on some jobs.

PR checklist

  • Checks all ✅
  • Update documentation No changes to behaviour.
  • Update doc/whatsnew.rst

@khaeru khaeru self-assigned this Aug 24, 2026
@khaeru khaeru added the ci Continuous integration & testing label Aug 24, 2026
@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.90%. Comparing base (504ca78) to head (0c8ea6f).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #66      +/-   ##
==========================================
- Coverage   95.91%   95.90%   -0.02%     
==========================================
  Files          69       69              
  Lines        3353     3366      +13     
==========================================
+ Hits         3216     3228      +12     
- Misses        137      138       +1     
Files with missing lines Coverage Δ
transport_data/ato/__init__.py 99.05% <100.00%> (+0.02%) ⬆️
transport_data/testing/__init__.py 100.00% <ø> (ø)
transport_data/tests/org/test_metadata.py 100.00% <100.00%> (ø)
transport_data/tests/test_ato.py 100.00% <100.00%> (ø)
transport_data/tests/util/test_ckan.py 100.00% <100.00%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@khaeru
khaeru marked this pull request as ready for review August 24, 2026 21:32
@khaeru
khaeru merged commit d4c9b7f into main Aug 24, 2026
19 of 20 checks passed
@khaeru
khaeru deleted the ci/2026-w35 branch August 24, 2026 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Continuous integration & testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant