fix(import-watcher): stop *arr import floods (backoff + give-up)#63
Merged
Conversation
The watcher only recognized an import once its stub infohash appeared in *arr's recent-history query. That history is time-windowed, so an import that completed long ago falls out of it and the row never leaves import_queued — and because manualImportCommandId stays null, every tick re-fires ManualImport at the destination, flooding it. Add an item-level check: ask *arr whether the target item (movieId / seriesId) already holds a file matching the release, and if so retire the row as imported instead of re-triggering. Matching prefers identical byte size (a move import never rewrites the file), then a normalized scene-title match, then release-group + quality as a last resort, so a different file on the same item won't false-match.
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Matching on release group + quality tier could retire a queued row when the target item merely held a *different* file of the same tier — a different edition of the same movie, or (via Sonarr's whole-series scan) a sibling episode. Both would skip the manual import for a release that is still missing. Keep only the per-file signals: byte-identical size (primary) and a normalized scene-title match (fallback). This leaves the whole-series scan safe, since neither signal collides across distinct files.
Reverts dropping the group+quality fallback — that signal is fine. The real hazard was importedReleasesFor returning every on-disk episode of a series, which let the coarse fallback match a *different* episode and retire the row before its own episode imported. importedReleasesFor now takes the queued release and Sonarr narrows to the release's own season/episode (empty when it can't be determined, so the normal history/command flow drives the import). Matching stays per-item.
A jack_manual trigger that keeps failing — e.g. *arr returns 500 because the movie's library folder is missing — was re-fired on every tick, flooding the destination. The watcher now tracks per-row trigger failures: it backs off exponentially between attempts (base/cap configurable) and, after maxManualImportAttempts, marks the row failed so it stops. State is in-memory (a restart is a fair signal to try again). The history and item-level checks still run each tick during back-off, so a genuine import is picked up the moment *arr gains the file. New config: maxManualImportAttempts (6), manualImportBackoffBaseMs (60s), manualImportBackoffMaxMs (30m).
roziscoding
changed the title
Remove hasImportedRelease / importedReleasesFor / releaseFilesMatch and the watcher's item-level "does *arr already hold this release?" check. The fuzzy size/title/group+quality matching was error-prone (it could retire a row when the item held a *different* release of the same tier). The watcher now relies on the two unambiguous signals: *arr's import history (recentlyImportedDownloadIds) and the tracked manual-import command status for success, and the backoff + give-up path to mark a row failed after N attempts. A stuck import simply fails and moves on, to be resolved by (future) user actions rather than guessed at here.
roziscoding
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stops the import watcher from flooding a destination Radarr/Sonarr when an
import_queuedrow can't be confirmed as imported.Problem
A jack_manual row re-fired
ManualImportat *arr on every tick when it couldn't be confirmed imported — either because the import aged out of *arr's recent-history window, or because the trigger itself kept failing (e.g. *arr returns 500 because the movie's library folder is missing). That hammered the destination indefinitely.Approach
Try to import; if *arr doesn't report success after a few attempts, mark the row failed and move on. Success and failure are read from unambiguous signals only:
recentlyImportedDownloadIds) or the tracked manual-import command status (we poll the exact command id we created — no fuzzy matching).maxManualImportAttemptsthe row is markedfailedwith a clear error. In-memory backoff state (a restart is a fair signal to retry). New config (all defaulted):maxManualImportAttempts(6),manualImportBackoffBaseMs(60s),manualImportBackoffMaxMs(30m).What changed vs the first cut
An earlier revision added an item-level reconciliation — asking *arr whether the target movie/episode already held a file "matching" the release via size / scene-title / release-group+quality. That heuristic was dropped: the coarse group+quality fallback (and the size-only match across a series) could retire a row when the item held a different release of the same tier (e.g. a proper/repack upgrade), silently skipping the import we actually wanted. Confirmation now comes only from history + command status; anything unconfirmed after N attempts fails and is left for future user-driven recovery actions.
Tests
bun testgreen. Watcher tests cover: history-hash flip, single push while absent from history, no re-trigger after restart once a command id is persisted, backoff (no re-fire every tick), and give-up (marksfailedafter N). Removed the matcher/item-reconcile tests along with the code.Requires the gitignored generated schema types (
bun run generateinpackages/schemas) for typecheck — unchanged from current CI expectations.Greptile Summary
This PR updates the import watcher to stop repeated manual import pushes. The main changes are:
Confidence Score: 5/5
This looks safe to merge.
Reviews (5): Last reviewed commit: "refactor(import-watcher): drop item-leve..." | Re-trigger Greptile
Context used (5)