Add support for spatie/crawler v9#7
Open
markvaneijk wants to merge 1 commit into
Open
Conversation
Crawler v9 was a full API rework (Crawler::create() takes the URL, fluent setters renamed, observer signatures changed), so the crawl is now delegated to a version-specific runner selected at runtime: - CrawlerV8Runner keeps the existing v8 behaviour unchanged - CrawlerV9Runner uses the v9 API; the default logging observer is replaced by v9's closure hooks, custom observers from config are passed through (they must target the installed major's signatures) - Version detection via method_exists(Crawler::class, 'addObserver'), introduced in v9 Composer now allows "^8.0 || ^9.0": PHP 8.1-8.3 keeps resolving v8 (v9 requires PHP 8.4), PHP 8.4+ picks v9 — and consumers can combine this package with spatie/laravel-sitemap >= 8.0.1 again. Supporting changes: - phpstan: exclude version-specific files (only one crawler major is installed at analysis time), pin CI to PHP 8.4, baseline two pre-existing findings surfaced by larastan 3 - phpunit.xml.dist migrated off the PHPUnit 9 schema; the stale coverage report config made PHPUnit 12 run zero tests silently - test matrix refreshed to PHP 8.2-8.4 x Laravel 11-13 (was PHP 8.1 / Laravel 9, which current constraints can no longer install); dev dependencies widened accordingly - README: document the dual-version support and the custom observer caveat Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
spatie/crawler v9 requires PHP 8.4 and reworked the public API. Staying pinned to
^8.0blocks consumers from combining laravel-static with anything that needs crawler v9 — most notablyspatie/laravel-sitemap >= 8.0.1, a very common companion package (that's how this surfaced: markvaneijk.com couldn't install both).What
composer.jsonnow allowsspatie/crawler: ^8.0 || ^9.0:Since v9 renamed the fluent API (
Crawler::create($url),crawlProfile(),concurrency(),defaultScheme(),followNofollow(),start()) and changed the observer contract, the crawl instatic:buildis delegated to a version-specific runner picked at runtime viamethod_exists(Crawler::class, 'addObserver'):CrawlerV8Runner— the existing v8 code, extracted verbatimCrawlerV9Runner— v9 API; the default logging observer is replaced with v9's closure hooks (onCrawled/onFailed/onFinished), and a customcrawl_observerfrom config is passed through viaaddObserver()(it must extend the installed major'sCrawlObserver— documented in the README)PHPStan can only analyse against the single installed crawler major, so the three version-specific files are excluded from analysis.
Supporting fixes (needed to get CI meaningful again)
phpunit.xml.distwas still on the PHPUnit 9 schema — under PHPUnit 12 the suite silently ran zero tests and exited 0. Migrated the schema and dropped the stale coverage-report config that caused the silent abort.env()calls in config, deprecatedforceRootUrl()).Verified
pest: 7 passed (18 assertions) against crawler 9.3.2 (Laravel 13, PHP 8.4)pest: 7 passed (18 assertions) against crawler 8.5.0 (same environment, pinned)StaticCrawlObserverstill loads fine under v8phpstan: no errorspint: touched files clean🤖 Generated with Claude Code