Add missing dataset download to Qwen-Image validate examples - #1667
Open
rakhimovv wants to merge 1 commit into
Open
Add missing dataset download to Qwen-Image validate examples#1667rakhimovv wants to merge 1 commit into
rakhimovv wants to merge 1 commit into
Conversation
The validate_full and validate_lora scripts read data/example_image_dataset/ but never fetch it. Nothing in examples/qwen_image/ downloads the edit/ or eligen/ subdirectories, so those scripts fail even under the ordering in examples/dev_tools/unit_test.py; the canny/ and layer/ ones only succeed because the matching model_inference scripts fetched that data earlier. Adds the dataset_snapshot_download call used by the three scripts in validate_lora that already fetch their data, following the same convention of naming a single file directly and globbing only where several are read.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
validate_fullandvalidate_lorascripts underexamples/qwen_image/model_training/readdata/example_image_dataset/but never fetch it.For
edit/andeligen/this is not recoverable by running things in a particular order: at4dbf980, nothing underexamples/qwen_image/downloads either subdirectory. So these fail even under the ordering inexamples/dev_tools/unit_test.py, which runsmodel_inferencebeforevalidate_lora. Runningtest_qwen_image()on a clean checkout, 7 of the 18 scripts invalidate_lorafail like this:(
edit/image1.jpgis fetched byexamples/boogu_image/, so a full cross-family run rescues some of these;edit/image_color.jpgandeligen/are fetched nowhere in the repo.)How the 18
validate_lorascripts divide:canny, twolayer,layer_v2) happen to pass because the matchingmodel_inferencescripts fetch that data into the same directory first.dataset_snapshot_download(the Blockwise ControlNet scripts).validate_full/has the same problem in 7 more files. The siblingmodel_training/lora/*.shscripts don't help either way: they fetchdiffsynth_example_datasetto a different path.This adds the call the three existing scripts use, following their convention of naming a single file directly (
canny/image_1.jpg,depth/image_1.jpg) and globbing only where several are read (inpaint/*.jpg):Qwen-Image-Layered-Control-V2.pyis included because it importsdataset_snapshot_downloadbut never calls it, so a grep for the name makes it look fine.This may also be what #1009 is asking about.
Checking
All 11
validate_lorascripts were run on 1×H100 against LoRAs trained by themodel_training/lora/*.shscripts in the same tree, and all produced output (e.g.Qwen-Image-Edit.py80s,Qwen-Image-Edit-2509.py105s,FireRed-Image-Edit-1.0.py124s,Qwen-Image-EliGen.py50s).To confirm the download patterns themselves rather than relying on data an earlier script had left behind, I re-ran four of them in a fresh working directory with an empty
data/:Qwen-Image-Edit.py(edit/image1.jpg),FireRed-Image-Edit-1.0.py(edit/*.jpg),Qwen-Image-EliGen.py(eligen/*.png) andQwen-Image-Layered-Control.py(layer/image.png). Each fetched exactly its own inputs and produced an image, covering both pattern forms.I have not run the 7
validate_fullscripts — that needs a full fine-tune, and the accelerate configs are set up for 8 GPUs. Their change is the same one-call addition, but please treat those as unverified.The same pattern appears in
flux,wanvideo,z_image,movaandltx2(105 files repo-wide by the same static check, of which I have run 12). I kept this PR toqwen_imagerather than sending a cross-family change unasked; glad to follow up on the others if that would help.