Evaluation arguments

[Han8931]

Hi, I just finished training a Roberta model on IMDb like this

python train.py \
    --train imdb \
    --eval imdb \
    --model-type roberta \
    --model-save-path ./example \
    --num-epochs 4 \
    --num-clean-epochs 1 \
    --num-adv-examples 0.2 \
    --attack-epoch-interval 1 \
    --attack a2t \
    --learning-rate 5e-5 \
    --num-warmup-steps 100 \
    --grad-accumu-steps 1 \
    --checkpoint-interval-epochs 1 \
    --seed 42

and I tried to run the evaluation code

python evaluate.py \
    --dataset imdb \
    --model-type roberta \
    --checkpoint-paths ./example \
    --epoch 4 \
    --save-log \
    --accuracy \
    --robustness \
    --attacks a2t a2t_mlm textfooler bae pwws pso \
    --interpretability 

However, I got the following error

FileNotFoundError: [Errno 2] No such file or directory: './example/test_logs.json'

Did I miss something? In my example folder I can only see train_log.txt file with some checkpoint directories.

[Yogender11]

Hello,
I'm encountering the same error.
@Han8931, @qiyanjun, @jinyongyoo, @jakegrigsby could you please share your insights on how to resolve it?

[qiyanjun]

Will take a look

[jinyongyoo]

@Yogender11 I'm sorry looks like there was a mistake with the file name. The file test_logs.json at

TextAttack-A2T/evaluate.py

Line 259 in 91a2092

with open(os.path.join(path, "test_logs.json"), "r") as f:

and

TextAttack-A2T/evaluate.py

Line 367 in 91a2092

with open(os.path.join(path, "test_logs.json"), "r") as f:

both refers to the json log file saved here

TextAttack-A2T/evaluate.py

Line 554 in 91a2092

os.path.dirname(model_save_path), "accuracy_eval_logs.json"

There must have been some last minute renaming that resulted in this issue. Could you check if accuracy_eval_logs.json file exists and if so, try to change the filenames at L259 and L367 and see if that fixes the problem?