Post-processing failure

Post any problems / bugs / issues that are Mylar-related in here.
Post Reply
MaRzE
Posts: 8
Joined: Thu Feb 25, 2021 8:11 am

Post-processing failure

Post by MaRzE »

Hi,
I need some help working out why post processing of downloads has stopped work and is failing.
I am running:
git build from python3-dev [d48b26a61f6e826341373dd55a74dd07e5b34bcb]
on MacOS Catalina 10.15.7
Using SABNZB+ v3.4.2 (Python v3.9.7) on same machine to download files

MYLAR PROGRAM OPTIONS
Python Version : 3.8.2
Language : en_US.UTF-8

Mylar Data Directory : /Applications/Mylar
Mylar Program Directory : /Applications/Mylar
Mylar Cache Directory : /Applications/Mylar/cache
Mylar Config File :/Applications/Mylar/config.ini

Here is the section from the log with the ERROR:
10-Dec-2021 14:00:36 - INFO :: mylar.traverse_directories.1544 : ThreadPoolExecutor-0_4 : there are 8 files.
10-Dec-2021 14:00:36 - INFO :: mylar.Process.430 : ThreadPoolExecutor-0_4 : I have located 8 files that I should be able to post-process. Continuing...
10-Dec-2021 14:00:36 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Action Comics [18005]
10-Dec-2021 14:00:36 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Action Comics [91078]
10-Dec-2021 14:00:36 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Fantastic Four [112685]
10-Dec-2021 14:00:36 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Daredevil [86343]
10-Dec-2021 14:00:36 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Daredevil [116964]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Dune: House Atreides [131348]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: New Mutants [122666]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: New Mutants [26327]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Avengers [54428]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Avengers [110496]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Avengers [33227]
10-Dec-2021 14:00:37 - WARNING :: mylar.Process.585 : ThreadPoolExecutor-0_4 : Avengers [2010] is either Paused or in an Ended status with 100% completion. Ignoring for match.
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: Captain Marvel [116365]
10-Dec-2021 14:00:37 - INFO :: mylar.Process.570 : ThreadPoolExecutor-0_4 : Now checking: The Amazing Spider-Man [112161]
10-Dec-2021 14:00:37 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Action.Comics.1037.2022.Digital.Zone-Empire/Action Comics 1037 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:37 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:37 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [1/8] Starting Post-Processing for Action Comics issue: 1037
10-Dec-2021 14:00:37 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:38 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:38 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Fantastic.Four.038.2022.Digital.Zone-Empire/Fantastic Four 038 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:38 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:38 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [2/8] Starting Post-Processing for Fantastic Four issue: 38
10-Dec-2021 14:00:38 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:38 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:38 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Daredevil.036.2022.Digital.Zone-Empire/Daredevil 036 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:38 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:38 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [3/8] Starting Post-Processing for Daredevil issue: 36
10-Dec-2021 14:00:39 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:39 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:39 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Dune.-.House.Atreides.012.2021.digital.Son.of.Ultron-Empire/Dune - House Atreides 012 (2021) (digital) (Son of Ultron-Empire).cbr
10-Dec-2021 14:00:39 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:39 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [4/8] Starting Post-Processing for Dune: House Atreides issue: 12
10-Dec-2021 14:00:39 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:40 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:40 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/New.Mutants.023.2022.Digital.Zone-Empire/New Mutants 023 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:40 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:40 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [5/8] Starting Post-Processing for New Mutants issue: 23
10-Dec-2021 14:00:40 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:40 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:40 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Avengers.050.2022.Digital.Zone-Empire/Avengers 050 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:40 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:40 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [6/8] Starting Post-Processing for Avengers issue: 50
10-Dec-2021 14:00:41 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:41 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:41 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Captain.Marvel.034.2022.Digital.Zone-Empire/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:41 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:41 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [7/8] Starting Post-Processing for Captain Marvel issue: 34
10-Dec-2021 14:00:42 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:42 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:42 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Amazing.Spider-Man.080.2022.Digital.Zone-Empire/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr
10-Dec-2021 14:00:42 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_4 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
10-Dec-2021 14:00:42 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] [8/8] Starting Post-Processing for The Amazing Spider-Man issue: 80
10-Dec-2021 14:00:42 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_4 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
10-Dec-2021 14:00:43 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
10-Dec-2021 14:00:43 - INFO :: mylar.Process.1967 : ThreadPoolExecutor-0_4 : [POST-PROCESSING] Manual post-processing completed for 8 issues [FAILED: 8]
10-Dec-2021 14:00:43 - INFO :: mylar.run.3212 : ThreadPoolExecutor-0_4 : [FOLDER-CHECK] Finished checking for newly snatched downloads
10-Dec-2021 14:01:11 - WARNING :: mylar._eventStreamResponse.98 : CP Server Thread-4 : data_error: can only concatenate str (not "NoneType") to str

Files download fine to the downloads directory or to the mylar/cache directory when using DDL. And, there they stay. I have tried forcing a manual processing of the folder the downloads are in, no joy. I have checked the permission on the folders and for the mylar scripts and processes to access these folders, all good, no joy.

It seems to be an issue from scanning the logs that mylar thinks the archives are corrupt and won't process them. They open fine in BetterZip, YAC readers, preview, archive, etc. but Mylar can't process them.

I have uncompressed the .cbr or .cbz files are re-zipped them, renamed extension from .zip to .cbz and then mylar processes the file without a problem. So, I am assuming/guessing there is an issue with mylar reading compressed archives?

Any help please to point to how to fix. I have made a care package if you need me to upload it somewhere. let me know?

Thanks,
MaRzE
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Post-processing failure

Post by evilhero »

You need to enable debug logging and then pastebin the log / carepackage so that I can see what's actually happening. INFO level logging just isn't enough information to figure anything concrete out. If you don't know how to enable debug logging - you can read the wiki here.

Mylar has to have full permissions to the directory where it's copying the file from during post-processing (it copies it from your series folder location into the mylar cache, uncompresses it, inserts the new metadata, and then zips it up and moves it back and then deletes the original). Since all of that type of logging isn't available atm at least due to the logging level, I would guess that the problem is probably permissions related - if it can't access the file due to invalid permissions, it will consider it corrupt as it can't open it.

Because your original files reside on a USB drive - you have to ensure that enforce permissions are turned off within Mylar (it could possibly have some issues trying to enforce perms as it's not localized even though it's an actual mount point), which would mean that the correct user has full access (rwx) recursively to that specific location. I'm guessing when you manually uncompress, and then zip up the files you're changing the permissions on said files (or ownership) so that Mylar can then access them.

Mylar doesn't have a problem reading compressed files at all (that's the nature of the game right) - but what you're describing sounds alot like some sort of permission-type of problems. Can you verify what user is running Mylar, and the permissions and ownership of the files in the cache ?
MaRzE
Posts: 8
Joined: Thu Feb 25, 2021 8:11 am

Re: Post-processing failure

Post by MaRzE »

Thanks @evilhero for the reply.
Mylar has been given full permissions to both the download file and the external volume that is the final comic store.
Mylar is run as my user via a script when I log in.
Permissions on the downloads folder is 755. The downloads folder is local on the NVM SSD.
Permissions for the comics folder is 777. This is an external raid volume,
Enforce Permissions is not enabled.
Below is the debug feed from the log for two of the stuck issues:
12-Dec-2021 22:32:31 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
12-Dec-2021 22:32:31 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_2 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Captain.Marvel.034.2022.Digital.Zone-Empire/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:31 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_2 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
12-Dec-2021 22:32:31 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] [7/8] Starting Post-Processing for Captain Marvel issue: 34
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2429 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] issueid: 896125
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2430 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] issuenumOG: 34
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2510 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Zero Suppression set to : 0x
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2550 : ThreadPoolExecutor-0_2 : issue detected greater than 10, but less than 100
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2565 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Zero level supplement set to 0x. Issue will be set as : 034
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2582 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Pretty Comic Issue is : 034
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2585 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Issue Year : 2022
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2592 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Publisher: Marvel
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2597 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Series: Captain Marvel
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2605 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Year: 2019
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2608 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Comic Location: /Users/Shared/Comics/Marvel/Captain Marvel (2019)
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2611 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Comic Version: v8
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2629 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Not an annual - removing from filename parameters
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2630 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] New format: $Series $VolumeN $Issue ($monthname $Year)
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2697 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Metatagging enabled - proceeding...
12-Dec-2021 22:32:31 - DEBUG :: mylar.Process_next.2712 : ThreadPoolExecutor-0_2 : readingorder: []
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.26 : ThreadPoolExecutor-0_2 : [META-TAGGER] dirName:/Volumes/.../Downloads/complete/comics/
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.31 : ThreadPoolExecutor-0_2 : ComicTagger Path location for internal comictagger.py set to : /Applications/Mylar/comictagger.py
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.35 : ThreadPoolExecutor-0_2 : [META-TAGGER] Filename is : /Volumes/.../Downloads/complete/comics/Captain.Marvel.034.2022.Digital.Zone-Empire/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.50 : ThreadPoolExecutor-0_2 : Filepath: /Volumes/.../Downloads/complete/comics/Captain.Marvel.034.2022.Digital.Zone-Empire/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.51 : ThreadPoolExecutor-0_2 : Filename: Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.53 : ThreadPoolExecutor-0_2 : New_Folder: /Applications/Mylar/cache/mylar_rqapouus
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.55 : ThreadPoolExecutor-0_2 : New_Filepath: /Applications/Mylar/cache/mylar_rqapouus/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.73 : ThreadPoolExecutor-0_2 : [META-TAGGER] Paths / Locations:
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.74 : ThreadPoolExecutor-0_2 : [META-TAGGER] scriptname : mylar.py
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.75 : ThreadPoolExecutor-0_2 : [META-TAGGER] downloadpath : /Volumes/.../Downloads/complete/comics
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.76 : ThreadPoolExecutor-0_2 : [META-TAGGER] sabnzbdscriptpath : /Applications/Mylar
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.77 : ThreadPoolExecutor-0_2 : [META-TAGGER] comicpath : /Applications/Mylar/cache/mylar_rqapouus
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.78 : ThreadPoolExecutor-0_2 : [META-TAGGER] Running the ComicTagger Add-on for Mylar
12-Dec-2021 22:32:31 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_2 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.144 : ThreadPoolExecutor-0_2 : [META-TAGGER] ComicTagger v.134 being used - using personal ComicVine API key supplied via mylar.
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.156 : ThreadPoolExecutor-0_2 : [META-TAGGER] CR Tagging enabled.
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.161 : ThreadPoolExecutor-0_2 : [META-TAGGER] CBL Tagging enabled.
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.215 : ThreadPoolExecutor-0_2 : [META-TAGGER] Enabling ComicTagger script with options: ['--configfolder', '/Volumes/.../.ComicTagger', '-e', '--delete-rar', '/Applications/Mylar/cache/mylar_rqapouus/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr']
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.223 : ThreadPoolExecutor-0_2 : [META-TAGGER] Executing command: ['/Library/Developer/CommandLineTools/usr/bin/python3', '/Applications/Mylar/comictagger.py', '--configfolder', '/Volumes/.../.ComicTagger', '-e', '--delete-rar', '/Applications/Mylar/cache/mylar_rqapouus/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr']
12-Dec-2021 22:32:31 - DEBUG :: mylar.run.224 : ThreadPoolExecutor-0_2 : [META-TAGGER] Absolute path to script: /Library/Developer/CommandLineTools/usr/bin/python3
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.275 : ThreadPoolExecutor-0_2 : [META-TAGGER] Output: Archive failed to export!
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=990 got=51] for /Applications/Mylar/cache/mylar_rqapouus/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr:ComicInfo.xml attempt#1
Error reading in Comet XML for validation!
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=1968382 got=51] for /Applications/Mylar/cache/mylar_rqapouus/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbr:Captain Marvel 034-000.jpg attempt#1
Error while copying to /Applications/Mylar/cache/mylar_rqapouus/Captain Marvel 034 (2022) (Digital) (Zone-Empire).cbz:

12-Dec-2021 22:32:32 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
12-Dec-2021 22:32:32 - INFO :: mylar.duplicate_filecheck.2170 : ThreadPoolExecutor-0_2 : [DUPECHECK] Duplicate check for /Volumes/.../Downloads/complete/comics/Amazing.Spider-Man.080.2022.Digital.Zone-Empire/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:32 - INFO :: mylar.duplicate_filecheck.2302 : ThreadPoolExecutor-0_2 : [DUPECHECK] Duplication detection returned no hits. This is not a duplicate of anything that I have scanned in as of yet.
12-Dec-2021 22:32:32 - INFO :: mylar.Process_next.2426 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] [8/8] Starting Post-Processing for The Amazing Spider-Man issue: 80
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2429 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] issueid: 896091
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2430 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] issuenumOG: 80
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2510 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Zero Suppression set to : 0x
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2550 : ThreadPoolExecutor-0_2 : issue detected greater than 10, but less than 100
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2565 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Zero level supplement set to 0x. Issue will be set as : 080
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2582 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Pretty Comic Issue is : 080
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2585 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Issue Year : 2022
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2592 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Publisher: Marvel
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2597 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Series: The Amazing Spider-Man
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2605 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Year: 2018
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2608 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Comic Location: /Users/Shared/Comics/Marvel/The Amazing Spider-Man (2018)
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2611 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Comic Version: v5
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2629 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Not an annual - removing from filename parameters
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2630 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] New format: $Series $VolumeN $Issue ($monthname $Year)
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2697 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Metatagging enabled - proceeding...
12-Dec-2021 22:32:32 - DEBUG :: mylar.Process_next.2712 : ThreadPoolExecutor-0_2 : readingorder: []
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.26 : ThreadPoolExecutor-0_2 : [META-TAGGER] dirName:/Volumes/.../Downloads/complete/comics/
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.31 : ThreadPoolExecutor-0_2 : ComicTagger Path location for internal comictagger.py set to : /Applications/Mylar/comictagger.py
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.35 : ThreadPoolExecutor-0_2 : [META-TAGGER] Filename is : /Volumes/.../Downloads/complete/comics/Amazing.Spider-Man.080.2022.Digital.Zone-Empire/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.50 : ThreadPoolExecutor-0_2 : Filepath: /Volumes/.../Downloads/complete/comics/Amazing.Spider-Man.080.2022.Digital.Zone-Empire/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.51 : ThreadPoolExecutor-0_2 : Filename: Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.53 : ThreadPoolExecutor-0_2 : New_Folder: /Applications/Mylar/cache/mylar_at_koesy
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.55 : ThreadPoolExecutor-0_2 : New_Filepath: /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.73 : ThreadPoolExecutor-0_2 : [META-TAGGER] Paths / Locations:
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.74 : ThreadPoolExecutor-0_2 : [META-TAGGER] scriptname : mylar.py
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.75 : ThreadPoolExecutor-0_2 : [META-TAGGER] downloadpath : /Volumes/.../Downloads/complete/comics
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.76 : ThreadPoolExecutor-0_2 : [META-TAGGER] sabnzbdscriptpath : /Applications/Mylar
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.77 : ThreadPoolExecutor-0_2 : [META-TAGGER] comicpath : /Applications/Mylar/cache/mylar_at_koesy
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.78 : ThreadPoolExecutor-0_2 : [META-TAGGER] Running the ComicTagger Add-on for Mylar
12-Dec-2021 22:32:32 - INFO :: mylar.run.135 : ThreadPoolExecutor-0_2 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.144 : ThreadPoolExecutor-0_2 : [META-TAGGER] ComicTagger v.134 being used - using personal ComicVine API key supplied via mylar.
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.156 : ThreadPoolExecutor-0_2 : [META-TAGGER] CR Tagging enabled.
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.161 : ThreadPoolExecutor-0_2 : [META-TAGGER] CBL Tagging enabled.
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.215 : ThreadPoolExecutor-0_2 : [META-TAGGER] Enabling ComicTagger script with options: ['--configfolder', '/Volumes/.../.ComicTagger', '-e', '--delete-rar', '/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr']
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.223 : ThreadPoolExecutor-0_2 : [META-TAGGER] Executing command: ['/Library/Developer/CommandLineTools/usr/bin/python3', '/Applications/Mylar/comictagger.py', '--configfolder', '/Volumes/.../.ComicTagger', '-e', '--delete-rar', '/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr']
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.224 : ThreadPoolExecutor-0_2 : [META-TAGGER] Absolute path to script: /Library/Developer/CommandLineTools/usr/bin/python3
12-Dec-2021 22:32:32 - DEBUG :: mylar.run.275 : ThreadPoolExecutor-0_2 : [META-TAGGER] Output: Archive failed to export!
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=858 got=51] for /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr:ComicInfo.xml attempt#1
Error reading in Comet XML for validation!
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=1165892 got=51] for /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr:Amazing Spider-Man 080-001.jpg attempt#1
Error while copying to /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbz:

12-Dec-2021 22:32:32 - ERROR :: mylar.Process_next.2734 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] This is a corrupt archive - whether CRC errors or it is incomplete. Marking as BAD, and not post-processing.
12-Dec-2021 22:32:32 - INFO :: mylar.Process.1967 : ThreadPoolExecutor-0_2 : [POST-PROCESSING] Manual post-processing completed for 8 issues [FAILED: 8]
12-Dec-2021 22:32:32 - INFO :: mylar.run.3212 : ThreadPoolExecutor-0_2 : [FOLDER-CHECK] Finished checking for newly snatched downloads
12-Dec-2021 22:32:32 - DEBUG :: mylar.job_management.3811 : ThreadPoolExecutor-0_2 : ReScheduled job: Folder Monitor to 2021-12-12 12:37:32
btx
Posts: 7
Joined: Mon Mar 02, 2020 12:28 am

Re: Post-processing failure

Post by btx »

Hopefully you won't mind some additional data suggestions from someone other than EvilHero. I have a good deal of experience working with comci archives under python + linux / macOS, but it's up to you.

Anyway, a clear error appears to be on the "Failed the read enough data" lines:
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=858 got=51] for /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr:ComicInfo.xml attempt#1
Error reading in Comet XML for validation!
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=1165892 got=51] for /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr:Amazing Spider-Man 080-001.jpg attempt#1
I've used rarfile a bunch but never ran into this 51 byte error thing. It's possible that the "copy to cache" function is broken in this case, but in a weird way that may prove enlightening.

1. I suggest including the output of:

stat "/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr"

2. Assuming you do still have that file, try to get a reasonable representation of its contents - but only if it really is 51 bytes. If that cbr file still exists in the cache dir, it would be useful to see what the file looks like on disk. If there really is a 51 byte file. Unsure what you have available on your cmdline so maybe try:

xxd "/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr"
or
strings "/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr"

3. Something to consider .. your source drive is likely fat32 or some other MSDOS rig. The main thing that causes trouble with mounted msdos filesystems is permissions. Make sure you do this step after the other two ... but see if you can copy the file to your local filesystem when running as the mylar user.

Good luck!
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Post-processing failure

Post by evilhero »

Exactly what @btx mentioned above - I was typing In a whole whack of stuff, but he hit it on the head.

The only thing I could add is to make sure your rar program is up to date. There was another issue a few days ago by someone who had the same type of error output (might have even been on a Mac), and their version of rar was out of date. They updated and it fixed their problem.
MaRzE
Posts: 8
Joined: Thu Feb 25, 2021 8:11 am

Re: Post-processing failure

Post by MaRzE »

btx wrote: Sun Dec 12, 2021 5:16 pm Hopefully you won't mind some additional data suggestions from someone other than EvilHero. I have a good deal of experience working with comci archives under python + linux / macOS, but it's up to you.

Anyway, a clear error appears to be on the "Failed the read enough data" lines:
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=858 got=51] for /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr:ComicInfo.xml attempt#1
Error reading in Comet XML for validation!
Unexpected exception in readArchiveFile(): [Failed the read enough data: req=1165892 got=51] for /Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr:Amazing Spider-Man 080-001.jpg attempt#1
I've used rarfile a bunch but never ran into this 51 byte error thing. It's possible that the "copy to cache" function is broken in this case, but in a weird way that may prove enlightening.

1. I suggest including the output of:

stat "/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr"
The file above does not exist any more. the mylar_at_koesy/ dir does not exist either.

2. Assuming you do still have that file, try to get a reasonable representation of its contents - but only if it really is 51 bytes. If that cbr file still exists in the cache dir, it would be useful to see what the file looks like on disk. If there really is a 51 byte file. Unsure what you have available on your cmdline so maybe try:
The file in the downloads directory is 32Mb, much bigger than 51bytes
stat for this file:
stat ~/Downloads/complete/comics/Amazing.Spider-Man.080.2022.Digital.Zone-Empire/Amazing\ Spider-Man\ 080\ \(2022\)\ \(Digital\)\ \(Zone-Empire\).cbr
16777229 69796895 -rwxrwxrwx 1 mark staff 0 31954227 "Dec 6 17:43:14 2021" "Dec 6 17:43:14 2021" "Dec 12 22:45:21 2021" "Dec 6 17:43:11 2021" 4096 62904 0 /Volumes/.../Downloads/complete/comics/Amazing.Spider-Man.080.2022.Digital.Zone-Empire/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr


xxd "/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr"
too much to post here same for command below
or
strings "/Applications/Mylar/cache/mylar_at_koesy/Amazing Spider-Man 080 (2022) (Digital) (Zone-Empire).cbr"

3. Something to consider .. your source drive is likely fat32 or some other MSDOS rig. The main thing that causes trouble with mounted msdos filesystems is permissions. Make sure you do this step after the other two ... but see if you can copy the file to your local filesystem when running as the mylar user.
Source drive is APFS, destination is HFS+ - Mac OS Extended (Journaled)

Good luck!
MaRzE
Posts: 8
Joined: Thu Feb 25, 2021 8:11 am

Re: Post-processing failure

Post by MaRzE »

evilhero wrote: Sun Dec 12, 2021 5:47 pm ...
The only thing I could add is to make sure your rar program is up to date. ...
The only version of unrar on my system is the one that ships with SABNZB+ that is located here:
/Applications/SABnzbd.app/Contents/Resources/osx/unrar/unrar

I will look at installing another version into the system in general. Normally the MacOS archive utility handles rar files natively. Is this what Mylar uses?
MaRzE
Posts: 8
Joined: Thu Feb 25, 2021 8:11 am

Re: Post-processing failure

Post by MaRzE »

Good news (- I think).
After installing RAR via brew, and running a manual Post process task ... it worked!
Problem seems to have been with RAR and UNRAR not being directly present. Not sure why it worked previously and then stopped, maybe some investigation required here??
Might be a good idea to add RAR/UNRAR to the list of dependencies for macOS Catalina 10.15.7 at a minimum. Not sure if this problem would exist on other macOS versions.

Thanks for all the help!

Cheers,
MaRzE
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Post-processing failure

Post by evilhero »

Good that you got it going.

Anything other than unrar / WinRAR won't be recognized by the metatagger, and this includes any OS native package as the calls are different.

We do list unrar as being a requirement for installation in our wiki here, but I'm also aware that most people never read the wiki. If unrar was not in your PATH, and it wasn't specified in the settings file, mylar wouldn't know about it.

There are some newer rar files that are having a problem being opened due to the version of rar that was used to compress it, which this may have fallen in line with.
Post Reply