Copying remote torrent downloads

Information and Tutorials on features in Mylar and how to use it
goin3d
Posts: 21
Joined: Sun Dec 08, 2019 5:34 am

Re: Copying remote torrent downloads

Post by goin3d »

in this example i started with a net new series that i hadn't worked with before. The series had 12 issues. The below is a "debug on" example of issue #5 that has been pulled down but fails to post process. i'll try and find a successful.

Code: Select all

28-Apr-2020 08:58:29 - DEBUG :: mylar.listFiles.144 : Thread-89 : [FILENAME]: Marvel Action Spider-Man 005 (2018) (Digital) (Zone-Empire).cbr
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.225 : Thread-89 : [CORRECTION] Sub-directory found. Altering path configuration.
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.227 : Thread-89 : [SUB-PATH] Checking Folder Name for more information.
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.229 : Thread-89 : [SUB-PATH] Original Path : /share/Comics/incoming
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.230 : Thread-89 : [SUB-PATH] Sub-directory : /share/Comics/incoming/Marvel Action Spider-Man 001-011 (2018-2019) (Digital) (Zone-Empire)
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.243 : Thread-89 : [SUB-PATH] subpath set to : Marvel Action Spider-Man 001-011 (2018-2019) (Digital) (Zone-Empire)
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.325 : Thread-89 : Scanner group tag discovered: Zone-Empire
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.436 : Thread-89 : NEWLY SPLIT REORGD: ['Marvel', 'Action', 'Spider-Man', '005', '(2018)', '(Digital)']
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.471 : Thread-89 : checking date : 2018
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.474 : Thread-89 : date: 2018
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.802 : Thread-89 : datecheck: [{'date': '2018', 'position': 4, 'mod_position': 30}]
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.808 : Thread-89 : sctd: 2020
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.816 : Thread-89 : year verified as : None
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.824 : Thread-89 : year verified as: 2018
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.856 : Thread-89 : highest_series_position: 4
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.876 : Thread-89 : possible_issuenumbers: [{'number': '005', 'position': 3, 'mod_position': 25, 'validcountchk': False}]
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.882 : Thread-89 : hyphen located at position: 20
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.884 : Thread-89 : yearposition: 4
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.925 : Thread-89 : issue number :005
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.954 : Thread-89 : issue_position: 3
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.981 : Thread-89 : issue verified as : 005
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.1116 : Thread-89 : sf_highest_series_pos: ['Marvel', 'Action', 'Spider-Man']
28-Apr-2020 08:58:29 - DEBUG :: mylar.parseit.1168 : Thread-89 : series title possibly: Marvel Action Spider-Man
28-Apr-2020 08:58:29 - INFO :: mylar.Process.415 : Thread-89 : I have located 3 files that I should be able to post-process. Continuing...
28-Apr-2020 08:58:29 - INFO :: mylar.Process.549 : Thread-89 : Now checking: Marvel Action: Spider-Man [115545]
28-Apr-2020 08:58:29 - WARNING :: mylar.Process.554 : Thread-89 : Marvel Action: Spider-Man [2018] is either Paused or in an Ended status with 100% completion. Ignoring for match.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Copying remote torrent downloads

Post by evilhero »

Code: Select all

28-Apr-2020 08:58:29 - WARNING :: mylar.Process.554 : Thread-89 : Marvel Action: Spider-Man [2018] is either Paused or in an Ended status with 100% completion. Ignoring for match.
It's still saying the series you're post-processing against is 100% complete. I noticed you're downloading packs, when it downloads a pack it post-processes every issue in the pack if it's available immediately after the torrent is downloaded back to the local machine.

How are you initiating the post-processing ? Are you sure the issues haven't already been post-processed when you're starting the post-processing ? If you're on the series page while it's post-processing, the issue status won't update - you need to refresh the page for it to show up (or not be on that page while it's post-processing so it will load up to display the correct counts when you visit it next time).
goin3d
Posts: 21
Joined: Sun Dec 08, 2019 5:34 am

Re: Copying remote torrent downloads

Post by goin3d »

Sorry if this is a dumb question, but how does Mylar define complete? In my case the handful of series i have tried have had both 0 issues in anything other than a snatched status, or a mixture of files that are downloaded or snatched.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Copying remote torrent downloads

Post by evilhero »

Mylar will only count the issues that are in a Downloaded / Archived status as part of the total counts. Unless the series is in a Paused status, there should be no reason for it to not post-process unless it's already 100% complete and in an Ended status (meaning the series isn't being published any longer). If it's Active (series is currently being published), the 100% complete restriction doesn't apply.

Your best bet at this point is to provide the carepackage. Since you're already got the debug logging on, go into the Configuration screen. In the upper right corner there's a button labelled Care Package. Hit that, let it generate the zip and you'll see it download in your browser (it will also keep it in the mylar cache directory). I'll need to see that file in order to see what's going on (plus it attaches your existing db file so I can see exactly what your watchlist contains and what might be causing any problems).
goin3d
Posts: 21
Joined: Sun Dec 08, 2019 5:34 am

Re: Copying remote torrent downloads

Post by goin3d »

let me know if you have any trouble grabbing it from:

https://drive.google.com/file/d/18YwnsW ... 320xH/view

Hopefully i did it correctly!!! Right before generating it i tried to manually post process two items that got pulled down from torrent, were listed as being "snatched" but failed to process.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Copying remote torrent downloads

Post by evilhero »

Ok, I can kinda see what's going on now - thanks for the zip, that helped alot.

The problem going on seems to be that it looks like you did a whack of imports, which all seemed to work fine and import properly. However, in some of these (I'll use GLOW as an example in all of this) the file count says its 4/4, but looking at the series itself in the GUI, there are only 2 issues in a Snatched state, the other 2 are in a Skipped state. So you'd think that'd be 0/4 on the counts and it would post-process. I'll provide a detailed 'long' explanation, then a 'short' explanation/fix so you can get the whole jist:

Long explanation:
However, Mylar currently at least, doesn't immediately update the file counts - so when you marked the 4 issues as Skipped, and then snatched 2 of them (I'm guessing that's kinda the workflow based on the dB) it didn't update the count from 4/4. So because the file counts weren't updated when post-processing ran against those 2 snatches, it read the counts as 4/4 from the dB, and thus it assumed 100% completion with the series being in an Ended state.

So basically, whenever you change the status of an issue on the series detail page you need to do a ReCheck Files (or you can wait til the series gets auto-refreshed, but that could be anywhere from 5mins - 24hrs). The idea being that the little progress bar shown beneath the image on the series detail page, should reflect the correct issue counts (how many you have / total issues).

If the counts are never in-line with what you see from eyeing the Status' on the same page, you need to either do a ReCheck Files or Refresh Series. The difference btwn the 2 is that a ReCheck Files will only scan the series location on the hard drive for issues and change the status as appropriately discovered. A Refresh Series will repoll all the information from CV (so it will pull down any new / missed issue information , update the image to the latest cover if you have that option enabled, etc) in addition to doing a ReCheck Files at the end of it.

It's a shortcoming not automatically updating the counts when the status changes, that we'll try and figure out a clean way to do it, but it's one that's haunted the program for awhile. The problem with doing it that was found, is that if a person is changing the status of an issue in a series like say 2000AD (which has literally 2000+ issues), a ReCheck Files scan can take anywhere from a full minute to up to 5 depending on the machine it's run on. So we opted to not scan after every status change based on that - as to the end user it almost appears as if it's locked up the GUI, but it in fact is scanning the issues.

Short Explanation:
Mylar thinks the series is 100% completed because the file counts in the DB haven't been updated since some issues have had their status' change and those changes aren't reflected in the DB.

[FIX]: Perform a ReCheck files after any manual status change from a Downloaded state to anything else.

Hope that helps!
goin3d
Posts: 21
Joined: Sun Dec 08, 2019 5:34 am

Re: Copying remote torrent downloads

Post by goin3d »

man, you're awesome, and that explanation helps a ton!!! not many people would have taken the time to detail it all out like you did and i truly appreciate it!
Post Reply