Failed download handling

Information and Tutorials on features in Mylar and how to use it
Post Reply
Aimhere
Posts: 76
Joined: Mon Apr 06, 2015 2:32 pm

Failed download handling

Post by Aimhere » Sun Oct 30, 2016 3:47 pm

Hey evilhero,

I'm still trying to wrap my brain around the failed-download handling options. If I'm interpreting this correctly, the options break down like this:

a) If NEITHER "Failed download handling" nor "Automatic re-tries" enabled: Mylar tries to grab the first release it finds, but the download fails. Mylar marks the issue as "Skipped" and no further processing occurs.

b) If "Failed download handling" ENABLED, but "Automatic re-tries" NOT enabled: Mylar tries to grab the first release it finds, but the download fails. Mylar marks the release as "Failed" and tries to re-download the SAME release the next time a "Wanted" search occurs.

c) If "Failed download handling" AND "Automatic re-tries" BOTH ENABLED: Mylar tries to grab the first release it finds, but the download fails. Mylar marks the release as "Failed" and immediately searches for a DIFFERENT release matching the issue desired. If all of the available/matching releases fail, Mylar marks them all "Failed" but will re-try them all (presumably in the same order?) the next time a "Wanted" search occurs.

[also, d) what happens if "Failed download handling" is NOT enabled, but "Automatic re-tries" IS enabled?]

Do I have this right?

I am only using Usenet (not torrents) for downloading. It seems to me that, for the comics I follow at least, issues tend to be released (by one or more groups) shortly after they are published, then are rarely (if ever) re-released. (Sure, sometimes there will be exceptions, but by and large it's "grab it the first time or don't get it at all".) If the existing release[s] fail the first time around (usually due to incomplete/missing articles on my Usenet provider(s)), then they're likely to never succeed.

What I want is for Mylar to NOT re-try any given release that has failed (because it's never going to succeed), but DO keep searching for other releases until it either finds one that is good, or exhausts all the possible matches. In any event, if a given release has failed, I would prefer that Mylar NEVER try downloading the same release again. Why else bother tracking failed releases in the first place?

The reason I bring all this up again is, I'm finding Mylar keeps trying to re-download releases that have failed (they are on the "Failed" management page), yet often those same issues are marked as "Skipped" on the series' page. I suspect something got messed up as I kept changing the Failed Downloads options. At this point, would it be best if I manually set all the missing issues (that I know are never going to successfully download) as "Skipped", then cleared all the "Failed" locks on the management page?

Aimhere

User avatar
evilhero
Site Admin
Posts: 2002
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Failed download handling

Post by evilhero » Mon Oct 31, 2016 5:29 pm

Aimhere wrote:Hey evilhero,

I'm still trying to wrap my brain around the failed-download handling options. If I'm interpreting this correctly, the options break down like this:

a) If NEITHER "Failed download handling" nor "Automatic re-tries" enabled: Mylar tries to grab the first release it finds, but the download fails. Mylar marks the issue as "Skipped" and no further processing occurs.

b) If "Failed download handling" ENABLED, but "Automatic re-tries" NOT enabled: Mylar tries to grab the first release it finds, but the download fails. Mylar marks the release as "Failed" and tries to re-download the SAME release the next time a "Wanted" search occurs.

c) If "Failed download handling" AND "Automatic re-tries" BOTH ENABLED: Mylar tries to grab the first release it finds, but the download fails. Mylar marks the release as "Failed" and immediately searches for a DIFFERENT release matching the issue desired. If all of the available/matching releases fail, Mylar marks them all "Failed" but will re-try them all (presumably in the same order?) the next time a "Wanted" search occurs.

[also, d) what happens if "Failed download handling" is NOT enabled, but "Automatic re-tries" IS enabled?]

Do I have this right?
b) Mylar tries to grab the first release it finds, but the download fails. Mylar marks the release as 'Failed'. It does not continue the search for this issue. Subsequent searches for given issue will still have said release as Failed so it will be ignored from all comparisons.
c) Mylar tries to grab the first release it finds, but the download fails. Mylar marks the release as "Failed" and immediately searches for a DIFFERENT release matching the issue desired. If all of the available/matching releases fail, Mylar marks them all "Failed". Subsequent searches for given issue will still have said release as Failed so they will be ignored from all comparisons.
d)Cannot happen via the GUI (it's a toggle-option). Automatic retries can only be enabled if 'Failed Download Handling' is enabled. Logistically within the code, if Failed Download Handling is not enabled, it will ignore the entire Failed Download Handling/Automatic Retries workflow option and just mark issues as 'Skipped' if not found with a warning within the log about using Failed Downloading handling.
The reason I bring all this up again is, I'm finding Mylar keeps trying to re-download releases that have failed (they are on the "Failed" management page), yet often those same issues are marked as "Skipped" on the series' page. I suspect something got messed up as I kept changing the Failed Downloads options. At this point, would it be best if I manually set all the missing issues (that I know are never going to successfully download) as "Skipped", then cleared all the "Failed" locks on the management page?
Mylar shouldn't be re-downloading anything in a Failed status. Mylar will not retry any given releases that have already been marked as Failed based on provider (so if A has comic.nzb and B has comic.nzb, then it tries A, marks it as failed, then tries B and marks it as failed), but like you said it should be retrying the general search for issues as long as it's in a Failed status. I'm trying to recall now if Mylar just stop searching entirely once it's in a Failed status (meaning it has to be manually done as a safeguard), or if it just assumes Failed = a form of Wanted, and queues it up with the other Wanted issues when perform backlog searches.

Your best bet at this point is to do what you've said - switching back and forth between all the options of Failed handling has probably caused some abandoned failed issues and or incorrect settings that Mylar might be having problems interpreting the correct way. Clearing all the 'Failed' locks on the management page should be enough to get things lined up again for you. I wouldn't mark all of the issues as Skipped initially - do a few and see what happens when you have the toggles set to how you actually desire them instead of experimenting (in your case you would want Failed Download Handling and Automatic Retries enabled). If something happens that shouldn't then please let me know - I know I worked through some of the Failed problems that we had discussed awhile ago, I just can't recall if those changes made it into the Master branch as of yet (which from what I recall is what you're running?)

Aimhere
Posts: 76
Joined: Mon Apr 06, 2015 2:32 pm

Re: Failed download handling

Post by Aimhere » Wed Nov 16, 2016 12:06 pm

Actually, I'm running the development version (have been for a while). Just updated to the latest build, in fact.

I have cleared all the failed locks and enabled both "Failed download" options. We'll see how it goes this time...

Aimhere
Posts: 76
Joined: Mon Apr 06, 2015 2:32 pm

Re: Failed download handling

Post by Aimhere » Fri Nov 18, 2016 12:16 pm

Okay, after taking the above steps (enabling both failed download options, clearing all failed locks), I'm finding that Mylar is once again re-trying the same failed issues over and over and over again.

For example, my NZBGet download log has a dozen identical entries for "Teen Titans 07 2012 Minutemen-DTs". They all failed because the release is incomplete on my Usenet providers, and the log shows ComicRN successfully marked them as "failed". Yet on Mylar's next automated search for "wanted" issues, it ends up sending the exact same release back to NZBGet again.

Multiply this by however many dozens of issues I'm missing, and you can imagine the number of download hits this is making on my NZB and Usenet providers.

I had the NZB search interval in Mylar set to 360 min (e.g. 6 hours), I'm increasing it now to 1440 minutes (24 hours) to try to mitigate the problem and avoid hitting my daily download limits.

(I don't have the Mylar debug logs available at the moment, I'm going to have to try to find the time to log the searches and wade through the logs for the relevant bits.)

It seems to me that Mylar doesn't go far enough in its failed download handling. I wish that, if a given download fails, Mylar would attempt to grab all available releases in turn, and if they ALL fail, just stop (and mark the issues as "Skipped" or something), rather than re-trying them indefinitely.

Aimhere

Post Reply