Unable to find or download new issues

Post any problems / bugs / issues that are Mylar-related in here.
Post Reply
Jackson
Posts: 14
Joined: Thu Nov 27, 2014 1:37 am

Unable to find or download new issues

Post by Jackson »

Hi,

I'm running into a problem with a new install and I'm pretty sure I'm missing something obvious. I'm hesitant to open a GitHub issue if I can't verify that isn't my problem.

Some quick notes:
  • I'm using 32p in Auth Mode as my torrent source
  • The install is new, the config is new, the database file is imported from an old version
  • I've reproduced the problems with both local watchdir and Transmission download clients
  • Running mylar master branch with python 2.7.14 but behaviour is the same in development branch
When I do a manual search for a wanted issue that I can verify is available on 32Pages, Mylar will refuse to find it, or if it does find it, the log tells me the download fails but isn't giving me any reason why the download is failing. I've copied an except from my log below as verbose mode is really verbose. If more is necessary please let me know.

Code: Select all

20-Sep-2017 12:08:12 - INFO :: Thread-13 : entry: {'length': u'80693111', 'link': u'54924', 'site': u'32P', 'pubdate': u'Wed, 20 Sep 2017 13:45:07 +0000', 'title': u'Poe Dameron vol0 #19 (Minutemen-Midas)'}
20-Sep-2017 12:08:13 - DEBUG :: Thread-13 : issues match!
20-Sep-2017 12:08:13 - INFO :: Thread-13 : Found Poe Dameron (2017) issue: 19 using 32P [RSS]
20-Sep-2017 12:08:13 - DEBUG :: Thread-13 : link given by: 32P
20-Sep-2017 12:08:13 - INFO :: Thread-13 : prov  : 32P [RSS][54924]
20-Sep-2017 12:08:13 - INFO :: Thread-13 : [FAILED_DOWNLOAD_CHECKER] result has a status of FAIL which indicates it is not a good choice to download.
20-Sep-2017 12:08:14 - INFO :: Thread-13 : [FAILED_DOWNLOAD_CHECKER] continuing search for another download.
20-Sep-2017 12:08:14 - DEBUG :: Thread-13 : [FAILED_DOWNLOAD_CHECKER] [32P [RSS]] Marked as a bad download : 54924
Eventually it gives up, telling me it can't find any issues from RSS or API.

Thanks for any help.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Unable to find or download new issues

Post by evilhero »

In the config.ini there are 2 specific variables that directly affect how mylar searches 32p in auth mode.

Search_32p {0/1} - this should be set to 0. Setting it to 1 will force mylar to directly search against 32p for the issue, it because of the changelog of the series page therein it won't return any valid results. 0 makes it hit an intermediary site that basically has all the correct series markers that exist on 32p in an available format for mylar to properly use.

deep_search_32p {0/1} - this is a new option but for series that have non alphanumeric characters in the series title (ie. ':'), it normally submits the series for search up to the special character in order to ensure that all results are tabulated. Unfortunately before this option, it would only take the 1st March and if it didn't match would return no results for a match. With this enabled (setting it to 1), it will parse every series in the result list for the proper series title and issue, but it might take more time to parse large titles (think how many different titles there are to say Batman on 32p and you'll see what I mean).

Normally, with your problem the search_32p = 0 solves things, but the deep_search_32p is now another option to try (in conjunction with the search_32p).

Give either /both a try and let me know how it works for you, or any other problems and I'll be glad to help out.
Jackson
Posts: 14
Joined: Thu Nov 27, 2014 1:37 am

Re: Unable to find or download new issues

Post by Jackson »

So after your suggestion I looked at the two lines in the config file based on how 32pages searches run. When I started, both were set at 0.

I tried different combinations of 0 and 1 for both options with no success. I started doing manual searches for a missing issue of a series with only alphabetic characters in the name. I manually verified the missing issue was available on 32p.

I ended up not getting anywhere, still running into the "failed download error" which I've copied from my web-ui log page below (reverse order). This is with verbose OFF but with verbose on the output is almost identical.

Code: Select all

2017-09-20 21:05:38	INFO	Could not find Issue 29 of Nightwing (2016) using 32P [api]
2017-09-20 21:05:38	INFO	[FAILED_DOWNLOAD_CHECKER] continuing search for another download.
2017-09-20 21:05:38	INFO	[FAILED_DOWNLOAD_CHECKER] result has a status of FAIL which indicates it is not a good choice to download.
2017-09-20 21:05:38	INFO	prov : 32P[54916]
2017-09-20 21:05:37	INFO	Found Nightwing (2017) issue: 29 using 32P
2017-09-20 21:05:37	INFO	entry: {'seeders': u'41', 'scanner': u'Minutemen-Slayer', 'pubdate': 'Wed, 20 Sep 2017 04:49:11', 'format': u'cbr', 'chkit': {'series': u'Nightwing', 'id': u'583'}, 'leechers': u'0', 'link': u'54916', 'int_pubdate': 1505904551.0, 'issues': u'29', 'language': u'English', 'title': u'Nightwing v4 #29', 'filesize': u'36531577', 'pack': u'0'}
2017-09-20 21:05:37	INFO	successfully retrieved 4 search results.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Unable to find or download new issues

Post by evilhero »

I thought I made mention of this in my first post, but looking back now it was never said. So I have to apologise for neglecting to mention it until now.

If an issue goes into Failed status, Mylar will ignore it as a search result. There are various reasons why it go into Failed status, such as unable to download a torrent, unable to post-process it successfully due to file corruption, and also if it's unable to connect to the client in order to send the torrent (this one will be fixed hopefully soon, it's a glitch in how it handles things).

You need to go to the Manage tab / Manage Failed Links. Once there, reverse sort by Date (so that the most recent is at the top), and then just batch select the ones that were mistakenly marked as Failed and choose Clear Failed Lock from the drop-down menu. Then subsequent searches will snatch it if it's a valid match to the given issue, but if it fails for one of the reasons above, it will get marked as Failed.

If you're having problems with Mylar not being able to send to your client (ie. it goes into Failed status after the torrent is snatched), post the debug logs showing what happens after it's successfully found an issue, along with any errors and we can go from there and get things working properly.
Jackson
Posts: 14
Joined: Thu Nov 27, 2014 1:37 am

Re: Unable to find or download new issues

Post by Jackson »

Aha, that is what I was missing. Clearing the failed downloads from that section (never noticed that before) allows me to get some more information out of my log files when in verbose mode.

Looking at that information my problem is with the handoff to Transmission. If I set the download client to a local watch dir everything works great, but when I put it back to Transmission then I get the following:

Code: Select all

21-Sep-2017 18:15:25 - DEBUG :: Thread-13 : filename for torrent set to : /home/jackson/app-data/mylar/cache/Nightwing_29_2017.torrent
│21-Sep-2017 18:15:25 - DEBUG :: Thread-13 : [32P] Verify SSL set to : True
│21-Sep-2017 18:15:25 - DEBUG :: Thread-13 : [32P-AUTHENTICATION] 32P (Auth Mode) Authentication already done. Attempting to use existing keys.
│21-Sep-2017 18:15:26 - DEBUG :: Thread-13 : [32P] Saved torrent file to : /home/jackson/app-data/mylar/cache/Nightwing_29_2017.torrent
│21-Sep-2017 18:15:26 - ERROR :: Thread-13 : incomplete format
│21-Sep-2017 18:15:26 - ERROR :: Thread-13 : Unable to send torrent to client. Assuming incomplete link - sending to Failed Handler and continuing search.
│21-Sep-2017 18:15:26 - INFO :: Thread-13 : [FAILED-DOWNLOAD] Marking as a Failed Download.
My Transmission settings are:

Transmission Host: http://host.domain:9091
User: <none>
Password: <none>
Directory: /home/jackson/path

My interpretation is that the Directory is local to Transmission and determines where the download gets saved too. I'm also not sure if the information in Host is in the correct format. I've found some issues in Github but nothing that hasn't declared to be fixed already.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Unable to find or download new issues

Post by evilhero »

I think with Transmission if I recall correctly, you don't to put in the http:// or https:// - just the address. If it's the default web interface port for transmission (which is 9091), you don't even have to put that in as it will default to it.

So if everything is local (both Mylar & Transmission), you could probably just set your Transmission host to localhost and leave it at that. Otherwise, you could just put the ip with no port and try that.

As far as the default directory, it's local to Transmission yes - it's where the final torrent contents will be saved to if it's specified. If it's exactly what you have as the default for Transmission anyways, there's no need to specify it since it will land there anyways. The idea behind this initially, was that if both clients were local - you could set a download folder path to be the folder that Mylar monitors for new releases, thereby automatically post-processing them when they complete.
Jackson
Posts: 14
Joined: Thu Nov 27, 2014 1:37 am

Re: Unable to find or download new issues

Post by Jackson »

I was able to get it working as expected, thank you for all your help.

The issue ended up being the port in transmission_host. Once I only had the hostname or IP address, everything was passed properly. As soon as I defined the port (even though it is the default) it failed. I'm not really ready to set up a second Transmission instance to test a non-default port, but I will create an issue in Github later today.

Again, thank you for all the time developing and helping users.
Post Reply