Mylar not finding any comics

[Jeroen]

Hey,

I would appreciate some help again

For a couple of days now Mylar hasn't been finding any comic on either KAT or 32P even though searching for these manually on the websites always comes up with results. It's almost like no search is actually performed. This is with wanted items on my pull list and manual searches.

An example of a typical log entry after doing a manual search:

Code: Select all

13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%                
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                     
13-Oct-2015 19:38:54 - INFO    :: Thread-12 : Could not find Issue 3 of Old Man Logan(2015) using 32P [rss]
13-Oct-2015 19:38:54 - INFO    :: Thread-12 : Shhh be very quiet...I'm looking for Old Man Logan issue: 3 (2015) using KAT [RSS]
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : calpha/cnumber: None / None
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : RSS Check: yes                                                            
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : nzbprov: KAT                                                                                    
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : comicid: 82208                         
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Sending request to [KAT] RSS for Old Man Logan : 003
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : ComicID: 82208                          
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%       
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                               
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : RSS Check: yes                                                            
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : nzbprov: KAT                                                                       
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : comicid: 82208                                                 
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Sending request to [KAT] RSS for Old Man Logan : 03
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : ComicID: 82208                                         
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                    
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : RSS Check: yes                          
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : nzbprov: KAT                            
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : comicid: 82208                 
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Sending request to [KAT] RSS for Old Man Logan : 3               
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : ComicID: 82208       
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%                                                  
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                                                                                  
13-Oct-2015 19:38:54 - INFO    :: Thread-12 : Could not find Issue 3 of Old Man Logan(2015) using KAT [rss]
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Could not find issue doing a manual search via : rss
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Initiating Search via : api             
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : 32P backlog searching is not currently supported.

The same happens with comics that are on the rss feeds right now (or on my pull list), so I can't imagine that latest line to be relevant every time.
Is it normal KAT is listed as "nzbprov"?

I did do a reinstall of Mylar after some experimenting but I can't for the life of me imagine what I could be forgetting or overlooking. I also tried to use a proxy for KAT because of recent blocking of kat.cr in several browsers lately but it doesn't seem to make a difference.

Any ideas? Other people experiencing similar problems lately?

My config.ini should it be of any help:
http://pastebin.com/AjBwXfem

[evilhero]

Code: Select all

13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%                
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                     
13-Oct-2015 19:38:54 - INFO    :: Thread-12 : Could not find Issue 3 of Old Man Logan(2015) using 32P [rss]
13-Oct-2015 19:38:54 - INFO    :: Thread-12 : Shhh be very quiet...I'm looking for Old Man Logan issue: 3 (2015) using KAT [RSS]
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : calpha/cnumber: None / None
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : RSS Check: yes                                                            
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : nzbprov: KAT                                                                                    
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : comicid: 82208                         
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Sending request to [KAT] RSS for Old Man Logan : 003
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : ComicID: 82208                          
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%       
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                               
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : RSS Check: yes                                                            
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : nzbprov: KAT                                                                       
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : comicid: 82208                                                 
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Sending request to [KAT] RSS for Old Man Logan : 03
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : ComicID: 82208                                         
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                    
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : RSS Check: yes                          
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : nzbprov: KAT                            
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : comicid: 82208                 
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Sending request to [KAT] RSS for Old Man Logan : 3               
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : ComicID: 82208       
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : tsearch : old%man%logan%                                                  
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : seriesname_alt:None                                                                                  
13-Oct-2015 19:38:54 - INFO    :: Thread-12 : Could not find Issue 3 of Old Man Logan(2015) using KAT [rss]
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Could not find issue doing a manual search via : rss
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : Initiating Search via : api             
13-Oct-2015 19:38:54 - DEBUG   :: Thread-12 : 32P backlog searching is not currently supported.

The same happens with comics that are on the rss feeds right now (or on my pull list), so I can't imagine that latest line to be relevant every time.
Is it normal KAT is listed as "nzbprov"?

The nzbprov indicator is just for logging purposes - it has no bearing on anything other than just to log the provider name (it should be changed so it logs it properly, but currently torrent/nzb providers are all logged as 'nzbprov')

Mylar will only hit the required feeds every 20 minutes, and cache said entries into it's local db. So if you turn off mylar, or don't leave it running 24/7 - the cache isn't as complete as it could be if it was left running. The fact that it's not returning any hits whatsoever off the local cache (when it says 'Sending request to [KAT] RSS...' it's searching the local cache, which gets populated with the current feed(s) prior to the search), would seem to indicate it's a problem with it being able to retrieve the feeds properly.

Don't use a proxy for KAT, as using a proxy will cause problems as alot of the proxy sites don't return results in a proper format for Mylar to interpret, and others aren't as up-to-date either. For 32P, if you're using the non-auth mode the only feed Mylar monitors is the new releases feed for 32P. Your personal feeds aren't monitored at all, so it could be fairly easy that an item is on your personal feed, but not on the new release feed - which would mean it wouldn't show up in the cache results when searching. Although it looks like you're using the auth-mode, which is your username/password.

When you do a Force RSS on the configuration/Search Providers tab, and check the logs - does it list how many items it's successfully storing within the cache ?

32P backlog support isn't supported yet, so if you don't have it in your cache / rss feeds then Mylar would not be able to retrieve any results properly. Mylar is able to do backlog searches for KAT however, so it's strange that you're not getting any results especially if you're not using a proxy for KAT.

[Jeroen]

I always use Auth mode for 32P, but I tried Legacy mode too but that didn't give me results either. Also tried after getting a new passkey from 32P.

This is a log after doing a force rss:

13-Oct-2015 20:58:58 - DEBUG :: CP Server Thread-7 : Trying to execute: "git rev-parse HEAD" with shell in /volume1/Mylar
13-Oct-2015 20:58:58 - DEBUG :: CP Server Thread-7 : Git output: 60140e8c900ae6c2acc3c9bf319506b6514fbed4

13-Oct-2015 20:59:02 - INFO :: CP Server Thread-7 : Attempting to run RSS Check Forcibly
13-Oct-2015 20:59:02 - INFO :: Thread-14 : RSS Feed Check was last run at : 2015-10-13 20:56:43
13-Oct-2015 20:59:02 - INFO :: Thread-14 : RSS Feed Check First Ever Run.
13-Oct-2015 20:59:02 - DEBUG :: Thread-14 : Updating RSS Run time to : 2015-10-13 20:59:02
13-Oct-2015 20:59:02 - INFO :: Thread-14 : [RSS] Initiating Torrent RSS Check.
13-Oct-2015 20:59:02 - INFO :: Thread-14 : [RSS] Initiating Torrent RSS Feed Check on KAT.
13-Oct-2015 20:59:03 - INFO :: Thread-14 : [KAT] there were 25 results from the New Releases RSS Feed for comics
13-Oct-2015 20:59:03 - DEBUG :: Thread-14 : Completed adding new data to RSS DB. Next add in 20 minutes
13-Oct-2015 20:59:05 - INFO :: Thread-14 : [KAT] there were 25 results from the New Releases for category Other RSS Feed that contain comics
13-Oct-2015 20:59:05 - DEBUG :: Thread-14 : Completed adding new data to RSS DB. Next add in 20 minutes
13-Oct-2015 20:59:05 - INFO :: Thread-14 : [RSS] Initiating Torrent RSS Feed Check on 32P.
13-Oct-2015 20:59:05 - DEBUG :: Thread-14 : [RSS] 32P mode set to Auth mode. Monitoring all personal notification feeds & New Releases feed

I only see the 25 results from KAT?

[evilhero]

Yep, that's correct - it can only pull in 25 items at a time from the rss feed. For KAT, it monitors the new releases feed for Books/Comics and it also monitors the 'other' RSS feed for any items that have yet to be categorized but are of cbr/cbz extensions.

Does it say anything else after the last line regarding 32p? Is it able to pull x items from the personal feeds for 32p (it will show in the logs just right after the last line you pasted in here).

[Jeroen]

No, that was the last line after that check.


But I was triggered by what you said about

Mylar is able to do backlog searches for KAT however, so it's strange that you're not getting any results especially if you're not using a proxy for KAT.

I noticed earlier I didn't have a proxy set for KAT in my config.ini even though I couldn't remember having removed it already. So that got me thinking maybe my changes to the config weren't actually saved before because of file permissions. And maybe that could also mean Mylar hasn't been able to build up a local cache... So I corrected permissions and immediately I see my personal feeds appearing in the log, which weren't before.

13-Oct-2015 21:09:44 - INFO :: Thread-12 : [KAT] there were 25 results from the New Releases for category Other RSS Feed that contain comics
13-Oct-2015 21:09:45 - DEBUG :: Thread-12 : Completed adding new data to RSS DB. Next add in 20 minutes
13-Oct-2015 21:09:45 - INFO :: Thread-12 : [RSS] Initiating Torrent RSS Feed Check on 32P.
13-Oct-2015 21:09:45 - DEBUG :: Thread-12 : [RSS] 32P mode set to Auth mode. Monitoring all personal notification feeds & New Releases feed
13-Oct-2015 21:09:45 - DEBUG :: Thread-12 : [32P] Verify SSL set to : True
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] Authkey found: ***
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] Userid found: ***
13-Oct-2015 21:09:46 - INFO :: Thread-12 : [32P-AUTHENTICATION] Atttempting to integrate with all of your 32P Notification feeds.
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] Auth:***
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] Passkey:***
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] [NOTIFICATION: All] Notification ID: torrents_notify_2181
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] [NOTIFICATION: Follow+List] Notification ID: torrents_notify_2182
13-Oct-2015 21:09:46 - DEBUG :: Thread-12 : [32P-AUTHENTICATION] [NOTIFICATION: Phonogram] Notification ID: torrents_notify_2975

I also see books appearing in the "upcoming" tab again.

So I think this could very well have been the cause?

[evilhero]

Could very well have been - If it was stalling at the 32P auth part on startup, then it would never be able to continue doing some of the other things (populating the pullist/upcoming tab, etc), as it would have failed earlier in the process.

From that log though it looks like it's able to poll KAT and 32P now successfully and like you said, it's populating the Upcoming as well now which is always a good thing

[Jeroen]

Great, I have several books expected to be available soon so I'll see if those get picked up.

I just tried creating a test notification feed at 32P. So far it didn't retrieve anything but maybe I should just let Mylar build up it's cache first and do it's magic

I'll report back. Thanks again for your help as always, I'll be sending some coffee your way

[Jeroen]

Allright, Mylar is grabbing comics again Ultimately the last part of the solution was clearing the "provider_order" in the config.ini. It immediately started grabbing comics after that.

[evilhero]

Allright, Mylar is grabbing comics again Ultimately the last part of the solution was clearing the "provider_order" in the config.ini. It immediately started grabbing comics after that.

Thanks for the helping the coffee fund

Glad to hear you got it working properly - the provider_order line has been a problematic part of the entire process for a lengthy time unfortunately. Something I hope to finally remedy in the next few commits ...

[Jeroen]

And seemingly out of the blue I am having trouble with this again

Code: Select all

01-Nov-2015 11:38:24 - INFO :: Thread-12 : [RSS] Initiating Torrent RSS Check.
01-Nov-2015 11:38:24 - INFO :: Thread-12 : [RSS] Initiating Torrent RSS Feed Check on KAT.
01-Nov-2015 11:38:34 - INFO :: FOLDERMONITOR : [FOLDER-CHECK] Checking folder /volume1/download/process/ for newly snatched downloads
01-Nov-2015 11:38:35 - INFO :: FOLDERMONITOR : [POST-PROCESSING] No matches for Manual Run ... exiting.
01-Nov-2015 11:38:35 - INFO :: FOLDERMONITOR : [FOLDER-CHECK] Finished checking for newly snatched downloads
01-Nov-2015 11:38:40 - INFO :: Thread-12 : [KAT] there were 0 results from the New Releases RSS Feed for comics
DiskStation> python Mylar.py 01-Nov-2015 11:38:55 - INFO :: Thread-12 : [KAT] there were 0 results from the New Releases for category Other RSS Feed that contain comics
01-Nov-2015 11:38:55 - INFO :: Thread-12 : [RSS] Initiating Torrent RSS Feed Check on 32P.
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [32P-AUTHENTICATION] Atttempting to integrate with all of your 32P Notification feeds.
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [32P] there were 0 results from the New Releases RSS Feed for comics
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [32P] there were 0 results from your Personal Notification Feed : All
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [32P] there were 0 results from your Personal Notification Feed : Follow+List
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [32P] there were 0 results from your Personal Notification Feed : Mylar+Test
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [32P] there were 0 results from your Personal Notification Feed : Pull+List
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [RSS] Initiating RSS Feed Check for NZB Providers.
01-Nov-2015 11:38:56 - INFO :: Thread-12 : [RSS] RSS Feed Check/Update Complete

Mylar is not finding releases I can manually verify being available.

I tried removing the provider order line from the config.ini again, but it doesn't seem to solve it this time around. Any ideas

On a side note, after updating to the latest commit Mylar refused to start, throwing this log message:

Code: Select all

DiskStation> python Mylar.py
Traceback (most recent call last):
  File "Mylar.py", line 26, in <module>
    import mylar
  File "/volume1/Mylar/mylar/__init__.py", line 25, in <module>
    import webbrowser
ImportError: No module named webbrowser

I managed to start Mylar by simply remove the mentioned line but I can imagine this not being the best solution

Not sure if this can be related to Mylar not finding releases. Can you advise about how to adress this? If this has anything to do with actually launching a browser, I am using Mylar on a NAS without any webbrowser available.