0-day doesn't seem to download unless I restart Mylar

Post any problems / bugs / issues that are Mylar-related in here.
leaderdog
Posts: 377
Joined: Sun Apr 26, 2015 1:52 pm

Re: 0-day doesn't seem to download unless I restart Mylar

Post by leaderdog »

Hi Evilhero

I don't restart the program at all, only for updates and the one time the power went off in the past several months. Other than today for testing purposes.
Activity/Jobs Scheduler

Code: Select all

DB Updater 	2019-03-28 17:30:02 	5 mins 	2019-03-28 17:25:02 	Waiting 	Pause Force
Folder Monitor 	2019-03-28 17:28:37 	5 mins 	2019-03-28 17:23:37 	Waiting 	Pause Force
RSS Feeds 	2019-03-28 11:34:30 	20 mins 	2019-03-28 11:43:08 	Running 	Pause Force
Check Version 	2019-03-28 23:14:31 	360mins 	2019-03-28 17:14:31 	Waiting 	Pause Force
Weekly Pullist 	2019-03-28 20:04:26 	4 hrs 	2019-03-28 16:04:26 	Waiting 	Pause Force
Auto-Search 	2019-03-28 18:32:34 	360 mins 	2019-03-28 12:32:34 	Waiting 	Pause Force
That all looks correct I think?

Nothing has downloaded in the 5 hours I was out.

At the bottom of the Weekly Pull List page:

last updated: 2019-03-28 16:04:26

Neat though, I never clicked any of those tabs/pages before haha. When it worked it just worked. Could some other setting have broke in the config file some how?

No one else seems to be chiming in as to they're having this problem as well. So it may be some weird configuration error on my end? or windows 7 is stopping something maybe? I don't know what or how. Like I said, radarr and sonarr as well as Lidarr and lazylibrarian are just doing their thing without any problems. Do I need to update cherrypy? I wasn't sure how or if I could, or if the new version was compatible with 2.7. Or another dependency perhaps that didn't get updated that may be an issue?

Thanks.
Hopefully we can narrow this down. :)
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: 0-day doesn't seem to download unless I restart Mylar

Post by evilhero »

Yeah no need to update cherrypy - it's bundled with Mylar so if you upgrade to a newer version, it might break Mylar as it's coded against the version that's bundled with it.


Try refreshing the Shazam! series - but before you do that take note of what the date says for the given issue. What has happened in the past is if some items aren't fully populated, when Mylar passes the information to the search queue it's expecting a certain amount of variables to be there - it might be that it not having one of those particular variables is causing Mylar to just stop (and because it's a threaded job, the logger probably wouldn't catch the exception).

Once you've refreshed Shazam, try to manually snatch that new issue again (via the magnifying glass) and see what happens.
leaderdog
Posts: 377
Joined: Sun Apr 26, 2015 1:52 pm

Re: 0-day doesn't seem to download unless I restart Mylar

Post by leaderdog »

Hi Evilhero,

Ok, that did work.

It said last update
2019-03-27

after refresh it showed
2019-03-29
and it updated the Date to 2019-05-01 for the file itself. which was an earlier date but I didn't write it down.

Then it went ahead and snatched it and downloaded it.

So the comicvine guys, goof, fix it later and then it updates correctly if I refresh it?

Would it be helpful then, since it's user error/relying on comicvine for information, to have the 0day auto refresh a day or two after comic Wednesday because there will always be errors since it's out of Mylar's hands? Or at least refresh the issues that didn't download off the list (still wanted).

The reason I ask was I had 1000 issues on my wanted list, many months of missed books, Many of them I did find on the newsgroups, so it wasn't doing its job, possibly because of the date error? Mylar does go through and refresh series, I see it doing it on long ended series, so when there are 3500 series in the list, that's a tremendous amount of time spent refreshing series that don't need to be. I'd think it would be nice if ended series wouldn't be searched unless a similar comic name is newly released, and it could check against it to see if it's being renewed or if it's a new series entirely. (Not sure if I explained that correctly).

Example: Vampblade (2016) is being refreshed right now and it ended it 2017-01-31. It says Last updated 2018-04-29 (and it's taking quite some time to refresh it, the page keeps flashing/refreshing then the little spinner just keeps spinning like it's having trouble pulling the info - sat here watching it for the better part of 5 minutes. More than that now, but I'm done typing the email and it's not completed the refresh yet)

I see the Pause comic button on the top of the page, maybe, an "ended" button could be there to speed Mylar up, and then it would not "refresh" the series for no reason. or is that what the pause button does? but as I mentioned above, the odd series does get resurrected so if the name title is brought back up it could automatically start it up.

Ideally of course it would be nice for Mylar to know when a series is ended, but I don't know if that's possible with comicvine.

But do you think the date issue is why I had many months of missed comics? Because arguably, as they were put on newsgroup and althub got them, it should have done the check vs rss to grab them as they're being released?
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: 0-day doesn't seem to download unless I restart Mylar

Post by evilhero »

Well the CV guys don't goof, really - they just don't populate all the data at the same time. So on Tuesday usually they will create the issue on CV (and thereby an issueid), but with no date - but it will have a cover and that's it. Then when it gets released (on the Wed), then on that date or lately it's been Thursdays they populate the rest of the issue information like the dates, description, authors, etc.

There's no need to have the 0day stuff auto-refresh a day / two after as that could be an endless thing (if an issue was supposed to come out, but didn't say for an example). However, the backend API site that Mylar pulls the pullist information from has all that and it's built into it. So when there's issue data present, that means there's a valid issueid that Mylar can assign. Since it knows this, and the backend API site pushes down comicid/issueid information, when Mylar sees that there's an issueid and it's on your watchlist (match via comicid) it does a refresh of the series so that the issue data can be populated properly.

I think the problem in all of this was that your pull-list wasn't retrieving all the information from the API site for some reason - if it had, it would have pulled down the new issueid and thereby the new information, dates and it wouldn't be an issue.

Now you can set it to only refresh active/continuing series via the config.ini (updated_ended variable, set it to True) but there are caveats with that as well. If it's a rebooted series that's marked as no longer continuing - you'll never get the new information. It has happened in awhile granted, but a few years ago they were rebooting all the old series with the original numbering which would mean that in some cases you might not get the relevant issue information to know there's a new issue with this particular option enabled.

The Ended option has been asked before, it's just a matter of trying to build it in as Mylar has to determine if a series is continuing or not based on the available dates it has (since CV will not indicate that a series is ended anywhere for us to track). The Paused button at the top of the page will skip Mylar refreshing the series in the dbupdates, as well as for searches/post-processing.

As far as to why it hasn't updated for you - the only reason I can think of is that you had a search error at one point and that locked up the search queue. Since you said you don't restart Mylar very often, the traceback could have been lost in the logs (or rotated out) by the time you sat down to look - and you wouldn't know any different. Once the traceback occurs, it literally kills the entire search queue. Same goes for any of the other queues available (there's one for search, one for post-processing, one for CDH, and one for DDL) - so if say the DDL queue encounters an error, you won't be able to perform any ddl searches properly since it's encountered an error.
leaderdog
Posts: 377
Joined: Sun Apr 26, 2015 1:52 pm

Re: 0-day doesn't seem to download unless I restart Mylar

Post by leaderdog »

thanks for clearing that up.

So should I be doing something? remake the config file?

Since files weren't being download because of a search error, does that fix itself when I do an update to Mylar and it restarts, or is there something still messed up somewhere?

and you or bbqsauce said I was having nzb errors from 6box. I removed them for the moment as a provider. Would that have been the search que issue?

And out of curiosity, why would it take around 10 minutes for Mylar to refresh an old series? Vampblade... well sheeet it is still doing a refresh on it. that's almost an hour later now. still has the circle thing by the name and says:
(Comic information is currently being loaded)

would something be "freezing" up or is it queued where it would cause issues? I know often when I go to look at a series when downloading files manually sometimes it just hangs and I just wait for 10 minutes or get bored and go do something else.

and while I was sitting here typing, Mylar did this :
500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

Traceback (most recent call last):
File "C:\Mylar\lib\cherrypy\_cprequest.py", line 670, in respond
response.body = self.handler()
File "C:\Mylar\lib\cherrypy\lib\encoding.py", line 217, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "C:\Mylar\lib\cherrypy\_cpdispatch.py", line 61, in __call__
return self.callable(*self.args, **self.kwargs)
File "C:\Mylar\mylar\webserve.py", line 97, in comicDetails
skipno = len(mylar.COMICSORT['SortOrder'])
TypeError: object of type 'NoneType' has no len()


My PC cursed? :)
leaderdog
Posts: 377
Joined: Sun Apr 26, 2015 1:52 pm

Re: 0-day doesn't seem to download unless I restart Mylar

Post by leaderdog »

Just to post an update on the Vampblade (2016) situation.

1) I killed Mylar instance last night,
2) git pull the new updates,
3) restarted Mylar
4) went to bed
5) checked now at 7:40am
6) Vampblade is still stuck in "refreshing" mode.

That's crazy. is anyone else having vampblade (2016) stuck in a refresh loop?

If I click on the link to go to vampblade that works fine. Is it Mylar that can't finish, or is it comivine's api that is messed up?

Should I just hit pause and hope it stops?

Here's the log in case there is something in it. Aw shoot, I didn't toggle debug after I restarted it last night. It hung long before that mind you and debug should be on then.
https://www.dropbox.com/s/469xolbkc2f15 ... g.rar?dl=0
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: 0-day doesn't seem to download unless I restart Mylar

Post by evilhero »

After you said that, I realized what was happening.

In your Folder Format, you have a / in there that's not the correct slash for your OS. The Folder Format is a directory structure, so if on Windows, you have a / instead of a \, the / designates a control character and it will break the structure.

You need to change your folder format string to the proper slash designation.

This was changed a few months ago, so that things were checked prior - due to people having incorrect slashes in their Folder Format, and it literally messing up directory structures when it goes to post-process. In your case, when it tries to refresh a series, it can't because the / is basically the equivalent of doing a ctrl+c during a running process.
leaderdog
Posts: 377
Joined: Sun Apr 26, 2015 1:52 pm

Re: 0-day doesn't seem to download unless I restart Mylar

Post by leaderdog »

oooooh you are good!

That fixed it. I see there are a whole bunch of titles that are like that. So I guess I'll get to work changing them.

Would that cause Mylar to be sluggish? because it has felt very sluggish over that past few months.
leaderdog
Posts: 377
Joined: Sun Apr 26, 2015 1:52 pm

Re: 0-day doesn't seem to download unless I restart Mylar

Post by leaderdog »

Hi Evilhero,

I went through all the "Active spinning Status" books and changed the / to \
killed the instance
did 'git pull' because 9 new updates
started Mylar, hoping that it would just reset the refreshing process.

I manually refreshed some after I changed the/ and it stopped it from spinning.

However,

There is now 103 books that are blue and the status is "spinning" stuck in refresh. Many of the ones were that way before, but when I checked them, they didn't have the wrong \ mark.

Is there anyway to reset the refreshing process?
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: 0-day doesn't seem to download unless I restart Mylar

Post by evilhero »

There's no way to reset the Refresh of a series, other than just refreshing the series. Until that time the series is considered incomplete as some issue data might not be present for some of those series, depending on the stage where it aborted the process.

You could go into the Manage tab / Manage Comics and sort it by Status so that all the Loading items are at the top. Then select all of those Loading items, and select a Refresh Series from the dropdown so it will refresh them all. I'd suggest doing it in groups of 25 or so just to be save, but I mean you could do all 100+ if you wanted to - just might make things harder to monitor is all.
Post Reply