NZB Search Interval locked

Post any problems / bugs / issues that are Mylar-related in here.
capGrundy
Posts: 51
Joined: Thu Nov 09, 2017 12:25 am

NZB Search Interval locked

Post by capGrundy »

I changed NZB Search Interval while trying to find issues I was having with CPU usage (fixed), but I can't seem to change it now. I even changed it manually in the ini, but when I restart, it sets it back to 360. Is this coded/stored somewhere else?
capGrundy
Posts: 51
Joined: Thu Nov 09, 2017 12:25 am

Re: NZB Search Interval locked

Post by capGrundy »

I found the issue, kind of. Once you touch the download_scan_interval, it automatically changes the search_interval to 360. Once you do that, it's unchangeable from there. No matter how much you edit config, it keeps setting it back.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: NZB Search Interval locked

Post by evilhero »

The 360 is a minimum that value that Mylar enforces. So even if you edit the config.ini and it loads up with your loaded value, it should only be adhering to the 360 minute minimum. This is due so that things don't get hammered constantly by Mylar and goes back to several applications hammering nzb providers in order to get the results faster which either resulted in a) the provider banning the user, b) the provider banning the use of the application.

Honestly, as long as you have the RSS Feeds option enabled and set properly so your provider doesn't have an issue (normally every 20 minutes is good as comics don't roll in every minute), it should be more than enough to get the comics as they're being released - provided you have Mylar on 24/7. This is because of the way Mylar caches the feeds so that it can continually search against it's own cache.

Over the next few months, I will be making some changes to the ways that Mylar searches and how often it searches for items prob (kinda like a tiered Wanted list - anything marked Wanted less than a week ago would get searched every 6hrs, anything more than a week every 24hrs, and anything more than a week would be a manual search). Of course the numerics might change, but as a rough start that's what I'm looking at in order to allow some more granular control over the Wanted list, while keeping it as automated as possible without running into issues where it takes forever to run because of the general size of the Wanted list)
capGrundy
Posts: 51
Joined: Thu Nov 09, 2017 12:25 am

Re: NZB Search Interval locked

Post by capGrundy »

Ok, that's fine. I was only thrown because I think it used to be 20min, and I only noticed the change when I was adjusting the 5min search interval for directories. I don't think it was the actual activity itself, but changing that stopped some constant CPU activity I was running into.

I'm fine with higher limits on check times, as other than right at release time, there isn't a lot of rss/file activity going on. It would be a nice tweak to be able to adjust all the intervals on the tasks page for those like myself don't need to it be as hawkish on some checks.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: NZB Search Interval locked

Post by evilhero »

Well the search interval has always been 360 minutes - like from the beginning of Mylar (at least as I can remember it being that). The RSS Feeds though has always been a default setting of 20 minutes. And the tweaking of the schedulers is coming - the biggest hurdle in the new config stuff and the schedulers was getting everything working properly. There are still some items that need to get fixed, and once those do - then the ability to change the times on the schedulers tab will probably show up.

As far as the constant CPU activity, the monitor running every few minutes if you have a large amount of files in the monitor folder would cause things to spike as it has to load all the items into memory, parse and then compare. Do you have a large amount of files in the monitor folder and/or are you using the copy option due to seeding? There was a patch that went into development a day or so ago, that involved items that are supposed to stay in the monitor folder not going through the parsing process as they've already been determined to have been post-processed already (based on crc), which would in theory lower the amount of cpu usage at that point.
capGrundy
Posts: 51
Joined: Thu Nov 09, 2017 12:25 am

Re: NZB Search Interval locked

Post by capGrundy »

I'm watching about 150-200 titles, and they have about 20-30 objects per dir on average. It seems like DB Updater and Status Updater are perpetually running.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: NZB Search Interval locked

Post by evilhero »

capGrundy wrote: Mon Dec 04, 2017 5:01 pm I'm watching about 150-200 titles, and they have about 20-30 objects per dir on average. It seems like DB Updater and Status Updater are perpetually running.
Well the number of titles you're watching doesn't really have an impact on things per se (there are those with well over 1000 in their watchlists) - nor does the number of items in the directory of each series. Only when the series gets Refreshed and/or the Recheck Files option will Mylar actually run thru the files in the directory. I was actually inquiring about the number of items in the folder that Mylar monitors for new issues (ie. the folder monitor, which is what the download scan interval applies to) as this would play a role in spiking usage if there are a large amount of files in that particular folder.

The DB Updater is what Mylar runs to keep things in sync with CV. It runs every 5 minutes and simply refreshes the oldest series in your watchlist to ensure that items are up-to-date, if everything is up-to-date then it doesn't start refreshing again until the 5hr mark has elapsed and every other series has been updated. The idea being that by the time new issues are released, the series are updated and ready to work with the new pull each week. The pull itself however, will also force update series it notes as having issues in a given week, but aren't available in the series as of yet (because CV doesn't update stuff until Tues/Wed usually).

I have some logistics going in soon, that will determine the amount of time to run the updater based on the number of items in your watchlist. So if you have say 700 titles, then it will run every 5-7 minutes, whereas if you have 50 it will run every 15-20 mins (not exact time obviously, but you get the gist I'm sure).

The Status Updater is just the Recheck Files running - which is run at the end of every Refresh of the series (which is what is run when the DB updater runs).
capGrundy
Posts: 51
Joined: Thu Nov 09, 2017 12:25 am

Re: NZB Search Interval locked

Post by capGrundy »

I'm not sure if I'm missing the distinction. In my setup, I only have active titles added (I removed ended series...and sometimes the "ended" flag in mylar is inaccurate). Each title dumps into the folder for that title, and not a new issues folder. So if I have it watching Saga, the path is to that title in my comics folder. So the new issues folder is just the default folder for that specific title.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: NZB Search Interval locked

Post by evilhero »

Well then it sounds like you're not even using the Folder Monitor option at all, so changing the download_scan_interval setting wouldn't affect anything. The Folder Monitor is just meant to be a place where you can dump files in and mylar will scan it and if they match an existing series in your watchlist, it wil post-process them.

The 'ended' flag might be inaccurate either due to the series not being refreshed at a normal rate, or CV hasn't updated the items on there end to reflect current dates (sometimes it's both).

Unfortunately CV doesn't indicate if a series is Active/Ongoing vs. Ended, so all the calculations to determine if a series is Ended in Mylar is done by Mylar itself. It usually takes the most recent issue of a series as the ending point, and then compares it to the current date. If it's more than the given threshold, it will mark it as Ended, otherwise it assumes it's in an Ongoing status. The threshold is determined by the biggie_pub and the indie_pub values, which are set to 55 and 75 days respectively by default. These settings really shouldn't be adjusted too much, as in doing so you can really change how effective Mylar is at determining if a series is continuing or not, and thus the post-processing/pullist would be affected by incorrect values since both rely on dates to determine items.
capGrundy
Posts: 51
Joined: Thu Nov 09, 2017 12:25 am

Re: NZB Search Interval locked

Post by capGrundy »

So what are DB Updater and Status Updater doing? Those seem to be the ones I find running constantly, sometimes. Like I said, it may not be causing a problem, as that went away after I played with the times. By and large cpu and r/w don't seem to be that big of a problem anymore.

WIth the ended, I kind of guessed that was the case. There are some smaller titles that can span months between releases, so I'm used to that being the case. I try to double-check through Comicvine, and that also have it's problems. They are only better at being able to find limited vs extended series most often.
Post Reply