Request: Better Instructions to setup Post Processing

Information and Tutorials on features in Mylar and how to use it
totallytroy
Posts: 3
Joined: Thu Dec 05, 2013 5:37 am

Request: Better Instructions to setup Post Processing

Post by totallytroy »

I'm having real trouble getting the Post Processing to work. The instructions on the github site are vague. It would be great to get instructions that a 5 year old could follow. Hopefully with clearer instructions I could find my problem :)

Some spots in the instructions that seem unclear:
- "Edit (put in your Mylar host, port, login and password (if required), and ssl(0 for no, 1 for yes) and rename the autoProcessComics.cfg.sample to autoProcessComics.cfg."
- I understand what do do here, but web_root= is never mentioned and I'm not sure if that has to be set.
- "Copy autoProcessComics.py, autoProcessComics.cfg and ComicRN.py into your SABnzbd scripts directory (or wherever your SABnzbd stores it's scripts)."
- If the user has never setup scripts with SABnzbd, they have no idea what you are talking about. Maybe a link SABnzbd would help
- "Make sure SABnzbd is setup to have a 'comic-related' category that points it to the ComicRN.py script that was just moved."
- If the user has never setup scripts with SABnzbd, they have no idea what you are talking about. Maybe a link SABnzbd would help
- For the newb, what does "points it to the" mean? I'm assuming you mean folder path. Any idea if you need a slash at the end for the scripts directory?
- What should Priority and Processing be in the Category settings?

Anyway, love the application. I just haven't been able to get it fully working correctly. I got it to download using the blackhole method, but it's not moving the files over :(

Thanks, Troy
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Request: Better Instructions to setup Post Processing

Post by evilhero »

I totally agree that the documentation has been severely lacking for a very long time while, and I have no one else to blame but myself for the lack of it. Between my personal life commitments (which I won't get into) and fixing bugs within Mylar as well as adding some improvements along the way, the documentation has always been dropping down in priorities for whatever reason.

I do plan on doing something soon in that regards, and yes 'dumbing it down' for novice users is an essential part of any computer documentation. I also plan on having a graphical config.ini option within Mylar in the future, where users can edit the config.ini variables (via text boxes or checkboxes) so they don't have to manually edit anything. As well, an automated installation system is coming in the future, where it will ask for the locations you wish to install and it will do the rest (auto-populating all the required data within Mylar so there's less chance of data errors). These are just a few of things that I have in my mind for the very near future of Mylar (a big one is the almost complete overhaul of the main home page into a more graphical bookshelf-type view).

But to answer you questions about the ReadMe...here's the answers ;)
"Edit (put in your Mylar host, port, login and password (if required), and ssl(0 for no, 1 for yes) and rename the autoProcessComics.cfg.sample to autoProcessComics.cfg."
- I understand what do do here, but web_root= is never mentioned and I'm not sure if that has to be set.
The web_root is for those users who want to change where Mylar mounts its files on the web server. If you're using a reverse proxy you would use the web_root option to remap the root for Mylar. Keep in mind that if you're using this option, you need to make sure that you edit your web server accordingly and then enter in the root in the config.ini. If you're using Mylar for post-processing, you'll have to set the web_root option within the autoProcessComics.cfg to the same value.
- "Copy autoProcessComics.py, autoProcessComics.cfg and ComicRN.py into your SABnzbd scripts directory (or wherever your SABnzbd stores it's scripts)."
- If the user has never setup scripts with SABnzbd, they have no idea what you are talking about. Maybe a link SABnzbd would help
This was just done under the assumption that the user knows SABnzbd, or sickbeard or headphones or the like where they require the scripts directory to be present. If you're on linux, you can always symlink the directory to the location where your SABnzbd scripts directory is located. But yes, I can understand where the lack of knowledge would cause some problems there. Here's what mine looks like for the record:

Image
For the newb, what does "points it to the" mean? I'm assuming you mean folder path. Any idea if you need a slash at the end for the scripts directory?
- What should Priority and Processing be in the Category settings?
The "points it to" means to have the script field within the Category screen in SABnzbd set to ComicRN.py. This is just a drop-down list showing all the scripts available within your SABnzbd script location folder. If the ComicRN.py script isn't listed, then either the .py script isn't copied into the correct location, or you have the wrong location stated for your script location within SABnzbd.

Image

In reality, the ComicRN.py script is hardcoded within Mylar as the script it will run when post-processing is enabled (this will change at a future point, but currently it's hard-coded). So you don't really need to specify a script for the Comics category, but the ComicRN.py script still needs to be within the SABnzbd scripts folder.
What should Priority and Processing be in the Category settings?
This is entirely dependent on what your personal preferences are. Priority can be set within Mylar, but you can also set it based on the category itself - the default is fine for normal operations, however if you are doing a huge backlog of stuff and you're trying to download movies or shows or whatever, then your comics will take precedence in the order they were queued if everything has the same priority (which if you have alot of comics queued, it may take awhile depending on your bandwidth).

Processing is how you want SABnzbd to handle the download after it completes. I have mine set to +delete (as a note SABnzbd downloads to your Temporary Download Directory first, which is stated within the SABnzbd configuration), however with that being said very rarely are comics 'rar'd up so you could really leave the Processing field blank since the +unrar/+unpack/+delete options are geared more towards downloads that encompass more than 1 file. In a typical situation, SABnzbd will download the comic to the Temporary Download Directory, run the Processing option on it (which will be zilch normally), and then will move it to the path specified for 'comics', and then run the post-processing script (ComicRN.py) on that final directory. During the post-processing is where Mylar renames, and moves the file to the proper series directory.

Hope that helps - and it's not too long winded, but I had some time so figured I'd try to be as in depth as possible in a reply ;)

The blackhole method won't be able to post-process unless you set the ComicRN.py script to run on the comics category. This is because when you're using blackhole it drops the nzb into the SABnzbd watch directory, but in doing so it can't assign a category or priority as it's just dropping the file into the directory - it's not sending the request directly to SABnzbd. So if you want the blackhole to work with post-processing you need to make sure you have a comics category setup in the following way :

Image

What this does is set the ComicRN.py script to run on any download that comes from a.b.comics.dcp or a.b.comics.reposts. Now you could do fancier stuff (like use a pre-postprocessing script which assigns a category based on the nzbname itself, but the above should work just fine).

Of course with that being said, you probably have no real need to use the blackhole method unless you were using another client, wanted to retain the .nzb files for some reason, or have Mylar and SABnzbd on different machines. I'm just assuming you were using the blackhole because you couldn't get normal post-processing via sending directly to SABnzbd working...
Radius
Posts: 2
Joined: Fri Jan 17, 2014 10:15 pm

Re: Request: Better Instructions to setup Post Processing

Post by Radius »

How about some post-processing help for the poor people that use torrents? ;)
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Request: Better Instructions to setup Post Processing

Post by evilhero »

Radius wrote:How about some post-processing help for the poor people that use torrents? ;)
Well with torrents, post-processing isn't a simple step because you can't get a script to auto-run as easily after a torrent has completed. It's a bit more involved, but it can be done currently (albeit with some restrictions on some file handling options).

Now I use torrents for the occasional thing, so I'll tell you how have mine setup and maybe it will give you an idea how to setup yours as well (if it's a similar system of course).

I run Ubuntu 12.04LTS, and I use ruTorrent (rTorrent GUI basically).

I have 3 separate directories within a 'files' directory for using with just torrents. A 'Downloading' directory where all my downloads go as they are being downloaded (my default directory for torrents), a 'Completed' directory where all my downloads get moved to upon completion, and a 'Watch' directory for watching for .torrent files.

When Mylar sends the torrent file to my rTorrent, it gets throw into my Watch directory. Which is then read and starts to download, saving the file(s) into my Downloading directory. Once the file(s) have been downloaded, a script runs on my rTorrent that moves the completed files into the Completed folder. This is just an execute on completion script, that's nothing special at all - it just serves to separate currently downloading files from already completed ones so that when the move happens it won't get any file locks or incomplete files. Most clients have some sort of option to 'move completed' torrents after completion so that seeding can continue.

Now I have a script that runs every 10-15 minutes that checks the Completed folder for new cbr/cbz files and copies them (because I want to continue seeding) into my 'Incoming' comics folder.

Here's the catchy part - Mylar can monitor a directory for new files and automatically post-process them accordingly (so long as the series exist and are in your watchlist). It's currently a manual setting within the config.ini - so you have to shutdown Mylar first (very important), then open up the config.ini in an editor and find the following line:

Code: Select all

check_folder = ""
Set the check_folder to the full path for the directory that you want Mylar to monitor.

By default it checks it every 20 minutes, but it also can be changed within the config.ini:

Code: Select all

download_scan_interval = 20
just change the 20 to whatever you want in minutes.

Now if you wanted to save a step, you could literally just set the check_folder to your torrents completed directory (keeping in mind that it will move the file and your seed will stop seeding because the files will have been physically moved). Mylar will only move cbr/cbz files that match on your watchlist and it will ignore all the other files. However, it does have to scan the entire directory so if you're seeing lots of torrents, it could get to be a time-intensive as Mylar continually scans the directory every X minutes for new comics.

Hope that helps abit, I've been working on the post-processing for torrents but because of the various clients and the different methods it's not a simple or easy task especially when you want to continue seeding, so it's a continual process and unfortunately I've been doing other things within Mylar with regards to bugfixes that takes away from that kind of time.
Radius
Posts: 2
Joined: Fri Jan 17, 2014 10:15 pm

Re: Request: Better Instructions to setup Post Processing

Post by Radius »

It sure helps. I use rutorrent with exactly the same folder configuration as you, so I won't have any problems. Thank you!

Just one rather off topic question. Do you know how to make rtorrent prevent the computer going to sleep while there are torrents still active?
Lish
Posts: 7
Joined: Sun Feb 02, 2014 8:18 am

Re: Request: Better Instructions to setup Post Processing

Post by Lish »

Hi,

I'm supposed to have the same config than the both of you....
So I've followed the instruction with the config.ini.
Looks I have a problem.

The monitored folder is : D:\Mes Documents\Mes images\EBooks\Comics a Classer\Scrap Mylar
The structure inside it is : All-New Invaders (2014)\All-New Invaders #01 ( could be anything else if it helps ).

Mylar checks this folder every 20mn, but nothing's done. I have this message in the logs :
Timestamp Level Message
2014-02-02 09:23:20 INFO Cannot determine issue number from given issue #:
Exactly the same with the manual post processing.

Any ideas ?

I can post a log file if it's neeeded.


PS : Thanks for your amazing software !!!
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Request: Better Instructions to setup Post Processing

Post by evilhero »

Well at least I know you're using the latest development build from that error. :)

Update to the latest development commit (as of last night). It should fix the issue you're having with Mylar not being able to recognize issues that have the # denoted before the issue number. If you're still having a problem, paste in the log file, but just showing the file checking portion as that's where the problem would lie in this case.

It's also possible that if the series doesn't have an issue year within the filename(s) it may be the cause of the problem. It's a problem for the manual post-processor ( which is used when you 'monitor a folder' ) as it takes just the filename(s) from within the folder regardless of the folder structure - so if the filename doesn't have an issue year within the filename itself, it can't determine the issue properly.
Lish
Posts: 7
Joined: Sun Feb 02, 2014 8:18 am

Re: Request: Better Instructions to setup Post Processing

Post by Lish »

ok,

Tried to update but no changes
Here a log :
https://dl.dropboxusercontent.com/u/1641313/mylar.log

Thx :)
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Request: Better Instructions to setup Post Processing

Post by evilhero »

It looks like you have 2 threads running the same process simultaneously....not that it would make a difference until you get to the point where it's importing - but having it run the same thing at the same time isn't gonna bode well at some point (that's not what's causing the problems for you right now I believe, but just a heads' up).

I'm curious though, do you have each of these series' within subfolders of the folder you're trying to import ?
As an example, if the folder you entered into Mylar to scan was :

Code: Select all

c:/home/comics/import/ 
and you have the comics in a folder structure like :

Code: Select all

c:/home/comics/import/comicseries1/
c:/home/comics/import/comic_series2/
c:/home/comics/import/some_series/
Is that guess about right ?

I had thought that Mylar drops down directory levels when doing manual post-processing, but perhaps I was mistaken or thought it was in another process.

I'll take a look at the logs and give you a heads-up when I figure out what's going on or if I need more information :)
Lish
Posts: 7
Joined: Sun Feb 02, 2014 8:18 am

Re: Request: Better Instructions to setup Post Processing

Post by Lish »

The folder to scan is :
D:\Mes Documents\Mes images\EBooks\Comics a Classer\Scrap Mylar\

The folder inside it is :
D:\Mes Documents\Mes images\EBooks\Comics a Classer\Scrap Mylar\All-New Invaders (2014)\All-New Invaders #01.cbz
D:\Mes Documents\Mes images\EBooks\Comics a Classer\Scrap Mylar\Hawkeye (2012)\Hawkeye #16

I could easyly change the structure if it's made things easier for you, maybe something like that :
D:\Mes Documents\Mes images\EBooks\Comics a Classer\Scrap Mylar\All-New Invaders 01 (2014).cbz
D:\Mes Documents\Mes images\EBooks\Comics a Classer\Scrap Mylar\Hawkeye 16 (2012).cbz

Thanks
Post Reply