SABNZBD Temp Download Folder Not Deleting in Post Processing

Post any problems / bugs / issues that are Mylar-related in here.
Post Reply
icegunner
Posts: 3
Joined: Wed Jan 27, 2021 1:58 am

SABNZBD Temp Download Folder Not Deleting in Post Processing

Post by icegunner »

Hi,

I have recently setup Mylar3 in a Docker container along with SABnzbd. I have gotten them to work together without much issue. However, the one annoying problem that I can't seem to get around is that the NZB folder in Sab's completed folder is not being deleted. I can add a comic, it sends it to SAB, gets downloaded, seen by Mylar, post processing begins, comic is moved to final destination, but the temp folder remains. Log shows this:

Code: Select all

2021-01-26 19:31:59	DEBUG	[POST-PROCESSING] Failed to remove directory - Processing will continue, but manual removal is necessary
2021-01-26 19:31:59	DEBUG	[POST-PROCESSING] Tidying up. Deleting original folder location : /downloads/comics/Eternals.001.2021.Digital.Zone-Empire
I can't seem to figure it out. I've checked permissions and the SAB download/completed folder is set at 777, the temp folder is also 777, everything is at 777. So, I'm at my wit's end.

Currently running Version: v0.4.9.3 (master) from hotio/mylar3 Docker image. My Docker is running on Windows 10 and the bound folders are all wide open for Everyone.

Surely, I'm missing something simple here. Help?

Cheers...
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: SABNZBD Temp Download Folder Not Deleting in Post Processing

Post by evilhero »

Well the error being returned is from the OS level - so it's either a permissions or ownership problem of some kind.

Now the error could pop up due to other issues related to that - such as, because you're on Windows it could be that the file is being locked up for some reason (the python module Mylar uses to remove directories can't remove them if they're either in use or read-only from my understanding).

You don't make mention of it, but how is SAB doing the post-processing with Mylar. Is it using the CDH aspect or are you using the ComicRN scripts? CDH wouldn't lock up the file, but it's possible the scripts could if you have additional scripts running after a download.

You've also mentioned the 777 aspect, is the completed folder on a different machine? Are you using ACL at all on the completed folder for SAB?
icegunner
Posts: 3
Joined: Wed Jan 27, 2021 1:58 am

Re: SABNZBD Temp Download Folder Not Deleting in Post Processing

Post by icegunner »

Thanks for the quick response.
evilhero wrote: Wed Jan 27, 2021 2:50 am Well the error being returned is from the OS level - so it's either a permissions or ownership problem of some kind.
I thought the same, but couldn't think if why based on what I was seeing and was hoping someone would see something I wasn't.
evilhero wrote: Wed Jan 27, 2021 2:50 am You don't make mention of it, but how is SAB doing the post-processing with Mylar. Is it using the CDH aspect or are you using the ComicRN scripts? CDH wouldn't lock up the file, but it's possible the scripts could if you have additional scripts running after a download.
I'm using the CDH. Based on what I've read, that was the better option. So, no scripts running. I tried to set this up as simply as possible.
evilhero wrote: Wed Jan 27, 2021 2:50 am You've also mentioned the 777 aspect, is the completed folder on a different machine? Are you using ACL at all on the completed folder for SAB?
In SAB, I'm setting the permissions on the Completed folder to 777. I found that if I didn't do that, Mylar had issues moving and would just abort outright. Since I have both SAB and Mylar running in Docker, they are also running in the same bridged network and in the same bound folder structure (Docker Compose). So, they both have the same bindings and the folders are in the same hierarchy. I actually based a lot of this on TRaSH's guide to try and avoid these kinds of issues. However, I'm not specifically doing hardlinks. In Mylar, I'm just moving.
I wonder if this isn't just some weird Windows issue with Docker? The move actually happens where the CBR/CBZ file gets moved to the final destination. It's just the parent folder that doesn't want to go. SAB wouldn't be holding on to it, right?
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: SABNZBD Temp Download Folder Not Deleting in Post Processing

Post by evilhero »

So within the directory that it's trying to delete, what are the permissions / ownership like therein? Is there just the cbz file or are there additional files in there as well (I'm talking about a individual download location, not the root of the SABnzbd completed download location btw).

So because Mylar is in docker and SABnzbd is in docker, it's considered to be on different machines - this is more from a general standpoint that one specific to your exact install. We usually say tell individuals running dockers to make sure that the ownership/perms are fully open to both users running Mylar and SABnzbd to the SABnzbd completed download folder and to point the Completed Download Location for SABnzbd (within the Mylar program) to the accessible folder location from within the given docker.

The only other thing you can try to do is generate a carepackage which will hold all the relevant data and we can look more into it (there are a few docker experts that hang around). You can generate a carepackage from the configuration page (button is in the upper right corner). It will generate a zip file containing your db, environment, and cleaned logs & ini. We might be able to decipher what's going on from the logs and the attached files.

The error in question is being thrown because the shutil.rmtree (which is a bundled python module) is failing for some reason, none of which seems to be logged unfortunately - so we need to figure out why the function is failing to remove (which could be for any reason really).
icegunner
Posts: 3
Joined: Wed Jan 27, 2021 1:58 am

Re: SABNZBD Temp Download Folder Not Deleting in Post Processing

Post by icegunner »

TL:DR: I think I've got it fixed. But, wanted to answer questions in case others search and need help with similar issues.

evilhero wrote: Wed Jan 27, 2021 3:27 pm So within the directory that it's trying to delete, what are the permissions / ownership like therein? Is there just the cbz file or are there additional files in there as well (I'm talking about a individual download location, not the root of the SABnzbd completed download location btw).
Everything's 777; open to everyone for anything. It was just the CBZ file. Mylar would see it, move it, and when it tried to delete the parent folder, error. So, it should've been pretty simple I would think.
evilhero wrote: Wed Jan 27, 2021 3:27 pm So because Mylar is in docker and SABnzbd is in docker, it's considered to be on different machines - this is more from a general standpoint that one specific to your exact install. We usually say tell individuals running dockers to make sure that the ownership/perms are fully open to both users running Mylar and SABnzbd to the SABnzbd completed download folder and to point the Completed Download Location for SABnzbd (within the Mylar program) to the accessible folder location from within the given docker.
I thought about this too. I even tried to check the box for SAB on a different machine and that actually made it worse. Once I did that, it wouldn't even find the file much less attempt to move it. Errors abound. Perhaps if I was running them as independent containers, I could see having to specify they're on different machines (different networks, file systems, etc.). In a stack, they should see each other easily (and they do for the most part).
evilhero wrote: Wed Jan 27, 2021 3:27 pm The error in question is being thrown because the shutil.rmtree (which is a bundled python module) is failing for some reason, none of which seems to be logged unfortunately - so we need to figure out why the function is failing to remove (which could be for any reason really).
I saw this in some of the code when I was troubleshooting. Perhaps adding some additional logging for such a case? I saw in the code that this particular piece of code didn't have it's own try/catch even though most everything else in there did. The error that I see in the logs seems to be a catch-all at the end of all the code.

-----------------------------------

So... Somehow, I fixed it. And I'm not really sure how.
I mentioned before that I was using the Docker image from Hot.io. I thought that I might try the image from linuxserver instead. So, I updated my Docker Compose file to use that image. Unfortunately, it wiped my config when it updated the container. Probably due to rebuilding of the bindings? I'm not sure. Either way, I had to go through and setup everything again. I tried to exactly replicate what I had before. Once I got it all setup, I tried that comic again. And it worked! :o Moved the comic and deleted the parent folder. I tried a few more comics just to make sure and it's all copacetic now.
So, what was it? I have no idea. Perhaps the new image? The SAB image I was using was also from linuxserver. Maybe that's it? As far as I can tell, everything is the same as it was with the Hot.io image: permissions, settings, folders, etc. Maybe it's just better not to mix images from different repos.

I'm good now. Wanted to at least report on what I went through in case someone else runs into this too. Thanks so much for the speedy replies and trying to help out. It is much appreciated. And a great program.

Cheers...

EDIT: One other thing I wanted to mention that I noticed... When a comic fails to download in SAB, it seems to stick around even though Mylar sees it failed and tries again later. Sonarr/Radarr detect these and generally wipe them from SAB so that SAB's log doesn't get full of failed downloads. Is that something that Mylar should be doing that I'm not seeing? Or is this a feature request?
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: SABNZBD Temp Download Folder Not Deleting in Post Processing

Post by evilhero »

Well if that's all it took - lol. I'm glad you were able to figure it out regardless. Something like that I'm at a loss for usually as I don't run docker at all, so I'm a bit distant from some of the convo and troubleshooting specifics.

As far as the Failed downloads, It would probably be a feature request - albeit not quite sure of a timeline when something like that could/would be implemented. I've generally taken the approach that I don't want to be removing items from other clients or manipulating them too much, in order to avoid excessive problems. I know there are users that would like to see the failed downloads in the queue so they can see at a glance what failed and if any reasoning was behind it. Of course, others like yourself, would like to see the failed downloads removed from the queue for cleaner aesthetics. So it would have to be a configurable option.

We should be able to do it at some point, but like I mentioned earlier, unsure of a timeline for when it would get introduced as we are currently trying to work-thru a massive backlog of PR's and updates that we need to get out before trying to accomplish this safely.
Post Reply