Page 1 of 1

Duplicate Prevention

Posted: Mon Sep 09, 2019 4:45 pm
by kevintothecore
Instead of dumping duplicates into a Duplicates folder, is there an option to simply NOT download duplicates? I mean, if a duplicate is detected, is there a way to then skip downloading it? Or just trash it all together, if the duplicate detection takes place after download...

Re: Duplicate Prevention

Posted: Mon Sep 23, 2019 3:59 pm
by evilhero
kevintothecore wrote: Mon Sep 09, 2019 4:45 pm Instead of dumping duplicates into a Duplicates folder, is there an option to simply NOT download duplicates? I mean, if a duplicate is detected, is there a way to then skip downloading it? Or just trash it all together, if the duplicate detection takes place after download...
The duplicate detection takes place after the download, as it's done during the post-process phase of the workflow (search, download, post-process, cleanup). In order for it to not download the duplicate it would have to be done at the search phase, which I mean it could theoretically do it - it just would add some additional complexity to the workflow model due to all the different methods that are available for searching.

I do like the idea however, of just not downloading it due to it hitting a duplication match - however there are a lot of hurdles to overcome with this method, mainly due to how the search operates. During the post-process phase, it's definitely easier to just wipe the download if it's a duplicate (I thought that's what actually happens when you turn off the duplicates folder option - but it's been ages since I last looked at that portion of code)

Re: Duplicate Prevention

Posted: Mon Sep 23, 2019 4:11 pm
by kevintothecore
Oooh, excellent. Toggling the Duplicates folder option would be the best solution then. Thanks