downloaded nzb isn't correct format

Post any problems / bugs / issues that are Mylar-related in here.
Post Reply
ILikeVoltron
Posts: 2
Joined: Sun Sep 29, 2013 8:50 pm

downloaded nzb isn't correct format

Post by ILikeVoltron »

I've used what mylar finds in the logs for this entry:
03-Mar-2014 01:22:50 - DEBUG :: CP Server Thread-9 : search-url: https://INSERT_SITE_HERE/api?t=search&q=black%20science%20004%20&o=xml&cat=7030&apikey=xxxxxxxxxxxxxxxxxxx&maxage=1500
The XML that page returns is as follows:

Code: Select all

<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:newznab="http://www.newznab.com/DTD/2010/feeds/attributes/">
	<channel>
		<atom:link href="https://INSERT_SITE_HERE/api?t=search&q=black%20science%20004%20&o=xml&cat=7030&apikey=xxxxxxxxxxxxxxxxxxxxxxx&maxage=1500" rel="self" type="application/rss+xml"/>
		<title>INSERT_SITE_HERE</title>
		<description>INSERT_SITE_HERE Feed</description>
		<link>https://INSERT_SITE_HERE/</link>
		<language>en-gb</language>
		<webMaster>admin@INSERT_SITE_HERE (INSERT_SITE_HERE)</webMaster>
		<category/>
		<image>
			<url>https://INSERT_SITE_HERE/</url>
			<title>INSERT_SITE_HERE</title>
			<link>https://INSERT_SITE_HERE/</link>
			<description>Visit INSERT_SITE_HERE</description>
		</image>
		<newznab:response offset="0" total="1"/>
		<item>
			<title><REMOVED></title>
			<guid isPermaLink="true">https://INSERT_SITE_HERE/details/172d55961612c5559580a246f2261267</guid>
			<link>https://INSERT_SITE_HERE/getnzb/172d55961612c5559580a246f2261267.nzb&i=3400&r=xxxxxxxxxxxxxxxxxxxxxxx</link>
			<comments>https://INSERT_SITE_HERE/details/172d55961612c5559580a246f2261267#nzb-info-comments</comments>
			<pubDate>Thu, 27 Feb 2014 04:53:37 +0100</pubDate>
			<category>Books > Comics</category>
			<description><![CDATA[<div><ul><li>ID: 172d55961612c5559580a246f2261267</li><li>Name: <a href="https://INSERT_SITE_HERE/details/172d55961612c5559580a246f2261267">Black Science 004 (2014) (digital) (Son of Ultron-Empire)</a></li><li>Size: 75.79 MB </li><li>Attributes: Category - <a href="https://INSERT_SITE_HERE/browse?t=7030">Books > Comics</a></li><li>Groups: <a href="https://INSERT_SITE_HERE/browse?g=alt.binaries.comics.dcp">alt.binaries.comics.dcp</a></li><li>Poster: [email protected] (Bluerobe)</li><li>PostDate: Thu, 27 Feb 2014 00:49:57 +0100</li><li>Password: None</li></ul></div><div style="clear:both;">
			]]></description>
			<enclosure url="https://INSERT_SITE_HERE/getnzb/172d55961612c5559580a246f2261267.nzb&i=3400&r=xxxxxxxxxxxxxxxxxxxxxxx" length="79469379" type="application/x-nzb"/>
			<newznab:attr name="category" value="7000"/>
			<newznab:attr name="category" value="7030"/>
			<newznab:attr name="size" value="79469379"/>
			<newznab:attr name="guid" value="172d55961612c5559580a246f2261267"/>
		</item>
	</channel>
</rss>

Using the <link> attribute, I've downloaded it myself, and it works fine, but when Mylar sends it to nzbget, it's not the correct link. I think maybe the "&amp" or something is getting messed up in the url? Actually, the 'nzb' that gets sent is just html, and it's a redirect to the login page, which means that the params are not being passed in properly to the download url.

This is what it looks like:
03-Mar-2014 01:23:52 - DEBUG :: CP Server Thread-9 : link to retrieve via api:https://INSERT_SITE_HERE/getnzb/172d55961612c5559580a246f2261267.nzb%26i=3400%26r=xxxxxxxxxxxxxxxxx
So what it looks like is the & is getting translated into "%26" twice, so the URL is never parsed properly and send down the calling applications properly. Let me know if you need anything else on this, I suppose I can look into the code myself.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: downloaded nzb isn't correct format

Post by evilhero »

TBH, I don't use NZBget so I would have a hard time troubleshooting this one, but I know the URL parsing was an issue a few commits ago when I noticed that for any newznab entry it was tacking on an extra APIKEY parameter that was nullifying alot of results, but that was with regards to searching as far as I know.

Are you on the development or master build of Mylar ?

The '%26' does have to be encoded twice, once for the i, and another for the r within the url. I've always thought that because it's a query string url that the '%' has to be url-encoded to the '%26', whereas if it was using the url-request method and thereby the params would require the url-syntax of '&'.

I don't use NZBget (I use SABnzbd), and I have several sites that use the format you've indicated and haven't come across any URL problems sending to SABnzbd - which makes me tend to think it may be some issue with NZBGet not handling the URL for whatever reason, or possibly you're on the Master build of Mylar. That and this hasn't come to

I can look into changing the URL syntax to use the request methodology, but I don't have much time to dedicate to this and I'd have no way to test it out before releasing it.

This was also discussed here : https://github.com/evilhero/mylar/issues/573 but was never responded to - I tested the same site and same nzb against SAB and it worked, but NZBGet wouldn't download it for the exact reason you've eluded to.
ILikeVoltron
Posts: 2
Joined: Sun Sep 29, 2013 8:50 pm

Re: downloaded nzb isn't correct format

Post by ILikeVoltron »

I get the same error sending it to blackhole directory, it's still a malformed nzb file. And it's only when connecting to newznab server, when the experimental works it doesn't exhibit the same issue.

This resolved my issue:

Code: Select all

diff --git a/mylar/search.py b/mylar/search.py
index b7c992b..ca4b575 100755
--- a/mylar/search.py
+++ b/mylar/search.py
@@ -1238,7 +1238,8 @@ def NZB_SEARCH(ComicName, IssueNumber, ComicYear, SeriesYear, Publisher, IssueDa
                         #following is JUST for nzb.su
                             if nzbprov == 'nzb.su' or nzbprov == 'newznab':
                                 linkit = os.path.splitext(entry['link'])[1]
-                                linkit = linkit.replace("&", "%26")
+                               logger.fdebug(linkit)^M
+                                #linkit = linkit.replace("&", "%26")^M
                                 linkapi = str(linkstart) + str(linkit)
                             else:
                                 # this should work for every other provider
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: downloaded nzb isn't correct format

Post by evilhero »

ILikeVoltron wrote:
This resolved my issue:

Code: Select all

-                                linkit = linkit.replace("&", "%26")
+                               logger.fdebug(linkit)^M
+                                #linkit = linkit.replace("&", "%26")^M
                                 linkapi = str(linkstart) + str(linkit)

Ok, well then there's a differing issue between SABnzbd and NZBget. You're just basically taking the linkit variable as is and pumping it through without url-encoding anything. This breaks it when using it with SABnzbd with any newznab-based provider. But if it works with NZBget, then I need to add in a simple if-then on how to handle it if it's going to SABnzbd or NZBget.

It's either the SAB/NZBget differences on how it's handling the '&' vs the '%26' or how your nzb provider is handling it. Do you have another newznab-based provider you could try with just to rule out where it's having the problem ?

But if you're saying you blackhole'd it, and it's still giving a malformed nzb, then it would point to it being a provider thing for some reason. Can you say what nzb provider you're using ? (if you don't want to post it, just pm me it). It might require it's own entry within Mylar.
Post Reply