One-Off comics are found, "+Add" are stalled with CV lookup

Information and Tutorials on features in Mylar and how to use it
Post Reply
MirthWiz
Posts: 4
Joined: Mon Apr 26, 2021 8:11 pm

One-Off comics are found, "+Add" are stalled with CV lookup

Post by MirthWiz »

Hi,

And thanks!

I followed the documentation for setting up Mylar and almost have it working. One-Off selections are found and downloaded using DDL (GetComics.info I think) and put in the post-processed folder. Yay!
When "+Add" is selected, the process seems to stall when contacting CV. But the API key is good, and I haven't used it too frequently to hit the limit. Checking CV API site it informs me the key has no restrictions.

I read your note regarding recently added items to the weekly pull list are not yet in CV, so I went back months to find comics that are in CV and also in GetComics. Those will work in one-off but not when +Add (adding the series).

Setup:
  • New installation of Mylar using Mac terminal.
    ComicVine API key and Mylar API key area correct. (ComicVine appears to work in post-processing.)
    SABnzbd setup on same Mac and test passed (Successfully verified API Key).
    Usenet provider - NewsHosting.com (annual subscription)
    Indexer - https://nzbgeek.info (but could only get trial for 3 days, will subscribe to a different indexer (or more) when I get the kinks worked out.)
    No RSS yet.
    NZB.SU - Off for now
    Use Experimental Search On
    Enable DDL On
From the logs:

Code: Select all

26-Apr-2021 13:27:08 - INFO    :: mylar.queueissue.1639 : Thread-16 : [PULL-LIST] Now Queueing The Cimmerian: The Frost-Giant's Daughter (2021) #3 for search
26-Apr-2021 13:27:09 - INFO    :: mylar.search_queue.3180 : SEARCH-QUEUE : [SEARCH-QUEUE] Now loading item from search queue: {'issueid': '827455', 'comicname': "The Cimmerian: The Frost-Giant's Daughter", 'seriesyear': '2021', 'comicid': '132475', 'issuenumber': '3', 'booktype': 'None'}
26-Apr-2021 13:27:09 - INFO    :: mylar.search_init.86 : SEARCH-QUEUE : [SEARCH] Special Characters exist within Series Title. Enabling search-safe Name : The Cimmerian The Frost-Giants Daughter
26-Apr-2021 13:27:09 - INFO    :: mylar.NZB_SEARCH.807 : SEARCH-QUEUE : Shhh be very quiet...I'm looking for The Cimmerian: The Frost-Giant's Daughter issue: 3 (2021) using DDL.
26-Apr-2021 13:27:11 - INFO    :: mylar.search_results.220 : SEARCH-QUEUE : There are 1 results
26-Apr-2021 13:27:14 - INFO    :: mylar.search_results.220 : SEARCH-QUEUE : There are 1 results
26-Apr-2021 13:27:17 - INFO    :: mylar.search_results.220 : SEARCH-QUEUE : There are 1 results
26-Apr-2021 13:27:19 - INFO    :: mylar.search_results.220 : SEARCH-QUEUE : There are 1 results
26-Apr-2021 13:27:21 - INFO    :: mylar.searcher.3555 : SEARCH-QUEUE : Found The Cimmerian: The Frost-Giant's Daughter (2021) #3 using DDL
26-Apr-2021 13:27:21 - INFO    :: mylar.searcher.3563 : SEARCH-QUEUE : nzbid: 183249
26-Apr-2021 13:27:21 - INFO    :: mylar.searcher.3564 : SEARCH-QUEUE : IssueID: 827455
26-Apr-2021 13:27:21 - INFO    :: mylar.searcher.3565 : SEARCH-QUEUE : oneoff: True
26-Apr-2021 13:27:23 - INFO    :: mylar.parse_downloadresults.522 : SEARCH-QUEUE : Only one item discovered, changing queue length to accomodate: {'series': 'The Cimmerian – The Frost-Giant’s Daughter #3', 'site': 'Download Now', 'year': '2021', 'issues': None, 'size': ' 44 MB', 'link': 'https://getcomics.info/run.php-urls/1SmTsQUiszh90ouaj+EkxPG1kYmW7PvXqF4xPHWYfbxM4KwU+EjVtjryQnHCq7tLorQgeNFdeIi7iNh39keqbjqablEKrZmJXYxyp1Y2tYMWnYC9eTV7XtDDCAE6G8T16S1/E5OTg2UNxo4IegTxEohsF0Wik2ip/cBE44n6K4W4veYDLm1pfhgWU23EAJ8ROGhJrW3C6IbuGBXNcxTm2v7MjgZ5J37y61IHI+CEJlg=:RDgQ1aR6uym3goECdeCYVw=='} [<class 'dict'>]
26-Apr-2021 13:27:23 - INFO    :: mylar.searcher.3848 : SEARCH-QUEUE : Successfully snatched The.Cimmerian.-.The.Frost-Giants.Daughter.#3.(2021) from DDL site. It is currently being queued to download in position 1
26-Apr-2021 13:27:23 - INFO    :: mylar.nzblog.733 : SEARCH-QUEUE : setting the nzbid for this download grabbed by DDL in the nzblog to : 183249
26-Apr-2021 13:27:23 - INFO    :: mylar.foundsearch.802 : SEARCH-QUEUE : [UPDATER] Updating status to snatched
26-Apr-2021 13:27:23 - INFO    :: mylar.foundsearch.904 : SEARCH-QUEUE : [UPDATER] Updated the status (Snatched) complete for The Cimmerian: The Frost-Giant's Daughter Issue: 3
26-Apr-2021 13:27:24 - INFO    :: mylar.ddl_downloader.3087 : DDL-QUEUE : Now loading request from DDL queue: The Cimmerian – The Frost-Giant’s Daughter #3
26-Apr-2021 13:27:57 - INFO    :: mylar.ddl_downloader.3116 : DDL-QUEUE : The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr successfully downloaded - now initiating post-processing for /Users/mac/Documents/mylar3/cache/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr
26-Apr-2021 13:28:01 - INFO    :: mylar.postprocess_main.3144 : POST-PROCESS-QUEUE : Now loading from post-processing queue: {'nzb_name': "The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr", 'nzb_folder': "/Users/mac/Documents/mylar3/cache/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr", 'failed': False, 'issueid': '827455', 'comicid': '132475', 'apicall': True, 'ddl': True}
26-Apr-2021 13:28:01 - INFO    :: mylar.Process.1368 : Thread-17 : watchmatch: {'process_status': 'match', 'sub': None, 'volume': None, 'match_type': None, 'comicfilename': "The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr", 'comiclocation': "/Users/mac/Documents/mylar3/cache/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr", 'series_name': "The Cimmerian - The Frost-Giant's Daughter", 'series_volume': None, 'alt_series': 'The Cimmerian', 'alt_issue': 'The Frost-Giantg11s Daughter', 'issue_year': '2020', 'issueid': None, 'justthedigits': '003', 'annual_comicid': None, 'scangroup': 'NeverAngel-Empire', 'booktype': 'issue'}
26-Apr-2021 13:28:01 - INFO    :: mylar.Process.1695 : Thread-17 : [PPINFO-POST-PROCESSING-ATTEMPT] {'comicid': '132475', 'comicname': "The Cimmerian: The Frost-Giant's Daughter", 'issuenumber': '3', 'publisher': 'Ablaze', 'issueid': '827455', 'comiclocation': "/Users/mac/Documents/mylar3/cache/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbr", 'sarc': None, 'oneoff': True}
26-Apr-2021 13:28:01 - INFO    :: mylar.nzb_or_oneoff_pp.1798 : Thread-17 : [POST-PROCESSING] Could not detect as a standard issue - checking against annuals.
26-Apr-2021 13:28:01 - INFO    :: mylar.nzb_or_oneoff_pp.1801 : Thread-17 : [POST-PROCESSING] issuenzb not found.
26-Apr-2021 13:28:01 - INFO    :: mylar.nzb_or_oneoff_pp.1819 : Thread-17 : [POST-PROCESSING] [ONE-OFF POST-PROCESSING] One-off download detected. Post-processing as a non-watchlist item.
26-Apr-2021 13:28:01 - INFO    :: mylar.nzb_or_oneoff_pp.1850 : Thread-17 : [POST-PROCESSING] One-off mode enabled for Post-Processing. Will move into Grab-bag directory: /Volumes/Ubooq/mylardownloads/Grabbag
26-Apr-2021 13:28:03 - INFO    :: mylar.run.123 : Thread-17 : ct_check: b'ComicTagger 1.3.4 [ninjas.walk.alone / SHURIKEN]\nCopyright (c) 2012-2021 ComicTagger Team\nDistributed under Apache License 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n'
26-Apr-2021 13:28:05 - INFO    :: mylar.run.217 : Thread-17 : b"Archive exported successfully to: The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz (Original deleted) \n"
26-Apr-2021 13:28:05 - INFO    :: mylar.run.218 : Thread-17 : None
26-Apr-2021 13:28:05 - INFO    :: mylar.run.197 : Thread-17 : [META-TAGGER] ComicRack tagging meta-tagging processing started.
26-Apr-2021 13:28:06 - INFO    :: mylar.run.217 : Thread-17 : b"URL can't contain control characters. '/api/issue/4000-827455/?api_key=f263ff6c3822e46cbca57fb9fed8c2a03b9cf16f None&format=json' (found at least ' ')\n\nNetwork error while getting issue details.  Save aborted\n\nNetwork Data Fetch Failures:\n------------------\n/Users/mac/Documents/mylar3/cache/mylar_1mwqtzeu/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz\n"
26-Apr-2021 13:28:06 - INFO    :: mylar.run.218 : Thread-17 : None
26-Apr-2021 13:28:06 - INFO    :: mylar.run.281 : Thread-17 : [META-TAGGER][COMIC-TAGGER] Successfully wrote ComicRack tagging [/Users/mac/Documents/mylar3/cache/mylar_1mwqtzeu/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz]
26-Apr-2021 13:28:06 - INFO    :: mylar.nzb_or_oneoff_pp.1936 : Thread-17 : [POST-PROCESSING] Sucessfully wrote metadata to .cbz (The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz) - Continuing..
26-Apr-2021 13:28:06 - INFO    :: mylar.nzb_or_oneoff_pp.1947 : Thread-17 : [POST-PROCESSING] Source Path : /Users/mac/Documents/mylar3/cache/mylar_1mwqtzeu/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz
26-Apr-2021 13:28:06 - INFO    :: mylar.validateAndCreateDirectory.1723 : Thread-17 : [POST-PROCESSING][DIRECTORY-CHECK] Found comic directory: /Volumes/Ubooq/mylardownloads/Grabbag
26-Apr-2021 13:28:06 - INFO    :: mylar.nzb_or_oneoff_pp.1983 : Thread-17 : [POST-PROCESSING] Destination Path : /Volumes/Ubooq/mylardownloads/Grabbag/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz
26-Apr-2021 13:28:06 - INFO    :: mylar.nzb_or_oneoff_pp.1984 : Thread-17 : [POST-PROCESSING][move] The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz into directory : /Volumes/Ubooq/mylardownloads/Grabbag/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz
26-Apr-2021 13:28:06 - INFO    :: mylar.postprocess_main.3163 : POST-PROCESS-QUEUE : Another item is post-processing still...
26-Apr-2021 13:28:10 - INFO    :: mylar.nzb_or_oneoff_pp.2020 : Thread-17 : [POST-PROCESSING] IssueID is : 827455
26-Apr-2021 13:28:10 - INFO    :: mylar.nzb_or_oneoff_pp.2023 : Thread-17 : [POST-PROCESSING] Writing to db: {'Status': 'Downloaded'} -- {'IssueID': '827455'}
26-Apr-2021 13:28:10 - INFO    :: mylar.nzb_or_oneoff_pp.2025 : Thread-17 : [POST-PROCESSING] Updated status to Downloaded
26-Apr-2021 13:28:10 - INFO    :: mylar.nzb_or_oneoff_pp.2027 : Thread-17 : [POST-PROCESSING] Updated history for one-off's for tracking purposes
26-Apr-2021 13:28:10 - INFO    :: mylar.nzb_or_oneoff_pp.2028 : Thread-17 : [POST-PROCESSING] Post-Processing completed for: [ The Cimmerian: The Frost-Giant's Daughter #3 ] /Volumes/Ubooq/mylardownloads/Grabbag/The Cimmerian - The Frost-Giant's Daughter 003 (2020) (digital) (NeverAngel-Empire).cbz
26-Apr-2021 13:28:35 - INFO    :: mylar.addbyid.567 : CP Server Thread-8 : Attempting to add directly by ComicVineID: 132475
26-Apr-2021 13:28:38 - WARNING :: mylar.GetComicInfo.408 : Thread-18 : Unable to retrieve URL for volume. This is usually due to a timeout to CV, or going over the API. Retrying again in 10s.
26-Apr-2021 13:28:48 - WARNING :: mylar.GetComicInfo.408 : Thread-18 : Unable to retrieve URL for volume. This is usually due to a timeout to CV, or going over the API. Retrying again in 10s.
If you have any ideas I will try them out. I have created a care package and will find a way to attach.
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: One-Off comics are found, "+Add" are stalled with CV lookup

Post by evilhero »

Think you might be abit confused as to what the various options do.

+Add (from the weekly pullist) --> will add the series to your watchilst. For a series to be monitored for newly released issues, and to have Mylar do it's automation thing - the series has to be added to the watchlist so it knows what it's looking for. It will not auto-search for items unless it knows the issue is present (which is what we need CV for, to verify that the information is what it says it is). You will notice that once you click on +Add, after a few seconds, the colour of the row will change to Red and the Status will have changed from Skipped to Wanted. You can them click on the series title and it will bring you into the series detail page for the given series - since it has now been added to your watchlist (and will appear on the main home page).

+One-Off (from the weekly pullist) --> will automatically attempt to search for the given issue. It will not keep track of any issues in the series or anything related to said series. It will only track that one particular issue. This has more limited options for searching as it's only searching for the exact series naming - if things deviate from that, the one-off aspect probably won't match up as well (it can't verify anything other than the store date). This is handy if it's just a one-and-done kinda deal with the given issue, or you're just wanting to see what the issue is like before adding the series to your watchlist.

What's been happening as of late, is that CV doesn't update their end until sometimes Sunday which means that Mylar can't auto-download issues until then - it knows the issue should exist, but it can't verify anything so it waits until it can be verified. When it knows a series should have a new issue, but it can't verify it marks it as Wanted, but it holds it in an upcoming queue of sorts ( Wanted tab / Upcoming section at the bottom ). Mylar can only search for items that are in your watchlist when the issue data is present on CV and once present it will move it from the Upcoming section into the Wanted section ( the section above the Upcoming on the same page ). But until that information is present, it stays in the Upcoming section awaiting CV. Mylar updates the pull every 4 hrs automatically - so when CV updates, Mylar knows and will auto-update your watchlist accordingly and change the status of any issues as required.

That's the blurb - hope it helps!
MirthWiz
Posts: 4
Joined: Mon Apr 26, 2021 8:11 pm

Re: One-Off comics are found, "+Add" are stalled with CV lookup

Post by MirthWiz »

Thanks for the reply and details. I read similar info in other threads where you're teaching us how this works and think I understand the basic ideas. Sorry I wasn't more precise in describing the problem I seem to have with my setup. Knowing new comics may not be populated in CV for a number of days I looked back at previous weekly pull lists, back a few months. I tried a comic that is in CV, and I already have and was able to call CV (with my API key) and get the details using a different script outside of Mylar. So the comic is in CV and is found in Mylar when I use the One-Off option. But it errors when trying to use the +Add option.

No searches work for me, except DDL Download using One-Off.
Even typing in "Hulk" or "Thor" in the main "Search" field has no results.

Code: Select all

27-Apr-2021 12:06:53 - DEBUG   :: mylar.findComic.147 : CP Server Thread-10 : there are 0 search results...
27-Apr-2021 12:07:22 - DEBUG   :: mylar.findComic.147 : CP Server Thread-4 : there are 0 search results...
I think I don't have the indexer setup correctly. Will try and work on that today.

Here's a care package, just in case you have time or think it's interesting.
https://megafile.cc/d/BppY/carepackage

Thanks again Hero!
MirthWiz
Posts: 4
Joined: Mon Apr 26, 2021 8:11 pm

Re: One-Off comics are found, "+Add" are stalled with CV lookup

Post by MirthWiz »

I setup purchased an NZB.su account and configured Mylar with username and API key. The test was successful. (Love the test server options BTW, well done)

Turned off all other search options so only NZB.su is enabled.
Restarted Mylar.

Swiped back in the weekly pull lists to "Weekly Pull list for week 51 :
December 20, 2020 - December 26, 2020"

Tried "Luna" with One-Off.
One-Off successfully found a comic and is currently trying to download with sabnzbd.

Code: Select all

27-Apr-2021 12:30:26 - INFO    :: mylar.nzblog.737 : SEARCH-QUEUE : setting the alternate nzbname for this download grabbed by nzb.su in the nzblog to : Luna0012021digitalSonofUltron-Empire
27-Apr-2021 12:30:26 - INFO    :: mylar.foundsearch.802 : SEARCH-QUEUE : [UPDATER] Updating status to snatched
27-Apr-2021 12:30:26 - INFO    :: mylar.foundsearch.904 : SEARCH-QUEUE : [UPDATER] Updated the status (Snatched) complete for Luna Issue: 1

Tried "Batman:White Knight Presents Harley Quinn (2020)" with the "Add" option.
Same error as before.

Code: Select all

27-Apr-2021 12:38:07 - INFO    :: mylar.addbyid.567 : CP Server Thread-8 : Attempting to add directly by ComicVineID: 131310
27-Apr-2021 12:38:09 - WARNING :: mylar.GetComicInfo.408 : Thread-13 : Unable to retrieve URL for volume. This is usually due to a timeout to CV, or going over the API. Retrying again in 10s.
27-Apr-2021 12:38:19 - WARNING :: mylar.GetComicInfo.408 : Thread-13 : Unable to retrieve URL for volume. This is usually due to a timeout to CV, or going over the API. Retrying again in 10s.
27-Apr-2021 12:38:29 - WARNING :: mylar.GetComicInfo.408 : Thread-13 : Unable to retrieve URL for volume. This is usually due to a timeout to CV, or going over the API. Retrying again in 10s.
27-Apr-2021 12:38:39 - WARNING :: mylar.GetComicInfo.408 : Thread-13 : Unable to retrieve URL for volume. This is usually due to a timeout to CV, or going over the API. Retrying again in 10s.
27-Apr-2021 12:38:49 - ERROR   :: mylar.GetComicInfo.265 : Thread-13 : Unable to add / refresh the series due to inablity to retrieve data from ComicVine. You might want to try abit later and/or make sure ComicVine is up.
27-Apr-2021 12:38:49 - ERROR   :: mylar.excepthook.315 : Thread-13 : Uncaught exception: Traceback (most recent call last):
  File "/Users/mac/Documents/mylar3/mylar/cv.py", line 405, in GetComicInfo
    comic['ComicURL'] = dom.getElementsByTagName('site_detail_url')[trackcnt].firstChild.wholeText
IndexError: list index out of range
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: One-Off comics are found, "+Add" are stalled with CV lookup

Post by evilhero »

Those errors messages pretty much say that you can't reach ComicVine using the Api Key that you supplied for Mylar. So it's either one of two things:

- you mistyped the apikey into Mylar (you have a space at the end, beginning, control character hidden possibly even). If you go to the config.ini you should see the comicvine apikey provided not within double quotation marks. If it is within double quotation marks, then you have one of the above in there and you need to clear it out (with Mylar not running of course).

- went over the hourly CV API limit. CV imposes strict limits on Api Key access - if you go over so many / hr (depending on what your endpoint is), or if you request more than once every second (or odd second, can't recall), they'll temporarily ban you. Based on the fact that you mention you were using an external script to test things, using the same api key I'm guessing as well, you possibly hit their anti-flooding rule, or went over the limit depending on how many times you were testing things out. You can go to https://comicvine.gamespot.com/api/ and enter in your credentials and it will tell you your rough api hits.

Every error in those logs are all CV related. If you can fix the CV problem, you won't have those problems any longer.

And again searching for a series as a One-Off doesn't call CV at all so you wouldn't get an error related to that (you do encounter them tho when metatagging based on your logs, which calls CV). Using the +Add adds the series to your watchlist, which calls CV multiple times in order to get all the information for the series and the relevant issues.
MirthWiz
Posts: 4
Joined: Mon Apr 26, 2021 8:11 pm

Re: One-Off comics are found, "+Add" are stalled with CV lookup

Post by MirthWiz »

Hero,
That was precisely what it was. It had my key, followed by the word "none". So when I copy and pasted my key, I thought the text field was selected so a replace would happen, but alas. Thanks for your help with my avoidable error. Hopefully this post will help someone else if they have similar issues. The clues were there, took the master to shed light on them.
Cheers,
Mirth
Post Reply