Comics not importing

Post any problems / bugs / issues that are Mylar-related in here.
Post Reply
arqcan
Posts: 4
Joined: Sun Feb 25, 2018 9:58 am

Comics not importing

Post by arqcan »

I have about 900 different comic book series that i have organised with comicrack and i am trying to import into Mylar. Everytime i try to import them in i get an error message and nothing get imported. These are the errors;

2018-02-25 09:57:11 ERROR Uncaught exception: Traceback (most recent call last):
File "c:\Mylar\mylar\logger.py", line 177, in new_run
old_run(*args, **kwargs)
File "c:\Python27\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "c:\Mylar\mylar\webserve.py", line 4101, in preSearchit
logger.fdebug('[IMPORT] [' + str(len(sresults)) + '] search results')
TypeError: object of type 'bool' has no len()

2018-02-25 09:57:11 WARNING Error fetching data from ComicVine: ('Connection aborted.', error(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

I have set my Scan Comic Library path to the location of my comics. When i click on 'Import reults management' It takes a long time to load and then it shows me all the comics it has scanned. When i tick all of the boxes and press 'Mass Import' a red box pops up and says there was an error however the only entry in the log is;

2018-02-25 16:01:25 INFO Initiating mass import.

If i tick everything again and choose 'Start Import' from the drop down menu i get a progress box in the bottom corner however in the log it looks like its working and then fails;

2018-02-25 16:08:07 ERROR Uncaught exception: Traceback (most recent call last):
File "c:\Mylar\mylar\logger.py", line 177, in new_run
old_run(*args, **kwargs)
File "c:\Python27\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "c:\Mylar\mylar\webserve.py", line 3929, in preSearchit
logger.fdebug('[IMPORT] [!none] dynamicname: ' + DynamicName)
TypeError: cannot concatenate 'str' and 'list' objects

2018-02-25 16:08:07 INFO [IMPORT][Punisher: The Platoon (v2017)] Now preparing to import. First I need to determine the highest issue, and possible year(s) of the series.

2018-02-25 16:08:07 DEBUG [IMPORT] I cannot find any results for the given series. I should remove this from the list.

2018-02-25 16:08:07 DEBUG [IMPORT] [!none] volume: v2015

2018-02-25 16:08:07 DEBUG [IMPORT] [!none] dynamicname: ghostracers

2018-02-25 16:08:07 INFO [IMPORT][Ghost Racers (v2015)] Now preparing to import. First I need to determine the highest issue, and possible year(s) of the series.

2018-02-25 16:08:07 DEBUG [IMPORT] I cannot find any results for the given series. I should remove this from the list.

2018-02-25 16:08:07 DEBUG [IMPORT] [!none] volume: v2013

2018-02-25 16:08:07 DEBUG [IMPORT] [!none] dynamicname: painkillerjanepriceoffreedom

2018-02-25 16:08:07 INFO [IMPORT][Painkiller Jane: The Price of Freedom (v2013)] Now preparing to import. First I need to determine the highest issue, and possible year(s) of the series.

Any ideas where i am going wrong?
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Comics not importing

Post by evilhero »

Well I think the first problem is that you're having problems hitting ComicVine for some reason.

The connection attempt failed error that got logged would indicate that either you hit the API limit there (altho it would have logged that as the error usually), or the fact that you don't have a CV API Key ? The subsequent errors (at least the first one you posted, is due to not being able to hit CV and get back a valid response (it should trap that error better, so I'll be fixing that). CV does limit the number of hits/hr to a maximum amount (if you go to the gamespot.comicvine.com/api url and signon, it would tell you what you've hit and what your limits are /hr). So the first thing that needs to happen is figure out why you're not able to get to CV properly - can you get there via a browser on the same machine ?

With Mylar, if you're doing an import you can't have your ComicLocation set to be the same value as where you're importing the comics from. What then happens is that it goes to move the files to the new ComicLocation, and ends up rescanning them again during the process and hits some weird never-ending loop. I do think if you set the 'do not move' option in the import main page, it should just mark the issues as Archived and proceed with the import without moving anything around (if that's your goal).

One other thing to try, is to not import the whole 900 series at one time. The problem with importing that many series is that if you have metadata within each issue and you have metatagging scanning enabled in the import, it will scan EACH issue for metadata - which involves copying the file to the cache dir, unzipping the issue in order to get at the metadata as comicrack has the xml embedded in the zip, reading the data, then cleaning up the cache and doing the same thing for each subsequent issue. I mean, with 900 series, multiply that by the # of issues per series you have and you're talking about some heavy time constraints. That and then having to narrow down any subsequent problems that occur in the entire dataset is like a needle in a haystack.

I would suggest initially at least, trying to import 10 series just to see if that works, any problems occur, etc. This way if you can get thru the 10 with no problems, you should be able to get thru the remainder easily enough provided you can reach ComicVine ;) If you encounter errors with the small dataset, by all means paste in the full debug log showing what's happening (pasting in the errors themselves, while useful doesn't give me a full understanding of what lead up to the error - so I'd be asking for the lines leading up to said error anyways), and I can help out.
arqcan
Posts: 4
Joined: Sun Feb 25, 2018 9:58 am

Re: Comics not importing

Post by arqcan »

Thanks for getting back to me, as for the comicvine issue, it looks like their site is down at the minute so that explains that issue.

I have altered the location and am giving the import another go doing 10 at a time and i'll let you know how i get on.
arqcan
Posts: 4
Joined: Sun Feb 25, 2018 9:58 am

Re: Comics not importing

Post by arqcan »

I tried to do 10 at a time but it added about 3 and stopped so i settled for one at a time for now. I tried to import Action Comics (v2016) and it said that it needed manual intervention, i choose the correct version and then it failed when it added it. Here is the log;

2018-02-25 22:54:13 ERROR Uncaught exception: Traceback (most recent call last):
File "c:\Mylar\mylar\logger.py", line 177, in new_run
old_run(*args, **kwargs)
File "c:\Python27\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "c:\Mylar\mylar\importer.py", line 413, in addComictoDB
moveit.archivefiles(comicid, comlocation, imported)
File "c:\Mylar\mylar\moveit.py", line 78, in archivefiles
imported = ast.literal_eval(imported)
File "c:\Python27\lib\ast.py", line 49, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "c:\Python27\lib\ast.py", line 37, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "", line 1
{'Volume': u'v2016', 'filelisting': [{'import_id': u'82079637', 'issuenumber': u'966', 'comicfilename': u'Action Comics Vol.2016
^
SyntaxError: EOL while scanning string literal
2018-02-25 22:54:13 INFO Mass import - Moving not Enabled. Setting Archived Status for import.
2018-02-25 22:54:13 INFO Updating complete for: Action Comics
2018-02-25 22:54:11 INFO Now adding/updating issues for Action Comics
2018-02-25 22:54:11 DEBUG Finished Annual checking.
2018-02-25 22:54:11 DEBUG [IMPORTER-ANNUAL] - tradeback/collected edition detected - skipping 53508
2018-02-25 22:54:11 DEBUG [IMPORTER-ANNUAL] - there are 2 results.
2018-02-25 22:54:11 DEBUG [RESULT][None] ComicName:Action Comics Annual -- 2012 [Series years: [u'2012']]
2018-02-25 22:54:11 DEBUG [RESULT][None] ComicName:Action Comics Annual -- 1987 [Series years: [u'1987']]
2018-02-25 22:54:11 DEBUG there are 2 search results...
2018-02-25 22:54:08 DEBUG [IMPORTER-ANNUAL] - Annual Year:2016
2018-02-25 22:54:08 DEBUG checking annual db
2018-02-25 22:54:08 DEBUG issuetype: None

Any ideas what may be causing this?
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Comics not importing

Post by evilhero »

Can you paste in the complete line that's saying is in error? It got cut off it looks like, and I need to see the complete thing in order to see where it might be tripping up. It does look like it's due to the use of the Archive option, but without seeing the full line it's difficult to guess.

The line that contains the Vol.2016 - I need to see the complete line intact:
{'Volume': u'v2016', 'filelisting': [{'import_id': u'82079637', 'issuenumber': u'966', 'comicfilename': u'Action Comics Vol.2016
^
SyntaxError: EOL while scanning string literal
arqcan
Posts: 4
Joined: Sun Feb 25, 2018 9:58 am

Re: Comics not importing

Post by arqcan »

I don't have the log for that particular issue as i cleared it and have tried a few more times with different setups. I have had comicrack rename all of my comics, it had the publication year and the volume year in the file name so i tidied that up. I have tried again with 10 at a time and i get the following error;

2018-02-27 06:57:59 ERROR Uncaught exception: Traceback (most recent call last):
File "c:\Mylar\mylar\logger.py", line 177, in new_run
old_run(*args, **kwargs)
File "c:\Python27\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "c:\Mylar\mylar\webserve.py", line 3929, in preSearchit
logger.fdebug('[IMPORT] [!none] dynamicname: ' + DynamicName)
TypeError: cannot concatenate 'str' and 'list' objects
2018-02-27 06:57:59 INFO [IMPORT][Punisher: The Platoon (v2017)] Now preparing to import. First I need to determine the highest issue, and possible year(s) of the series.
2018-02-27 06:57:59 INFO Successfully imported : Painkiller Jane: The Price of Freedom
2018-02-27 06:57:59 INFO [FILE-RESCAN] I have physically found 0 issues, ignored 0 issues, snatched 0 issues, and accounted for 0 in an Archived state [ Total Issue Count: 0 / 1 ]
2018-02-27 06:57:59 INFO [FILE-RESCAN] Total files located: 0
2018-02-27 06:57:59 INFO [FILE-RESCAN] Updated the status of 1 issues for Painkiller Jane: The Price of Freedom (2014) that were not found.
2018-02-27 06:57:58 INFO there are 0 files.
2018-02-27 06:57:58 INFO [FILE-RESCAN] Now checking files for Painkiller Jane: The Price of Freedom (2014) in E:\OneDrive\Comics\newcomics\Painkiller Jane The Price of Freedom (2014)
2018-02-27 06:57:58 INFO [FILE-RESCAN] I have physically found 0 issues, ignored 0 issues, snatched 0 issues, and accounted for 0 in an Archived state [ Total Issue Count: 0 / 1 ]
2018-02-27 06:57:58 INFO [FILE-RESCAN] Total files located: 0
2018-02-27 06:57:58 INFO [FILE-RESCAN] Updated the status of 1 issues for Painkiller Jane: The Price of Freedom (2014) that were not found.
2018-02-27 06:57:58 INFO there are 0 files.
2018-02-27 06:57:58 INFO [FILE-RESCAN] Now checking files for Painkiller Jane: The Price of Freedom (2014) in E:\OneDrive\Comics\newcomics\Painkiller Jane The Price of Freedom (2014)
2018-02-27 06:57:58 INFO Now scanning in files.
2018-02-27 06:57:58 INFO [FILE-RESCAN] I have physically found 0 issues, ignored 0 issues, snatched 0 issues, and accounted for 0 in an Archived state [ Total Issue Count: 0 / 1 ]
2018-02-27 06:57:58 INFO [FILE-RESCAN] Total files located: 0
2018-02-27 06:57:58 INFO [FILE-RESCAN] Updated the status of 1 issues for Painkiller Jane: The Price of Freedom (2014) that were not found.
2018-02-27 06:57:58 INFO there are 2 files.
2018-02-27 06:57:58 INFO [FILE-RESCAN] Now checking files for Painkiller Jane: The Price of Freedom (2014) in E:\OneDrive\Comics\newcomics\Painkiller Jane The Price of Freedom (2014)
2018-02-27 06:57:58 INFO Updating issue information and setting status to Archived for location: \\SERVER\OneDrive\Comics\Painkiller Jane - The Price of Freedom (2014)
2018-02-27 06:57:58 WARNING [malformed string] Error encountered converting import data
2018-02-27 06:57:58 INFO Mass import - Moving not Enabled. Setting Archived Status for import.
2018-02-27 06:57:58 INFO Updating complete for: Painkiller Jane: The Price of Freedom

I hope that's enough info.

Thanks
User avatar
evilhero
Site Admin
Posts: 2883
Joined: Sat Apr 20, 2013 3:43 pm
Contact:

Re: Comics not importing

Post by evilhero »

Unfortunately, because you're getting:
2018-02-27 06:57:58 WARNING [malformed string] Error encountered converting import data
it makes it difficult to figure out what's exactly going on - especially since you're only logging the INFO levels.

If you can enable debug/verbose logs prior to starting the import of a small group, I can see what it is that it's trying to parse and giving back that resulting error. I know it's having some problem converting, but unless I can see more of debug log, it's almost impossible to figure out (especially since I've tried like 6/7 times now, and get no such errors when doing various types of imports)
Post Reply