Incomplete import

[jan_axhell]

Hi, I'm totally new to Mylar. I have installed latest Mylar3 on a Raspberry Pi 4 from Portainer and configured supposedly well. I have a 100,000 comics collection, all tagged with ComicTagger using ComicVine API key, which I have also added to Mylar.
My collection is on a mapped external HDD organized as Publisher\Series(Year)\[Issues].cbz (same config I set in Mylar).
I have scanned my collection and after a few hours of rolling ring Mylar showed the Manage Import Results button (or something like that).
I have pressed that button and again after hours of ring rolling, it showed me a huge page with a list of all my comics.
As I've read in other posts, I have pressed the Mass Import button. After many more hours it returned the same list split in 463 pages with some Comics flagged as Imported, but only a tiny fraction. I have repeated the extremely slow process and it *seems* that the amount of imported comics has raised, but I'm not sure of that, I have no way to tell for sure.

If my comics are all tagged with the same ComicVine info that Mylar looks up, why doesn't it import all my comics at once?
Also why does it import them in an apparently random order? I see no alphanumeric criteria in the comics it decided to import.

[evilhero]

The importer is broken whke it comes to mass imports, and in that sense I mean that if you have TPB's, GNs or files with no issue numbers it won't be able to import any of those items and it will crap out the importer. If you have any of those types, you need to temporarily remove them from the location where you're scanning. If you are using mylar with the intention of using it with manga, I'd probably try to find another option as manga isn't well supported on both CV and in mylar itself.

Importing 100,000 comics is a huge task of time - it has to open and analyze each and every file, so that will take time. If the metadata doesn't line up with CV, or the metadata is different btwn issues of the same series, it will not be able to match up to anything. Let alone thay you'll probably hit the CV hourly API limit well before you get anywhere near 25% of your collection as it has to reverse lookup the id's as the comicids aren't stored in the XML files.

Also if your files aren't named appropriately, and don't have metadata accordingly (ie. 1.cbz) it's not gonna work well in most cases.

[jan_axhell]

Hi,
the collection I'm scanning is exclusively USA Comics, I knew it might mess up if I added other countries, so for a start I limited to that.
I have TPBs and GNs along with the rest, and it's totally impractical to temporarily exclude those, they're nested in hundreds of Publisher/Series branches. I wouldn't know hot to do that, nor wish to. It's been a pain to organize my collection.
My comics are named as "Series Title v1 123.cbz". Some dupes has ALT1/2/3 tag after issue number, but I don't care if those are not imported.
Why would metadata be different between issues and/or not line up with CV? I have tagged them using CV on ComicTagger..

I didn't know comicsids aren't stored inside XML, I wonder what's the point not to do that.

[evilhero]

If you enabled the "are series in folders option" , and you have more than one volume in the same directory, it's not going to work. By volume, I mean if you have TPB's mixed in with normal issues, but they're all part of the same series. If CV has them as different entities, that's how they need to be directory wise if you enable that option. Not enabling that option does the opposite, but then you lose the grouping aspect of like series in the same directory.

if you have weird numbering that doesn't conform to CV numbering (IE. ALT), how is mylar supposed to understand that when it has no basis to go from? Any type of info like alternate covers or the like belongs in the metadata, not in the filename, especially since you are using your own naming scheme.

Again, it is what it is. If you wish to use the importer, you should try to move out those TPB's, GNs etc, if not - then import based on series folder and see if that works Bette (use a smaller sample set).

The importer will get fixed eventually, but atm there are far bigger fish that we're trying to address. Importing your collection initially is the worst part of the process, and it sounds like you would have to do some organizing anyways if you're using Mylar and you have TPB's mixed in the same directories as normal issues (meaning TPB volumes and normal issues preside in the slexact same location). Mylar can search /track / monitor TPB's etc once they're added, it's only the import aspect that has yet to be updated due to some workflow issues.

I didn't design the comicinfo.xml spec, you'd have to talk to someone that did as to why comicids aren't included, only issueids are included and those are in the comment field of the XML (not even it's own field)

[jan_axhell]

Ok,
my TPBs and GNs are in their own folders, not in same series folders, also because many of them have no "parent" series to be added to.

Yes, I understand "ALT" issues won't be imported, I wrote I don't care about those, I just notified the only "anomaly" in my naming scheme.
I will search the info about correct Mylar naming scheme and see if I can fix that.

I know you didn't design comicinfo.xml, I was just expressing my puzzlement about it.

I will try to scan again a small Publisher and see what happens. Can I do that in parallel to what I've already scanned, like add a new folder to scan, or do I have to "factory reset"?

[evilhero]

To keep things clean, I'd do a Flush All Imports from the Import Results Management page. That will wipe out all the relevant information from the db pertaining to your previous imports and let you start from new. That way you'll be able to see/track things abit easier instead of having to go through possibly thousands of items.