[DONE] Database Dump Request

I cannot do a database dump of it, Total CRC-32 is not stored within the database, its generated on the fly each time you review a disc page.

hmm. this presents a problem. We're working on trying to build a scraper for this but had really hoped grabbing the CRC-32 would have been far easier for the verification process.

Its the first one.

@AMOK.

Might be worth having a look at the code that combines the ~CRC32 values if iR0bo0t is willing to share or writing something from scratch, pretty sure it cant be that hard and maybe looking to script something (Python/PHP?) to generating the combined CRC32 from the DAT files themselves.  at least that way you wont need to scrape the pages in cases of corrections / updates.   

Slightly off topic:

from the sounds of it you are probably heavily invested in the the Phoebe/Rhea ODE but maybe worth a look at the upcoming Satiator from professor abraisive,

I am currently awaiting the beta board for testing.

one of the main advantages being his cue parser is specifically designed for split bin/cue images along with leaving the actual optical drive untouched obviously.  plus there in nothing in Saturn Subs that isnt covered by the redump .cue

I'm well aware of JHL's project and am one of the biggest critics of it as it doesn't allow for use of every title on the system (anything MPEG card optional/required can't be used). While there aren't a lot of detractors out there, my understanding is that he's not going to be implementing an mpeg card into the unit and therefore the unit misses the point to me. In my mind, any ODE product should enhance the capabilities and user experience of the original product without reducing compatibility or functionality.

While the rhea/phoebe does remove the ability to play optical discs, i have yet to run into a single disc image it can't handle that hasn't been fixed in some way (i believe there were a couple of discs early on that couldn't play but to my knowledge those have since been fixed via firmware). This seems a fair trade to me. Though i'd much prefer an ability to switch easily between using the ODE and using the original drive, i'm not willing to sacrifice titles out of the library for that feature.

In any case, I'm still much more of a fan of using CCD over BIN/CUE.

We're going to continue work on the scraper for now, but yes, having an alternative method of getting the total CRC-32 without needing to build the scraper would probably be ideal.

Also, i'm in agreement that the need i have doesn't necessarily justify an additional table, but i'm not exactly sure how one could convert the existing bin/cue sets over to CCD in batch and verify the images retained integrity without doing what i'm doing. it would be a different story if a CCD based set already existed, or if the raw files generated by DIC were what were distributed instead of bin/cue, but for end users that would likely be a nightmare.

So yes, i'm open to any and all suggestions to make this process more self contained. No matter what, some online update is required (either a new dat file must be downloaded and compared, or the site needs to be date checked and re-scraped each time a new disc is added).

Sure!

While we're here, @iRobot, i didn't see anything back from you about the calculation. I'm stuck in limbo for at least one project until we make progress with Total CRC32.

If we skip that one value and find a way to add it in later, i'd still like the rest of the data dump.

I apologize, i thought my question was fairly straightforward.

I don't know how you're calculating the total CRC-32. my attempts at using various calculators to take 2 or more CRC values out of the dat/webpage result and sum them up doesn't result in the value you have as total CRC.

I'm not interested in reinventing the wheel here. i need a solution that allows me to check the site pretty much any time and see if there are changes or updates to any disc (not just saturn mind you) and if there is a change against what was last checked/calculated, obtain the total CRC again and verify it against the dumps i currently have for that title.

I'd MUCH prefer some semi static listing of Total CRC, either in the database or in the dat, but upon opening the dat, i saw the value wasn't listed there either. I don't want to factor in your on the fly calculation into my programs/scripts as doing that will add an extra potential point of failure.  If you won't incorporate Total CRC as a static stat into either the dat or the webpages, i've got not choice but to do that on my own, but i still would rather be handed the code/algorithm than try to "figure it out".

And yes, the data base dump would also be super awesome since a lot of the requested data isn't in the dat file anyway.

Thank you!