You are not logged in. Please login or register.
Redump Forum → General discussion → Inclusion of SBI files in the dat?
I was just wondering - is there any plans to include the SBI subchannel files in the dat for each game/disc?
Many thanks.
Nope, because you won't be able to read them from the CD again with the same checksum, so the checksums for these files are useless.
Nope, because you won't be able to read them from the CD again with the same checksum, so the checksums for these files are useless.
What do you mean? 
Have you found a way to dump subchannels correctly? 
He's talking about .sbi.. they can be downloaded from the site, so I think he means why don't we add those to the .dat if they are needed anyway?
Cues can also be downloaded from the site, but you can get them from your own CDs as well. You can't get the same sbi files from the CDs, though, so it's totally useless to include them into the dats - you can download them from the site, you can download them elsewere, you can dump them from your own CDs, they will be different every time, but they will work (most likely).
I don't see any logic in your answers.. nobody makes their .cue's manually because of the filenames etc... and .sbi only contains the libcrypt sectors, so it wouldn't be difficult to implement this in psxt001z using the fast option (if that's what you're aiming at) and then it would be possible to dump .sbi files. The point is that the dump is useless in emulators without this file. It would be the same as having to download a .cue file each time they wanted to play a game.
Then again, it's not even real dump data, and there's no such thing as a .sbi burn tool, so it doesn't make much sense to only include it for emulators. I don't think we'll ever get a way to generate these subs (it's possible, but Dremora clearly has no time).
It too think including the 'official' crcs of the sbi-files would be a nice ideal. It won't hurt filesize and would make the dump more 'complete'. Maybe you always get different sbi's, but why not use the ones availble here as a reference? I think hardly anyone creates these files by theirself, I even don't know how to create them and don't care because I can get them here.
Having the sbi-files in the games' archive would also help to avoid confusion when a partiular game won't run in an emulator. Is it because it's copyprotected and therefor needs a sbi-file, or simply because the emulator doesn't support that game yet or maybe it does need a special config? Without any hint, you first have to look up infos about the game...
You're theory about the sbi files being needed because of copy protection is correct. It's called LibCrypt and is mostly found on European PlayStation games (it is found on other regions as well but is not as frequent as anti-mod protection). This copy protection modifies entries in the subchannel data of the disc so that if a burned copy were made without also cloning the subchannel data the game will freeze at random points when the check is being performed.
I can see where everyone is going with this and I think many people here have lost track of what this project's actual goal is. We are trying to collect accurate checksum data from disc based games for the purpose of making a database to be used to check games that people may get by whatever means. Emulation of the dumps SHOULD NOT be considered when thinking of new ways to improve the database only the accuracy of the information displayed here. The dats are there simply as a quick way to check many images at once and and the sbi files that are downloadable are generated on the site when dumpers post LibCrypt data using a script created for the site. The only modification to the cue files that are posted in the dumps are the file names and that can very easily be done without downloading a new cue.
One thing I can suggest though is that we do need something added to the site to give us a list of the games that do have sbi files on them so that we can quickly know what games have been proven to have LibCrypt protection on them without having to look at each game separately. Perhaps we should be able to download all of them at once much like we now seem to have download links to download all cue and gdi files for all the games in the database but adding them to the dat is up to the admins at this point.
I can see where everyone is going with this and I think many people here have lost track of what this project's actual goal is. We are trying to collect accurate checksum data from disc based games for the purpose of making a database to be used to check games that people may get by whatever means. Emulation of the dumps SHOULD NOT be considered when thinking of new ways to improve the database only the accuracy of the information displayed here. The dats are there simply as a quick way to check many images at once and and the sbi files that are downloadable are generated on the site when dumpers post LibCrypt data using a script created for the site. The only modification to the cue files that are posted in the dumps are the file names and that can very easily be done without downloading a new cue.
Sorry, but I still don't understand the problem you have with this small and simple addition.
AFAIK, providing the subchannel-data where needed is already mandatory when submitting a dump of a protected game. Thus this data is already stored in the db and viewable on the disks' page. There would be no improvement/change to data and/or the dumping/submitting procedure at all, it just means to alter the 'create-dat-script' to include the crc-hashes of the sbi-files.
Above said, I can't think of a reason why incorperating the sbi in the dats would harm the goal of the project.
Hm...ok now that I think about it a little more I guess it wouldn't hurt. I'm just seeing way too much stuff around here with people wanting to make this more "emulator friendly" when that was never what this started out to be in the first place but yes I guess a CRC of the sbi file is possible since the site generates the checksums for the cue files as well. I just wish Dremora (or someone here that can program) would make something that can inject the SBI data in to a generated CloneCD sub file for people like me who want to burn the images and run them on the real system.
I only asked this question out of curiousity (since I'm new and all). Ideally I just wanted the SBI's available with the dump in the archive when I run it through Clrmame - only so I have them locally if needed for emulation and because they relate to the modified subchannel data of the original disc.
Hm...ok now that I think about it a little more I guess it wouldn't hurt. I'm just seeing way too much stuff around here with people wanting to make this more "emulator friendly" when that was never what this started out to be in the first place but yes I guess a CRC of the sbi file is possible since the site generates the checksums for the cue files as well. I just wish Dremora (or someone here that can program) would make something that can inject the SBI data in to a generated CloneCD sub file for people like me who want to burn the images and run them on the real system.
Indeed, directly onthefly injection of the subchannel-data when mounting/burning a dump would be the best solution, but until someone comes up with way to use them, we are stuck to the sbi-files and therefor should be referenced in the dats.
Just like some MAME versions ago, where you need to use some special files (xor tables) to break protection of CPS3 games. Ofcourse MAME had references to those files in it's internal database. Now that MAME can emulate the protection itself without the need to 'crack' it, the xor-tables arn't needed anymore and has been removed from the rom-db.
I believe SBI info should not be in the Dat file if it can't be verified, as that was not what i remember this site purpose was.
for me having info about packiso images CRC or SBI info in a dat is not right, and it should be the responsibilities of the uploaders to download those SBI files and share them with the dumps. also if someone have some packiso images generating a new dat file for those take just few clicks, having that info and regularly update it when ever there is a change related to that image in redump.org DB will only make maintaining the DB hard.
PS: i would rather like the dat file to be updated so that the naming scheme of the files can be updated to reflect more proper game name then having these info.
SBI files should always be the same, else the libcrypt protection would fail.
SBI files are needed to actually preserve the digital data, so it should be included when replicating data. (Though it is bad that it is done in a binary format which is poorly documented)
One thing I can suggest though is that we do need something added to the site to give us a list of the games that do have sbi files on them so that we can quickly know what games have been proven to have LibCrypt protection on them without having to look at each game separately.
http://redump.org/discs/system/psx/libcrypt/2/
btw, do those SBIs even work?
there appears to be no CRC stored within them,
so it's supposed to be recreated i guess
but since there are multiple patterns,
how would you tell whether to XOR with 0x0080 or with 0x8001?
if it's byte right after MSF, something's wrong then - it's never different from 0x01
for example:
same changes on CDs with different patterns:
C/A TNO IND M S F Zro aM aS aF CRC Unmd LC1 CRC Real LC2
MSF: 03:08:05 Q-Data: 41 01 01 07* 06 05 00 *23 08 05 ffb8 xor b838 = 4780 | ffb8 xor ff38 = 0080
MSF: 03:08:05 Q-Data: 41 01 01 07* 06 05 00 *23 08 05 3839 xor b838 = 8001 | 3839 xor ff38 = c701
in SBI CRC is lost:
C/A TNO IND M S F Zro aM aS aF CRC Unmd LC1 CRC Real LC2
MSF: 03:08:05 Q-Data: 41 01 01 07* 06 05 00 *23 08 05 ???? xor b838 = 4780 | ???? xor ff38 = 0080
MSF: 03:08:05 Q-Data: 41 01 01 07* 06 05 00 *23 08 05 ???? xor b838 = 8001 | ???? xor ff38 = c701so to recalculate CRC it could be either one of those algorithms:
a) Real XOR 0080
b) Unmd XOR 8001
but only one of them is valid for each CD
i.e.
this is as far as SBI goes:
MSF: 03:08:05 Q-Data: 41 01 01 07* 06 05 00 *23 08 05
those bytes are the same for both CDs
http://redump.org/disc/592/
http://redump.org/disc/798/
but each has different pattern in DB
it can't be right...
edit:
from what i can tell from cdr sources
0x01 following MSF indicate both time values being modified
and that's as much as SBI can hold
other options are 0x02 and 0x03 - for storing only relative or absolute MSFs
so i'd say ther's no way for SBI to hold CRCs - it's an unfortunate limitation of format
so what are those SBIs then good for at all?
CRCs are unneeded for ingame libcrypt validation, only MSFs and AMSFs are important.
but CRCs are modified as well -
it's either (CRC from clean block or CRC from modified block) XORed with one of predefined values:
0x0080 or 0x8001
so what is this all about? why would Sony bother XORing then?
to tell people where LC is, so it could be copied with ease?
maybe emulators cheat? have you tested on console?
i don't think SBI author did really understand how LibCrypt works,
since ther's that possiblility to include only one of MSFs (saving 7 bytes?) -
on LibCrypt this seems to never happen - it's always both or none, afaict
so CRC can even be sole value modified:
http://redump.org/disc/1128/
those sectors are known to be LC,
with them pattern looks complete: 32 total sectors divided in groups of 16
3rd entry is @offset 0x20 in SBI, containing only valid bytes
how can modified data be recreated from unmodified values?
btw such blocks would be skipped with original plugin - it only stores ones with modified MSFs
so this complexity of format - it appears completely irrational and quite useless,
it could have been just MSF+entire Q
overcomplication often is a result of lack of understanding
well ok,
i've made a program that would convert .sbi -> .sub taking magic value as an input parameter
sbi2sub
processed following images with it:
Crash Bash
Speed Freaks
and tested with ePSXE 1.70
plugins that do not appear to read subcode at all (they crashed @LC):
SaPu's CD-ROM
Xeven's Cdr
plugins that would have to work but didn't (both games hanged prior LC with those):
P.E.Op.S. CDR
Pete's CDR
plugins that did worked:
ePSXe CDR <- only one that passed and would run actual CDs instead of images
Moby2 cd disk image driver
and additionaly ePSXe's Run ISO command
without subcode:
Crash Bash would crash on loading screen after character select & intro, before stage select
Speed Freaks hangs while Neon City stage is loading
with subcode those parts passed on both XOR patterns ($8001 & $0080)
i didn't test further though:
maybe there are later checks that wouldn't
maybe there is certain threshold, so for example:
byte from each MSF + 2 CRC bytes = 4 - maybe half of them passing is sufficient
but CD-R recorded this way would degrade much faster
maybe hardware or even other emulators act different
so i didn't test all those things - i'll take a closer look later with debugger - it's quite interesting
but consider this:
if those CRCs are there for the kicks only - completely unneeded to pass LibCrypt
it's a huge flaw in protection then, imho - this pattern gives away those special blocks
so what if Sony removed it later?
what if there are LC versions without modified CRCs or at least without constant (maybe even for USA or Japan)?
we wouldn't pick those currently, right?
the only way would be to clear subchannel a lot - with multiple rereads
and compare it later on with the ones from different CD, made in a similar way
psxt001z is capable of doing this, afaik, but nobody would - it takes forever
it's such a drag, not everyone would even test CDs with fast option
lol, Dremora can't manage to write such a tool in 3 years time and you do it in 1 day (and you both live in Latvia? )
So the sbi format is missing relevant data that cannot be generated again? How about a tool which loads the libcrypt sectors from the database instead of using the sbi file? 
Usage: sbi2sub file.cue file.sbi XOR [file.dat]
Where: XOR is either $0080 or $8001
Example:
sbi2sub Game.cue Game.sbi $8001
sbi2sub Game.cue Game.sbi $0080 PSX.dat
What's that?
.dat is meant as a container from where to retrieve sizes, instead of actual .bin files, when they're not accessible
or when 1st pass is done & .img, etc. created,
execution of program with .dat will overwrite only .sub & .ccd, not .img itself
so it's slightly faster and less HDD killing, for those cases when only testing of $8001 vs $0080 is required
lol, Dremora can't manage to write such a tool in 3 years time and you do it in 1 day (and you both live in Latvia?
So the sbi format is missing relevant data that cannot be generated again? How about a tool which loads the libcrypt sectors from the database instead of using the sbi file?
no, no, Dremora had most of it done in psxt001z for a long time - there is SUB generator, it just doesn't take TOC as an input
i guess he had his reasons not to make it or really is very busy
and so did i - i had written very similar program recently, to make .SUBs for those saturn ring tests
so just had to make few little changes here and there - it's not a big deal
thanks though
did few more tests - i've masked first one MSF column, then 2nd in Crash Bash
and it didn't pass both of the times, no matter the CRC value
so it isn't threshold (any CRCs pass, but moded MSFs do not)
it's strange - i really thought this would be it
maybe there really is nothing more to this?
and so far all records in db belong to one of those two patterns, afaict,
so it's 50:50 for about 100 records,
not that bad if somebody wants to just burn some CDs
but for sake of preservation SBI, as it is now do not fit, imho
belonging to exact pattern is lost, so if there would be no more DB
guessing would be all that's left and still even if both values would pass on PSX
this information itself would be lost
batch reading from db is possible, cHrI8l3 would know better than i
but i think it wouldn't be good to to that - ther's no simple way, imho
it would retrive those 100 records whole, every time
things like that basically could kill server
one single reading to save manual labour making a backup in .txt or so - for later processing,
on the other hand, would do no harm
Why not to make XOR optional, so it will be possible to use non-libcrypt .sbi files for input?
to be honest, i don't want to
i could have made .sbi optional at all so it would rather be .cue->.ccd+.bin
but it could get out of control, then, as i see it
people might convert images and reupload them without referencing redump.org at all
and those would likely get more popular than ours because of application ease
so while they'd still be valid images
a lot of project's meaning would be lost and redump.org would have to compete with it's own subset
i don't want that to hapen,
i think this also might be one of reasons why Dremora didn't create such program
so i chose to make it as specific as possible
...
You're sick. No comments.
Redump Forum → General discussion → Inclusion of SBI files in the dat?
Powered by PunBB 1.4.4, supported by Informer Technologies, Inc.
Currently installed 6 official extensions. Copyright © 2003–2009 PunBB.