Topic: Message for themabus/gigadeath - Ys 4 PCE help!

Hello, I just noticed that Ys 4 PCE dump has been submitted.
I have many (20-30) PCE games here but this is the first one submitted in the database that I also have!
One first question: how did you calculate PCE games offset? Please remember that I have to rely heavily on D8 command because my drive offset is very limited. I tried a few experiments in these days but nothing good...

Now, I have dumped my Ys 4 copy with perfectrip, using as sample correction value 1782 (1752 + 30 of drive offset value).
Perfectrip returned 0 errros and the usual message "congrats, you got a perfect rip" for every track, but I checked the image with CD Mage and it reports 494 read errors! I tried to fix them but it can't.
Furthermore I got different gaps with perfectrip and EAC too (3.00 sec gaps are 2.74 in my dump).
Maybe my copy is different and dumping with +1782 EAC is wrong (I'll try to calculate the offset by myself once I understand how)

The ring serial code is different then the one of themabus' disc:

HCD3051-4-1108-R2F V    <-- themabus
HCD3051-4-1108-R1F V    <-- mine

I'll post the dump info so maybe you can detect what's wrong:

cue:
FILE "01.- ys4.track.pcm" BINARY
  TRACK 01 AUDIO
    INDEX 01 00:00:00
FILE "02.- ys4.track.img" BINARY
  TRACK 02 MODE1/2352
    INDEX 00 00:00:00
    INDEX 01 00:02:74
FILE "03.- ys4.track.pcm" BINARY
  TRACK 03 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:02:00
FILE "04.- ys4.track.pcm" BINARY
  TRACK 04 AUDIO
    INDEX 01 00:00:00
FILE "05.- ys4.track.pcm" BINARY
  TRACK 05 AUDIO
    INDEX 01 00:00:00
FILE "06.- ys4.track.pcm" BINARY
  TRACK 06 AUDIO
    INDEX 01 00:00:00
FILE "07.- ys4.track.pcm" BINARY
  TRACK 07 AUDIO
    INDEX 01 00:00:00
FILE "08.- ys4.track.pcm" BINARY
  TRACK 08 AUDIO
    INDEX 01 00:00:00
FILE "09.- ys4.track.pcm" BINARY
  TRACK 09 AUDIO
    INDEX 01 00:00:00
FILE "10.- ys4.track.pcm" BINARY
  TRACK 10 AUDIO
    INDEX 01 00:00:00
FILE "11.- ys4.track.pcm" BINARY
  TRACK 11 AUDIO
    INDEX 01 00:00:00
FILE "12.- ys4.track.pcm" BINARY
  TRACK 12 AUDIO
    INDEX 01 00:00:00
FILE "13.- ys4.track.pcm" BINARY
  TRACK 13 AUDIO
    INDEX 01 00:00:00
FILE "14.- ys4.track.pcm" BINARY
  TRACK 14 AUDIO
    INDEX 01 00:00:00
FILE "15.- ys4.track.pcm" BINARY
  TRACK 15 AUDIO
    INDEX 01 00:00:00
FILE "16.- ys4.track.pcm" BINARY
  TRACK 16 AUDIO
    INDEX 01 00:00:00
FILE "17.- ys4.track.pcm" BINARY
  TRACK 17 AUDIO
    INDEX 01 00:00:00
FILE "18.- ys4.track.pcm" BINARY
  TRACK 18 AUDIO
    INDEX 01 00:00:00
FILE "19.- ys4.track.pcm" BINARY
  TRACK 19 AUDIO
    INDEX 01 00:00:00
FILE "20.- ys4.track.pcm" BINARY
  TRACK 20 AUDIO
    INDEX 01 00:00:00
FILE "21.- ys4.track.pcm" BINARY
  TRACK 21 AUDIO
    INDEX 01 00:00:00
FILE "22.- ys4.track.pcm" BINARY
  TRACK 22 AUDIO
    INDEX 01 00:00:00
FILE "23.- ys4.track.pcm" BINARY
  TRACK 23 AUDIO
    INDEX 01 00:00:00
FILE "24.- ys4.track.pcm" BINARY
  TRACK 24 AUDIO
    INDEX 01 00:00:00
FILE "25.- ys4.track.pcm" BINARY
  TRACK 25 AUDIO
    INDEX 01 00:00:00
FILE "26.- ys4.track.pcm" BINARY
  TRACK 26 AUDIO
    INDEX 01 00:00:00
FILE "27.- ys4.track.pcm" BINARY
  TRACK 27 AUDIO
    INDEX 01 00:00:00
FILE "28.- ys4.track.pcm" BINARY
  TRACK 28 AUDIO
    INDEX 01 00:00:00
FILE "29.- ys4.track.pcm" BINARY
  TRACK 29 AUDIO
    INDEX 01 00:00:00
FILE "30.- ys4.track.pcm" BINARY
  TRACK 30 AUDIO
    INDEX 01 00:00:00
FILE "31.- ys4.track.pcm" BINARY
  TRACK 31 AUDIO
    INDEX 01 00:00:00
FILE "32.- ys4.track.img" BINARY
  TRACK 32 MODE1/2352
    INDEX 00 00:00:00
    INDEX 01 00:02:74

Clrmame info:
    rom ( name "01.- ys4.track.pcm" size 7916832 crc 22144d0f md5 ad1067d0bcc2ffeea540d3b985c51188 sha1 3b472f5e80bbf8259e09f8e4f547567bddb02490 )
    rom ( name "02.- ys4.track.img" size 77954688 crc 8324bcd5 md5 c441e993980d545420331374f0368776 sha1 2ace12bb3426eeb5c82da6e1ee536789e66efd04 )
    rom ( name "03.- ys4.track.pcm" size 21245616 crc 1f3e5d6c md5 8ca53ea6f78da0bcec2250237d52ab12 sha1 908a1edd2d7da3b937642c67633b048f23d0c74c )
    rom ( name "04.- ys4.track.pcm" size 20702304 crc 21bdbdcd md5 646903b17eb6a4b03bf69be5e62004b7 sha1 26ad247c08f02a853896072e1c5da2263008ca62 )
    rom ( name "05.- ys4.track.pcm" size 31149888 crc de995bce md5 48e25509b400b0e8591f7db8c144c1f7 sha1 21336c819e6ecb67663b97c596f24e16df71ba26 )
    rom ( name "06.- ys4.track.pcm" size 30211440 crc 4237bd43 md5 80c5e9a1b46ca2c62cb657911f00c17c sha1 1826f4caba38f888291bc0f5883d3441959ba423 )
    rom ( name "07.- ys4.track.pcm" size 18347952 crc 66af6612 md5 a656b9c95addb4fa26a5ee8810491e77 sha1 5b2b4bf57d6fe9d1b1cb4c272a01e75502561650 )
    rom ( name "08.- ys4.track.pcm" size 21363216 crc 32623eb4 md5 6017cf64f57fe24fd023f004d2b3b316 sha1 89cfac82c0d431ae039d00ef52892f4ee914b3cb )
    rom ( name "09.- ys4.track.pcm" size 19519248 crc b3bd7535 md5 b2a8590d4511516914c8f1078d6fe951 sha1 8447f56811e84c2a72997c30b6641706cbc101a4 )
    rom ( name "10.- ys4.track.pcm" size 20544720 crc 65853bbf md5 3968e4caadc7243542f6825a9e485228 sha1 f8e53de9991084c71f99c0975be683e00858a86e )
    rom ( name "11.- ys4.track.pcm" size 20093136 crc 6be0df2c md5 3c34fddea1d38c669f9a79d78ce849a8 sha1 f440d8a4b34398966ff84d4402af44b1f448f2f6 )
    rom ( name "12.- ys4.track.pcm" size 19923792 crc e6cfa922 md5 96e81539e23fa40172ba31670df34758 sha1 f96abc7c85018a1e4045203f8fe063c45fc8024c )
    rom ( name "13.- ys4.track.pcm" size 21539616 crc 755565b7 md5 f11fa4340dfcd08429e2665878115257 sha1 4e525a86991889bd252ec022cb5d06178adfa839 )
    rom ( name "14.- ys4.track.pcm" size 21565488 crc 05ca7e64 md5 9bf62cbb1ad3d97b7dada1dfea488a2f sha1 d2999aec46d4d6a344b9a4cbf5cd09317e21fe5f )
    rom ( name "15.- ys4.track.pcm" size 22040592 crc abff117e md5 fd458871bed8e1576706ccfe5aa30240 sha1 5c6a8fd91cd03e2a81b4438b8b7374bfbe07c854 )
    rom ( name "16.- ys4.track.pcm" size 20798736 crc d3481af5 md5 0d869c7c4eaa0e6fce4d9cfb1d798dae sha1 0b98ddeea5a440f26f5f42d01ede251aac6805dd )
    rom ( name "17.- ys4.track.pcm" size 22198176 crc 5563f869 md5 47fe5007a2c3543d752a218052651632 sha1 f5e612748f59d62b6c27ba1bb5ec195b657dad54 )
    rom ( name "18.- ys4.track.pcm" size 22193472 crc 0ecae187 md5 0f2d132c15b0f1479d8cda8469eac4ab sha1 738ef0aac03436f6376e05e3962190f418fd6921 )
    rom ( name "19.- ys4.track.pcm" size 20892816 crc 2ade8e25 md5 21d806c3c0531c2f202f75b60c20d784 sha1 d6c406337a271e2b72e261ea959da913967bb8f8 )
    rom ( name "20.- ys4.track.pcm" size 32570496 crc f3a0781c md5 60addca2e63b42ce0b13815c950898d6 sha1 40d4681a46b3249b25451f3ae20843316501d9b0 )
    rom ( name "21.- ys4.track.pcm" size 17783472 crc 1004ca6d md5 1cd5aebf500adbe4825e044e958dfd29 sha1 36e8570ea23e9b7119eb555205d7667a5180140c )
    rom ( name "22.- ys4.track.pcm" size 20963376 crc bbdae5bb md5 29f9ed80137f593ed7e13bd0313ac6d8 sha1 113655c76a073fda8ffae489532e0823923c2f5c )
    rom ( name "23.- ys4.track.pcm" size 31415664 crc 08311e69 md5 07091b2b93a6505726ec64084060fc3d sha1 cb3aeac80035d063d8215b66f0795ce208fe5c38 )
    rom ( name "24.- ys4.track.pcm" size 14267232 crc 6a95b675 md5 2104b0678fca46fda824f2606bbf2256 sha1 333e87aaa44ba775adb03983d752fd21f2894307 )
    rom ( name "25.- ys4.track.pcm" size 18180960 crc f73ae3d2 md5 f92181caf68226bdde79b0f1e29029fe sha1 68dc29a6edc6682c1cb210e87ee3176ac7c3e360 )
    rom ( name "26.- ys4.track.pcm" size 13434624 crc 6ac7d52c md5 c494add5a5cb05ee87661cca14a2707e sha1 01a098312eda1b82cf5fb2b73ac3aaf9ae4e47a9 )
    rom ( name "27.- ys4.track.pcm" size 11230800 crc 8a12e31b md5 803a05e0af682db388df1d7c02ce52a1 sha1 7db51c80e79b4d51648618675d651c13a4dc30c4 )
    rom ( name "28.- ys4.track.pcm" size 19265232 crc ebb8ed90 md5 036d537323e085fbda3f358a838795b1 sha1 6dee6bdcb147dc9a1c7f27f4cef00ddf9ab788be )
    rom ( name "29.- ys4.track.pcm" size 28096992 crc a4bdddca md5 c7d991c17ccaa173a8f9b4a93499c316 sha1 9dfc87cc0f3a9ab8033524b076487bf12c1cd7a2 )
    rom ( name "30.- ys4.track.pcm" size 55869408 crc 2814bc9d md5 241a79f16e7870b298ca0d1f2e78a110 sha1 a3ec2043b4eb6bf0ac6dd3ed4db2b55a9870c257 )
    rom ( name "31.- ys4.track.pcm" size 10275888 crc b7740db3 md5 301d0fe7cc45e415fc4baafc403548ca sha1 05f37bbefd7b603fa57b5a42c15400477510f7d1 )
    rom ( name "32.- ys4.track.img" size 39069072 crc a4963538 md5 6244046bc4c774ee68ad64e532c5c6af sha1 40f7fffd43769216eb5a48513bd3d78f350f7a6d )

Re: Message for themabus/gigadeath - Ys 4 PCE help!

If game revision is different, then probably disc offset is different too. You have to calculate your disc offset with D8.

We set 2.74 gaps to 3.00 becasue PCE disc are non-standard and there's all kind of things interfering with gaps, with 3.00 gaps the first track CRC matches between games.

Re: Message for themabus/gigadeath - Ys 4 PCE help!

gigadeath wrote:

You have to calculate your disc offset with D8.

How? I try to calculate on the first data track (setting the read sector where the track starts) but nothing.
And how can you set the gaps to 3.00? What is the theory behind this?
If the gap is 3.00, the tracks have to be dumped with a "manual" procedure

Re: Message for themabus/gigadeath - Ys 4 PCE help!

Use this in D8 batfile

@echo off
px_d8 x 4350 0
pause

So you'll read inside the data track

Only the data data pregap has only to be shifted, and that can done after the dump, first you'll have to dump your disc with the correct offset to see if the audio tracks match.

5 (edited by themabus 2008-02-09 21:16:05)

Re: Message for themabus/gigadeath - Ys 4 PCE help!

hi, xenogears
i was more like testing so far myself and intended to write something like a brief summary next week (about 10% would be dumped), but well, since also gigadeath already joined in, and so far everything's seems fine, so ok..
but if some of mods or anyone think ther's something wrong with this, please let me know, ok? i just really hope it's allright


1)you can get offset using D8 as described in Vigi's guide , but difference is that ther's audio @LBA 0 for PCE, so you must use other one, to hit on data track, gigadeath's suggested 4350 will work most of the time. (i think this is the best way to go)

2)you still can do sector view method: for positive offseted CDs it's the same as for psx and the rest, only now you should look for scrambled audio at the track 02 -> track 03 boundary;
for negative (most PCE CDs) you should subtract amount of data sectors in audio->data transition area +1 from data track's LBA.
most often track 02 has LBA 3590 and 150 sectors in TA: 3590-151=3439. now on, math is just the same, just that you look for a scrambled data backwards.

3)you can try to use this awk script, it'll wrap px_d8 and do math, but if ther's something wrong with it, we will not know, so maybe you shouldn't.. smile
to make it work you must place it in the same directory where ps_d8 is and execute command:
offset.bat <drive> <LBA> <DriveOffset>
like: offset.bat d 3600 30
and it'll do 3 readings and return result like this:
------------------------------
DRIVE Correction    30
------------------------------
LBA    Left    Rel.    Abs.
------------------------------
3600    135    -1311    -1341
3750    135    -1311    -1341
3900    135    -1311    -1341
------------------------------
PLEXTOR    ASUS    LITE-ON
------------------------------
-1311    517    -1335
------------------------------

last column: -1341, that repeats x3, it's absolute offset to submit to DB, it should always match. those at footer are offsets i'd use for my drives in EAC, to set up your drives you'd have to change 3 variables in offset.awk to you'r drive offsets (EACish, not reversed):
D1=30
D2=1858
D3=6


on audio:
gaps for PCE often are interpretable, and will wary depending on software. when those differences, as seen in EAC, are little (do not exceed 2 frames) from: 3 seconds audio->data; 2 seconds data->audio; 0 seconds audio->audio: it's so far always was either because of a different interpretation of subcode data or because one audio silence sector is left unmarked as pause. in all such cases we agreed to map gaps to this 3-2-0 scheme.
to do so:
1.1) audio->data gap < 3":
we cut corresponding count of sectors from the end of this dumped audio track:
for example: 2.74 gap for Track 02 in EAC - cut 1sector=2352 bytes from the end of Track 01
1.2)audio->data gap > 3":
we'd dump it with zero gap instead (<- this is very important! because otherwise data is lost), and then cut 3 seconds (=225 sectors) from the end.
**worst possible case it's when you have an audio track that is enclosed by data tracks (so you can't just ignore detected gaps) and this following data track gets gap detected larger than it should be, then you'd have to extract sector range or dump twice with and without gaps and then combine this data, but it's maybe on 1 cd from 100 or like so.
2.1)data->audio < 2":
add corresponding number of audio silence sectors to the beginning of affected audio track (psxt001z generated)
2.2)data->audio > 2":
remove that amount of sectors from beginning of this audio track.
3)audio->audio >0:
shift back those sectors from the beginning of track that follows gap, to the end of one that preceeds it. or alternatively just read such tracks without detecting gaps (like in 1.2 you can't ignore gaps if this 1st affected track is preceeded by data track).

(it's, maybe looks a bit bizarre, but actually most cd's are 3-2-0 or seldom with some minor changes, i mean, there can be 6 cds with correct gaps and then one with 6 messed up gaps, it's like that, so please don't worry about it, if ther's something very strange you can just skip it for now.)

from 50 CDs there's so far were, i think, 5 exceptions with different gaps (Ys 1&2/Ys 3 are two of them, btw), where this scheme would not work.

on data tracks:
data->audio gaps must be fixed the same way as always - removing audio sectors (so far always 150 on my tested CDs). if data track is the last one on CD, ther's no gap so this should not be done (<-again very important to remember).

and we did also agreed to fix audio->data gap structure. amount of sectors similar to the target gap must be added to data track that follows audio. but unlike data->audio gaps (where only audio part of transition area is set as pause), those consists of both parts: 1st audio silence (psxt001z --gen output); 2nd mode 1 empty sectors - my brother and i, we made a very basic program for those, that you can use.

so for Ys 4:
it's 2.74 audio->data gaps (case 1.1):
1.dump everything with IsoBuster/EAC as usually
2.remove 1sector=2352 bytes from the end of Track 01 and Track 31 (would be great, if you could check if this sector is silence)
3.remove 150 sectors from the end of Track 02 only. (again, it's better to always check if ther's no data, when you cut something, tho there can be scrambled audio that may appear somewhat like a data, depending on drive)
3.generate 75 audio silence sectors with psxt001z:
psx --gen "Track 02.000" 176400
psx --gen "Track 32.000" 176400
4.generate 150 empty mode 1 sectors for Track 02 and 32 (75+150=225=3"=our target gap):
GenEDCECC.exe "Track 02.ecc" 3440 150
GenEDCECC.exe "Track 32.ecc" 311960 150
(1st parameter is target file name, it'll be overwriten without warning, so be careful, please. 2nd parameter is DataTrack LBA-number of Mode 1 sectors in transition area (150 always so far). 3rd: number of Mode 1 sectors in transition area. and i assumed here, in this example, that you have the same LBAs as i did)
5.recombine data tracks:
copy /b "Track 02.000" + "Track 02.ecc" + "Track 02.bin" "Track 02.out"
copy /b "Track 32.000" + "Track 32.ecc" + "Track 32.bin" "Track 32.out"
6.you must modify .cue manually with target gaps, like 3;2;0... in this case.

7? would be great if you could error check data tracks @this point, i'd use a custom program for this but you can open this moded .cue with cdmage, there will be errors on those audio sectors at the beginning (75 for each track, usually), and sometimes there can be few errors at the very end of last data track (if ther's happen to be mode 0 sectors), i think, but other than that it should be ok. please don't fix it, just scan, ok? wink.

8? finally to be sure that everything's reassembles correct, it would be great to dump a .cue with IsoBuster from original CD and compare it with .cue you'd get with IsoBuster from virtual drive where those tracks are mounted (Alcohol or such). it's because IsoBuster has LBAs in .cues so those two must byte match (fc /b original.cue redump.cue) or have the same crc. (although it does not contain LBA of leadout or total CD size so it's still possible to mess up last track and not notice it, but this should not happen as long, as you remember not to slice those 150 sectors off from the end of it smile )

EAC will usually return same gaps on all drives, if they all use method A, so you can verify audio tracks from EAC gui and only need to do realigning once. btw it's important to do reading on different drives, i had a CD that would not return any errors but two output audio tracks was always different by few bytes on each of 3 drives (Burst mode/Paranoid/C2&cache on/off.. it didn't matter - always the same crc on the same drive but different among them), so i didn't submit it, but on one drive it's impossible to tell.

so i think this pretty much sums it all up.
this way you should have all meaningful data that there was on a cd dumped to images; audio offset is realigned to 0; TOC stays true to original; transition areas are reconstructed, as they were in original; subchannel will be recreated when burned/mounted - it's the only thing that can differ a bit, well because of those gaps.. about as perfect as it gets.
would EAC create .cue files for PCE CDs and use different algorithms for gaps (at least on those ADDRESS=2 subcode sections) it'd be much easer then, and not much different from dumping as it's in the guide, so i really just hope these problems will be fixed soon.

i'll try to write that resume next week, on some stats and exceptions that was there so far, but they are very rare..

6 (edited by gigadeath 2008-02-10 14:28:05)

Re: Message for themabus/gigadeath - Ys 4 PCE help!

themabus wrote:

finally to be sure that everything's reassembles correct, it would be great to dump a .cue with IsoBuster from original CD and compare it with .cue you'd get with IsoBuster from virtual drive where those tracks are mounted (Alcohol or such). it's because IsoBuster has LBAs in .cues so those two must byte match (fc /b original.cue redump.cue) or have the same crc. (although it does not contain LBA of leadout or total CD size so it's still possible to mess up last track and not notice it, but this should not happen as long, as you remember not to slice those 150 sectors off from the end of it smile )

Actually this passage is not only great, it is necessary, I noticed comparing LBAs in IsoBuster between actual CD and dump that there's always 1/2/3/4/5 sectors shifting when data track -> audio track (in other words, between 2nd and 3rd tracks). That has to be corrected manually, taking out 348096 bytes of silence at the end of the data track and adding 348096 bytes of silence at the beginning of the audio track, if there's a 2 sectors shifting (I dump with PerfectRip). It's important to not delete sectors starting with the sync (00FF etc).

And note that this happens for CDs that EAC reports as having perfect 3:00/2:00 gaps, not only odd ones with 2.74/3.01/0.01 gaps. So the LBA comparing has to be done always.

PCE dumping is the most difficult and boring task in the world, I suggest Xenogears to dump only discs already in the database to see if they match, we have to prepare a guide ad-hoc for PCE discs.

7 (edited by gigadeath 2008-02-10 14:28:44)

Re: Message for themabus/gigadeath - Ys 4 PCE help!

Another thing that has to be note, if you dump PCE disc data tracks has to be resized at the end only by 348096 bytes, not 352800! Taking out 2 full seconds you'll cut 2 sectors of data! And when dumping with PerfectRip you have to add only 348096 bytes of 00 silence at the beginning of the following audio track, otherwise you'll have a bigger file than the actual one. It's not always 348096, depends on the shifting.

Themabus can you confirm?

Re: Message for themabus/gigadeath - Ys 4 PCE help!

gigadeath wrote:

PCE dumping is the most difficult and boring task in the world, I suggest Xenogears to dump only discs already in the database to see if they match, we have to prepare a guide ad-hoc for PCE discs.

OK, I'll wait a proper dumping guide for PCE then!
Thanks for your support!

Re: Message for themabus/gigadeath - Ys 4 PCE help!

gigadeath wrote:

Another thing that has to be note, if you dump PCE disc data tracks has to be resized at the end only by 348096 bytes, not 352800! Taking out 2 full seconds you'll cut 2 sectors of data! And when dumping with PerfectRip you have to add only 348096 bytes of 00 silence at the beginning of the following audio track, otherwise you'll have a bigger file than the actual one. It's not always 348096, depends on the shifting.

Themabus can you confirm?

mmm, if ignoring subcode oddities, i always had this gap 2 seconds so far, i think... if total offset (cd+drive) end up positive it's possible to get a  scrambled audio silence here, just like when determining offset in sector view, so it will wary on drives. and my Lite-On would often return a whole gap filled up with some scrambled noise (it doesn't descramble to 00, but rather has some bytes set). but you can tell because prior to audio silence data tracks conclude with an empty data sectors, just like those generated, should be sync+header+00000..+edc+ecc if ther's a lot of data, it's a junk, i think. and, headers, there aren't any at all or they do no follow in sequence. maybe you get something similar? i think it's data up until it matches on two drives or audio silence sectors start, FileSize-352800 so far. do you get this on Human Sport Festival also?