26

(17 replies, posted in General discussion)

Ok, here's another update based on the discs I purchased and dumped myself. Initially that was asked by Fireball as he has experience with them but this is really good shitty audio usecases and some general info on what we can encounter.

Dracula: Music Collection
http://redump.org/disc/14890/

Nothing special about this disc other than two masterings differ by the offset, same happens here: http://redump.org/disc/77301/
Specified possible write offsets +390 and +684 cut into tracks, I don't think they are relevant. True offset should be in perfect range [-4926 .. -3731] based on my redumper algo.

We can offset match such discs using two possible approaches:
1. Always shift each dump left-most or right-most so regardless of the offset each dump will match each other. Pros are that it's totally automatic. But a big con is that we will need to redump all audio entries which is unrealistic.
2. By introducing something I'd call a "universal checksum". Basically upon successfull dump, redumper calculates right-most (or left-most) data shift checksums and outputs crc/md5/sha-1 checksum the usual way: <rom offset="+123" size="68701920" crc="060bb712" md5="47393f188ff00fafbdf77b5eb771dbd3" sha1="ef991d90b284b0c92ab2b4eb0eb77942e32bb98c" /> and notes the offset value needed to right-most/left-most shift. We store this information somewhere for the future reference. Every time a potential different offset verification title is dumped, we compare universal checksums and if they match, we add another ringcode line to the matched entry with deduced offset in relation to the previous entry.
The pros of such approach is that we don't dramatically change the way we dump comparing to method (1) so already added dumps stays the way they are. The cons are that it's not 100% automatic.

Personally, I'm in for (2), this is easy to implement and we can set a precedent that will be used in the audio dumping world.


Tenbu Mega CD Special Mini Audio CD
http://redump.org/disc/6695/

This is very clean, redumper shifts out 13 samples left from lead-out and everything still fits in pre-gap nicely.


Micronet Music Collection Vol. 1
http://redump.org/disc/30335/

This has huge non-zero chunk (22006 samples or 88024 bytes) in the lead-out. According to the proposed rules, we shift the data out of there left by the amount of 22006 samples. This will get rid of the lead-out data but will spill over 16 non zero samples into pre-gap. Not ideal but it's close to the truth and perfect range for this disc is [+8423 .. +21155]. IMO the best solution given that we preserve whole data in one file.


Oyaji Hunter Mahjong
http://redump.org/disc/39873/

This is exactly as comments say. I stand corrected, this is more horrible. There is 68 sectors of data in lead-out, there is 150 sectors of data in-pregap and there is ~670 sectors (1574524 bytes) of non-zero data in TOC before pre-gap. I capture everything in redumper and it seems to be consistent in the scram file. Offset 0 is used by default and I extract leadout.bin as is and getting same checksums calculated by Fireball, everything matches. 150 sectors data in pre-gap are fully preserved in Track 1, but what to do with the data in TOC? I don't know. Well, in fact I will propose a solution later but that requires everybody to be open minded smile


Other Considerations
Now, with all these examples in mind, I have a modified idea which will let us capture every byte and be mostly redump compatible (including site and the current DB).
What if we never shift audio e.g. always use offset 0 but store spillover lead-in and lead-out data in separate tracks? Something like pregap.bin / leadout.bin that we don't currently "preserve" but in a more generalized way.
This fits in a very elegant way with lead-out as internally, leadout is just another disc track with AA track number and it has all the track properties such as mode, data, positional subchannel etc. As in reality lead-out track spans the whole disc, we trim all the zeroed data and make it sector aligned. If there is no data in the lead-out - we don't create a file and that will satisfy 99% of all the use cases. But at the same time we accomodate for the case where there is something there. As other two big benefits I see that we can preserve Dreamcast logo data which is session 1 lead-out and I sometimes see lead-out audio spillover in PSX discs where it's not currently being preserved in any way. We can have the track defined in the CUE-sheet with all the appropriate properties thus this data will be preserved by "data hungry" preservationists, whoever they are. The similar approach will go for the non zeroed lead-in track. If it's empty, like it usually is in 99% cases, it won't exist. If it does, it's zero trimmed at front and sector aligned. No data is lost ever, redump track compatibility is all time high as it's CUE tied and we add it to the website like a usual tracklist with hashes.


Oyaji Hunter Mahjong example:

FILE "Oyaji Hunter Mahjong (Japan) (3DO Game Bundle) (Track 1#00).bin" BINARY
  REM REDUMP LEADIN
  TRACK 00 AUDIO
    INDEX 00 00:00:00
FILE "Oyaji Hunter Mahjong (Japan) (3DO Game Bundle) (Track 1).bin" BINARY
  TRACK 01 AUDIO
    INDEX 01 00:00:00
FILE "Oyaji Hunter Mahjong (Japan) (3DO Game Bundle) (Track 2).bin" BINARY
  TRACK 02 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:12:45
FILE "Oyaji Hunter Mahjong (Japan) (3DO Game Bundle) (Track 3).bin" BINARY
  TRACK 03 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:09:60
FILE "Oyaji Hunter Mahjong (Japan) (3DO Game Bundle) (Track 4).bin" BINARY
  TRACK 04 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:11:63
FILE "Oyaji Hunter Mahjong (Japan) (3DO Game Bundle) (Track 4@AA).bin" BINARY
  REM REDUMP LEADOUT
  TRACK 05 AUDIO
    INDEX 01 00:00:00

Or variation naming/numbering schemes. I specifically chosen # and @ for filenames as the symbols sort before and after number entry thus you get a nice look and this scheme supports multisession pre-gaps/lead-out as we don't have to renumerate anything.
We could use simply "Track 00" for lead-in and "Track 05" for lead-out but there has to be a good way of supporting this for multisession discs where there can be session lead-out/lead-in between two tracks with adjacent numbers.
Or, we don't have to add it to the CUE-sheet at all but in my opinion having it there ties all the files together for the preservation. We could even have special redump CUE tags for that, plenty of ways.

27

(20 replies, posted in General discussion)

https://i.ibb.co/MsQGdRz/Screenshot-from-2022-11-11-18-52-31.png
From PAL 70004 BIOS, all that's available.
This is most likely excluding list of problematic games which are either disabled or patched. I checked a couple serials, there are multi disc games for instance. The list obviously not helping much.
As most is unaware, PS1 games are half-emulated because PS2 doesn't have PS1 GPU, hence much more games not included in the list have problems, as a good read I recommend PS1 emulation engineer interview: https://freelansations.medium.com/the-s … 39cf5a0353

Myria wrote:

* If a disk’s first TOC has no B0 entry, the TOC has a sufficiently small A2 entry, and the TOC’s own timestamps in sub-Q use positive encoding (00:00:00 start rather than 99:59:74 end) fake that there is a B0 entry around 08:00:00 or so.

There are usually no TOC B0 entries on 99% of the CD's. I saw them only on some multisession pressed CD's. From what I know it's used for CD-R.


Myria wrote:

This allows dumping the “PRODUCED BY SEGA ENTERPRISES” area—yes, that area is actually readable in CD audio mode.  (It uses SafeDisc 2-like weak sectors.)

Can you elaborate? I never heard of "weak sectors" before.

29

(17 replies, posted in General discussion)

Just an update on this, I ordered a couple more Japanese audio CD's which have audio in pregap/leadout which Fireball suggested to check.
They are on the way here. After I receive them and redump, I will report my findings here and we will reiterate on the final audio CD rules and preservation format as I really want this finalized.

30

(17 replies, posted in General discussion)

I want to try some CD's suggested by Fireball to see if we're covered there.

31

(17 replies, posted in General discussion)

Jackal wrote:

And where does this discussion leave us with discs like those PSX with audio in lead-out? I'm against appending lead-out data to the last track, because it's just not part of the main tracks. Also, I dont think we should shift audio data out of the lead-out for mixed mode discs, because the combined offset correction overrules it. So the only solutions for such discs imho is to put the data in leadout.bin or do nothing with it.

I totally agree with that. I purposely haven't mentioned that yet to get Fireball's opinion on pregap.bin/leadout.bin and focus on one issue at a time wink.
In all situation where we have an offset determined by a data track, we shouldn't extend last track, saving leadout separately for cases like this would be possibly a best solution?
TL;DR, shifting data out of lead-out / pre-gap should happen only for the discs where we can't figure out an offset in a deterministic way (based on data track sync/MSF or anything similar).

32

(17 replies, posted in General discussion)

Ok, so some cool down period passed, let's regroup.

Let's say, we remove perfect track split out of consideration. I'll have redumper output perfect audio offset range anyway just for reference but will not apply it by default. The concept would be super useful for perfectionist audio folks so I'm happy I have that implementation for them.

What's basically left for us @redump is to define a clear approach to how we handle audio discs with non zero pre-gap and lead-out data.

I guess we have a consensus on the next two rules:
1. if there is non zero data in lead-out and that data can be fully shifted out of there (left) without spanning non zero data into pre-gap, correct offset with a minimum shift required
2. if there is non zero data in pre-gap and that data can be fully shifted out of there (right) without spanning non zero data into lead-out, correct offset with a minimum shift required

How to define situations where it's impossible to shift out data from lead-out/pre-gap (data is wider than allocated TOC space for it), several options:
1. Use offset 0 as a base, dump non zero pre-gap data to pregap.bin, dump non zero lead-out data to leadout.bin
2. Fully shift data out of lead-out if needed, dump non zero pre-gap data to pregap.bin
3. Use offset 0 as a base, prepend non zero pre-gap data to the first track, append non zero data to the last track
4. Fully shift data out of lead-out if needed, prepend non zero pre-gap data to the first track

My insight:
I don't like (1) or (2) because preserving data to external pregap.bin and leadout.bin files will usually be "lost" because it's unreferenced from the CUE-sheet and I don't see a good way of linking the files to the other cue/bin set.
I also don't like (1) or (2) because in all cases that I saw, non zero pre-gap data genuinely belongs to the first track. It's either HTOA index 0 entry or non zeroed mastering "silence" which is still part of the first track.
Lastly, I didn't find a proof anywhere in the Red Book standard that pre-gap data of an audio disc should be zeroed. I found such a requirement only for the pre-gap of a data track in ECMA-130.

That said, I personally would lean towards (4).

Let me know what do you think.

33

(3,497 replies, posted in General discussion)

ehw wrote:

Hey sarami I have a question about your .c2 format.

According to your readme, 1 bit inside the .c2 file represents 1 byte in the disc image. But there isn't any additional information besides that. I can assume that 0 means no c2 error and 1 means c2 error, but I'm having a difficult time trying to determine how the bits correlate to the exact locations of the bytes in the disc image.

Please see my reply here: http://forum.redump.org/post/99760/#p99760

ehw wrote:

I assume that the bits correlate to the bytes in LSB order. So since $01 is 00000001 in binary, there should be an error that affects just 1 byte around 0x618-0x61F, in this case the c2 error should be on 0x61F. But when comparing both image dumps, there are TWO bytes that differ rather than one.

C2 is usually big endian (MSB) with some drive exceptions.
Also, you see two damaged bytes per 1 C2 bit set only for sectors that are scrambled, I guess this is how scrambling implemented on hardware level.

Talk to me on Discord (superg#9200)

34

(17 replies, posted in General discussion)

Jackal wrote:

So my question for this discussion is:
Do we really need a "perfect offset" correction if there is no data loss with 0 offset (and no common write offset can be detected)? After all, we wont know how the disc was mastered and if the gaps are supposed to be silent.

Yeah, this is legit question, if no data is lost, we just end up with imperfect split sometimes. But that technically can be corrected just from BINs if needed.

35

(17 replies, posted in General discussion)

F1ReB4LL wrote:

Wrong, dumpers must tweak the offset value to include all the data if possible, also DIC should do this automatically, IIRC.

True, we want dumpers to do that, but the status quo is that they really don't. They do whatever DIC does by default.

F1ReB4LL wrote:

Sometimes both pregap and leadout have data, needs manual analysis in this case (usually it's about tweaking the offset to include the leadout data + extracting the pregap, like it's done in http://redump.org/disc/6695/, but I guess it depends on which of the pregap and the leadout portions of data is smaller - the smaller one needs the offset tweaking, the larger one needs the extraction).

Even if we separately extract pregap.bin, it can be automated. I don't necessarily like this approach as pregap data usually belongs to track 1. It's fine if it's "almost silence" but Intothisworld owns a disc where it's HTOA in there.

F1ReB4LL wrote:

There are thousands of games with the same problem, so it's mostly about their bad mastering rather than direct offset issues.

True, but at least we have a way for them to determine a proper offset value using data track.

F1ReB4LL wrote:

Crazy. I think that matching EAC and AccurateRip databases for the 'offset 0' audio tracks is a better benefit compared to some custom aligning.

We can still have hashes or some other info calculated for offset 0 to satisfy the compatibility things like EAC/AccurateRip.

F1ReB4LL wrote:

Also, what happens if all the tracks have big portions of silence at their beginnings and endings? Does it shift the offset to force the first track to start from its first nonzero byte?

No no, it's not like that. The current logic doesn't align to first non-zero sample. I calculate perfect offset range, but if offset 0 belongs to that range, it's has a higher priority. The disc you describle with the big portion of zeroes at the beginning and end will have a big slack which will most likely include offset 0.

F1ReB4LL wrote:

It will NOT work for Jaguar discs at all, because its 'data' (the INDEX 01 of the second track of the second session) must start with its magic (according to the official Jaguar CD format description documents), but if you align according to the magic, the first track of the first session for a good half of Jaguar CD releases gets shifted to the first pregap - and in your case, if you align according to the non-zero data and silences, its magic will be way off its proper position.

It works for the Myst Demo I own. I can't speak for the generic case. My Myst Demo aligns to that magic using the logic I described:

detecting offset
audio silence detection... done
perfect audio offset (silence level: 0): [-2201 .. -401]
disc write offset: -401
warning: pre-gap audio is incomplete (session: 2, errors: 371)
detection complete (time: 14s)

As offset 0 is not included in the perfect range here, -401 is closest to 0 so it gets chosen and that perfectly coincides with aligned by magic.
By default I have it implemented using magic, like DIC does. I just remember you mentioned you're not fond of the current way so if you think we can do something better for Jaguar CD - let me know.

But the algo is still useful for PSXGS as if you align by magic, it cuts through other data portions.

36

(17 replies, posted in General discussion)

Jackal wrote:

The logic as described sounds good to me. I'm assuming the offset correction is always in samples, so a multiple of 4 bytes?

Yes, I analyze it as multiples of 4 bytes (two signed 16-bit sample values).

Jackal wrote:

I would also like to know if your method is able to find the true "perfect" offset for discs such as these: http://redump.org/discs/quicksearch/wri … /audio-cd/ + http://redump.org/discs/quicksearch/det … region/Eu/ where aligning either left or right to the first byte gives a "common" write offset value that is consistent with the ringcode.

Don't have any from this list unfortunately, you can try redumper if you have any of these, for each audio disc it will output something like

detecting offset
audio silence detection... done
perfect audio offset (silence level: 5): [+736 .. +22675]
disc write offset: +736
warning: pre-gap audio contains non-zero data, preserving (session: 1, leading zeroes: 87473, non-zeroes: 727/88200)
detection complete (time: 75s)

And after that "redumper split --force-offset=22675 --overwrite" to align it.

Jackal wrote:

Maybe this should be an extra step? So validate for common offset values when shifted to first or last non-zero byte before going with 0 offset or whatever else the logic would decide on. Would be great if you could test that using some of the images.

I also thought about something like this. Left shift (or right shift) to the first non zero byte and calculate some hash and use it as a universal checksum which allows us to match same audio discs mastered with a different offset. For instance for two such discs from here: http://redump.org/disc/77301, if I right-align, both dumps match.

37

(17 replies, posted in General discussion)

EDIT 12/03/2022:

Final Audio CD offset correction algorithm (technical):
1. if there is one and only one possible disc write offset, applying which perfectly aligns audio silence (level: 0) ranges with TOC index 0 ranges, use it
2. else if there is non-zero data in lead-out and that data can be fully shifted out (left) without spanning non-zero data into lead-in, correct offset with a minimum shift required
3. else if there is non-zero data in lead-in and that data can be fully shifted out (right) without spanning non-zero data into lead-out, correct offset with a minimum shift required
4. else apply offset 0

Dump format specification notes:
Regardless of the applied offset, we always operate on LBA [ 0 .. lead-out ) range when we perform a track split. This is also true for data discs. If, as a result of mastering, there is non-zero data either in lead-in ( -inf .. 0 ) or lead-out [ lead-out .. +inf ), this is preserved in separate files. For discs with data tracks, non-zero data means the descrambled data portion of a sector (fields such as sync, MSF, mode, subheader, ECC/EDC is excluded). The resulting lead-in/lead-out files should be trimmed to the minimum non-zero data size, lead-in from the front and lead-out from the back, but they should remain sector-aligned (size divisible by 2352).

Considerations for matching Audio CD with different write offsets
For each disc, dumping software can generate a checksum/hash for the non-zero sample span of the data e.g. aligned to 4-byte CD sample size. Such hash can be used for the disc identification purpose as well as for Audio CD different write offset matching as for such cases the hash would be the same. This applies to both audio and data discs. I suggest to use a longer SHA-1 hash and not CRC32 just to be future proof. It's quite likely that we will get a couple of CRC32 collisions in ~100K discs currently in the DB. As a side perk, it can also serve as a unique disc ID which can be easily looked up in the database, if such capability will be ever implemented at redump.org. In any way, I would like to have such a hash in Audio CD entries just for write offset matching.

END OF EDIT, the rest information here is kept for the reference and archival purpose.




This topic is to clarify and decide on how we manage disc write offset for audio discs.

The current status quo is that we always dump audio discs with offset 0 as there is no reliable reference point in the audio stream (in contrary to a data track) which can be used to determine the offset. This approach has a number of disadvantages, such as:
* shifted audio data in pre-gap / lead-out which we don't currently preserve
* ocassional imperfect track split which cuts in the middle of audio tracks e.g. you hear a bit of the next track in the end of a current one or previous track in the beginning of the current one

I believe I solved both of these problems in redumper. Let me define some terminology first.
Perfect Audio Offset is a disc write offset which guarantees that no data is shifted into lead-out and guarantees track split which doesn't cut into a middle of a track.

Perfect Audio Offset implementation details
For a given audio disc, I build a silence map based on TOC/subchannel information, essentially it's INDEX 00 CUE entries which are almost always empty (silent). As a next step, I build a similar silence map based on an audio stream. Finally, For each offset within [-150 .. lead-out] constrained range, I try to line up these two maps in a way that TOC/subchannel based one fits into an audio stream. If it fits, it's a perfect audio offset.

The current audio offset logic
favor offset value from perfect offset range if available
if multiple perfect offset values available (range), try to shift data out of pre-gap (if needed) if still within a perfect range
otherwise if no data in pre-gap, favor offset 0 if it belongs to the perfect range
finally, if offset 0 doesn't belong to the perfect range, use a value closest to 0 within a perfect range
if no any perfect offset available, try to shift data out of lead-out and pre-gap (only if we can get rid of full pre-gap data) if that doesn't lead to data loss.

or pseudocode:

if(perfect_audio_offset_available)
{
    if(perfect_offset_single_value)
        use_offset_value();
    else if(perfect_offset_value_range)
    {
        if(data_in_pregap)
        {
            if(enough_space_to_get_rid_of_whole_data_in_pregap AND still_within_perfect_range)
                move_minimum_data_right();
        }
        else if(zero_offset_belongs_to_perfect_range)
            use_zero_offset();
        else
            choose_the_offset_closest_to_zero();
    }
}
else
{
    if(data_in_leadout)
        move_minimum_data_left();
    else if(data_in_pregap AND enough_space_to_get_rid_of_whole_data_in_pregap)
        move_minimum_data_right();
}

Pre-Gap notes
Based from the discs I own and discs I have an access to, most audio discs with data in pre-gap is not a result of a write offset but rather a way it was mastered. It's often values close to silence but not zeroed and sometimes it's a part of a hidden track (HTOA). In cases like this, there is no way to move out that data fully out of pre-gap without it shifting into lead-out as there is not much space (it's common to have 1-2 seconds of pre-gap data audio which is a lot). I can definitely say it worth preserving by extending track 1 fixed 150 sectors back for all such cases, with, optionally, marking that in CUE? Before anyone says it's stupid, DIC already does a similar track 1 extend for CDi-Ready discs.

Statistics
With new method, I redumped all my audio discs and sadikyo redumped some of his. I shared new dumper version with Intothisworld and he shared it with ppltoast but I have yet to get some results from them. The current merged detail statistics is available here:
https://docs.google.com/spreadsheets/d/ … sp=sharing

TL;DR
73 discs - match redump (offset 0 is one of the perfect offsets)
9 discs - no perfect offset found (no distinctive silence in index 0 or no index 0 entries at all), offset 0 used so matches redump
7 discs - only one perfect offset found and it's true offset value used to master disc with, will require DB update
27 discs - perfect offset range excludes 0, will require DB update
3 discs - have pre-gap data which is impossible to fully shift out and it's not currently preserved, will require DB update

Side Effects
Given method works really well for PSX GameShark Update discs and Jaguar discs without relying on magic numbers and we get a perfect splits there too.

More Side Effects
If we left-align (or right-align) offset based on a perfect offset range, we will be getting matches for audio discs with the same data but different write offset, such as:
http://redump.org/disc/77301

I would like to hear your opinions on what do you think of it, let's discuss.

38

(3,497 replies, posted in General discussion)

sarami, just FYI, two CDi DIC dumps have scrambled sectors close to the end of the track:
http://redump.org/disc/24962/ - 1 sector
http://redump.org/disc/78871/ - 2 sectors
We've corrected that in the database but I guess that have to be fixed on the DIC side.

39

(2 replies, posted in General discussion)

scsi_wuzzy wrote:

It would be nice if we could ultimately end up with CD images like the ones Near described. Near's article about the CD format hypothesizes about a CD format that just stores all the lead-in sectors (including subchannels) so that we could have a single file raw image format that embedded the TOC the same way the real disc does. Such an image format would handle multisession discs easily.

I already have something very close to that but it's not redump scope really. You can't get all the lead-in sectors because of the mechanical limitations of the drive (how close laser head assembly can get to the center of the disc). Plextor allows us to start reading from some place in the TOC and that is good enough given that TOC is repeating itself.
Custom drive firmware would be an ambitious project for sure but that's very complex task.

scsi_wuzzy wrote:
F1ReB4LL wrote:

If you're using one of the latest DIC versions, it has both scrambled and descrambled image checksums in the "_disc.txt" file, you can scramble the descrambled image back and verify its checksum, if it matches - no reason to store the scrambled file itself.

I've thought about doing it this way and then just storing deltas for when the scrambled data doesn't match exactly (since the deltas would allow creation of the scrambled data from the unscrambled data and would typically be much smaller than the entire scrambled image). That's probably what I'll end up doing, but I also may look into adding metadata to an archival format like Aaru if it's possible.

I wanted to make sure there wasn't a better way to do it using some existing, standardized approach before I came up with my own solution. It sounds like there's not, unfortunately.

Scrambling is a simple math involving shift register, it's a trivial implementation.
Delta is ineffective here and it will be as big as the data track (data is scrambled, audio is unscrambled).
The most annoying thing in this conversion process is to actually know which sector is audio and which is data, scm doesn't have that info so you will have to extract it from TOC to be absolutely sure (you can go by data sync header but there is no guarantee there won't be such sequence in audio sector).

I want to share my findings about general layout of a multisession disc.
Apparently what I see here contradicts with common redump knowledge so this might be important.

In particular, my findings show that each session has it's own TOC in lead-in which lists track entries only for that session.
I will demonstrate that on http://redump.org/disc/75764/

The disc has the following TOC:

TOC:
  session 1
    track 01 { audio, LBA:      0 ..  22641, length:  22642, MSF: 00:02:00-05:03:66 }
    track 02 { audio, LBA:  22642 ..  62829, length:  40188, MSF: 05:03:67-13:59:54 }
    track 03 { audio, LBA:  62830 ..  85925, length:  23096, MSF: 13:59:55-19:07:50 }
    track 04 { audio, LBA:  85926 .. 108706, length:  22781, MSF: 19:07:51-24:11:31 }
    track 05 { audio, LBA: 108707 .. 127440, length:  18734, MSF: 24:11:32-28:21:15 }
    track 06 { audio, LBA: 127441 .. 149520, length:  22080, MSF: 28:21:16-33:15:45 }
    track 07 { audio, LBA: 149521 .. 185511, length:  35991, MSF: 33:15:46-41:15:36 }
    track 08 { audio, LBA: 185512 .. 198852, length:  13341, MSF: 41:15:37-44:13:27 }
    track 09 { audio, LBA: 198853 .. 208969, length:  10117, MSF: 44:13:28-46:28:19 }
    track 10 { audio, LBA: 208970 .. 219014, length:  10045, MSF: 46:28:20-48:42:14 }
  session 2
    track 11 {  data, LBA: 230415 .. 262720, length:  32306, MSF: 51:14:15-58:24:70 }

I extracted both session lead-ins using negative PLEXTOR readings and here are the snippets of both sessions decoded subchannel Q to illustrate.

session 1:

[LBA:   -235] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:58:65, zero: 00, A/P MSF: 41:15:37, crc: DCEE (+)
[LBA:   -234] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:58:66, zero: 00, A/P MSF: 41:15:37, crc: 0E00 (+)
[LBA:   -233] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:58:67, zero: 00, A/P MSF: 41:15:37, crc: 5FAA (+)
[LBA:   -232] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -231] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -230] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -229] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:58:71, zero: 00, A/P MSF: 44:13:28, crc: DB86 (+)
[LBA:   -228] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:58:72, zero: 00, A/P MSF: 44:13:28, crc: 0968 (+)
[LBA:   -227] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:58:73, zero: 00, A/P MSF: 44:13:28, crc: 58C2 (+)
[LBA:   -226] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -225] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -224] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -223] control: 0000, ADR: 1, tno: 00, P/I: 10, MSF: 99:59:02, zero: 00, A/P MSF: 46:28:20, crc: 9562 (+)
[LBA:   -222] control: 0000, ADR: 1, tno: 00, P/I: 10, MSF: 99:59:03, zero: 00, A/P MSF: 46:28:20, crc: C4C8 (+)
[LBA:   -221] control: 0000, ADR: 1, tno: 00, P/I: 10, MSF: 99:59:04, zero: 00, A/P MSF: 46:28:20, crc: 10AF (+)
[LBA:   -220] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -219] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -218] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -217] control: 0000, ADR: 1, tno: 00, P/I: A0, MSF: 99:59:08, zero: 00, A/P MSF: 01:00:00, crc: 76AC (+)
[LBA:   -216] control: 0000, ADR: 1, tno: 00, P/I: A0, MSF: 99:59:09, zero: 00, A/P MSF: 01:00:00, crc: 2706 (+)
[LBA:   -215] control: 0000, ADR: 1, tno: 00, P/I: A0, MSF: 99:59:10, zero: 00, A/P MSF: 01:00:00, crc: 01AA (+)
[LBA:   -214] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -213] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -212] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -211] control: 0000, ADR: 1, tno: 00, P/I: A1, MSF: 99:59:14, zero: 00, A/P MSF: 10:00:00, crc: 8710 (+)
[LBA:   -210] control: 0000, ADR: 1, tno: 00, P/I: A1, MSF: 99:59:15, zero: 00, A/P MSF: 10:00:00, crc: D6BA (+)
[LBA:   -209] control: 0000, ADR: 1, tno: 00, P/I: A1, MSF: 99:59:16, zero: 00, A/P MSF: 10:00:00, crc: 0454 (+)
[LBA:   -208] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -207] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -206] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -205] control: 0000, ADR: 1, tno: 00, P/I: A2, MSF: 99:59:20, zero: 00, A/P MSF: 48:42:15, crc: 4F83 (+)
[LBA:   -204] control: 0000, ADR: 1, tno: 00, P/I: A2, MSF: 99:59:21, zero: 00, A/P MSF: 48:42:15, crc: 1E29 (+)
[LBA:   -203] control: 0000, ADR: 1, tno: 00, P/I: A2, MSF: 99:59:22, zero: 00, A/P MSF: 48:42:15, crc: CCC7 (+)
[LBA:   -202] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -201] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -200] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -199] control: 0000, ADR: 1, tno: 00, P/I: 01, MSF: 99:59:26, zero: 00, A/P MSF: 00:02:00, crc: 02FB (+)
[LBA:   -198] control: 0000, ADR: 1, tno: 00, P/I: 01, MSF: 99:59:27, zero: 00, A/P MSF: 00:02:00, crc: 5351 (+)
[LBA:   -197] control: 0000, ADR: 1, tno: 00, P/I: 01, MSF: 99:59:28, zero: 00, A/P MSF: 00:02:00, crc: AA34 (+)
[LBA:   -196] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -195] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -194] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -193] control: 0000, ADR: 1, tno: 00, P/I: 02, MSF: 99:59:32, zero: 00, A/P MSF: 05:03:67, crc: AB7A (+)
[LBA:   -192] control: 0000, ADR: 1, tno: 00, P/I: 02, MSF: 99:59:33, zero: 00, A/P MSF: 05:03:67, crc: FAD0 (+)
[LBA:   -191] control: 0000, ADR: 1, tno: 00, P/I: 02, MSF: 99:59:34, zero: 00, A/P MSF: 05:03:67, crc: 2EB7 (+)
[LBA:   -190] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -189] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -188] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -187] control: 0000, ADR: 1, tno: 00, P/I: 03, MSF: 99:59:38, zero: 00, A/P MSF: 13:59:55, crc: 707D (+)
[LBA:   -186] control: 0000, ADR: 1, tno: 00, P/I: 03, MSF: 99:59:39, zero: 00, A/P MSF: 13:59:55, crc: 21D7 (+)
[LBA:   -185] control: 0000, ADR: 1, tno: 00, P/I: 03, MSF: 99:59:40, zero: 00, A/P MSF: 13:59:55, crc: DB62 (+)
[LBA:   -184] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -183] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -182] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -181] control: 0000, ADR: 1, tno: 00, P/I: 04, MSF: 99:59:44, zero: 00, A/P MSF: 19:07:51, crc: 3086 (+)
[LBA:   -180] control: 0000, ADR: 1, tno: 00, P/I: 04, MSF: 99:59:45, zero: 00, A/P MSF: 19:07:51, crc: 612C (+)
[LBA:   -179] control: 0000, ADR: 1, tno: 00, P/I: 04, MSF: 99:59:46, zero: 00, A/P MSF: 19:07:51, crc: B3C2 (+)
[LBA:   -178] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -177] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -176] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -175] control: 0000, ADR: 1, tno: 00, P/I: 05, MSF: 99:59:50, zero: 00, A/P MSF: 24:11:32, crc: 5B3E (+)
[LBA:   -174] control: 0000, ADR: 1, tno: 00, P/I: 05, MSF: 99:59:51, zero: 00, A/P MSF: 24:11:32, crc: 0A94 (+)
[LBA:   -173] control: 0000, ADR: 1, tno: 00, P/I: 05, MSF: 99:59:52, zero: 00, A/P MSF: 24:11:32, crc: D87A (+)
[LBA:   -172] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -171] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -170] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -169] control: 0000, ADR: 1, tno: 00, P/I: 06, MSF: 99:59:56, zero: 00, A/P MSF: 28:21:16, crc: B92F (+)
[LBA:   -168] control: 0000, ADR: 1, tno: 00, P/I: 06, MSF: 99:59:57, zero: 00, A/P MSF: 28:21:16, crc: E885 (+)
[LBA:   -167] control: 0000, ADR: 1, tno: 00, P/I: 06, MSF: 99:59:58, zero: 00, A/P MSF: 28:21:16, crc: 11E0 (+)
[LBA:   -166] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -165] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -164] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -163] control: 0000, ADR: 1, tno: 00, P/I: 07, MSF: 99:59:62, zero: 00, A/P MSF: 33:15:46, crc: B4CD (+)
[LBA:   -162] control: 0000, ADR: 1, tno: 00, P/I: 07, MSF: 99:59:63, zero: 00, A/P MSF: 33:15:46, crc: E567 (+)
[LBA:   -161] control: 0000, ADR: 1, tno: 00, P/I: 07, MSF: 99:59:64, zero: 00, A/P MSF: 33:15:46, crc: 3100 (+)
[LBA:   -160] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -159] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -158] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -157] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:59:68, zero: 00, A/P MSF: 41:15:37, crc: 068A (+)
[LBA:   -156] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:59:69, zero: 00, A/P MSF: 41:15:37, crc: 5720 (+)
[LBA:   -155] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:59:70, zero: 00, A/P MSF: 41:15:37, crc: 718C (+)
[LBA:   -154] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -153] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -152] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -151] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:59:74, zero: 00, A/P MSF: 44:13:28, crc: 2CE0 (+)
[LBA:   -150] control: 0000, ADR: 1, tno: 01, P/I: 00, MSF: 00:01:74, zero: 00, A/P MSF: 00:00:00, crc: B9AA (+)

session 2:

[LBA: 230242] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:11:67, zero: 00, A/P MSF: 51:14:15, crc: C00F (+)
[LBA: 230243] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:11:68, zero: 00, A/P MSF: 51:14:15, crc: 396A (+)
[LBA: 230244] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:11:69, zero: 00, A/P MSF: 51:14:15, crc: 68C0 (+)
[LBA: 230245] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:11:70, zero: 00, A/P MSF: 11:20:00, crc: A806 (+)
[LBA: 230246] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:11:71, zero: 00, A/P MSF: 11:20:00, crc: F9AC (+)
[LBA: 230247] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:11:72, zero: 00, A/P MSF: 11:20:00, crc: 2B42 (+)
[LBA: 230248] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:11:73, zero: 00, A/P MSF: 11:00:00, crc: 4FA9 (+)
[LBA: 230249] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:11:74, zero: 00, A/P MSF: 11:00:00, crc: 9BCE (+)
[LBA: 230250] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:00, zero: 00, A/P MSF: 11:00:00, crc: FB94 (+)
[LBA: 230251] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:01, zero: 00, A/P MSF: 58:24:71, crc: 77D1 (+)
[LBA: 230252] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:02, zero: 00, A/P MSF: 58:24:71, crc: A53F (+)
[LBA: 230253] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:03, zero: 00, A/P MSF: 58:24:71, crc: F495 (+)
[LBA: 230254] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:12:04, zero: 00, A/P MSF: 51:14:15, crc: 2E36 (+)
[LBA: 230255] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:12:05, zero: 00, A/P MSF: 51:14:15, crc: 7F9C (+)
[LBA: 230256] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:12:06, zero: 00, A/P MSF: 51:14:15, crc: AD72 (+)
[LBA: 230257] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:12:07, zero: 00, A/P MSF: 11:20:00, crc: 1AB2 (+)
[LBA: 230258] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:12:08, zero: 00, A/P MSF: 11:20:00, crc: E3D7 (+)
[LBA: 230259] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:12:09, zero: 00, A/P MSF: 11:20:00, crc: B27D (+)
[LBA: 230260] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:10, zero: 00, A/P MSF: 11:00:00, crc: A190 (+)
[LBA: 230261] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:11, zero: 00, A/P MSF: 11:00:00, crc: F03A (+)
[LBA: 230262] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:12, zero: 00, A/P MSF: 11:00:00, crc: 22D4 (+)
[LBA: 230263] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:13, zero: 00, A/P MSF: 58:24:71, crc: AE91 (+)
[LBA: 230264] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:14, zero: 00, A/P MSF: 58:24:71, crc: 7AF6 (+)
[LBA: 230265] control: 0100, ADR: 1, tno: 11, P/I: 00, MSF: 00:01:74, zero: 00, A/P MSF: 51:12:15, crc: 26C5 (+)

As you can see, session 1 lead-in lists only tracks 1-10 and session 2 lead-in lists only track 11.
(TOC entries in lead-in are usually cyclically repeated in a pack of 3)

I see this on all other pressed multisession discs I own (up to 10) and to me it makes total sense as this looks like compatibility thing for earlier players which don't support multisession and session 2 tracks (usually data) are totally "invisible" there.
When you request disc TOC on a multisession supported drive, it gets both sessions TOC and merges it.

Another directly related side effect is that drives often have problems with reading multisession CD-TEXT (CD-TEXT data is stored in TOC R-W subchannels). I have one such disc with CD-TEXT defined in both sessions and earlier PLEXTOR drives (PX-W5224TA) are able to get only CD-TEXT stored in the first session while later DVD PLEXTOR drives have no problem extracting both session data.

Whole disc decoded subchannel Q for the reference:
https://www.dropbox.com/s/h5cubuo6m95wo … q.zip?dl=0

42

(3,497 replies, posted in General discussion)

sarami, some Plextor DVD drives have different C2 offsets which are not sector aligned. I took a couple of C2 discs with negative and positive write offsets and a mix of audio / data and tested it on the drives I have here. I compared C2 dump with known good dump of the same disc and matched it with C2 error vector.
Starting from PX-712A, it takes drive 2 more samples (1 C2 byte is 8 bit) to calculate C2:

PLEXTOR - CD-R PX-W4012A v1.07, C2 offset: 294
PLEXTOR - CD-R PX-W4824A v1.07, C2 offset: 294
PLEXTOR - CD-R PX-W5224A v1.04, C2 offset: 294
PLEXTOR - DVDR PX-708A v1.12, C2 offset: 294
PLEXTOR - DVDR PX-712A v1.09, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.11, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.58, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.59, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.5A, C2 offset: 295
PLEXTOR - DVDR PX-755A v1.08, C2 offset: 295
PLEXTOR - DVDR PX-760A v1.07, C2 offset: 295

(C2 offset is in bytes)

This is a problem because for PX-712A or later, any C2 error in the last 8 bytes of any sector will slip unnoticed into the final dump.

Should be an easy fix, instead of pre-reading 2 sectors like you do for Plextor, you can pre-read 3 sectors at a time and offset it based on a drive model.

43

(3,497 replies, posted in General discussion)

sarami,
There is a CD-TEXT DIC issue I want to report.
Due to CD-TEXT space limitations (stored in R-W subchannel), sometimes they use TAB symbol "\t" ASCII and "\t\t" WIDE to refer to previous non-empty text, this is not implemented in DIC, thus such fields have spaces instead.
Example: http://redump.org/disc/87491/

There are more things that I have to report but I'll do that a little bit later.

44

(3,497 replies, posted in General discussion)

sarami wrote:

CD-TEXT can store 8 languages.
BLOCK 1 language uses _alt.cue, BLOCK 2 language uses _alt2.cue ... BLOCK 7 language uses _alt7.cue

Got it. So I guess for this particular disc English block is empty and all the meaningful data is in the Japanese CD-TEXT block.
My next question is, what encoding gets dumped to the BLOCK 1 cue?
I'm getting garbage there but I don't have JP fonts installed and no Japanese locale. We would need that corrected (I guess manually in a cue sheet) in order to be added to redump.

EDIT: Nevermind, notepad++ displays everything correctly, it was my viewer which had a problem, thank you!

45

(3,497 replies, posted in General discussion)

sarami, I got "_alt.cue" and "_imgAlt.cue" when dumping "Suikoden Tierkreis: Digital Artbook & Soundtrack" Enhanced CD.
The cue difference is empty TITLE / PERFORMER vs what I think is Japanese encoding.
Can you please take a look?

Logs: https://www.dropbox.com/s/fexbx20talw1h … 29.7z?dl=0

46

(3,497 replies, posted in General discussion)

sarami wrote:
superg wrote:

When I use DIC supplied with MPF, Build 20210401T101950

I only support the latest version. It's now 20211001T112852.

I downloaded the newest DIC version from the website and the problem is still there. I did 2 dumps of clean GameShark 2 disc and I'm getting different results (4476 vs 4477) errors.
Let me know if logs would be useful, I'll upload them.

47

(3,497 replies, posted in General discussion)

sarami wrote:
superg wrote:

When I use DIC supplied with MPF, Build 20210401T101950

I only support the latest version. It's now 20211001T112852.

I will use that.

48

(3,497 replies, posted in General discussion)

sarami wrote:

What is the problem? My unlicensed is no problem.

There are two problems:
When I use DIC supplied with MPF, Build 20210401T101950, if that helps, and dumping PS2 GameShark disc with /sf flag, after initial pass it tries to re-read some C2 sectors from BIG.DAT and as it never gets a match, dumping fails.
For other PS2 unlicensed I often get improper dumps with wrong error count, it deviates from 4475 errors count for such discs. All these discs are clean and I currently dump them using the earlier DIC version which always yields the exact 4475 errors. That version you built after I mentioned this earlier error count deviation and it always works for me.

49

(3,497 replies, posted in General discussion)

matura713 wrote:

Total errors: 4497

This error count looks familiar to me. We get a very similar number of C2 errors at start for many PS2 unlicensed discs.
Did you try to dump with /sf?

50

(3,497 replies, posted in General discussion)

sarami wrote:

"UPX" is shown in IMAGE_SECTION_HEADER of SETUP.EXE. Please check if it is compressed by UPX or not.
If yes, DIC is not supported it yet. 20210401 version does not output IMAGE_EXPORT_DIRECTORY yet.

Is there any way to disable this checking so DIC doesn't crash?