51 Reply by superg

(17 replies, posted in General discussion)

F1ReB4LL wrote:

Wrong, dumpers must tweak the offset value to include all the data if possible, also DIC should do this automatically, IIRC.

True, we want dumpers to do that, but the status quo is that they really don't. They do whatever DIC does by default.

F1ReB4LL wrote:

Sometimes both pregap and leadout have data, needs manual analysis in this case (usually it's about tweaking the offset to include the leadout data + extracting the pregap, like it's done in http://redump.org/disc/6695/, but I guess it depends on which of the pregap and the leadout portions of data is smaller - the smaller one needs the offset tweaking, the larger one needs the extraction).

Even if we separately extract pregap.bin, it can be automated. I don't necessarily like this approach as pregap data usually belongs to track 1. It's fine if it's "almost silence" but Intothisworld owns a disc where it's HTOA in there.

F1ReB4LL wrote:

There are thousands of games with the same problem, so it's mostly about their bad mastering rather than direct offset issues.

True, but at least we have a way for them to determine a proper offset value using data track.

F1ReB4LL wrote:

Crazy. I think that matching EAC and AccurateRip databases for the 'offset 0' audio tracks is a better benefit compared to some custom aligning.

We can still have hashes or some other info calculated for offset 0 to satisfy the compatibility things like EAC/AccurateRip.

F1ReB4LL wrote:

Also, what happens if all the tracks have big portions of silence at their beginnings and endings? Does it shift the offset to force the first track to start from its first nonzero byte?

No no, it's not like that. The current logic doesn't align to first non-zero sample. I calculate perfect offset range, but if offset 0 belongs to that range, it's has a higher priority. The disc you describle with the big portion of zeroes at the beginning and end will have a big slack which will most likely include offset 0.

F1ReB4LL wrote:

It will NOT work for Jaguar discs at all, because its 'data' (the INDEX 01 of the second track of the second session) must start with its magic (according to the official Jaguar CD format description documents), but if you align according to the magic, the first track of the first session for a good half of Jaguar CD releases gets shifted to the first pregap - and in your case, if you align according to the non-zero data and silences, its magic will be way off its proper position.

It works for the Myst Demo I own. I can't speak for the generic case. My Myst Demo aligns to that magic using the logic I described:

detecting offset
audio silence detection... done
perfect audio offset (silence level: 0): [-2201 .. -401]
disc write offset: -401
warning: pre-gap audio is incomplete (session: 2, errors: 371)
detection complete (time: 14s)

As offset 0 is not included in the perfect range here, -401 is closest to 0 so it gets chosen and that perfectly coincides with aligned by magic.
By default I have it implemented using magic, like DIC does. I just remember you mentioned you're not fond of the current way so if you think we can do something better for Jaguar CD - let me know.

But the algo is still useful for PSXGS as if you align by magic, it cuts through other data portions.

Go to forum: General discussion

52 Reply by superg

(17 replies, posted in General discussion)

Jackal wrote:

The logic as described sounds good to me. I'm assuming the offset correction is always in samples, so a multiple of 4 bytes?

Yes, I analyze it as multiples of 4 bytes (two signed 16-bit sample values).

Jackal wrote:

I would also like to know if your method is able to find the true "perfect" offset for discs such as these: http://redump.org/discs/quicksearch/wri … /audio-cd/ + http://redump.org/discs/quicksearch/det … region/Eu/ where aligning either left or right to the first byte gives a "common" write offset value that is consistent with the ringcode.

Don't have any from this list unfortunately, you can try redumper if you have any of these, for each audio disc it will output something like

detecting offset
audio silence detection... done
perfect audio offset (silence level: 5): [+736 .. +22675]
disc write offset: +736
warning: pre-gap audio contains non-zero data, preserving (session: 1, leading zeroes: 87473, non-zeroes: 727/88200)
detection complete (time: 75s)

And after that "redumper split --force-offset=22675 --overwrite" to align it.

Jackal wrote:

Maybe this should be an extra step? So validate for common offset values when shifted to first or last non-zero byte before going with 0 offset or whatever else the logic would decide on. Would be great if you could test that using some of the images.

I also thought about something like this. Left shift (or right shift) to the first non zero byte and calculate some hash and use it as a universal checksum which allows us to match same audio discs mastered with a different offset. For instance for two such discs from here: http://redump.org/disc/77301, if I right-align, both dumps match.

Go to forum: General discussion

53 Topic by superg

(17 replies, posted in General discussion)

EDIT 12/03/2022:

Final Audio CD offset correction algorithm (technical):
1. if there is one and only one possible disc write offset, applying which perfectly aligns audio silence (level: 0) ranges with TOC index 0 ranges, use it
2. else if there is non-zero data in lead-out and that data can be fully shifted out (left) without spanning non-zero data into lead-in, correct offset with a minimum shift required
3. else if there is non-zero data in lead-in and that data can be fully shifted out (right) without spanning non-zero data into lead-out, correct offset with a minimum shift required
4. else apply offset 0

Dump format specification notes:
Regardless of the applied offset, we always operate on LBA [ 0 .. lead-out ) range when we perform a track split. This is also true for data discs. If, as a result of mastering, there is non-zero data either in lead-in ( -inf .. 0 ) or lead-out [ lead-out .. +inf ), this is preserved in separate files. For discs with data tracks, non-zero data means the descrambled data portion of a sector (fields such as sync, MSF, mode, subheader, ECC/EDC is excluded). The resulting lead-in/lead-out files should be trimmed to the minimum non-zero data size, lead-in from the front and lead-out from the back, but they should remain sector-aligned (size divisible by 2352).

Considerations for matching Audio CD with different write offsets
For each disc, dumping software can generate a checksum/hash for the non-zero sample span of the data e.g. aligned to 4-byte CD sample size. Such hash can be used for the disc identification purpose as well as for Audio CD different write offset matching as for such cases the hash would be the same. This applies to both audio and data discs. I suggest to use a longer SHA-1 hash and not CRC32 just to be future proof. It's quite likely that we will get a couple of CRC32 collisions in ~100K discs currently in the DB. As a side perk, it can also serve as a unique disc ID which can be easily looked up in the database, if such capability will be ever implemented at redump.org. In any way, I would like to have such a hash in Audio CD entries just for write offset matching.

END OF EDIT, the rest information here is kept for the reference and archival purpose.




This topic is to clarify and decide on how we manage disc write offset for audio discs.

The current status quo is that we always dump audio discs with offset 0 as there is no reliable reference point in the audio stream (in contrary to a data track) which can be used to determine the offset. This approach has a number of disadvantages, such as:
* shifted audio data in pre-gap / lead-out which we don't currently preserve
* ocassional imperfect track split which cuts in the middle of audio tracks e.g. you hear a bit of the next track in the end of a current one or previous track in the beginning of the current one

I believe I solved both of these problems in redumper. Let me define some terminology first.
Perfect Audio Offset is a disc write offset which guarantees that no data is shifted into lead-out and guarantees track split which doesn't cut into a middle of a track.

Perfect Audio Offset implementation details
For a given audio disc, I build a silence map based on TOC/subchannel information, essentially it's INDEX 00 CUE entries which are almost always empty (silent). As a next step, I build a similar silence map based on an audio stream. Finally, For each offset within [-150 .. lead-out] constrained range, I try to line up these two maps in a way that TOC/subchannel based one fits into an audio stream. If it fits, it's a perfect audio offset.

The current audio offset logic
favor offset value from perfect offset range if available
if multiple perfect offset values available (range), try to shift data out of pre-gap (if needed) if still within a perfect range
otherwise if no data in pre-gap, favor offset 0 if it belongs to the perfect range
finally, if offset 0 doesn't belong to the perfect range, use a value closest to 0 within a perfect range
if no any perfect offset available, try to shift data out of lead-out and pre-gap (only if we can get rid of full pre-gap data) if that doesn't lead to data loss.

or pseudocode:

if(perfect_audio_offset_available)
{
    if(perfect_offset_single_value)
        use_offset_value();
    else if(perfect_offset_value_range)
    {
        if(data_in_pregap)
        {
            if(enough_space_to_get_rid_of_whole_data_in_pregap AND still_within_perfect_range)
                move_minimum_data_right();
        }
        else if(zero_offset_belongs_to_perfect_range)
            use_zero_offset();
        else
            choose_the_offset_closest_to_zero();
    }
}
else
{
    if(data_in_leadout)
        move_minimum_data_left();
    else if(data_in_pregap AND enough_space_to_get_rid_of_whole_data_in_pregap)
        move_minimum_data_right();
}

Pre-Gap notes
Based from the discs I own and discs I have an access to, most audio discs with data in pre-gap is not a result of a write offset but rather a way it was mastered. It's often values close to silence but not zeroed and sometimes it's a part of a hidden track (HTOA). In cases like this, there is no way to move out that data fully out of pre-gap without it shifting into lead-out as there is not much space (it's common to have 1-2 seconds of pre-gap data audio which is a lot). I can definitely say it worth preserving by extending track 1 fixed 150 sectors back for all such cases, with, optionally, marking that in CUE? Before anyone says it's stupid, DIC already does a similar track 1 extend for CDi-Ready discs.

Statistics
With new method, I redumped all my audio discs and sadikyo redumped some of his. I shared new dumper version with Intothisworld and he shared it with ppltoast but I have yet to get some results from them. The current merged detail statistics is available here:
https://docs.google.com/spreadsheets/d/ … sp=sharing

TL;DR
73 discs - match redump (offset 0 is one of the perfect offsets)
9 discs - no perfect offset found (no distinctive silence in index 0 or no index 0 entries at all), offset 0 used so matches redump
7 discs - only one perfect offset found and it's true offset value used to master disc with, will require DB update
27 discs - perfect offset range excludes 0, will require DB update
3 discs - have pre-gap data which is impossible to fully shift out and it's not currently preserved, will require DB update

Side Effects
Given method works really well for PSX GameShark Update discs and Jaguar discs without relying on magic numbers and we get a perfect splits there too.

More Side Effects
If we left-align (or right-align) offset based on a perfect offset range, we will be getting matches for audio discs with the same data but different write offset, such as:
http://redump.org/disc/77301

I would like to hear your opinions on what do you think of it, let's discuss.

Go to forum: General discussion

54 Reply by superg

(3,534 replies, posted in General discussion)

sarami, just FYI, two CDi DIC dumps have scrambled sectors close to the end of the track:
http://redump.org/disc/24962/ - 1 sector
http://redump.org/disc/78871/ - 2 sectors
We've corrected that in the database but I guess that have to be fixed on the DIC side.

Go to forum: General discussion

55 Reply by superg

(2 replies, posted in General discussion)

scsi_wuzzy wrote:

It would be nice if we could ultimately end up with CD images like the ones Near described. Near's article about the CD format hypothesizes about a CD format that just stores all the lead-in sectors (including subchannels) so that we could have a single file raw image format that embedded the TOC the same way the real disc does. Such an image format would handle multisession discs easily.

I already have something very close to that but it's not redump scope really. You can't get all the lead-in sectors because of the mechanical limitations of the drive (how close laser head assembly can get to the center of the disc). Plextor allows us to start reading from some place in the TOC and that is good enough given that TOC is repeating itself.
Custom drive firmware would be an ambitious project for sure but that's very complex task.

Go to forum: General discussion

56 Reply by superg

(4 replies, posted in General discussion)

scsi_wuzzy wrote:
F1ReB4LL wrote:

If you're using one of the latest DIC versions, it has both scrambled and descrambled image checksums in the "_disc.txt" file, you can scramble the descrambled image back and verify its checksum, if it matches - no reason to store the scrambled file itself.

I've thought about doing it this way and then just storing deltas for when the scrambled data doesn't match exactly (since the deltas would allow creation of the scrambled data from the unscrambled data and would typically be much smaller than the entire scrambled image). That's probably what I'll end up doing, but I also may look into adding metadata to an archival format like Aaru if it's possible.

I wanted to make sure there wasn't a better way to do it using some existing, standardized approach before I came up with my own solution. It sounds like there's not, unfortunately.

Scrambling is a simple math involving shift register, it's a trivial implementation.
Delta is ineffective here and it will be as big as the data track (data is scrambled, audio is unscrambled).
The most annoying thing in this conversion process is to actually know which sector is audio and which is data, scm doesn't have that info so you will have to extract it from TOC to be absolutely sure (you can go by data sync header but there is no guarantee there won't be such sequence in audio sector).

Go to forum: General discussion

57 Topic by superg

(2 replies, posted in General discussion)

I want to share my findings about general layout of a multisession disc.
Apparently what I see here contradicts with common redump knowledge so this might be important.

In particular, my findings show that each session has it's own TOC in lead-in which lists track entries only for that session.
I will demonstrate that on http://redump.org/disc/75764/

The disc has the following TOC:

TOC:
  session 1
    track 01 { audio, LBA:      0 ..  22641, length:  22642, MSF: 00:02:00-05:03:66 }
    track 02 { audio, LBA:  22642 ..  62829, length:  40188, MSF: 05:03:67-13:59:54 }
    track 03 { audio, LBA:  62830 ..  85925, length:  23096, MSF: 13:59:55-19:07:50 }
    track 04 { audio, LBA:  85926 .. 108706, length:  22781, MSF: 19:07:51-24:11:31 }
    track 05 { audio, LBA: 108707 .. 127440, length:  18734, MSF: 24:11:32-28:21:15 }
    track 06 { audio, LBA: 127441 .. 149520, length:  22080, MSF: 28:21:16-33:15:45 }
    track 07 { audio, LBA: 149521 .. 185511, length:  35991, MSF: 33:15:46-41:15:36 }
    track 08 { audio, LBA: 185512 .. 198852, length:  13341, MSF: 41:15:37-44:13:27 }
    track 09 { audio, LBA: 198853 .. 208969, length:  10117, MSF: 44:13:28-46:28:19 }
    track 10 { audio, LBA: 208970 .. 219014, length:  10045, MSF: 46:28:20-48:42:14 }
  session 2
    track 11 {  data, LBA: 230415 .. 262720, length:  32306, MSF: 51:14:15-58:24:70 }

I extracted both session lead-ins using negative PLEXTOR readings and here are the snippets of both sessions decoded subchannel Q to illustrate.

session 1:

[LBA:   -235] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:58:65, zero: 00, A/P MSF: 41:15:37, crc: DCEE (+)
[LBA:   -234] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:58:66, zero: 00, A/P MSF: 41:15:37, crc: 0E00 (+)
[LBA:   -233] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:58:67, zero: 00, A/P MSF: 41:15:37, crc: 5FAA (+)
[LBA:   -232] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -231] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -230] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -229] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:58:71, zero: 00, A/P MSF: 44:13:28, crc: DB86 (+)
[LBA:   -228] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:58:72, zero: 00, A/P MSF: 44:13:28, crc: 0968 (+)
[LBA:   -227] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:58:73, zero: 00, A/P MSF: 44:13:28, crc: 58C2 (+)
[LBA:   -226] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -225] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -224] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -223] control: 0000, ADR: 1, tno: 00, P/I: 10, MSF: 99:59:02, zero: 00, A/P MSF: 46:28:20, crc: 9562 (+)
[LBA:   -222] control: 0000, ADR: 1, tno: 00, P/I: 10, MSF: 99:59:03, zero: 00, A/P MSF: 46:28:20, crc: C4C8 (+)
[LBA:   -221] control: 0000, ADR: 1, tno: 00, P/I: 10, MSF: 99:59:04, zero: 00, A/P MSF: 46:28:20, crc: 10AF (+)
[LBA:   -220] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -219] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -218] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -217] control: 0000, ADR: 1, tno: 00, P/I: A0, MSF: 99:59:08, zero: 00, A/P MSF: 01:00:00, crc: 76AC (+)
[LBA:   -216] control: 0000, ADR: 1, tno: 00, P/I: A0, MSF: 99:59:09, zero: 00, A/P MSF: 01:00:00, crc: 2706 (+)
[LBA:   -215] control: 0000, ADR: 1, tno: 00, P/I: A0, MSF: 99:59:10, zero: 00, A/P MSF: 01:00:00, crc: 01AA (+)
[LBA:   -214] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -213] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -212] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -211] control: 0000, ADR: 1, tno: 00, P/I: A1, MSF: 99:59:14, zero: 00, A/P MSF: 10:00:00, crc: 8710 (+)
[LBA:   -210] control: 0000, ADR: 1, tno: 00, P/I: A1, MSF: 99:59:15, zero: 00, A/P MSF: 10:00:00, crc: D6BA (+)
[LBA:   -209] control: 0000, ADR: 1, tno: 00, P/I: A1, MSF: 99:59:16, zero: 00, A/P MSF: 10:00:00, crc: 0454 (+)
[LBA:   -208] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -207] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -206] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -205] control: 0000, ADR: 1, tno: 00, P/I: A2, MSF: 99:59:20, zero: 00, A/P MSF: 48:42:15, crc: 4F83 (+)
[LBA:   -204] control: 0000, ADR: 1, tno: 00, P/I: A2, MSF: 99:59:21, zero: 00, A/P MSF: 48:42:15, crc: 1E29 (+)
[LBA:   -203] control: 0000, ADR: 1, tno: 00, P/I: A2, MSF: 99:59:22, zero: 00, A/P MSF: 48:42:15, crc: CCC7 (+)
[LBA:   -202] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -201] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -200] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -199] control: 0000, ADR: 1, tno: 00, P/I: 01, MSF: 99:59:26, zero: 00, A/P MSF: 00:02:00, crc: 02FB (+)
[LBA:   -198] control: 0000, ADR: 1, tno: 00, P/I: 01, MSF: 99:59:27, zero: 00, A/P MSF: 00:02:00, crc: 5351 (+)
[LBA:   -197] control: 0000, ADR: 1, tno: 00, P/I: 01, MSF: 99:59:28, zero: 00, A/P MSF: 00:02:00, crc: AA34 (+)
[LBA:   -196] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -195] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -194] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -193] control: 0000, ADR: 1, tno: 00, P/I: 02, MSF: 99:59:32, zero: 00, A/P MSF: 05:03:67, crc: AB7A (+)
[LBA:   -192] control: 0000, ADR: 1, tno: 00, P/I: 02, MSF: 99:59:33, zero: 00, A/P MSF: 05:03:67, crc: FAD0 (+)
[LBA:   -191] control: 0000, ADR: 1, tno: 00, P/I: 02, MSF: 99:59:34, zero: 00, A/P MSF: 05:03:67, crc: 2EB7 (+)
[LBA:   -190] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -189] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -188] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -187] control: 0000, ADR: 1, tno: 00, P/I: 03, MSF: 99:59:38, zero: 00, A/P MSF: 13:59:55, crc: 707D (+)
[LBA:   -186] control: 0000, ADR: 1, tno: 00, P/I: 03, MSF: 99:59:39, zero: 00, A/P MSF: 13:59:55, crc: 21D7 (+)
[LBA:   -185] control: 0000, ADR: 1, tno: 00, P/I: 03, MSF: 99:59:40, zero: 00, A/P MSF: 13:59:55, crc: DB62 (+)
[LBA:   -184] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -183] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -182] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -181] control: 0000, ADR: 1, tno: 00, P/I: 04, MSF: 99:59:44, zero: 00, A/P MSF: 19:07:51, crc: 3086 (+)
[LBA:   -180] control: 0000, ADR: 1, tno: 00, P/I: 04, MSF: 99:59:45, zero: 00, A/P MSF: 19:07:51, crc: 612C (+)
[LBA:   -179] control: 0000, ADR: 1, tno: 00, P/I: 04, MSF: 99:59:46, zero: 00, A/P MSF: 19:07:51, crc: B3C2 (+)
[LBA:   -178] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -177] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -176] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -175] control: 0000, ADR: 1, tno: 00, P/I: 05, MSF: 99:59:50, zero: 00, A/P MSF: 24:11:32, crc: 5B3E (+)
[LBA:   -174] control: 0000, ADR: 1, tno: 00, P/I: 05, MSF: 99:59:51, zero: 00, A/P MSF: 24:11:32, crc: 0A94 (+)
[LBA:   -173] control: 0000, ADR: 1, tno: 00, P/I: 05, MSF: 99:59:52, zero: 00, A/P MSF: 24:11:32, crc: D87A (+)
[LBA:   -172] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -171] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -170] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -169] control: 0000, ADR: 1, tno: 00, P/I: 06, MSF: 99:59:56, zero: 00, A/P MSF: 28:21:16, crc: B92F (+)
[LBA:   -168] control: 0000, ADR: 1, tno: 00, P/I: 06, MSF: 99:59:57, zero: 00, A/P MSF: 28:21:16, crc: E885 (+)
[LBA:   -167] control: 0000, ADR: 1, tno: 00, P/I: 06, MSF: 99:59:58, zero: 00, A/P MSF: 28:21:16, crc: 11E0 (+)
[LBA:   -166] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -165] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -164] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -163] control: 0000, ADR: 1, tno: 00, P/I: 07, MSF: 99:59:62, zero: 00, A/P MSF: 33:15:46, crc: B4CD (+)
[LBA:   -162] control: 0000, ADR: 1, tno: 00, P/I: 07, MSF: 99:59:63, zero: 00, A/P MSF: 33:15:46, crc: E567 (+)
[LBA:   -161] control: 0000, ADR: 1, tno: 00, P/I: 07, MSF: 99:59:64, zero: 00, A/P MSF: 33:15:46, crc: 3100 (+)
[LBA:   -160] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -159] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -158] control: 0000, ADR: 5, 00 C0 00 00 00 00 95 00 00, crc: 17A5 (+)
[LBA:   -157] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:59:68, zero: 00, A/P MSF: 41:15:37, crc: 068A (+)
[LBA:   -156] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:59:69, zero: 00, A/P MSF: 41:15:37, crc: 5720 (+)
[LBA:   -155] control: 0000, ADR: 1, tno: 00, P/I: 08, MSF: 99:59:70, zero: 00, A/P MSF: 41:15:37, crc: 718C (+)
[LBA:   -154] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -153] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -152] control: 0000, ADR: 5, 00 B0 51 12 15 02 58 24 71, crc: 7EDC (+)
[LBA:   -151] control: 0000, ADR: 1, tno: 00, P/I: 09, MSF: 99:59:74, zero: 00, A/P MSF: 44:13:28, crc: 2CE0 (+)
[LBA:   -150] control: 0000, ADR: 1, tno: 01, P/I: 00, MSF: 00:01:74, zero: 00, A/P MSF: 00:00:00, crc: B9AA (+)

session 2:

[LBA: 230242] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:11:67, zero: 00, A/P MSF: 51:14:15, crc: C00F (+)
[LBA: 230243] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:11:68, zero: 00, A/P MSF: 51:14:15, crc: 396A (+)
[LBA: 230244] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:11:69, zero: 00, A/P MSF: 51:14:15, crc: 68C0 (+)
[LBA: 230245] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:11:70, zero: 00, A/P MSF: 11:20:00, crc: A806 (+)
[LBA: 230246] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:11:71, zero: 00, A/P MSF: 11:20:00, crc: F9AC (+)
[LBA: 230247] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:11:72, zero: 00, A/P MSF: 11:20:00, crc: 2B42 (+)
[LBA: 230248] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:11:73, zero: 00, A/P MSF: 11:00:00, crc: 4FA9 (+)
[LBA: 230249] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:11:74, zero: 00, A/P MSF: 11:00:00, crc: 9BCE (+)
[LBA: 230250] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:00, zero: 00, A/P MSF: 11:00:00, crc: FB94 (+)
[LBA: 230251] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:01, zero: 00, A/P MSF: 58:24:71, crc: 77D1 (+)
[LBA: 230252] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:02, zero: 00, A/P MSF: 58:24:71, crc: A53F (+)
[LBA: 230253] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:03, zero: 00, A/P MSF: 58:24:71, crc: F495 (+)
[LBA: 230254] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:12:04, zero: 00, A/P MSF: 51:14:15, crc: 2E36 (+)
[LBA: 230255] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:12:05, zero: 00, A/P MSF: 51:14:15, crc: 7F9C (+)
[LBA: 230256] control: 0100, ADR: 1, tno: 00, P/I: 11, MSF: 51:12:06, zero: 00, A/P MSF: 51:14:15, crc: AD72 (+)
[LBA: 230257] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:12:07, zero: 00, A/P MSF: 11:20:00, crc: 1AB2 (+)
[LBA: 230258] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:12:08, zero: 00, A/P MSF: 11:20:00, crc: E3D7 (+)
[LBA: 230259] control: 0100, ADR: 1, tno: 00, P/I: A0, MSF: 51:12:09, zero: 00, A/P MSF: 11:20:00, crc: B27D (+)
[LBA: 230260] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:10, zero: 00, A/P MSF: 11:00:00, crc: A190 (+)
[LBA: 230261] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:11, zero: 00, A/P MSF: 11:00:00, crc: F03A (+)
[LBA: 230262] control: 0100, ADR: 1, tno: 00, P/I: A1, MSF: 51:12:12, zero: 00, A/P MSF: 11:00:00, crc: 22D4 (+)
[LBA: 230263] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:13, zero: 00, A/P MSF: 58:24:71, crc: AE91 (+)
[LBA: 230264] control: 0100, ADR: 1, tno: 00, P/I: A2, MSF: 51:12:14, zero: 00, A/P MSF: 58:24:71, crc: 7AF6 (+)
[LBA: 230265] control: 0100, ADR: 1, tno: 11, P/I: 00, MSF: 00:01:74, zero: 00, A/P MSF: 51:12:15, crc: 26C5 (+)

As you can see, session 1 lead-in lists only tracks 1-10 and session 2 lead-in lists only track 11.
(TOC entries in lead-in are usually cyclically repeated in a pack of 3)

I see this on all other pressed multisession discs I own (up to 10) and to me it makes total sense as this looks like compatibility thing for earlier players which don't support multisession and session 2 tracks (usually data) are totally "invisible" there.
When you request disc TOC on a multisession supported drive, it gets both sessions TOC and merges it.

Another directly related side effect is that drives often have problems with reading multisession CD-TEXT (CD-TEXT data is stored in TOC R-W subchannels). I have one such disc with CD-TEXT defined in both sessions and earlier PLEXTOR drives (PX-W5224TA) are able to get only CD-TEXT stored in the first session while later DVD PLEXTOR drives have no problem extracting both session data.

Whole disc decoded subchannel Q for the reference:
https://www.dropbox.com/s/h5cubuo6m95wo … q.zip?dl=0

Go to forum: General discussion

58 Reply by superg

(3,534 replies, posted in General discussion)

sarami, some Plextor DVD drives have different C2 offsets which are not sector aligned. I took a couple of C2 discs with negative and positive write offsets and a mix of audio / data and tested it on the drives I have here. I compared C2 dump with known good dump of the same disc and matched it with C2 error vector.
Starting from PX-712A, it takes drive 2 more samples (1 C2 byte is 8 bit) to calculate C2:

PLEXTOR - CD-R PX-W4012A v1.07, C2 offset: 294
PLEXTOR - CD-R PX-W4824A v1.07, C2 offset: 294
PLEXTOR - CD-R PX-W5224A v1.04, C2 offset: 294
PLEXTOR - DVDR PX-708A v1.12, C2 offset: 294
PLEXTOR - DVDR PX-712A v1.09, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.11, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.58, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.59, C2 offset: 295
PLEXTOR - DVDR PX-716A v1.5A, C2 offset: 295
PLEXTOR - DVDR PX-755A v1.08, C2 offset: 295
PLEXTOR - DVDR PX-760A v1.07, C2 offset: 295

(C2 offset is in bytes)

This is a problem because for PX-712A or later, any C2 error in the last 8 bytes of any sector will slip unnoticed into the final dump.

Should be an easy fix, instead of pre-reading 2 sectors like you do for Plextor, you can pre-read 3 sectors at a time and offset it based on a drive model.

Go to forum: General discussion

59 Reply by superg

(3,534 replies, posted in General discussion)

sarami,
There is a CD-TEXT DIC issue I want to report.
Due to CD-TEXT space limitations (stored in R-W subchannel), sometimes they use TAB symbol "\t" ASCII and "\t\t" WIDE to refer to previous non-empty text, this is not implemented in DIC, thus such fields have spaces instead.
Example: http://redump.org/disc/87491/

There are more things that I have to report but I'll do that a little bit later.

Go to forum: General discussion

60 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:

CD-TEXT can store 8 languages.
BLOCK 1 language uses _alt.cue, BLOCK 2 language uses _alt2.cue ... BLOCK 7 language uses _alt7.cue

Got it. So I guess for this particular disc English block is empty and all the meaningful data is in the Japanese CD-TEXT block.
My next question is, what encoding gets dumped to the BLOCK 1 cue?
I'm getting garbage there but I don't have JP fonts installed and no Japanese locale. We would need that corrected (I guess manually in a cue sheet) in order to be added to redump.

EDIT: Nevermind, notepad++ displays everything correctly, it was my viewer which had a problem, thank you!

Go to forum: General discussion

61 Reply by superg

(3,534 replies, posted in General discussion)

sarami, I got "_alt.cue" and "_imgAlt.cue" when dumping "Suikoden Tierkreis: Digital Artbook & Soundtrack" Enhanced CD.
The cue difference is empty TITLE / PERFORMER vs what I think is Japanese encoding.
Can you please take a look?

Logs: https://www.dropbox.com/s/fexbx20talw1h … 29.7z?dl=0

Go to forum: General discussion

62 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:
superg wrote:

When I use DIC supplied with MPF, Build 20210401T101950

I only support the latest version. It's now 20211001T112852.

I downloaded the newest DIC version from the website and the problem is still there. I did 2 dumps of clean GameShark 2 disc and I'm getting different results (4476 vs 4477) errors.
Let me know if logs would be useful, I'll upload them.

Go to forum: General discussion

63 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:
superg wrote:

When I use DIC supplied with MPF, Build 20210401T101950

I only support the latest version. It's now 20211001T112852.

I will use that.

Go to forum: General discussion

64 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:

What is the problem? My unlicensed is no problem.

There are two problems:
When I use DIC supplied with MPF, Build 20210401T101950, if that helps, and dumping PS2 GameShark disc with /sf flag, after initial pass it tries to re-read some C2 sectors from BIG.DAT and as it never gets a match, dumping fails.
For other PS2 unlicensed I often get improper dumps with wrong error count, it deviates from 4475 errors count for such discs. All these discs are clean and I currently dump them using the earlier DIC version which always yields the exact 4475 errors. That version you built after I mentioned this earlier error count deviation and it always works for me.

Go to forum: General discussion

65 Reply by superg

(3,534 replies, posted in General discussion)

matura713 wrote:

Total errors: 4497

This error count looks familiar to me. We get a very similar number of C2 errors at start for many PS2 unlicensed discs.
Did you try to dump with /sf?

Go to forum: General discussion

66 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:

"UPX" is shown in IMAGE_SECTION_HEADER of SETUP.EXE. Please check if it is compressed by UPX or not.
If yes, DIC is not supported it yet. 20210401 version does not output IMAGE_EXPORT_DIRECTORY yet.

Is there any way to disable this checking so DIC doesn't crash?

Go to forum: General discussion

67 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:

279144 belongs to track 1.

Got it, I'll mark the old dump as bad and will add a new one.
Thanks for your help!

Go to forum: General discussion

68 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:

Try to use "/s 2"

/s 2 didn't help.
My primary concern is that in my new dump the split is 1 sector earlier comparing to what we have in the DB.
e.g. in the new dump the data track last sector is audio track but in the DB it's not and it seems more correct. I have to identify which dump is correct, old or new.

Go to forum: General discussion

69 Reply by superg

(3,534 replies, posted in General discussion)

sarami, is it normal to have a separate set of (Subs indexes) where everything is the same as in primary TOC based split?
Logs: https://www.dropbox.com/s/4clypf96lu0zq … OR.7z?dl=0
Asking because my verification has identical (Subs indexes) but doesn't match what we have in the DB: http://redump.org/disc/15537/
I compared both dumps and the new data track is 1 sector more padded with zeroes and audio track is 1 sector less comparing to the DB one.

Go to forum: General discussion

70 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:
superg wrote:

I am having a problem with PS2 GameShark disc.

http://www.mediafire.com/file/eq80y20l9 … st.7z/file
Test, please.

Didn't work sad I'm still getting slightly different data on each dump attempt.
New logs: https://www.dropbox.com/s/16zvbqdcggzr8 … st.7z?dl=0
I ended up picking up the most correct dump and hexediting the erroneous sector (they are all empty).

EDIT: I rechecked my dumps before the test version and after, error number definitely decreased with the new version so I guess if the disc is faulty this is still good DIC fix to have. Thank you!

Go to forum: General discussion

71 Reply by superg

(3,534 replies, posted in General discussion)

sarami, I am having a problem with PS2 GameShark disc. They usually have intentional C2 errors in 25-4499 sector range, 4475 total errors. I am using /sf flag but there are some C2 errors later (22K-25K range) and I am getting random results in a few sectors here and there.
As far as I understand, /sf affects all sectors, is there a way to specify that I want /sf enabled only for 25-4499 range or am I missing something?

Logs: https://www.dropbox.com/s/857nbsv9uaqardp/gs.7z?dl=0

Go to forum: General discussion

72 Reply by superg

(3,534 replies, posted in General discussion)

sarami wrote:
superg wrote:

I'm getting a duplicated (subs indexes) copy

It's real subs indexes.

Which one should be considered correct for the redump submission?

Go to forum: General discussion

73 Reply by superg

(3,534 replies, posted in General discussion)

I'm getting a duplicated (subs indexes) copy when dumping Fox Hunt soundtrack (Audio CD). Both copies tracks are the same but want to double check with you that everything is fine.

Logs:
https://www.dropbox.com/s/kld66kkis5xy2 … OR.7z?dl=0

EDIT: cue timestamps are different though.

Go to forum: General discussion

74 Reply by superg

(3,534 replies, posted in General discussion)

sarami, I spent some time dumping one of the Russian unlicensed discs and DIC without any success. The disc is clean and it locks up on some sectors when dumping (slow) and there are some C2 errors.
I almost lost hope but decided to give it a try with IsoBuster and everything dumped fine, and what is more important - I have 100% match with the internet image of the same game. That's quite alarming to me as I have a pile of ~250 discs undumpable with DIC and I wonder whether all of them are bad or not.

Logs: https://www.dropbox.com/s/p7fzgdlkq7yij … gs.7z?dl=0

Can you please look into it?

Go to forum: General discussion

75 Reply by superg

(3,534 replies, posted in General discussion)

rosewood wrote:

[ERROR] Number of sector(s) where mode2 NoEdc subheader(0x10 - 0x17) isn't same: 1
Total errors: 1

This is not a problem, afaik this check takes place after a successful dump and it just indicates that sector subheader copy is not the same. This is common for PSX discs.

Go to forum: General discussion