1 Topic by Urk (edited by Urk 2025-03-07 08:27:48)

  • Urk
  • Dumper
  • Offline
  • Registered: 2024-08-11
  • Posts: 104

Hi,

When I started submitting discs, I mostly used Redumper. Then, I saw that DIC was better, in particular with mixed Data-Audio discs, so...
I'm in the process of redumping all my discs with DIC & Redumper to ensure all my dumps are correct.

I need help to figure out a weird situation with 3 discs.

The two firsts are covermount games that have not been submitted yet. (iM1A2 Abrams & Take No Prisoners).
(Single track/Not mixed discs)

I dumped each game 4 times :
- 2 times with Redumper (1x & 4x)
- 2 times with DiscImageCreator (1x & 4x)

And... I get consistent different dumps.

Redumper consistent dumps differs from DiscImageCreator consistent ones.

The thrid game is Heroes of Might & Magic II (That have been submitted : http://redump.org/disc/118659/)
(Mixed disc)

I just redumped this game six times.
- 3 times with Redumper (1x, 4x & 8x)
- 3 times with DiscImageCreator (1x, 4x & 8x)

And again, I get consistent differents dumps between Redumper & DiscImageCreator.

All the discs are in a good shape, no scratches, no moisture.

The question is : Which dumps should I trust to be the good ones : DIC dumps or Redumper dumps ?

Your help would be appreciated smile

Regards,

  • From: Asia-Pacific
  • Registered: 2022-11-23
  • Posts: 700

In what way did you think that DIC is better?
If you can then dumping with both can help ensure the dump is good, but Redumper is recommended for the majority of discs.

As for why the dumps are different, the logs give hints as to non-zero errors, invalid sync sectors.
There are some scenarios that Redumper and DIC deal with unexpected sectors differently.
I would have to look at the logs closer to be sure.

Also, please update your MPF to the latest version (3.3.0) which contains the recommended version of Redumper and has fixed a lot of bugs.

Deterous's Website

  • Registered: 2011-02-20
  • Posts: 1,767

Urk wrote:

And again, I get consistent differents dumps between Redumper & DiscImageCreator.

Pregap length of track 30 and 32 is different.

Urk wrote:

All the discs are in a good shape, no scratches, no moisture.

Last subQ all tracks and second subQ from the last of all tracks is weird. It seems a mastering issue.

DiscImageCreator, UmdImageCreator, Conv2multiBin, bin2wav, PS3Auth (needs login), recompressTo7z_20150109.bat

sarami's Website

4 Reply by Urk (edited by Urk 2025-03-07 08:27:33)

  • Urk
  • Dumper
  • Offline
  • Registered: 2024-08-11
  • Posts: 104

In what way did you think that DIC is better?

Oh, I think I read that in some forums. I thought that it was the reason why Redumper was later accepted than DIC for dumps.
I assume that I'm wrong smile

Also, please update your MPF to the latest version (3.3.0) which contains the recommended version of Redumper and has fixed a lot of bugs.

Done ! I'm on 3.3.0 now !

I redumped these 3 discs with the new MPF version, but get the same results.

  • Urk
  • Dumper
  • Offline
  • Registered: 2024-08-11
  • Posts: 104

Pregap length of track 30 and 32 is different.

Last subQ all tracks and second subQ from the last of all tracks is weird. It seems a mastering issue.

Doesn't ReDump & DIC should read the same pregap lengh & SubQ infos as they read discs at block level ?

I'm not expert enough to decide which dumps are to be considered corrects ^^

  • Registered: 2011-02-20
  • Posts: 1,767

Urk wrote:

I'm not expert enough to decide which dumps are to be considered corrects ^^

As far as the logs are concerned, DIC is correct.

DiscImageCreator, UmdImageCreator, Conv2multiBin, bin2wav, PS3Auth (needs login), recompressTo7z_20150109.bat

sarami's Website

7 Reply by Deterous (edited by Deterous 2025-03-06 04:55:04)

  • From: Asia-Pacific
  • Registered: 2022-11-23
  • Posts: 700

Urk wrote:

I thought that it was the reason why Redumper was later accepted than DIC for dumps.

Redumper is new, that is why it is only recently accepted.

Both tools are dumping the exact same data from the discs, the difference is in how they are doing post-processing.
For the Might & Magic disc, they just split at different locations. DIC has identical pregap lengths for all tracks, Redumper has different pregap for Track 30 and 32. For this disc, it is due to the subcode desync messages, likely needs to be fixed (Redumper currently incorrect splits). I have fixed redump.org hashes to include the DIC splits (Tracks 29-32 have changed).

For the other two discs, they have some invalid sectors (one of which is EDC/ECC error). DIC and Redumper treat these errors differently, hence the different hashes. It is likely that redumper's method here is preferred. These are all the test cases that Redumper uses when determining what to do with invalid/unexpected sectors: https://github.com/superg/redumper/tree … unscramble

Deterous's Website

  • Registered: 2011-02-20
  • Posts: 1,767

To:Deterous

01_invalid_sync.0.pass
10_invalid_mode_zeroed_intermediate.0.pass
11_invalid_mode_non_zeroed_intermediate.0.pass
12_invalid_mode_non_zeroed_intermediate_last_byte.0.pass

Can you explain from the CD specifications why this sector is unscrambled?

03_mode0_zeroed_data.0.pass
04_mode0_non_zeroed_data.0.pass
05_mode0_non_zeroed_data_last_byte.0.pass

User area of the mode 0 sector is all zero. It's CD specifications. But I confirmed that other software also treats as no error sector. I'll fix this case.

16_msf_mismatch_invalid_sync.42.fail
:
:
45_no_msf_invalid_mode_no_intermediate_max.null.fail

What is the "msf_mismatch" and what is the "no_msf"? It seems msf of test data is normal (00:02:00).

DiscImageCreator, UmdImageCreator, Conv2multiBin, bin2wav, PS3Auth (needs login), recompressTo7z_20150109.bat

sarami's Website

  • From: Asia-Pacific
  • Registered: 2022-11-23
  • Posts: 700

I will see if superg can respond to your questions, better than I could.

Deterous's Website

10 Reply by Urk

  • Urk
  • Dumper
  • Offline
  • Registered: 2024-08-11
  • Posts: 104

Deterous, I think you forgot to update the Error Count on the page http://redump.org/disc/118659/

Error Count : 2

11 Reply by Deterous (edited by Deterous 2025-03-06 13:03:05)

  • From: Asia-Pacific
  • Registered: 2022-11-23
  • Posts: 700

The error count is still 0, as per redump.org rules around what errors get counted (invalid sync sectors are not counted in the error count). The only change is the index 1 position for two tracks.

Deterous's Website

12 Reply by Urk

  • Urk
  • Dumper
  • Offline
  • Registered: 2024-08-11
  • Posts: 104

Ah, ok !

So, I'll Keep (And I'll submit) Redumper dumps of the two others games (iM1A2 Abrams & Take No Prisoners)

Thanks both of you guys, for your time, informations and advices !

  • From: Asia-Pacific
  • Registered: 2022-11-23
  • Posts: 700

If you have the time to, submitting logs from both tools is good. But it is not required.

Deterous's Website

  • Registered: 2019-05-25
  • Posts: 1,228

sarami wrote:

01_invalid_sync.0.pass
10_invalid_mode_zeroed_intermediate.0.pass
11_invalid_mode_non_zeroed_intermediate.0.pass
12_invalid_mode_non_zeroed_intermediate_last_byte.0.pass

Can you explain from the CD specifications why this sector is unscrambled?

The only thing I know of that comes from the CD specification is that if there is sync header and track is data, it needs descramble. Most of these test cases came from bad mastering cases such as early masterings and Philips CDi.

sarami wrote:

03_mode0_zeroed_data.0.pass
04_mode0_non_zeroed_data.0.pass
05_mode0_non_zeroed_data_last_byte.0.pass

User area of the mode 0 sector is all zero. It's CD specifications. But I confirmed that other software also treats as no error sector. I'll fix this case.

Same is about mode0 sectors. Specs say that those should be empty, but in reality I see discs where

sarami wrote:

16_msf_mismatch_invalid_sync.42.fail
:
:
45_no_msf_invalid_mode_no_intermediate_max.null.fail

What is the "msf_mismatch" and what is the "no_msf"? It seems msf of test data is normal (00:02:00).

Each filename has a number suffix that indicates the expected LBA. When redumper descrambles, it knows the position of of the sector and if it matches, it's the strong check for the successful descramble.
So .42 means that expected LBA of the sector contents is 42, on the other hand .null means that there is no positional information.
For the details see https://github.com/superg/redumper/blob … ts.cc#L187

The bottom line is, DIC never followed CD specs in the first place (always descramble if there is sync header and it's data track). Long time ago I've been checking your descramble code and you have a lot of logic that checks if padding is empty, if mode is correct and some other things. So the current status quo is to have clean dumps without garbage and redumper is doing exactly that. The test cases were created solely to make sure that any subsequent redumper code change is not affecting whole redump.org database.

  • Registered: 2019-05-25
  • Posts: 1,228

redumper build_490 fixes the incorrect split for your Heroes 2, feel free to redump and it should match DIC.

16 Reply by Urk

  • Urk
  • Dumper
  • Offline
  • Registered: 2024-08-11
  • Posts: 104

It's a match ! Well done !

Here are the logs if you need to check anything : https://mega.nz/file/gDIEkYRI#x2jyworhw … wkAqricv3I

17 Reply by reentrant (edited by reentrant 2025-03-07 12:53:50)

  • Registered: 2014-10-06
  • Posts: 725

Unscrambling has always been a problem for me. It'd be great if both tools were 100% correct about this:
https://github.com/superg/redumper/tree … unscramble

Sarami: any chance you could adapt your unscramber code to pass all of those tests?

  • Registered: 2011-02-20
  • Posts: 1,767

To:superg
Thanks for the answer. I still have a question.
Redumper unscrambles the sector if sync is broken but msf and mode are normal. Why? Optical drive treats the broken sync sector as error, do not unscramble.

DiscImageCreator, UmdImageCreator, Conv2multiBin, bin2wav, PS3Auth (needs login), recompressTo7z_20150109.bat

sarami's Website

  • Registered: 2019-05-25
  • Posts: 1,228

sarami wrote:

To:superg
Thanks for the answer. I still have a question.
Redumper unscrambles the sector if sync is broken but msf and mode are normal. Why? Optical drive treats the broken sync sector as error, do not unscramble.

Yes, this again fixes some bad CDi masterings because they actually store data sectors without sync headers.

  • Registered: 2014-10-06
  • Posts: 725

I don't know if such rule (valid for CDi) should be applicable to all systems (especially IBM PC)...

Small suggestion: is it possible to emit some metadata describing mastering error of a sector - some info describing what's faulty and how it was corrected - for the purpose of being able to reverse repair operation (to have a possibility of creating unmodified dump)?

  • Registered: 2019-05-25
  • Posts: 1,228

reentrant wrote:

I don't know if such rule (valid for CDi) should be applicable to all systems (especially IBM PC)...

This rule is not affecting IBM PC at all.

reentrant wrote:

Small suggestion: is it possible to emit some metadata describing mastering error of a sector - some info describing what's faulty and how it was corrected - for the purpose of being able to reverse repair operation (to have a possibility of creating unmodified dump)?

It's possible but I don't think it's useful for general public. Descrambling is lossless operation and you can always restore any sector how you want, especially if descrambling algorithm is consistent and easy to understand.
If you really care about having unmodified dump, preserve .scram, that will never change.

  • Registered: 2011-02-20
  • Posts: 1,767

I decided to simply check edc/ecc if sync or mode was irregular. It is nonsense to change behavior depending on CD-i or not.

If mode bit1 is 0 and bit0 is 1, it deems to the mode1 and calc edc/ecc.
If mode bit1 is 1 and bit0 is 0, it deems to the mode2 form1/2 and calc edc/ecc.
If edc/ecc is correct, the sector is unscrambled, otherwise is not unscrambled.

If mode bit1 is 0 and bit0 is 0, it deems to the mode0 and the sector is unscrambled if sync and msf is correct.
If mode bit1 is 1 and bit0 is 1, it deems to unknown and the sector is not unscrambled.

The result would be the same as redumper's test(01..46), perhaps.

DiscImageCreator, UmdImageCreator, Conv2multiBin, bin2wav, PS3Auth (needs login), recompressTo7z_20150109.bat

sarami's Website

  • Registered: 2019-05-25
  • Posts: 1,228

sarami wrote:

It is nonsense to change behavior depending on CD-i or not.

Good luck and all the best!

  • Registered: 2011-02-20
  • Posts: 1,767

By the way, can anyone upload a sector without sync header of a cdi disc? I want to know if it's "mode2 form1" or "mode2 form2" or "mode2 no edc".

DiscImageCreator, UmdImageCreator, Conv2multiBin, bin2wav, PS3Auth (needs login), recompressTo7z_20150109.bat

sarami's Website