1 Topic by Feltzkrone (edited by Feltzkrone 2009-10-22 18:43:06)

  • Registered: 2009-10-13
  • Posts: 34

EDIT: Okay, there's still much things left to clarify - it seems the guide proposal is partially simply wrong as Fireball depicts here. I haven't yet fixed it but will do and rewrite parts (or maybe start all over again as Rockrnoms doesn't find it easy to read) once everything is clear.

---- Original text starting here ----

After heaving read what's going on in the thread "data inside pregap?" I believe we are heavily in need of a proper step-by-step ripping guide for producing correct dumps of CDs which have a pregap in front of Track 2.

The guide aims at producing dumps which contain scrambled pregap data sectors taken from CD in Track 2 binary file, which is, from what I understood so far, the form of preserving such CDs as F1ReB4LL would like to see them. Here is my proposal for such a guide and this guide involves the following tools:
Exact Audio Copy, IsoBuster, CDToImg (D8 hack), Px_D8 and HxD (any other hex editor properly used should be OK, too)

This proposal is built upon a notional example with easy to follow values. I don't know if this is the preferred style of explaining things, at least it is mine. smile

It would be nice if technically versed persons like F1ReB4LL for example read this guide and could confirm that it's correct and judge if it is easy to follow. For the case of something being wrongly described please post corrections and also please post additions when you think that something important is missing.


---- "Ripping guide for Mixed-Mode CDs with data sectors in Track 2 pregap" - proposal in original form ----


1. Getting knowledge of how the pregap is made up

We have a CD-ROM with one data track and two audio tracks. It has the following TOC.

Track 1 - Data  - LBA 0
Track 2 - Audio - LBA 10500, Pregap 00:03.00 (225 sectors)
Track 3 - Audio - LBA 21000, Pregap 00:02.00 (150 sectors)

We first try to determine the combined offset by using either IsoBuster's (or CDTool's) sector viewer. When browsing sectors we always keep in mind that whenever we see unexpected data or get read errors, we go some sectors forward then back to the wanted sector - or first go backward some sectors and then forward to the sector in question.

We substract 225 (as pregap is 03.00) from 10500 and seek to sector 10275. The sector looks like a normal data sector (easiliy recognized because of its sync mark - 00 FF FF FF...). So we go forward using the sector viewer in order to find the sector partially containing scrambled bytes and we finally find 408 scrambled bytes at sector 10350 (the last 408 bytes of the previous data sector in scrambled form).

At this point we know the following: The first 75 sectors of pregap are data sectors, the remaining 150 sectors are audio sectors. The combined offset which will be used to read the audio tracks is +408 bytes / 4 bytes per sample = +102 samples.


2. Doing things as they are normally done

Both audio tracks are read using EAC and the combined read offset of +102. We keep in mind that the first 75 sectors of Track 2 dump must be considered as containing undefined garbage. After that we dump the data track using IsoBuster. We remove the last 225 sectors from Track 1 dump using HxD. Number of bytes to remove is 225 sectors * 2352 bytes per sector = 529200 bytes.

For that purpose in HxD we open the Track 1 binary file, jump to its end by pressing Ctrl+End, choosing Edit / Mark block from the menu. In the dialog we first switch from hex to dec so our input is interpreted as decimal values. We then take the value shown as the start address (which is actually the size of the file), substract the number of bytes to remove from it and finally enter the result as start address and hit OK. By pressing the Del key the block is removed and we now can save the file.

At this point we have already partially finished the dump: The data track dump is correct and all audio track dumps are correct, but Track 2 is still not proper due to the mixed-mode pregap.


3a. Getting a dump of Track 2 as is present on CD: Deleting the current pregap data in Track 2 dump

The first thing we want to do is getting rid of the undefined pregap data contained in Track 2 dump made with EAC. We start by using HxD (or any other hex editor) and opening the Track 2 binary file. We then calculate the length of the pregap in bytes as follows:
Pregap length = number of pregap sectors * 2352 bytes per sector

On this particular disc the formula works as follows:
Pregap length = 225 sectors * 2352 bytes per sector = 529200 bytes

In HxD we now mark the block (Edit / Mark block) using 0 as the start address and the calculated pregap length as the length (after having selected decimal value format in the dialog). We now delete the block (by pressing del key) and don't close or save the file yet - just leave it open and modified as is for now.


3b. Getting a dump of Track 2 as is present on CD: Extracting real pregap data from CD

This step involves the usage of the D8 hack of CDToImg, Px_D8 and a drive capable of processing D8 vendor read commands (such as real Plextor models).

We use Px_D8 to determine the combined offset for the drive used (which is different to the one used in the previous steps, but doesn't matter at all). The combined offset is +30 samples = 120 bytes (or 0x78 in hex). Then we use CDToImg and make a full dump (we choose dump.raw as filename) which contains all data as is present on disc, i.e. all data sectors in this dump are present in scrambled form.

We now calculate the start address of the block/range containing the pregap data in the raw dump as follows:
Start address = (Track 2 LBA - number of pregap sectors) * 2352 bytes per sector + combined offset in bytes

In the current case this leads to the following value:
Start address = (10500 - 225) * 2352 + 120 = 24166920

We now use HxD to copy the wanted block to clipboard. We open the raw dump file (dump.raw) and again mark the wanted block/range using Edit / Mark block. This time we use the calculated start address as start address for this dialog and reuse the previously (step 3a) calculated pregap length as length. Again, before entering the values we check that decimal number format is selected.

Now in HxD we copy the selected block of dump.raw, then change to the still opened Track 2 binary file and insert the copied block right before the very first byte of it. We are now ready to save Track 2 binary file and so we do.


4. Recapitulation of what we have done

  • we extracted all tracks as usual using IsoBuster and EAC

  • we truncated the data track as usual because we had a pregap at Track 2 (audio)

  • we made a full disc dump with CDToImg

  • we removed the (wrongly read) pregap data from Track 2 binary file

  • we then extracted the correct pregap data from the full disc dump

  • we inserted the correct pregap before the remaining Track 2 binary data

In effect we replaced the bad pregap data in Track 2 (which has been read by EAC) with proper pregap data which has been read using CDToImg. That proper pregap data contains several data sectors in scrambled form (as on CD).


5. Handling EAC pregap detection oddities (*not sure if this should be included)

It might happen that EAC is unable to detect the correct pregap length and just returns 00:00.00. In exactly that case we need to use CDTool to browse the sectors as (opposed to IsoBuster) it's able to return subchannel data when viewing sector contents. When using the sector browser in CDTool we'll always have to choose subchannel reading mode "P-W RAW Interleaved (Code 001b)" and select "Deinterleave subch data".

At first we will have to determine the drive's subchannel read offsets for both data and audio sectors. So we start the sector viewer and view sector 75 (a confirmed data sector). In the subchannel data are (the bottom data area) we look at the first four digits of the second line and see "0102" which should be read as 1 second, 2 sectors. We now know that the subchannel data the drive reports is 2 sectors off, i.e. we have a subchannel offset of +2, in the following steps refered to as DSO for data subchannel offset.

Next we take Track 2 LBA from TOC and add 75 (=> 10575) then view that sector. We now read "0101" so we know that the drive has a subchannel offset of +1 when reading audio sectors, in the following steps referred to as ASO for audio subchannel offset. Note that this offset generally can be different to the DSO (and is in this example).

Generally DSO and ASO aswell might be negative. In this case you will find e.g. "0074" as a value in the second row of subchannel data, which reads as 0 seconds, 74 sectors and represents a subchannel offset of -1.

We now go to sector (Track 2 LBA - ASO - 2), in this case (10500 - 1 - 2 = 10497). Again we look at the second row in subchannel data and should find either "0001" or "0002" - and we find "0001". If additionally the last two digits of the first row in subchannel data is "00" there is a pregap on disc which EAC was unable to detect. In general as long as you the "00" at the end of the first line, it is an indicator for being in pregap.

Pregaps usually have lengths of 2, 3 or 4 seconds, in some cases these lengths are one, two or three sectors smaller or greater. These possible lengths should cover 99% of all cases, so instead of going backward sector by sector in the sector viewer, we go backward by 75 sectors. By the way: We do count the number of times we go backward 75 sectors. We now see "0101" which tells us that we still are in pregap and are one second ahead. As we want to find the beginning of the pregap we repeat this step.

Again we go back 75 sectors resulting in "0200" and we see a sync mark in the sector data in the text area above. We still are in the pregap but now the pregap contains data sectors so now the formerly calculated DSO applies instead of ASO. That's why we get "0200" and not the expected value of "0201". To compensate this, we calculate the difference from ASO to DSO as follows: +1 - (+2) = -1. That's the number of sectors we will go foward. In this case the number is negative, so we go one sector backward.

We now get "5357" which has nothing in common with the previously seen values. Additionally we don't see "00" anymore at the end of the first row, so it's an indicator for having left pregap => now being before pregap. We have gone backward 75 sectors for three times. That is 75 * 3 = 225 sectors.

The final step is moving forward (and counting) sector by sector until the last value in the first row becomes "00" again. We have to do this 2 times and now find "00" and in the second row "0274". Now we calculate the pregap length...

1. We have gone three times 75 backward => 225 sectors.
2. We have gone two times one sector forward => - 2 sectors => 223 sectors.
3. We started at Track 2 LBA - ASO - 2, so we skipped the last 2 sectors of pregap => + 2 sectors => 225 sectors

Just for proof we check our calculation by using the "0274" value that can be seen in subchannel data. It shall be read as 2 seconds, 74 sectors. Our first value was "0001" and guess what: We build the difference which is 2 seconds, 73 sectors. Then we add two sectors as we skipped the last 2 sectors of pregap and the circle closes => 3 seconds, 0 sectors.

Finally we go one sector forward and we should notice all those "00000..." values at the beginning of the first row in subchannel data change to "FFFFF...". That's the P-Channel indicating the end of the data track has been entered. The P-Channel usually switches to "FFFFF..." exactly one sector after the beginning of a pregap. So another indicator for having correctly determined the pregap length.

In this case we figured out that the pregap length is 03.00. As EAC left out the pregap completely when reading Track 2 we just skip step 3a and instead we have to manipulate the CUE file made with EAC and will replace "INDEX 01 00:00:00" with "INDEX 00 00:00:00 [CRLF] INDEX 01 00:03:00".

Please note that the remaining steps have to be performed as described. Step 5 is not ment as an addition to the regular dumping guide!

  • From: Hyrule
  • Registered: 2008-01-16
  • Posts: 1,440

Your explanation is quite confusing, sorry. Moreover we have some other tools that can simplify the job.
Also about point 5, if you are referring to 2 tracks disc (1 data 1 audio) you don't have to do all this job to get real pregap.
I'll test what you say on point 5 as soon as possible to fit it in exceptions.

My patch requests thread
--------------------------------
  • Registered: 2006-12-24
  • Posts: 3,847

I repeat, there are CDs, where those sectors are marked as data in the subs (track is audio, sectors are data AND marked as data in the subs) - I insist on leaving them descrambled and this case suggests different handling (in this case you should extract the data part of the gap from clonecd dump skipping the 1st track - first 176400 bytes, first 352800 bytes or first 529200 bytes, etc., because data sectors rarely fills the _whole_ gap, then, you should extract the audio part of the gap and the track itself, skipping [1st_track_size + combined_offset_size_in_bytes + data_part_of_the_gap_size], then glue the both parts). Of course, subchannels analyzing is necessary in both cases.

4 Reply by Feltzkrone (edited by Feltzkrone 2009-10-22 19:01:59)

  • Registered: 2009-10-13
  • Posts: 34

F1ReB4LL wrote:

I repeat, there are CDs, where those sectors are marked as data in the subs (track is audio, sectors are data AND marked as data in the subs) - I insist on leaving them descrambled and this case suggests different handling

That's why I have asked you implicitly to review the guide: to clarify things. It doesn't matter to me if I (or somebody else) needs to rewrite big parts of it - it doesn't matter if a first proposal might be wrong. All I would like is to see is a proper guide as the result, may it take months, I don't care... So let's use this thread as discussion platform in order to clarify things and simply write down how to detect different cases, what analyzations have to be made and finally how to store data in which form in the image. Once done I'll give it another try. smile

Ok, I understand that things will have to be done differently depending on how the data sectors are marked (as either data or audio), so I hope that I now got it right: If those data sectors are marked as data, keep them unscrambled in Track 2 binary. Is that correct now?

And what if those data sectors are marked as audio? (All following questions refer to audio-marked data sectors...)
1) Are they still in scrambled form on CD or might that depend on the mastering?
2) Does any drive unscramble them automatically if they are scrambled (= drive ignores that they are marked as audio)?
3) If they are not automatically unscrambled, does the factory write offset apply when reading them with READ CD commands, i.e. data is shifted when read, i.e. sync marks are not at the beginning of the returned sector data?
4) How should the data be kept in the image? Scrambled or unscrambled, or depending on certain circumstances?

When you are saying that subchannel data analyzing is necessary in both cases, isn't it that subchannel analysis is first necessary to distinguish both cases from each other and after that (again) necessary to figure out the number of sectors that actually are marked as data?

I'm convinced now that for those cases subchannel data inspection generally must be done. I still find it annoying to rely on CloneCD .sub files as their quality (remember Manic Karts) heavily depends on the quality of drives, i.e. as soon as a drive has changing subchannel data offsets when switching from data to audio sectors the resulting .sub file is crap when produced with CloneCD. But actually all you'd need is a drive just capable of reading pregap sectors.

How about a tool that just automates pregap (including its subchannel data) analyzing and prints the results - similar to Px_D8 which just prints the combined offset?

EDIT: If I may ask in all innocence - are F1ReB4LL and Rocknroms the only ones willing to discuss and clarify more complex cases like these or are other members, moderators and admins just very busy at the moment? (Don't get me wrong - no offense!)

  • Registered: 2006-12-24
  • Posts: 3,847

First of all, no need to use the bold symbols - I've marked some words from my post to notify all the newbies, that your guide isn't perfect and it's not that easy.

Feltzkrone wrote:

And what if those data sectors are marked as audio? (All following questions refer to audio-marked data sectors...)
1) Are they still in scrambled form on CD or might that depend on the mastering?

Sure. Unscrambled sectors are very rare, but it happens sometimes (see [SS] Sakura Tsuushin entry, for example)

Feltzkrone wrote:

2) Does any drive unscramble them automatically if they are scrambled (= drive ignores that they are marked as audio)?

When you dump a first data track, data sectors from pregap on its end will be descrambled (same for CloneCD dumps, etc.). Not sure what happens when you extract them as audio, though. EAC tweaks the first gap, excluding those sectors. PR extracts them totally wrong (not scrambled, not unscrambled, but screwed).

Feltzkrone wrote:

3) If they are not automatically unscrambled, does the factory write offset apply when reading them with READ CD commands, i.e. data is shifted when read, i.e. sync marks are not at the beginning of the returned sector data?

Usual READ CD command should return them unscrambled, because they "belong" to the previous (data) track, according to the drive's firmware's logic, I've already explained this. Next track "officially" starts from the 01 index according to the TOC, 00 index belongs to the same track according to the subs, but following the TOC ignores this.

Feltzkrone wrote:

4) How should the data be kept in the image? Scrambled or unscrambled, or depending on certain circumstances?

I repeat: in my opinion, all the sectors should be scrambled (even the data tracks), because that's how they are stored on CD. But in the current situation data sectors marked as audio should be scrambled, data sectors marked as data should be unscrambled. But in any case there should be a proper comment in the dump's entry describing all the abnormalities.

Feltzkrone wrote:

When you are saying that subchannel data analyzing is necessary in both cases, isn't it that subchannel analysis is first necessary to distinguish both cases from each other and after that (again) necessary to figure out the number of sectors that actually are marked as data?

Yes, you should find the proper gaps in subs at first, then you should check the mode for the gap sectors (audio/data), then you should count a number of data sectors in the gap. Btw, some of the sectors of the gap may be marked as data in the subs and some - as audio (don't have any examples yet, but I can't exclude a possibility of this).

Feltzkrone wrote:

How about a tool that just automates pregap (including its subchannel data) analyzing and prints the results - similar to Px_D8 which just prints the combined offset?

What results, exactly?

Feltzkrone wrote:

EDIT: If I may ask in all innocence - are F1ReB4LL and Rocknroms the only ones willing to discuss and clarify more complex cases like these or are other members, moderators and admins just very busy at the moment? (Don't get me wrong - no offense!)

Jackal is also able to do some researches, Dremora in some rare cases... Themabus is more on the hardware side (Saturn rings tests, etc.). Don't remember anyone alse.

  • Registered: 2009-10-13
  • Posts: 34

F1ReB4LL wrote:
Feltzkrone wrote:

How about a tool that just automates pregap (including its subchannel data) analyzing and prints the results - similar to Px_D8 which just prints the combined offset?

What results, exactly?


The tool would analyze Track 2 pregap in case of Mixed-Mode CDs and produce an output like this (example):

Subchannel offset data ... +2 sectors
Subchannel offset audio .. +1 sectors
Q-Channel pregap length .. 225 sectors (-00:02.74 to -00:00.00)
P-Channel flags .......... 225 sectors (-00:02.73 to +00:00.00)
Q/P data matches ......... yes
Combined offset .......... +8 samples

+----------------------------------------------------------------------------+
| Pregap layout                                                              |
+---------+--------+-------+-------------------------------------------------+
|  Offset | Length | Type  | Main channel analysis                           |
|---------+--------+-------+-------------------------------------------------+
|    -225 |     75 | Data  | Data (sync found, aligned)                      |
|    -150 |      1 | Audio | 32 scrambled bytes from data sector             |
|    -149 |    149 | Audio | Audio (sync not found)                          |
+---------+--------+-------+-------------------------------------------------+
  • Registered: 2006-12-24
  • Posts: 3,847

Writing a basic proper dumping tool won't take much longer time, so don't see much point in this.

  • Registered: 2009-10-13
  • Posts: 34

F1ReB4LL wrote:

Writing a basic proper dumping tool won't take much longer time, so don't see much point in this.

If this really was the case, why don't we already have one? wink


The current guide forces you to use three different tools:

  • IsoBuster

  • ExactAudioCopy

  • Resize

Of course it would make sense to have one tool instead of these three which handles 99% of all cases automatically. Combined offset detection as suggested in the guide is no hocus pocus, generally none of those three do anything which couldn't be automated using one tool. But at the moment we don't have anything like that.

I only wanted to provide a tool which would detect special cases of pregap layouts and simplify the task of determining the combined offset. What's bad about it? Just that we still don't have an all-in-one tool then?

Don't get me wrong, but if such a tool is really wanted you are one of those few people who will have to contribute their knowledge. Coding (half-)blindly might produce a usable all-in-one dumping tool but it won't be perfect then and will cause bad dumps, that's why knowledge is needed to be spread. For example by posting background info on how certain things found on a CD should be detected and interpreted, how errors in audio extraction could be detected and compensated (i.e. figuring out what is so 'magical' about EAC). Once all info needed for coding such a tool we actually can code it and the resulting tool will be reliable. So in my opinion it actually is a lot more work to code such a dumping tool compared to just coding a more or less simple pregap reader.

If you don't mind I'd like to alter the topic subject for something like "Technical discussion for a future dumping tool" and - as the new subject denotes - we could talk about technical backgrounds, tricks and abnormalities here - that the tool should be able to handle properly.

Also we would have to discuss if the tool still should rely on CUE/BIN as with this format mixed-mode pregaps cannot be preserved properly. Subchannel data would have to be preserved aswell where it is non-standard. Apart from that I wouldn't mind discarding Sync and ECC/EDC info in data sectors where they are built the standard way - finally giving an image of the CD which contains user data and abnormalities, from which (if needed) a full and clean CCD/IMG/SUB aswell as a clean CUE/BIN can be reconstructed. What do you think about that?

  • Registered: 2006-12-24
  • Posts: 3,847

Feltzkrone wrote:

If this really was the case, why don't we already have one? wink

Noone to code. I don't have enough time and no volunteers at all.

Feltzkrone wrote:

I only wanted to provide a tool which would detect special cases of pregap layouts and simplify the task of determining the combined offset. What's bad about it? Just that we still don't have an all-in-one tool then?

You offer to write a tool, which should be able to read some sectors, read the subs, correct the subs, analyze the subs, count the data offset, count the subs offset - man, you just need to read all the sectors instead of some gap and split the tracks - voila, a good dump. I only wanted to provide -- you're welcome to provide any tool smile

Feltzkrone wrote:

Don't get me wrong, but if such a tool is really wanted you are one of those few people who will have to contribute their knowledge. Coding (half-)blindly might produce a usable all-in-one dumping tool but it won't be perfect then and will cause bad dumps, that's why knowledge is needed to be spread.

Of course, I've already offered this (my ideas/algorithms, me as a tester, but someone else as a coder), noone is interested.

Feltzkrone wrote:

For example by posting background info on how certain things found on a CD should be detected and interpreted, how errors in audio extraction could be detected and compensated (i.e. figuring out what is so 'magical' about EAC).

Feltzkrone wrote:

If you don't mind I'd like to alter the topic subject for something like "Technical discussion for a future dumping tool" and - as the new subject denotes - we could talk about technical backgrounds, tricks and abnormalities here - that the tool should be able to handle properly.

Noone is interested.

Feltzkrone wrote:

Also we would have to discuss if the tool still should rely on CUE/BIN as with this format mixed-mode pregaps cannot be preserved properly. Subchannel data would have to be preserved aswell where it is non-standard. Apart from that I wouldn't mind discarding Sync and ECC/EDC info in data sectors where they are built the standard way - finally giving an image of the CD which contains user data and abnormalities, from which (if needed) a full and clean CCD/IMG/SUB aswell as a clean CUE/BIN can be reconstructed. What do you think about that?

Yes, with the .sub dumps it's possible to add the checksum for a single .img file, ccd can be generated from the .sub file (in 99% cases, at least - to cover the 100% we should also dump TOC into a standalone file and generate ccd and cue based on both TOC and .sub dumps, not only .sub). This would make many stupid people happy, who think, that our splitted dumps are bad and clonecd ones are perfect.

10 Reply by Feltzkrone (edited by Feltzkrone 2009-10-23 18:05:46)

  • Registered: 2009-10-13
  • Posts: 34

It's fine that you are willing to provide background info as soon as needed. And here is your volunteer - but with the following drawbacks (at least in the eyes of most coders):
- GUI tool will be coded in Delphi 7 (alternatively commandline-based Java with native DLL usage)
- It will first use SPTI only, so it probably won't work on Windows versions < 2000
- I will only be able to code 1-2 hours per day (employed as programmer, sometimes just being fed up with programming and weekends are there to care for other things)

If you wouldn't mind these drawbacks I'm willing to start, although with the feeling of reinventing the wheel but as all wheels available don't represent a perfect circle I won't care. smile


EDIT:

F1ReB4LL wrote:

This would make many stupid people happy, who think, that our splitted dumps are bad and clonecd ones are perfect.

From what I had to read so far from those people is that they simply don't trust anything other than CloneCD or Alcohol 120%, just because only they can handle newest forms of copy protection etc. If they got knowledge about we would be discarding certain data being present on the CD they immediately would lose their interest. But they don't get the point that tools like ECM discard data, too - in their eyes it's just a stronger form of compression.

  • Registered: 2008-03-10
  • Posts: 355

ECM and similar tools do only discard redundant data, that is the point - this data can be reconstructed smile

EAC itself does not do many magical things: offset correction and reading the audio sectors multiple times

  • Registered: 2006-12-24
  • Posts: 3,847

Yes, ECM won't touch any unusual/corrupted/whatever sectors, audio sectors, etc. Haven't checked its source, but I'm sure it reads 2048 bytes from each sector, then generates ECC/EDC fields, then compares with the ones in the sector - if matched, it cuts them out and leaves a mark that they should be regenerated during the UNECM routine. Anyway, it's a safe tool.

Feltzkrone, try to join our IRC channel and contact me there.

  • From: Hyrule
  • Registered: 2008-01-16
  • Posts: 1,440

Feltzkrone wrote:

or maybe start all over again as Rockrnoms doesn't find it easy to read

I think it's quite complicated to understand for most of people, moreover take a look at the forums also for reMove tool, you can find it in quite every forum faq by me. It can be more useful while moving sectors.
It will be great if you can code a tool that will avoid all those exceptions and that can handle them in a while.

My patch requests thread
--------------------------------
  • Registered: 2007-12-04
  • Posts: 445

Feltzkrone may I suggest that you go for Java command-line because in the future it would be easier to port the program to other OSs.

  • Registered: 2009-10-13
  • Posts: 34

ssjkakaroto wrote:

because in the future it would be easier to port the program to other OSs

Seconded. The only problem I have with this choice is that I'll have to familiarize myself with JNI which is used to glue native DLLs and the JVM together. But it shouldn't be too hard either and will make things easier in the end as Java does provide a lot of classes and functions to the programmer which are not available in Delphi 7. The critical point in this is that programs which require an installed Java runtime are not that widely accepted by the end users as native Win32 applications are (the same drawback goes for .NET). I'd like to see other opinions on this topic before committing myself to one or another.

I guess it would be sufficient to implement device enumeration and device access (send CDBs to a device and pass data buffer and data direction) in the native DLL. I don't have anything with to do with e.g. Linux, but I guess it shouldn't be a problem for some Linux programmer to build an analogous shared object (I think this is for Linux what DLLs are in Windows) which can be used then.

  • Registered: 2006-12-24
  • Posts: 3,847

ssjkakaroto wrote:

Feltzkrone may I suggest that you go for Java command-line because in the future it would be easier to port the program to other OSs.

Unless he's interested to modify PerfectRip (which is written on Delphi) instead of writing everything from scratch.

  • Registered: 2009-10-13
  • Posts: 34

F1ReB4LL wrote:

Unless he's interested to modify PerfectRip (which is written on Delphi) instead of writing everything from scratch.

First I don't have the feeling the author would hand out the source (I didn't ask and might be completely wrong here) and second rewriting from scratch gives us the possibility to do everything as we think should be done.

By the way, there is a not so unimportant thing to consider when rewriting from scratch: Many drives are able to disable error correction when reading data sectors. I believe if e.g. one bit in user data is wrong (on pressed CD) but ECC/EDC is intact, the drive would correct the wrong bit. If those wrong bits really are on CD shouldn't it be considered to write them (wrong) as they are into the image file, i.e. disabling error correction when dumping? Ofcourse, when disabling error correction more than one read per sector would have to be performed to really get sure we got what's on CD - as EAC does with audio data.

I already came over a CD (Al Unser, Jr. Arcade Racing) where a wrong bit is in subchannel data in one sector which is read as it is regardless what drive I used - meaning it really is on CD and not just a random error dropped in by the drive. I guess 1:1 preservation (which is aimed at redump.org AFAIK) should preserve this wrong bit, too. What's your opinion on this?

  • From: Hyrule
  • Registered: 2008-01-16
  • Posts: 1,440

Feltzkrone wrote:

I already came over a CD (Al Unser, Jr. Arcade Racing) where a wrong bit is in subchannel data in one sector which is read as it is regardless what drive I used - meaning it really is on CD and not just a random error dropped in by the drive. I guess 1:1 preservation (which is aimed at redump.org AFAIK) should preserve this wrong bit, too. What's your opinion on this?

Data is on CD only if it's in disc sector viewer. If the disc has some signs or scrathes and the drive is not good you will get errors, so if you cannot dump properly a disc, there's nothing to preserve.
If this error is something really on CD and if we are talking about PC stuff it could be some kind of protection so it has to be dumped differently.

My patch requests thread
--------------------------------
  • Registered: 2006-12-24
  • Posts: 3,847

Of course, there are random errors in subchannels, there are random errors already pressed on the disc and random errors when reading them. That's why fixing the subchannels isn't a perfect idea in terms of preservation.

About error correction - still waiting you on the IRC channel (or ICQ) to discuss the details and algorithms. IMO, discs should be dumped in scrambled form via D8, then descrambled via software (to bypass any firmware error correction). No C2 errors - good dump.

  • Registered: 2009-10-13
  • Posts: 34

I'll install ICQ or an IRC client on weekend. I have already finished the thin JNI DeviceIO DLL for Win32/SPTI and used it successfully in Java (plain dump of all sectors of a disc using READ CD CDBs). First I'll need to read and interprete TOC properly, I'll care about that until weekend, so I guess we won't lose any time when waiting (with knowledge exchange) until then.

Regarding D8 scrambled reads... It's another thing left to be defined: What about drives which are unable to read data via D8 command? I think we can't just exclude all users with average drives from submitting dumps - instead I would suggest something like a dump quality measurement where too low quality levels exclude dumps from being added as first-time dumps.

  • Registered: 2007-12-04
  • Posts: 445

F1ReB4LL wrote:

Unless he's interested to modify PerfectRip (which is written on Delphi) instead of writing everything from scratch.

Feltzkrone wrote:

... I have already finished the thin JNI DeviceIO DLL for Win32/SPTI and used it successfully in Java (plain dump of all sectors of a disc using READ CD CDBs). First I'll need to read and interprete TOC properly, I'll care about that until weekend, so I guess we won't lose any time when waiting (with knowledge exchange) until then. ...

I'm glad you went that way, because PerfectRip has that gap issue and didn't gave consistent results even when there were no C2 errors.

  • Registered: 2009-10-13
  • Posts: 34

A little update on this topic: I will first concentrate on dumping discs as F1ReB4LL suggests, need to get hold of a scratched CD (I think some bullshit early 90s audio CD which I have plenty of and a nail will do) in order to verify that the D8 command I found out probably returning subchannel data and C2 error pointers really does it. For anybody interested here is the CDB: D8 00 XX XX XX XX YY YY YY YY 08 00, where X = starting LBA and Y = number of sectors.

Would be nice if anybody could confirm this, I just increased the value at the second last position until the drive returned C2 pointers and subcode. As mentioned before, I personally prefer reading subchannel data aswell because there might be data to be preserved but not possible with CUE format, such as pause styles and MCN and ISRC interleaving and "errors" actually on disc.

By the way, this "document" is the only one found with Google which is at least telling a little about the D8 command. In fact it tells you how things where done back in 1995... ;-)
http://www.isomedia.com/homes/isomedia/ … aq_37.html

Nevertheless, once finished D8 ripping, which might take a few weeks due to proper handling and file output, I will extend the program to work with MMC-compliant and Accurate Stream capable (= no jitter) drives as good as possible (= as good as with IsoBuster + EAC, but a bit better regarding pregap detection).

  • From: Macondo
  • Registered: 2007-08-21
  • Posts: 966

hi Feltzkrone
i can confirm that Plextor Premium return 294 additional bytes with this parameter,
so it pretty much looks like c2
d8 command is described in SCSI-2 Command Reference Manual
but it does not list those options beyond 3, maybe in some later revisions,
but then i'd guess very few devices would support this

ECMa130 @20091225 :: friidump 0.5.3 :: PSXstuff :: SaturnPrograms :: MyStupidPrograms2 :: MyStupidPrograms

themabus's Website

24 Reply by Feltzkrone (edited by Feltzkrone 2009-11-03 23:04:56)

  • Registered: 2009-10-13
  • Posts: 34

Thanks for testing the command. I'm afraid I run into bigger problems right now: PX-716A returns inconsistent subchannel data between the mentioned D8 and normal READ CD commands.

D8 on PX-716A using my tool, viewing .sub file with HxD:

000D3C20  00 00 00 00 00 00 00 00 00 00 00 00 41 01 00 02  ............A...
000D3C30  00 35 00 02 02 35 6F FE 00 00 01 00 00 00 00 00  .5...5o.........
000D3C40  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
000D3C50  00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00  ................
000D3C60  00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00  ................
000D3C70  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................

READ CD on same sector (still PX-716A) using cdtool:

00000000000000000000000041010002  ............A...
0035000202356FFE0000000000000000  .5...5o.........
00000000000000000000000000000000  ................
00000000000000000000000000000000  ................
00000000000000000000000000000000  ................
00000000000000000000000000000000  ................

Same sector with my LG drive using cdtool:

00000000000000000000000041010102  ............A...
0035000202356FFE0000000000000000  .5...5o.........
00000000000000000000000000000000  ................
00000000000000000000000000000000  ................
00000000000000000000000000000000  ................
00000000000000000000000000000000  ................

Note how the dump using LG drive looks fine and how the dumps using PX-716A using D8 and normal READ CD command differ (look at those 01s and especially the INDEX = 00 values). If you XOR the first 0x30 bytes from the D8 dump with the second 0x30 bytes you get the same 0x30 bytes as using PX-716A along with READ CD. By the way: The results are 1:1 reproduced, even if medium removed and re-inserted.

This is very strange and in my opinion there are only two possibilites:
1. The LG drive cleans the data (maybe using the Q channels 16 bit checksum)
2. PX-716A does some hocus pocus or has problems when reading subs
2a. With D8 possibly the second 0x30 bytes act as error pointers (???)
2b. With READ CD those errors are partially cleaned, as INDEX is still returned as 00 instead of 01

Any ideas how to handle this properly? Ofcourse the dumping tool might make use of the 16-bit CRC value as generating Q channel data like it -should- look like, calculate CRC for that data and compare CRC with returned CRC. If there's a match and the returned data is suspicious, take the generated instead. I personally don't like recovery methods like these but if nobody else has an idea 1) what might be the reason for this strange behaviour and 2) how to handle it properly I wouldn't know what to do.

Thanks in advance.

EDIT: Perhaps this behaviour is related to Rocknrom's PerfectRip gap/index problem (here) as my PX-716A returns the very same (wrong) data with both subchannel mode 001b and 100b. I guess what Plextor returns here -should- really be handled as errors, since I would have to write index changes on the dumped sector aswell if the data would be taken seriously.

EDIT2: I have done a dump using PerfectRip and it doesn't produce any wrong INDEX entries, but look at this log extract...

Info    | 22:50:52 | Pause found: 410200000300001555597822
Error   | 22:50:52 | Error while burst reading sectors: 071590 to 071615 (15:54.40 to 15:54.65)
Info    | 22:50:52 | Retrying with 1 sector reads from: 071590 (15:54.40)
Error   | 22:50:53 | Error reading sector: 071609 (15:54.59)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector (ignoring): 071609 (15:54.59)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector: 071610 (15:54.60)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector (ignoring): 071610 (15:54.60)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector: 071611 (15:54.61)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector (ignoring): 071611 (15:54.61)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector: 071612 (15:54.62)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector (ignoring): 071612 (15:54.62)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/02/81 - Unknown sense key code combination
Error   | 22:50:53 | Error reading sector: 071613 (15:54.63)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/15/00 - Random positioning error
Error   | 22:50:53 | Error reading sector (ignoring): 071613 (15:54.63)
Error   | 22:50:53 | Sense hex (key/ASC/ASCQ): 03/15/00 - Random positioning error
Info    | 22:50:53 | 1 sector reads were ok, setting back to burst reading from: 071616 (15:54.66)
[...]
Info    | 22:53:48 | Total corrupt Q subcode blocks: 233
Warning | 22:53:48 | Replaced sector: 071609 (15:54.59)
Warning | 22:53:48 | Replaced sector: 071610 (15:54.60)
Warning | 22:53:48 | Replaced sector: 071611 (15:54.61)
Warning | 22:53:48 | Replaced sector: 071612 (15:54.62)
Warning | 22:53:48 | Replaced sector: 071613 (15:54.63)
Info    | 22:53:48 | Reading process completed successfully.

The sectors PerfectRip replaced (because it couldn't read them) are fully readable with CDTool. And please take a look at the Q subcode error count. I guess these come from exactly the strange subchannel data that I've read before.

  • From: Hyrule
  • Registered: 2008-01-16
  • Posts: 1,440

Do you invert cdtools sector views while posting? I ask because the different one above is LG and bot plextor are the same.

My patch requests thread
--------------------------------