1 (edited by ehw 2023-10-01 17:17:34)

So a long time ago, there was some interest in exploring the possibility of dumping DVDs 'raw'. Meaning, scrambled dumps with error detection and correction data along with header information. Some of you may be aware that unlike CDs, drive's don't like to give out data to the user that isn't needed for the typical user, like error correction/detection data. So instead, drives just give you the corrected, 2048 byte user data already processed. This is kind of a problem in some cases as you have to trust that the drive is performing the error correction and detection properly, which occurs either at the firmware level or at the DSP level. Having the ability to look at the raw data has some advantages, but it's limited to how much control the drive gives to the user, which isn't a lot.

Currently we're exploring using the raw information provided by various DVD drives for trying to get as much data from the disc as possible. Since DVD drives traditionally don't like to give you any sector data unless it's 100% 'correct', for bad or hard to read discs it leaves a lot on the table.

Luckily, most drives out there have support for the READ BUFF SCSI command which can help spill some unintended data to the user on request. If you get the parameters just right, you can get the data that was read by the drive before it's returned to the user. Since we know what a raw DVD sector consists of, we can easily find the right parameters needed for the command to find the right combination that returns the data we want.

However, there's a catch.

Because READ BUFF is implemented at the firmware level, despite every drive having some level of support for the command, the command varies from vendor to vendor, drive to drive. So one combination of parameters might not necessarily return the data you want in one drive even if it worked on another drive model. The second issue is that drives will store only what's necessary for the drive's firmware itself to perform what it needs to do on the data that's read from the disc. As such, some drives return variable, but somewhat fixed/standard data sizes per sector. So even if the data is found using READ BUFF, you need to determine how much data is returned per sector that's stored. Sometimes 2366 bytes with some padding are stored, sometimes 2064, etc. Sometimes data can be returned already unscrambled, sometimes data can be stored scrambled but with only error detection data. Sometimes you get everything. It's a mess.

Luckily, we can bruteforce the READ BUFF command and figure out what will give us the expected data. There was a tool written in the past that used to do just this, however the source was never released and doesn't seem to work correctly. It also doesn't take into account the various other memory dumping commands that exist as well, like F1 and E7.

That's where you come in. We have a lot of drives already, but not everything. We would like to create a database and keep track of drives that return raw DVD sector data onto the cache and the SCSI command parameter combo that gives you the data, along with what's actually contained within that data. We made a super streamlined version of the program that you can run on Windows machine that will take care of everything for you. Here's how it works.

1.) Get a DVD, preferably something on Redump.org, and put it in a DVD drive you'd like to test. The disc must be a standard DVD, no Xbox, GC, Wii, etc! A normal PC game on a DVD, PS2, or maybe a DVD Video disc should be okay.
2.) Run the program. The program will ask which drive letter contains the disc to use.
3.) The program will then perform some tests for feature capabilities, and then will begin bruteforcing parameters for various memory dumping commands. The program uses an embedded python script with calls to sg3 tools to issue various SCSI commands to the target drive while the media is inserted.
4.) It may take a while, but when it's done, it will generate an "uploadme.zip" containing memory dump fragments and a log file. Upload the uploadme.zip file here!
5.) I'll look at the dump and I'll add it to the table below.


Spreadsheet:
https://docs.google.com/spreadsheets/d/ … li=1#gid=0

Source code:
https://github.com/hiddenpalaceorg/DVDRawBruteforce

Grab the tool here:
https://github.com/hiddenpalaceorg/DVDR … e/releases

If you guys have any questions feel free to post here. Hopefully the zip files it generates doesn't take up a lot of space, as hopefully you can add the attachments here. But feel free to use another upload site if it becomes too big...

NOTE: if during execution the program takes way too long, feel free to kill it and just upload what it generated. Since it's bruteforcing many combinations of values it could take at least an hour or two depending on how well the drive works.

NOTE2: Even if it doesn't look like it's doing anything while it scans for values of 3C, it's working. So don't kill it right away. It will only print when data is returned by the drive using the SCSI command, not for when the command passes/fails.

EDIT: Program requires cygwin be installed but should have the .dlls needed to use it standalone. Let me know if there are any issues. If you were having issues before and didn't say anything, grab the zip and try it again.

I got an error that appeared twice saying that "cygwin1.dll" was missing, and then the program closed.

Post's attachments

sg_raw.png 7.62 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Lugamo wrote:

I got an error that appeared twice saying that "cygwin1.dll" was missing, and then the program closed.

Redownload the zip and try again, it needs cygwin installed and I forgot to include some of the .dlls inside. tongue

ehw wrote:
Lugamo wrote:

I got an error that appeared twice saying that "cygwin1.dll" was missing, and then the program closed.

Redownload the zip and try again, it needs cygwin installed and I forgot to include some of the .dlls inside. tongue

Now it works. I'll reply again with whatever it generated once it finishes. This is the disc I'm dumping right now, with my LG WH14NS40.

ehw wrote:

There was a tool written in the past that used to do just this, however the source was never released

Is there source for the new tool you've just linked?

ehw wrote:

Even if it doesn't look like it's doing anything while it scans for values of 3C, it's working

It would be nice to get an update as to where it's up to in the list of commands (even just "No sectors returned from 3C 08 __", i.e. a message every time a whole chunk of 256 commands were checked with nothing returned), so I can tell that *some* progress is happening and it isn't some hardware issue causing it to freeze.

Deterous wrote:
ehw wrote:

There was a tool written in the past that used to do just this, however the source was never released

Is there source for the new tool you've just linked?

ehw wrote:

Even if it doesn't look like it's doing anything while it scans for values of 3C, it's working

It would be nice to get an update as to where it's up to in the list of commands (even just "No sectors returned from 3C 08 __", i.e. a message every time a whole chunk of 256 commands were checked with nothing returned), so I can tell that *some* progress is happening and it isn't some hardware issue causing it to freeze.

Source code:
https://github.com/hiddenpalaceorg/DVDRawBruteforce

New release, added a progress bar:
https://github.com/hiddenpalaceorg/DVDR … 2023-09-30

smile

7 (edited by Lugamo 2023-10-01 04:37:31)

Have you considered adding a license?

Dump update: It has been stuck in F1 0C for a while now.

8 (edited by ehw 2023-10-01 06:15:06)

Lugamo wrote:

Have you considered adding a license?

Dump update: It has been stuck in F1 0C for a while now.

Not really, not interested for something like this.

BTW, if for whatever reason it seems like its way too slow (like 3+ hours) or seems like it stalled, feel free to stop it by using Ctrl+C and upload what you have. Some drives just don't like being bruteforced to that high value. I could probably restrain the values it's bruteforcing with but you never know if a vendor might be sneaky and put something at 3C FF FF or something, lol.

Had a couple of submissions already. Keep them coming! Tell your friends and spread the word and see if we can test as many drives as we can.

ehw wrote:

BTW, if for whatever reason it seems like its way too slow (like 3+ hours) or seems like it stalled, feel free to stop it by using Ctrl+C and upload what you have. Some drives just don't like being bruteforced to that high value. I could probably restrain the values it's bruteforcing with but you never know if a vendor might be sneaky and put something at 3C FF FF or something, lol.

Here you go. I will try with another DVD later.

Post's attachments

PCActual20UnfinishedDump.7z 94.66 kb, 2 downloads since 2023-10-01 

You don't have the permssions to download the attachments of this post.

Here's mine from a TSSTcorp SE-208DB.  Looks like 3C 02 00 returned 0x8BC bytes of scrambled sector data.  So did F1 84, which was offset by about 13.5 sectors (0x7800 bytes).  And F1 8A returned the PFI, funnily enough.
Disc used: [PC] FlatOut 2 (Europe) (En,Fr,De,Es,It) (Xplosiv)


Are PS2 discs also disallowed or not?  At first I started with a PS2 disc since I have way more of those on hand, but shortly after restarted with a PC disc.  I don't know if you look for specific patterns in testing, but my assumption was that maybe you expect the 0th sector to be empty, which is not the case on PS2 discs since those have the PlayStation 2 logo seen when booting a game XORed across the first 12 sectors.

Post's attachments

upload_me.zip 233.84 kb, 1 downloads since 2023-10-01 

You don't have the permssions to download the attachments of this post.

11

Lugamo wrote:
ehw wrote:

BTW, if for whatever reason it seems like its way too slow (like 3+ hours) or seems like it stalled, feel free to stop it by using Ctrl+C and upload what you have. Some drives just don't like being bruteforced to that high value. I could probably restrain the values it's bruteforcing with but you never know if a vendor might be sneaky and put something at 3C FF FF or something, lol.

Thanks! Added

Here you go. I will try with another DVD later.

Edness wrote:

Here's mine from a TSSTcorp SE-208DB.  Looks like 3C 02 00 returned 0x8BC bytes of scrambled sector data.  So did F1 84, which was offset by about 13.5 sectors (0x7800 bytes).  And F1 8A returned the PFI, funnily enough.
Disc used: [PC] FlatOut 2 (Europe) (En,Fr,De,Es,It) (Xplosiv)

Are PS2 discs also disallowed or not?  At first I started with a PS2 disc since I have way more of those on hand, but shortly after restarted with a PC disc.  I don't know if you look for specific patterns in testing, but my assumption was that maybe you expect the 0th sector to be empty, which is not the case on PS2 discs since those have the PlayStation 2 logo seen when booting a game XORed across the first 12 sectors.

You can use PS2 discs, absolutely. It's just that I've only ever tried standard DVDs that were mastered like a traditional DVD ISO disc, like something with a UDF or ISO partition as some other discs may be harder for older or other drives to read or might not have the same raw data that the program uses to identify it (like weird DRM or something).

Thanks for the submission! big_smile

BTW, new version pushed. This one adds a timeout of 20 seconds per bruteforce'd value to cover moments where it looks like the program is frozen. Hopefully it takes care of that...

I also noticed a small oversight with the F1 progress bar so hopefully that's fixed too.

https://github.com/hiddenpalaceorg/DVDR … 2023-10-01

I will also add my logs. I checked on BW-16D1HT, but with firmware 3.10 modded by RibShark

Post's attachments

upload_me.zip 754.17 kb, 1 downloads since 2023-10-01 

You don't have the permssions to download the attachments of this post.

More logs from two drives (SOHD-167T and SH-216BB)
BTW: Is it possible to add a function to the program code that allows the program to end when the drive is not responding? My SH-216BB hang up when reading 256 sectors (F1 I think). It happened twice and each time I had to restart the drive from the power supply. When the drive was suspended, the program was still waiting for a response from it. It would be nice if the program restarted the drive in such situations, or if the drive stopped responding after some time, it would terminate its operation

Post's attachments

upload_me.zip 6.18 kb, 5 downloads since 2023-10-01 

upload_me.zip 208.08 kb, 1 downloads since 2023-10-01 

You don't have the permssions to download the attachments of this post.

14

MrPepka wrote:

I will also add my logs. I checked on BW-16D1HT, but with firmware 3.10 modded by RibShark

Thanks! For some reason it should've found a working command but I think I might've boo boo'd the addition of a timeout because apparently sometimes when sg_raw timesout it causes the drive to reset which would clear the cache, which means we lose LBA 0 which makes everything after the first timeout potentially useless. I think that might've happened here. Do you remember when you were testing the drive when it exactly started to take a really long time for the progress bar to move?

I might have to revert the timeout thing and just have people CTRL+C it if it starts going really really slow. At least for the time being.

MrPepka wrote:

More logs from two drives (SOHD-167T and SH-216BB)
BTW: Is it possible to add a function to the program code that allows the program to end when the drive is not responding? My SH-216BB hang up when reading 256 sectors (F1 I think). It happened twice and each time I had to restart the drive from the power supply. When the drive was suspended, the program was still waiting for a response from it. It would be nice if the program restarted the drive in such situations, or if the drive stopped responding after some time, it would terminate its operation

It's supposed to timeout as of the latest version after 20 seconds and move on to the next number...

With the BW-16D1HT, the progress bar did not move for a very long time, but rather at a normal pace
As for SH-216BB, I tested the latest version, but when the drive hang up, the 20-second timeout did not work, the program was constantly waiting for a response from the drive. Only when I stopped the program and restarted the drive and then resumed the program, did the program continue to run

TSST TS-L632H doesn't support or what happens?

Post's attachments

logfile.log 2.71 kb, 1 downloads since 2023-10-01 

You don't have the permssions to download the attachments of this post.

Tested another drive I had laying around, as well as sent this in a chat with a couple of friends who were also interested in it, so there may be more to send soon.  The laptop HP SU-208CB drive I just tested here I'm like 99.1% sure is actually a rebranded TSSTcorp drive, so make of that what you will for which brand to put that under.  I also have a few more to test, those being an assortment of drives in other old PCs but that's for later.

It seemed to completely hang when it got to F1 05.  I couldn't kill the process or anything, so I ended up force shutting it down after like an hour which means it didn't flush anything to the log file either.  But 3C 02 00 got 0x8BC bytes of raw sector data, at least.  I re-ran just the beginning to get the basic drive info at least, but I don't feel like running the whole thing on that crappy laptop again, as it was excruciatingly slow.

Jason098 from the aforementioned group chat modified it slightly to run on Linux since almost everyone in my friend group uses it, which also means the get_dvd_drive_info() function had to be commented out for them.
There's also an odd edge case to consider: one of Jason's discs had "unexpected" data in the 1st byte of the PSN.  AFAIK, normally (if the PFI is anything to go by) the leftmost byte is unused so usually it's just 0x00, but that one disc had "random" values in there like 50030000, 20030001, etc.  It may be better to mask out the 1st byte and only check if the lower 24 bits equal to and increment from 0x030000 instead.  It's possible a field marked as "No" could actually be wrong and mislabelled due to this oddity.

There were some other drive issues he had too, like if the lower nybble of the 3C command ended with 0x0B, the drive would pretty much hang so we had to work around that as well (which I understand he also later brought up to you in another server.)

Post's attachments

HP SU-208CB.7z 7.24 kb, 1 downloads since 2023-10-01 

You don't have the permssions to download the attachments of this post.

18 (edited by ehw 2023-10-01 22:45:26)

I made a new version to try to deal with the drive timeouts and potential possibility that it clears the cache when it does so. I don't have any drives on hand that does this so it needs testing:
https://github.com/hiddenpalaceorg/DVDR … 023-10-01a

It will reread LBA 0 so it puts it back on the cache again if a specific error is returned from sg_raw. Hopefully this helps a bit...

Edness wrote:

Tested another drive I had laying around, as well as sent this in a chat with a couple of friends who were also interested in it, so there may be more to send soon.  The laptop HP SU-208CB drive I just tested here I'm like 99.1% sure is actually a rebranded TSSTcorp drive, so make of that what you will for which brand to put that under.  I also have a few more to test, those being an assortment of drives in other old PCs but that's for later.

It seemed to completely hang when it got to F1 05.  I couldn't kill the process or anything, so I ended up force shutting it down after like an hour which means it didn't flush anything to the log file either.  But 3C 02 00 got 0x8BC bytes of raw sector data, at least.  I re-ran just the beginning to get the basic drive info at least, but I don't feel like running the whole thing on that crappy laptop again, as it was excruciatingly slow.

Jason098 from the aforementioned group chat modified it slightly to run on Linux since almost everyone in my friend group uses it, which also means the get_dvd_drive_info() function had to be commented out for them.
There's also an odd edge case to consider: one of Jason's discs had "unexpected" data in the 1st byte of the PSN.  AFAIK, normally (if the PFI is anything to go by) the leftmost byte is unused so usually it's just 0x00, but that one disc had "random" values in there like 50030000, 20030001, etc.  It may be better to mask out the 1st byte and only check if the lower 24 bits equal to and increment from 0x030000 instead.  It's possible a field marked as "No" could actually be wrong and mislabelled due to this oddity.

There were some other drive issues he had too, like if the lower nybble of the 3C command ended with 0x0B, the drive would pretty much hang so we had to work around that as well (which I understand he also later brought up to you in another server.)

I'm not sure whether to trust when the left most byte is different because other bytes within the page might be altered randomly from different runs too. I'm making notes of when this happens in the notes field but I'm looking for when the first 4 bytes are exactly 00 03 00 00 to know that it's workable for sure.

Keep them coming lol.

MrPepka wrote:

TSST TS-L632H doesn't support or what happens?

Try again? Maybe the drive doesn't like the disc and can't read it.

Sony DDU1611
https://mega.nz/file/JjhDWa4J#yQ7NM5jwH … dujWrBMOYg (The logs weigh over 1 GB, it's worth having a good connection to download them!)

ehw wrote:

I made a new version to try to deal with the drive timeouts and potential possibility that it clears the cache when it does so. I don't have any drives on hand that does this so it needs testing:
https://github.com/hiddenpalaceorg/DVDR … 023-10-01a

It will reread LBA 0 so it puts it back on the cache again if a specific error is returned from sg_raw. Hopefully this helps a bit.

OK, these are the new logs

Post's attachments

upload_me.zip 756.27 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

21 (edited by MrPepka 2023-10-02 00:17:48)

ehw wrote:
MrPepka wrote:

TSST TS-L632H doesn't support or what happens?

Try again? Maybe the drive doesn't like the disc and can't read it.

The same. I tried with 2 different discs, the drive reads them correctly

Post's attachments

logfile.log 3.36 kb, 1 downloads since 2023-10-02 

You don't have the permssions to download the attachments of this post.

Lugamo wrote:
ehw wrote:

BTW, if for whatever reason it seems like its way too slow (like 3+ hours) or seems like it stalled, feel free to stop it by using Ctrl+C and upload what you have. Some drives just don't like being bruteforced to that high value. I could probably restrain the values it's bruteforcing with but you never know if a vendor might be sneaky and put something at 3C FF FF or something, lol.

Here you go. I will try with another DVD later.

It has finished, here it is. The dumped disc is Earth 2160.

Post's attachments

upload_me_2023-10-02_03.08.49.7z 124.92 kb, 1 downloads since 2023-10-02 

You don't have the permssions to download the attachments of this post.

23

Lugamo wrote:
Lugamo wrote:
ehw wrote:

BTW, if for whatever reason it seems like its way too slow (like 3+ hours) or seems like it stalled, feel free to stop it by using Ctrl+C and upload what you have. Some drives just don't like being bruteforced to that high value. I could probably restrain the values it's bruteforcing with but you never know if a vendor might be sneaky and put something at 3C FF FF or something, lol.

Here you go. I will try with another DVD later.

It has finished, here it is. The dumped disc is Earth 2160.

Thanks! Keep them coming.

MrPepka wrote:
ehw wrote:
MrPepka wrote:

TSST TS-L632H doesn't support or what happens?

Try again? Maybe the drive doesn't like the disc and can't read it.

The same. I tried with 2 different discs, the drive reads them correctly

Not sure what to say...the drive just doesn't like it..

24 (edited by MrPepka 2023-10-02 14:16:42)

Generally, when it comes to TS-L632H (or SN-S082H, it's the same construction), in addition to the log, this file is also created

Post's attachments

5a_mode_sense_page01.bin 14 b, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.

Disc: Split/Second (Argentina) (En,Fr,De,Es,It).
Drive: Plextor PX-755A.

Post's attachments

upload_me.zip 122.96 kb, 1 downloads since 2023-10-02 

You don't have the permssions to download the attachments of this post.