1 (edited by jamjam 2013-01-23 00:27:41)

Topic: New dreamcast dumping program (test please)

Here's a work in progress commandline alternative to CDRWIN, specific to dumping dreamcast games. Do the trap disc as normal then use this instead. The main benefit is it reads in ranges, and if a read error is encountered it retries. Tries to avoid manual partial sector range hell wink

Features:
+ Dumps in sections to avoid redumping good sections later
+ Redumps sections until there's a match (thus the section is deemed good) or after certain number of passes
+ Detects if sections are already written to file and doesn't try to read those sections again (resume skip)
+ Saves read hashes to file, for matching sections from different program executions (resume pickup)
+ When it is deemed that all sections are dumped correctly, creates dense.bin as the input for ice.exe

Usage: DCdumper.exe drive_letter [options]

[options]
-h      - This help.

-c#     - Cut-off: After which disc is dumped in one lump instead of sections
          Min: 44990 (all at once)  Max: 549150 (no cut-off)  Default: 549150

-dt -df - Delete temporary files: Delete sections and hashes after creating dense.bin
          Default true

-ft -ff - Fake reads: Fake read nearby to try and kick program out of read errors
          Default true

-i#     - Set number of sectors per section
          Min: 26  Max: 504161 (all at once)  Default: 10289

-p#     - Passes: Set maximum number of passes
          Min: 2  Max: 100  Default: 10

-st -sf - Step back: Fake reads after a pass to step to beginning
          Default: true

-t#     - Reread attempts: Max number of re-read attempts on a read error
          Min: 0  Max: 20  Default: 3

If using a "Lite-On SOHD-167T", use option -c400000 (may help with many other drives too).

Please test with existing dumps and report what happens, send the log file if possible. If it creates dense.bin, run it through ice.exe and see if dense.bin is valid (and output tracks match redump). If you can't get it to dump at all, let me know your machine, optical drive, os and anything else you think is relevant.

Suggestions and feedback welcome.

http://www.multiupload.nl/J5O2BNTD4P (0.42a)

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)

Re: New dreamcast dumping program (test please)

jamjam wrote:

Is the same true of "Lite-On SOHD-167T"?

No jamjam, SOHD-167T seems to be limited on the firmware side, it is returning the WRONG sectors in any way!

I guess they locked it for using LBA higher than ~450075 (99:59:74), or 0x6DE1B.
Using a higher LBA numbers leads to interruption and the drive reads wrong sector ranges in some way.

PX-760A (+30), PX-W4824TA (+98), GSA-H42L (+667), GDR-8164B (+102), SH-D162D (+6), SOHD-167T (+12)

3 (edited by amarok 2011-09-14 18:19:48)

Re: New dreamcast dumping program (test please)

Suggestions and feedback welcome. Will post source at a later date.

All I get is the following error message: "Could not get handle to CD. Check input." I'm sure I've used the correct parameters ('dcdump 3:0:0 J:'). I suppose my IDE to USB adapter is to blame again... It doesn't work with ddump either. Too bad, I think this might actually be the tool I've been waiting for for years tongue The only suggestion I have is... do it more like IsoBuster (which works for me), and less like ddump tongue

My "TSSTcorp DVD-ROM SH-D162C" has read errors without fail on partial ranges over 440000 initially, but after a few passes can read them.

My SH-D162D does that from ~495.000 onwards, but can't start reading above even after a few passes. I have to extract 495.000-549.150 in one go.

Re: New dreamcast dumping program (test please)

Thank you both.

@iRobot: I'll make a switch or check a device string to read the last bit in a big lump on the lite-on.

@amarok
Try just 'j', the second parameter is treated as a single character and passed to startstop.exe. Not handled very well at all, just a quick workaround until I can get to grips with windows api, and just use #:#:# OR drive letter.

If ddump doesn't work I doubt this will be any different, sorry. The usb probably is the problem. I'll look into getting usb drives working, probably over my head for now though.

If by do it like isobuster you mean work with your drive, it might be possible. If you mean the way it dumps or does something, let me know the difference.

The SH-D162D might work with partial reads past that point. The program does try reading sector ranges a lot, and the way it cycles through the sections every pass seems to help. If it's like the lite-on though, I'll handle it the same way.

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)

Re: New dreamcast dumping program (test please)

I want to test this but im very unclear as to how????

Please let me know, what to do, and could I have the required dlls, as im getting a dll error for starters

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

6 (edited by amarok 2011-09-15 01:07:57)

Re: New dreamcast dumping program (test please)

Try just 'j', the second parameter is treated as a single character and passed to startstop.exe. Not handled very well at all, just a quick workaround until I can get to grips with windows api, and just use #:#:# OR drive letter.

That was the first thing I tried. Same result.

If by do it like isobuster you mean work with your drive, it might be possible. If you mean the way it dumps or does something, let me know the difference.

Hehe, all I can say is that ddump is the only dumping tool that doesn't work with my USB adapter (well, okay, neither does dcdump, obviously tongue ). CloneCD, Alcohol, IsoBuster and so on, they all work just fine.

The SH-D162D might work with partial reads past that point. The program does try reading sector ranges a lot, and the way it cycles through the sections every pass seems to help. If it's like the lite-on though, I'll handle it the same way.

No, it's not like the Lite-On. The SH162D reads sectors above 450.000 correctly, it just refuses to start reading at ~495.000+.

Re: New dreamcast dumping program (test please)

tried to dump a game with Lite-On SOHD-167T
haven't had much luck, read errors just like with cdrwin hmm
it's strange the game discs don't have any scratches on them

when i first started the program i received msvcp100.dll missing (Win7 x64)
after i downloaded the vcredist_x86.exe it started fine

couldn't get the device id using the program so i had to look it up in device manager (1:0:0)

will try again with discs that i dumped successfully in the past (Sega Rally & Resident Evil 2 Leon's Disc)

thank you!

Re: New dreamcast dumping program (test please)

@tossEAC
Try what Pikmin says, vcredist_x86.exe. Google "visual c++ runtime" incase your computer needs a slightly different version. Beyond that, it should have everything (just startstop.exe and akrip32.dll in same directory as DCDump.exe). Works on xp sp3, haven't tested with anything else.

As for using:

  • Run ddump.exe with no parameters to get a list of all your devices. Find your dreamcast dumping device, and note the #:#:#

  • Also get the drive letter of your device

  • Then follow the dreamcast guide until it says use CDRWin, and instead use DCDump with parameters
    DCDump.exe #:#:# drive_letter

For example my drive is 0:0:0 with drive letter d so the commandline would be
DCDump.exe 0:0:0 d

It will then start dumping the disc in sections, you'll see it says what it's doing. The dump and temp files get saved to the same directory as the program.

@amarok
The program exits before using the drive letter so it isn't the current problem, but for future reference.

CloneCD, Alcohol and Isobuster are commercial grade programs, that need to be robust and work everywhere with everything. They probably have profiles to work well in every situation. It's wishful thinking to expect homemade programs to have the same level of robustness. The main problem here seems to be that you can't find a usb device with #:#:# without some extra steps. If I can't get it to work, maybe when source is available someone else can.

@Pikmin
Read errors for every read attempt, no successes at all? If there is a successful partial read you should see " - Initial dump" or " - Match: ...". Yes please try more tests, thank you smile

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)

9 (edited by tossEAC 2011-09-15 12:58:02)

Re: New dreamcast dumping program (test please)

working now thanks folks.

I have noticed, it gives read errors on, section 0,47 & 48, I dont think it has got anything to do with scratches, it seems its just a problem with the drive somehow, not being able to read the very start or the very end of the disc.

The first 3 I tried, had problems, but read in very fast, about 10 minutes each disc all passses, this 4th disc I am trying is reading in much slower, but didnt give read errors on the initial dump on any section.

And so far what is dumping is typically getting matches second time, apart from when it has scratch problems, which all my disc now im unable to dump sadly have got or had bad scratch problems.

Im sad, becasue the other day before i knew of this program, i cleaned two discs, really well, and both had one error when ice ran through the bin, and me being stupid tried a re-clean, then they ended up with a lot more errors

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

10 (edited by Pikmin 2011-09-15 13:27:21)

Re: New dreamcast dumping program (test please)

jamjam wrote:

As for using:

  • Run ddump.exe with no parameters to get a list of all your devices. Find your dreamcast dumping device, and note the #:#:#

  • Also get the drive letter of your device

  • Then follow the dreamcast guide until it says use CDRWin, and instead use DCDump with parameters
    DCDump.exe #:#:# drive_letter

For example my drive is 0:0:0 with drive letter d so the commandline would be
DCDump.exe 0:0:0 d

i have tried running it without any options and all i get is this:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Users\Pikmin\Desktop\DCDump (0.1a)>DCDump.exe
Usage: DCDump.exe #:#:#

Help: DCDump.exe -h

C:\Users\Pikmin\Desktop\DCDump (0.1a)>

(both on Win7 x86 & x64)

11 (edited by tossEAC 2011-09-15 15:25:37)

Re: New dreamcast dumping program (test please)

I managed to get one previously undumpable to dump and ice exe said no errors (single track game) smile Your my personal saviour smile

But Im having a lot of read error issues with section 48, even with the disc i dumped, I had to remove sections 31-47, and redump them before the drive would physically dump section 48. Can you work on the section 48 bug please jamjam.

And BTW DDump did nothing when i ran it, CDRWin said my drive was 3:0:0 and mthat didnt work, in the end i tried jamjams setting 0:0:0 and that worked.

Jamjam maybe you need to right a tool that does the DDump part to get the drive id.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

12 (edited by jamjam 2011-09-17 01:36:14)

Re: New dreamcast dumping program (test please)

Ok, new version (0.2a) which adds the ability to change some options. If resuming from a previous dump, don't change the commandline options from what they were (as it may not resume or clean up files if options result in different sections to original options).

Usage: DCdumper.exe #:#:# drive_letter [options]

[options]
-h      - This help.

-c#     - Set cut-off sector (after which disc is dumped in one lump)
          Min: 44990 (all at once)  Max: 549150 (no cut-off)  Default: 549150

-dt -df - Set whether or not to delete temporary files when creating dense.bin (default true)

-i#     - Set number of sectors per section
          Min: 26  Max: 504161 (all at once)  Default: 10289

-p#     - Set maximum number of passes
          Min: 2  Max: 100  Default: 10

@Pikmin
I was talking about ddump, the program to dump safedisc discs. I'll rename to DCdumper to avoid confusion.

@tossEAC
Try the new version with option -c400000. Also, if it gets to the end of execution and hasn't dumped everything, there's no need to delete anything (unless it ended badly and didn't write a file properly). The program will resume where it left off.

-c400000 should also be used by "Lite-On SOHD-167T", to avoid the LBA errors.

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)

Re: New dreamcast dumping program (test please)

Heres a fix that would be easy to implement with an option.

My drive struggles to initialise the dump from 44990, but it has no problem if it first trys from 50000, then i cancell, and jog back to 44990, then it will read no problem.

So the fix would be for Section:2 to be dumped first, then Section:1 then Section:3 --> onwards.

What do you think jamjam sound possible, ill try it out if you like.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

14 (edited by tossEAC 2011-09-15 16:49:08)

Re: New dreamcast dumping program (test please)

Another suggestion for an option aswell as the other suggestion, would be to have it read three sections, then verify three sections, to avoid the drive going from the end to the start, which it doesnt do very well.

I just got a disc to dump section 1-49 initial dump, no read errors, but then when it went back to the start to verify, section 1+2 had read errors, im hoping all other sections get matches, as dumping the first two sections is not as big a problem as dumping the last two.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

15 (edited by tossEAC 2011-09-15 16:48:45)

Re: New dreamcast dumping program (test please)

smile

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

I fail to see the difference between faking a read from 50000, or trying to dump a different section then going back to the section with a read error (which is what it does currently). It doesn't matter if there are read errors, as long as any two times it gets dumped with the same hash (after the match it doesn't try again). However I can implement it if it is necessary, and I can see how it may be if there is one section left which errors, as there is nothing left to jog it.

If it says initial dump, it read the section for the first time, hash stored. Nothing to compare against so it may have been read wrong.
If it says read error, the read error has been caught, hash not stored. Likely it errored in the same way CDRWin wouldn't start to read anything. Not a major problem.
If it says match, the read just done matches one of the previous successful reads. Rejoice etc
If it says non-match, it was read, and there were previous reads, but the current read doesn't match any of the previous. All or all but one of the reads so far have errors.

How about reading from start to finish, then from finish to start? I assume the big jump from end to start may screw it up sometimes atm, this way may be more gentle and have the desired effect.

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)

Re: New dreamcast dumping program (test please)

You sort of understand, but i dont think you see fully, and its not your programme at fault. Its great stuff smile

I'll explain a bit more later, at the moment im using a combination of cdrwin and your tool, cdrwin to dump the last sections.

I have been using it all day, and I understand how it works, i even have one good dump to prove it, its when it doesnt work, the main problem lies in going from the end of the disc to the start, this drive aint a native gdrom drive afterall.

I hope you can try those suggestions i made earlier at least try reading Section:2,1,3 in that order to see if it helps ill test it if you can writew the option, it wouldnt hurt to give it a try, then we can deal with the end sections later.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

DOES this help, If it says initial dump, that means the drive has read the disc thats all, but good, but may contain errors.

If on the verification pass it says match, match, match.... sit back and relax.

If it says no match, then this is a big problem as its 99% certain that the discs scratches are causing the problem.

But if it says read error, then this is not because of scratches, its more to do with the drive, gdrom and software.

If you have a read error on SECTION 1 and 49, I Really dont think the samsung drive can read section 1 then go to section 49, Im pretty sure this is impossible. Not without reading somewhere in between, which could be the fix you need. By maybe telling the programme if it needs section 49, then try to start reading before that, but only read to memory then once it has found its position the read could be moved up one section untill it finds its position and repeated untill you get to the section you actually need ie. 49

So if section 1 and 49 read error. Then even after the tenth pass, it will just keep saying read error on and on.

But the first section is easy to fix just read it with cdrwin.

The 49th Section, is more difficult.

You can tell CDRWin to read from 477128-549150 then split it into 24199728 byte packages, and just keep the ones you need.

477128 was the highest position I could find that CDRWIN would start from, higher and you just get an error.

Does that make a bit more sense now jamjam smile

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

Trying a disc now, 49 Initial Dumps, no rewad errors, goes back to verify, guess what read error, Non-Match, Non Match.

Basically I think these non matches are because the first read error cocked it up.

This programme still needs lots of work, this ones giving every section non matches, i have had enough for today.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

JamJam could you help me with this aswell if at all possible.

I have a xml file which i need to remove certain lines quickly

Example.

    <Event InTC="00:00:33:17" OutTC="00:00:33:19" Forced="False">
      <Graphic Width="1241" Height="966" X="47" Y="63">Ep 1_exp_exp_exp_0089.png</Graphic>
    </Event>
    <Event InTC="00:00:33:19" OutTC="00:00:33:20" Forced="False">
      <Graphic Width="1241" Height="966" X="47" Y="63">Ep 1_exp_exp_exp_0090.png</Graphic>
    </Event>
    <Event InTC="00:00:33:20" OutTC="00:00:33:20" Forced="False">
      <Graphic Width="1241" Height="966" X="47" Y="63">Ep 1_exp_exp_exp_0091.png</Graphic>
    </Event>
    <Event InTC="00:00:33:20" OutTC="00:00:33:22" Forced="False">
      <Graphic Width="1241" Height="966" X="47" Y="63">Ep 1_exp_exp_exp_0092.png</Graphic>
    </Event>

I would like a tool that could remove all the lines that have this one thing in common.

    <Event InTC="00:00:33:20" OutTC="00:00:33:20" Forced="False">
      <Graphic Width="1241" Height="966" X="47" Y="63">Ep 1_exp_exp_exp_0091.png</Graphic>
    </Event>

You see on this example I have InTC="00:00:33:20" & OutTC="00:00:33:20" and the times are both the same, I Want all lines from this xml to be removed if InTC = OutTC. You will probably need to find a way to remove the two lines below the lines with identical times to make it work. ie.

      <Graphic Width="1241" Height="966" X="47" Y="63">Ep 1_exp_exp_exp_0091.png</Graphic>
    </Event>

Would this be hard to make a programme to do this, if not do you need an example xml to try it on, this would save me hours and hours of work and make me very happy, and I'll find a few games I have dupes of and you can have them when you send me that drive, I have a few xbox games i could send if that will be a help, I dont have the space to keep all the games I have so getting rid of ten xbox games will at least make 10 spaces for games I havent got.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

I just buffed a disc a bit to much, has one deep mother of all scratches in it, so i put a load of displex polish on the edges of the disc, and buffed only for a bit, but that polish is powerfully sticky, and after only a bit of a buff, the disc started turning into jello (jelly) it went all soft and started warping in my hand, but funnilly enough I just sprayed on some other polish and buffeed the displex off and now its normal again. sad jello

BTW Jammy, I was using a SH-D162D before, but have just had a try on 162C, its a completely different ballgame, but I remember when i first started dumping dreamcast using the 162C, a lot of my dumps had the odd byte of data wrong, like FF instead of FE, and I think I saw that before when i used it.

Also is that SH-D162B (Black) any good for dreamcast, because my xbox/360/dreamcast drive i use the most is on its last legs, mainly becuase of dreamcast dumping recently. So i was thinking of replacing it with a dreamcast only drive, I dont casre if it doesnt do 360, just dreamcast. So let me know if the black one works on Dreamcast.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

hi, jamjam

i tried DCdumper.exe 0.2a but it would just crash on my PC (XP SP3)

usually programs would use SCSI Pass Through Interface (SPTI) to communicate with drive
(SCSI_PASS_THROUGH_DIRECT / SCSI_PASS_THROUGH_DIRECT_WITH_BUFFER)
it has least problems
and you can get handle with CreateFile, so only letter is needed - it shouldn't matter if its USB or ATA or SATA
there are good examples in Kris Kaspersky's book
http://forum.mobilism.org/viewtopic.php … ;view=next
(look for RAW reading through SPTI
there should be precompiled examples and .c files that came on CD with the book somewhere on web too,
but if you can't find those PM me and i'll send them to you)
and on Truman's homepage
http://www.cdtool.pwp.blueyonder.co.uk/workshop.htm

good luck

Re: New dreamcast dumping program (test please)

@themabus
Thank you, I'll look into it. My choice of akrip was just because it was the first thing I found that worked, but it seems people are having problems.

@tossEAC
I will try your suggestions if necessary, fake reads particularly sound potentially useful. No I will not be your xml code monkey bitch tongue
I have a black drive but I think it is C version.

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)

Re: New dreamcast dumping program (test please)

Ok, xml monkey bitch is not needed smile

And BTW I got that disc I turned to jello to dump perfectly using your amazing tool, thats one off the list, it was Urban Chaos btw.

He who controls the SPICE... controls the UNIVERSE!
The SPICE must flow.

Re: New dreamcast dumping program (test please)

New version (0.3a).

Usage: DCdumper.exe drive_letter [options]

20110917 (0.3a)
+No longer uses akrip32, now DeviceIO (SPTD)
+No longer uses startstop.exe, now DeviceIO (Load device)
+Section hashes stored to file as successful reads are completed (better resume function)
+Log file created (mirrors console)
+Pass drive letter, not scsi id

Thanks themabus, much cleaner now.

PS3Dec (decrypt ps3 images), PS3DumpCheck (check integrity), GetKey (dump PS3 metadata), DatSplit (split redump dats), GPack (compress related images together)