1 Reply by A Murder of Crows

(6 replies, posted in Guests & account requests)

This came from Knight0fDragon:


ok, now that I have some time, I can explain the issue. 

So I have developed an app that is designed to make patching Sega Saturn games as simple as possible. 

What it does is basically extracts all the files from a disc, and then rebuild the files to generate a new ISO.  One of my conditions was that if you just pressed build without applying any patch, it must rebuild the disc perfectly. 

Well, upon first working with MPEG games,  I would come to find out that the CUE SHEETS on REDUMP would lay out the track info incorrectly. 

A good example would be Lunar Silver Star Complete. 

If you take notice,  the last track would be listed with a pregap of 149 sectors (1 minute: 59 seconds, 74 frames).  This came across as weird to me.  Well upon further investigation,  it seems that the issue is with how XA data is handled.  With sega saturn games, Track 1 is always Mode 1 data.  Beyond that, it could be either Mode 2 XA, or audio.  Now the thing with Saturn, some games have files called CDDA files, and the MPEG games have obviously MPEG files.

If you try to open these files, your system should yell at you.  The reason behind it is because in the Mode 1 track's file table, these files are listed as XA files, meaning they have extra information on them, but Mode 1 has no idea how to read them. 

The Saturn, however, does. 

Now another thing:
- Mode 1 data is 2048 bytes. 
- Mode 2 XA and Audio are 2352 bytes. 

So you need to adjust the LBA and the file size to account for the difference from 2048 to 2352:

Basically you
- take the sector size and multiply it by 2352 to get the proper address,
- and you take the size, divide by 2048,
- and multiply by 2352 to get the correct size.) 

Now, when you do all this math,  it ends up giving the correct positions for the pregaps in the CUE sheet.

Since you guys are all about accurate preservation, I figured it may be a good idea to let you know that perhaps these issues are coming up. 

If you head over to Sega Xtreme you can grab a copy of my app
https://segaxtreme.net/resources/sega-s … atcher.73/ 

Test it with lunar star story MPEG and build a CCD.   

On my CCD, you will notice
- Track 3 index 1 points to 244,897, and
- Track 4 index 0 points to 255,744. 

If you go to the REDUMP,
- Track 3 index 1 points to 244,897, and
- Track 4 index 0 points to 255,745. 

In the File table,
- CDDA1 points to 0x03BCA1 (244,897) with a size of 0x0152F800 (22,214,656) 

Now if you divide this size (22,214,656) by 2,048, you get 10,847.
244897 + 10,847 gets you 255,744, which is where Track 4 Index 0 should point.

Go to forum: Guests & account requests

2 Reply by A Murder of Crows

(6 replies, posted in Guests & account requests)

Sarami, i can forward the logs of my dump if that helps, but Knight's issue isn't directly with the dump produced so much as an issue of all MPEG games. He'll be the person who needs to explain his issues.

Go to forum: Guests & account requests

3 Reply by A Murder of Crows

(3,497 replies, posted in General discussion)

Hey Sarami, got any feedback on this:

http://forum.redump.org/topic/33374/iss … peg-dumps/

Go to forum: General discussion

4 Reply by A Murder of Crows

(6 replies, posted in Guests & account requests)

When i did my dump of Lunar tonight, i got the following:

Track 3:
25867296

Track 4:
1872192

Sarami may need to be brought in

Go to forum: Guests & account requests

5 Reply by A Murder of Crows

(9 replies, posted in General discussion)

While i personally am okay with renaming the folder, i really feel it shouldn't be needed.

This particular title is simply way too long, and we should be able to make a decision here as to what it should be, not only to remain consistent with other named images, but to remain compliant within windows name protocols.

That said, i've attempted to use the \\?\x:\ workaround and it only works if you choose not to do a fix. the moment you do a fix, the folder gets renamed and the files never get seen again.

Go to forum: General discussion

6 Topic by A Murder of Crows

(1 replies, posted in General discussion)

Hey all

So, i'm at the stage where i need to create my own DAT file but want to emulate the settings/structure Redump's dat files use perfectly.

I'd like to know what dat file creation software is being used, and what settings are used on that software to create the dats, particularly for the saturn games.

the biggest things i'm seeing are Categories (clrmame doesn't have/use them) and the fact that my dat says "Machine Name" instead of "Game name"

thanks!

Go to forum: General discussion

7 Reply by A Murder of Crows

(9 replies, posted in General discussion)

ssjkakaroto wrote:

The emulator BizHawk (https://github.com/TASVideos/BizHawk/releases/) comes with a tool called DiscHawk that also merges bin+cue files into ccd+img. You can try that.


Thanks for that tip. I'm curious how it handles these strange multi-index track cue files that sbitools choked on.

Go to forum: General discussion

8 Reply by A Murder of Crows

(9 replies, posted in General discussion)

F1ReB4LL wrote:

I think it should be a feature request for rommanagers to shorten the filenames in such cases.

Honestly, i don't think it should be up to the rom manager.

We're the ones setting the title. If we can't commit to a title that fits within the confines of the file systems we use, there's a serious problem. If we ask the rom manager programs to shorten the file/folder names for us, there won't be any reason to expect consistency from the different programs. They could shorten the file/folder names any way they choose, leading to inconsistent results.

I think the more appropriate response is for us to decide on the proper title as well as a shortened title, then include a text file of the full title as part of the image. Those who want to deal with the ridiculously long name can fight it on their own while the rest of us breathe a sigh of relief that we can actually let a rom manager update our roms without hanging on stupidly long titles/file names.

Go to forum: General discussion

9 Reply by A Murder of Crows

(9 replies, posted in General discussion)

Aringon wrote:

Why not C:\Saturn\Zoku Hatsukoi Monogatari - Shuugaku Ryokou (Japan) (Disc 4) (Daigakusei Jidai & Koukou kara Daigaku ni Agaru ma.bin

Why do you need to put it in a folder with its name, doubling the length of the file path?

We aren't, but when using a rom manager like clmamepro for the dat file to do things like set name updating, you'll find that because of the title chosen in the dat, the folder is created automatically.

Go to forum: General discussion

10 Topic by A Murder of Crows

(9 replies, posted in General discussion)

Hey everyone.

there is at least 1 title in the Sega Saturn set that is so long, windows can't deal with it once it has been "properly" named:

Zoku Hatsukoi Monogatari - Shuugaku Ryokou (Japan) (Disc 4) (Daigakusei Jidai & Koukou kara Daigaku ni Agaru ma no Haruyasumi)


There are several others that have similar problems with some of their files if the folder structure is "properly" named as well.

I'm wondering, is there a way we could maybe shorten this "official" title to make it more friendly with NTFS?

Go to forum: General discussion

11 Reply by A Murder of Crows

(9 replies, posted in General discussion)

I was able to get quite a few images converted using Wiggy2k's method of sbitools.

Unfortunately, 41 images didn't get converted, with sbitools claiming that they had invalid cue files.

While i haven't looked at every single Cue file from those having issues yet, the one thing i'm seeing are individual tracks broken down into multiple indexes

FILE "Striker 96 (USA) (Track 1).bin" BINARY
  TRACK 01 MODE1/2352
    INDEX 01 00:00:00
FILE "Striker 96 (USA) (Track 2).bin" BINARY
  TRACK 02 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:02:00
    INDEX 02 03:12:19
    INDEX 03 06:08:69
    INDEX 04 10:35:57
    INDEX 05 13:31:22
    INDEX 06 16:49:55
FILE "Striker 96 (USA) (Track 3).bin" BINARY
  TRACK 03 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:02:00

============

FILE "Dark Savior (Europe) (Track 1).bin" BINARY
  TRACK 01 MODE1/2352
    INDEX 01 00:00:00
FILE "Dark Savior (Europe) (Track 2).bin" BINARY
  TRACK 02 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:02:00
    INDEX 02 01:37:43
    INDEX 03 02:14:12
    INDEX 04 02:35:21
    INDEX 05 02:54:51
    INDEX 06 03:15:05
    INDEX 07 03:30:59
    INDEX 08 03:55:71
    INDEX 09 08:48:48
    INDEX 10 11:59:63
    INDEX 11 16:14:21
    INDEX 12 20:10:44
    INDEX 13 24:37:74


===================

CATALOG 0000000000000
FILE "Honkaku Hanafuda (Japan) (Track 1).bin" BINARY
  TRACK 01 MODE1/2352
    INDEX 01 00:00:00
FILE "Honkaku Hanafuda (Japan) (Track 2).bin" BINARY
  TRACK 02 MODE2/2352
    INDEX 00 00:00:00
    INDEX 01 00:03:00
FILE "Honkaku Hanafuda (Japan) (Track 3).bin" BINARY
  TRACK 03 AUDIO
    INDEX 00 00:00:00
    INDEX 01 00:02:00
    INDEX 02 02:05:00
    INDEX 03 05:29:60
    INDEX 04 08:29:12
    INDEX 05 12:45:11



I'm wondering if there is something anyone can help me do to get these compatible with SBITools. I've already reached out to the author of that program, but he's got life issues that will likely prevent him from working on this tool further for the time being.

Go to forum: General discussion

12 Reply by A Murder of Crows

(17 replies, posted in General discussion)

I apologize, i thought my question was fairly straightforward.

I don't know how you're calculating the total CRC-32. my attempts at using various calculators to take 2 or more CRC values out of the dat/webpage result and sum them up doesn't result in the value you have as total CRC.

I'm not interested in reinventing the wheel here. i need a solution that allows me to check the site pretty much any time and see if there are changes or updates to any disc (not just saturn mind you) and if there is a change against what was last checked/calculated, obtain the total CRC again and verify it against the dumps i currently have for that title.

I'd MUCH prefer some semi static listing of Total CRC, either in the database or in the dat, but upon opening the dat, i saw the value wasn't listed there either. I don't want to factor in your on the fly calculation into my programs/scripts as doing that will add an extra potential point of failure.  If you won't incorporate Total CRC as a static stat into either the dat or the webpages, i've got not choice but to do that on my own, but i still would rather be handed the code/algorithm than try to "figure it out".

And yes, the data base dump would also be super awesome since a lot of the requested data isn't in the dat file anyway.

Thank you!

Go to forum: General discussion

13 Reply by A Murder of Crows

(17 replies, posted in General discussion)

While we're here, @iRobot, i didn't see anything back from you about the calculation. I'm stuck in limbo for at least one project until we make progress with Total CRC32.

If we skip that one value and find a way to add it in later, i'd still like the rest of the data dump.

Go to forum: General discussion

14 Reply by A Murder of Crows

(9 replies, posted in General discussion)

hmm, this sounds fantastic, and certainly a lot easier to deal with than what dave and i started with. i'll give it a go

Go to forum: General discussion

15 Topic by A Murder of Crows

(9 replies, posted in General discussion)

F1ReB4LL suggested i reach out to LedZep and/or Sarami regarding a "gluer" tool to glue the various bins/tracks of a bin+cue together to form a single img, or a straight up converter, that would do a conversion of cue+bin to to CCD+IMG+SUB.

I've had successful conversions of bin+cue to ccd+img+sub by mounting and redumping, but i've also had several that ended up with an img file that fails to match a previously dumped crc-32, and have run into some types of images that the tools i currently have are unable to deal with properly.

Is there any chance of getting a tool that would help automate this sort of process?


Thanks all

Go to forum: General discussion

16 Reply by A Murder of Crows

(17 replies, posted in General discussion)

am i able to get any forward momentum on at least the calculation for using the dats as a source?

Go to forum: General discussion

17 Reply by A Murder of Crows

(17 replies, posted in General discussion)

wiggy2k wrote:

@AMOK.

Might be worth having a look at the code that combines the ~CRC32 values if iR0bo0t is willing to share or writing something from scratch, pretty sure it cant be that hard and maybe looking to script something (Python/PHP?) to generating the combined CRC32 from the DAT files themselves.  at least that way you wont need to scrape the pages in cases of corrections / updates.   

Slightly off topic:

from the sounds of it you are probably heavily invested in the the Phoebe/Rhea ODE but maybe worth a look at the upcoming Satiator from professor abraisive,

I am currently awaiting the beta board for testing.

one of the main advantages being his cue parser is specifically designed for split bin/cue images along with leaving the actual optical drive untouched obviously.  plus there in nothing in Saturn Subs that isnt covered by the redump .cue

I'm well aware of JHL's project and am one of the biggest critics of it as it doesn't allow for use of every title on the system (anything MPEG card optional/required can't be used). While there aren't a lot of detractors out there, my understanding is that he's not going to be implementing an mpeg card into the unit and therefore the unit misses the point to me. In my mind, any ODE product should enhance the capabilities and user experience of the original product without reducing compatibility or functionality.

While the rhea/phoebe does remove the ability to play optical discs, i have yet to run into a single disc image it can't handle that hasn't been fixed in some way (i believe there were a couple of discs early on that couldn't play but to my knowledge those have since been fixed via firmware). This seems a fair trade to me. Though i'd much prefer an ability to switch easily between using the ODE and using the original drive, i'm not willing to sacrifice titles out of the library for that feature.

In any case, I'm still much more of a fan of using CCD over BIN/CUE.

We're going to continue work on the scraper for now, but yes, having an alternative method of getting the total CRC-32 without needing to build the scraper would probably be ideal.

Also, i'm in agreement that the need i have doesn't necessarily justify an additional table, but i'm not exactly sure how one could convert the existing bin/cue sets over to CCD in batch and verify the images retained integrity without doing what i'm doing. it would be a different story if a CCD based set already existed, or if the raw files generated by DIC were what were distributed instead of bin/cue, but for end users that would likely be a nightmare.

So yes, i'm open to any and all suggestions to make this process more self contained. No matter what, some online update is required (either a new dat file must be downloaded and compared, or the site needs to be date checked and re-scraped each time a new disc is added).

Go to forum: General discussion

18 Reply by A Murder of Crows

(17 replies, posted in General discussion)

Jackal wrote:

If this is for a personal list of sorts then you're better off making one manually in Excel (I've been doing the same to keep track of my IBM PC collection). There would be no benefit in automating anything, you only have to update the list when a new dump is added to the db.

hardly. a friend and i have developed a tool to automatically convert Redump images in Bin+Cue to cloneCD format, primarily because Bin+Cue is not supported by some ODEs and it makes little sense to maintain 2 separate image sets. Also, converting 2000+ images takes a LOT of time (approximately 50 hours last time i did it) and each time we still run in to certain errors, indicating that something in the conversion process is going wrong, but not on all images.

What i want to do is add in CRC-32 checking after the image conversion process completes on a per image basis and flag the ones that don't match up.

Either way, i do not want to attempt to manually visit 2000+ pages to pull a single point of data, much less 10 points.

There certainly is benefit to automating. Even if a new dump is added to the DB, the date added can be checked and if the date doesn't match or surpass the last time the tool was run, no update is needed for that title. Otherwise, snag updated info and flag for attention.

there are very few tasks i can think of where automation isn't welcome. Heck, even though i haven't been able to dump in a while due to various issues, the only reason i can dump is because my friend and i looked at the dump process and developed tools to automate the creation of the command for DIC as well as for moving and zipping the resulting files.

anyway, it appears that using a scraper is the only way currently to obtain the CRC-32 info.  I hope that changes in the future as verification of successful and accurate transition between formats is pretty important to preservation efforts and the prevention of inaccurate images getting out to the wild.

Go to forum: General discussion

19 Reply by A Murder of Crows

(17 replies, posted in General discussion)

iR0b0t wrote:

10. Total CRC-32

I cannot do a database dump of it, Total CRC-32 is not stored within the database, its generated on the fly each time you review a disc page.

hmm. this presents a problem. We're working on trying to build a scraper for this but had really hoped grabbing the CRC-32 would have been far easier for the verification process.

Go to forum: General discussion

20 Topic by A Murder of Crows

(17 replies, posted in General discussion)

Hey everyone

I've been advised to ask iR0b0t for a database query/dump.

I'm trying to do a comparison between submitted Total CRC-32 values for dumps against dumps that I converted to CloneCD format.

I'm hoping to get a csv with the following info:

1. Title
2. Disc Number/Total Discs for title
3. System
4. Region
5. Serial
6. Build Date
7. Version
8. Edition
9. Bar Code
10. Total CRC-32

I primarily need this for Sega Saturn discs, but it would be wonderful to get output like this for all discs i might dump in the future.

Thank you

Go to forum: General discussion

21 Reply by A Murder of Crows

(3,497 replies, posted in General discussion)

sarami wrote:
Enker wrote:

2. Both your program and PerfectRip have the file name issue. I don't think there is any setting I can change to fix this. Other programs I use don't have this issue.

It may be beside the point...

If you use Windows7, check off below
explorer -> tool(T) -> folder option(O) -> show tab 'Do not show extensions that are registered'
(I don't use English version, so may not be accurate.)

Seems to be the same problem i'm having. Did this ever get addressed?

Go to forum: General discussion

22 Reply by A Murder of Crows

(3,497 replies, posted in General discussion)

Seems i'm too much of a newb to figure out the context for this. How should i formulate the command?

wiggy2k wrote:

Manually specifying the .bin file extension at the end of the command should do it.

alternatively just dump it with an arbitrary name like dump.bin and rename it with the DAT.
it should match http://redump.org/disc/29848/

Glad to see you finally got your hands on a copy.
hope it wasnt too pricey, it's gone insane lately.

Go to forum: General discussion

23 Reply by A Murder of Crows

(3,497 replies, posted in General discussion)

EDIT:

Seems i am having a problem formulating a command that would work to dump a game with periods in the title.

>DiscImageCreator.exe cd E: "D:\dump\E\Daytona USA C.C.E. Net Link Edition [(Star)2S(Star) 81218 B A1]" 40 /c2

This should create a set of files called

Daytona USA C.C.E. Net Link Edition [(Star)2S(Star) 81218 B A1].*

but instead the final period is seen as the point of extension.

is there any command arguments that would help me avoid this issue?

Go to forum: General discussion

24 Reply by A Murder of Crows

(3,497 replies, posted in General discussion)

iR0b0t wrote:

@A Murder of Crows
he pretty much did what you asked him for, you should be able to run multiple DIC instances now.

>> Test version 20170930


ah, i didn't see the new test version

thanks all!

Go to forum: General discussion

25 Reply by A Murder of Crows

(3,497 replies, posted in General discussion)

sarami wrote:
A Murder of Crows wrote:

Is there any way to run 2 copies of DIC at the same time on different drives?  I have a LOT of discs to dump in a short amount of time, but keep running into the following error:

Disabled mutex.

Thanks for the reply.  Please excuse my ignorance however.  I don't understand what you mean by that.

Go to forum: General discussion