1 (edited by Jackal 2019-07-07 21:55:24)

Are there any coders who could have a look at this? http://xboxdevwiki.net/XDVDFS#Random_blocks

https://github.com/JayFoxRox/xbox-dvd-compress

It would be great to have a tool developed similar to nNASOS for dramatically reducing the compressed size of Xbox images (by scrubbing the random padding and allowing it to be regenerated)

2 (edited by Sniik 2019-07-07 20:52:52)

Jackal wrote:

Are there any coders who could have a look at this? http://xboxdevwiki.net/XDVDFS#Random_blocks

It would be great to have a tool developed similar to nNASOS for dramatically reducing the compressed size of Xbox images (by scrubbing the random padding and allowing it to be regenerated)

Yeah I would like to. Is this known to be correct?

3 (edited by LedZeppelin68 2019-07-08 15:55:50)

great, thanks for the link, very interesting for research
will take a look after figuring out the 3ds decryption

It works!

https://pastebin.com/vfC9GSLX

Tested it with the game from the example and known seed

Compressed size of archive 65 mb, against 6.4Gb with "garbage"

Great, Jackal, thanks for the info!

5 (edited by LedZeppelin68 2019-07-10 13:58:27)

OK, test it on "Petit Copter (Japan).iso"

2 executables

1) xbox_dernd.exe - derandomizer

like "nasos deux", it doesn't change the structure of the file, just replaces "random data block" with block full of "JUNK", it is like zeroing but it serves as a mark

you will get a file called "Petit Copter (Japan).f401863e.iso"
f401863e is a seed for the random generator

2) xboxRegen.exe - regenerator of random data

replaces blocks full of "JUNK" with random data blocks, generated by using seed number


xbox_dernd is tweaked to be compatible with the Petit Copter for test purpose
the tweak is, that it bruteforces only 1/16 part of full range
from 0xf0000000 to 0xffffffff (f401863e)
to speed up the test process, it tooks about 8 seconds on my rig
brutforcing the full range will took about 8 minutes

P.S. 7zipped iso is 48,7 Mb

Post's attachments

xbox_tools_test_alpha1.zip 7.6 kb, 10 downloads since 2019-07-10 

You don't have the permssions to download the attachments of this post.

6 (edited by wiggy2k 2019-07-10 16:15:48)

wow, nice work, that does work really quickly too.

https://i.ibb.co/MsQch5X/xbox.png

That's a massive space saving.

Great work, keep on saving me a loooooot of space on my rigs! smile

8 (edited by Jackal 2019-07-10 18:14:58)

Hi,

great job! 3 questions:

- Is there only 1 seed value for the entire disc? So it shouldn't take long to brute force every disc once they are supported?
- Does nNASOS also fill with "JUNK" pattern, or something else?
- I tried 2 random games (Crimson Skies - High Road to Revenge (USA) (En,Fr,De,Zh,Ko) + Tony Hawk's Pro Skater 2x (USA) ), but it couldn't find a seed value. So just this 1 game is supported for now?

9 (edited by LedZeppelin68 2019-07-10 18:55:13)

Hello!

1) 1 seed value for entire disc, since current tool uses only one thread brutrforcing takes around 8 minutes on modern PC, I'll try to speed it up

2) I cannot tell for sure about nNASOS, if it uses the first version of NASOS, no it fills garbage blocks with zeroes and should have header block with marks which block uses garbage padding.
NASOS deux replaces garbage with "JUNK", and the image is playable in emulator
http://forum.redump.org/topic/17831/nasos-2/

3) Yes, this version supporting one game only

10 (edited by LedZeppelin68 2019-07-10 18:56:55)

https://i.imgur.com/7bA0cki.png

this is time taken to brute the seed value for "Silent Hill 2 - Inner Fears (Europe) (En,Ja,Fr,De,Es,It).iso"

11 (edited by LedZeppelin68 2019-07-10 19:10:42)

one more thing we need is a security sectors ranges, they are unique for every game, and we have to specify it

since redump does have this info, this is not a problem smile

I included sample "ss.txt" in archive and updated xbox_dernd tool so it can support it
also i unlocked brutforce range to full range

test it! smile

P.S. Silent Hill 2 is 1,77 GB 7zipped
P.S.S. you should place "ss.txt" in the folder with iso

Post's attachments

xbox_tools_test_alpha2.zip 8.19 kb, 8 downloads since 2019-07-10 

You don't have the permssions to download the attachments of this post.

12 (edited by Jackal 2019-07-10 20:01:59)

Hi, I tested "Tony Hawk's Pro Skater 2x (USA)". It finds a seed:

Seed found: 0xe4e92289
Time elapsed: 00:04:55.8389074

But when I use xboxregen, the pattern that is added does not match the original file:

https://i.imgur.com/KpHztQO.png

I did put the correct ss.txt

Looking forward to a solution smile

LOL, i got it, because xboxregen also needs "ss.txt" and i forgot to add the support, lame i am

Post's attachments

xbox_tools_test_alpha3.zip 8.45 kb, 9 downloads since 2019-07-10 

You don't have the permssions to download the attachments of this post.

14 (edited by jhmiller 2019-07-10 21:29:23)

I get an exception: "divide by zero".
And it takes about 5 minutes to complete the process, not 5 seconds sad
My system: Win10 X64

http://forum.redump.org/misc.php?action=pun_attachment&item=3117
The game is the "UEFA Champions League 2004-2005 (Europe) (Es,It)".
I made an ss.txt with the sectors from the Redump game.

Edit:
Now I tried with the "2002 FIFA World Cup" and found the seed !
The torrentziped iso went from 6,18GB to 2,25GB smile
Maybe the UEFA is not compatible?

Post's attachments

Captura.JPG 55.18 kb, file has never been downloaded. 

You don't have the permssions to download the attachments of this post.
I love my XKey, my WODE and my 3Key.
Cerrar MegaUpload sólo es el comienzo de la censura, será el fin de la libertad.
Nada es verdad, todo está permitido.

15 (edited by Jackal 2019-07-10 21:34:32)

Of course it's possible that either the dump is corrupt or the ss.txt is bad. Plz paste the contents of the ss.txt.

Can you also use xboxregen on the 2002 FIFA World Cup "cleaned" ISO and confirm it restores to the original hashes?

I will do more tests tomorrow night.

I think "UEFA Champions League 2004-2005 (Europe) (Es,It)" is a late game and uses another padding generator

http://xboxdevwiki.net/XDVDFS#Random_blocks
Version 4831 - 5849 algorithm
i didn't implement it yet, but i know which game to test smile

Maybe is the padding generator.
In any case, this is the "ss.txt" I made with the SS from the redump page of the UEFA http://redump.org/disc/52675/:

291148,295243
442992,447087
681096,685191
908456,912551
1066974,1071069
1218274,1222369
1372342,1376437
1530846,1534941
2214524,2218619
2371192,2375287
2526846,2530941
2676146,2680241
2832990,2837085
3067338,3071433
3222090,3226185
3454438,3458533

I tried to restored the iso of the "2002 FIFA World Cup" and YES, the iso restored matches the original iso !!!

One doubt, is there any restriction when naming the iso?
Is it necessary to keep the seed in the name?
Or could it be kept separate?

I love my XKey, my WODE and my 3Key.
Cerrar MegaUpload sólo es el comienzo de la censura, será el fin de la libertad.
Nada es verdad, todo está permitido.

One doubt, is there any restriction when naming the iso?
Is it necessary to keep the seed in the name?
Or could it be kept separate?

since we need a bunch of information to restore the padding, seed and SS ranges
it is useless to store the seed in the name, it is a temporary workaround

we have two ways
1) store all data in a separate file
2) store data inside the iso, for example, in the first sector

but there is bad news: JayFoxRox (our member and person who discovered the algorithms) described it there
https://www.reddit.com/r/crypto/comment … eforce_on/

in two words:
isos compiled with SDK between versions 3926 - 4721 - easy to brutforce
isos compiled with SDK between versions 4831 - 5849 - will take forever

Is it possible to generate the real SS sectors contents, without padding them with zeroes? What is the logic in different amounts of readable/unreadable sectors in different blocks?

20 (edited by Jackal 2019-07-11 16:52:38)

LedZeppelin68 wrote:

but there is bad news: JayFoxRox (our member and person who discovered the algorithms) described it there
https://www.reddit.com/r/crypto/comment … eforce_on/

in two words:
isos compiled with SDK between versions 3926 - 4721 - easy to brutforce
isos compiled with SDK between versions 4831 - 5849 - will take forever

Bad news indeed sad

It seems to be using a "filetime" value for generating the seed value? So it it's possible somehow to pinpoint the approximate time when the disc was mastered and only try values within a limited time range, then maybe the bruteforce time could be reduced dramatically?
There seem to be timestamps in various places (also in the DMI and SS) that could be used for this: https://xboxdevwiki.net/Xbox_Game_Disc

I will test some ISO's and see for which ones the seed value can't be found, and then look for some timestamps.

And I really think we need to start storing the seed values and also SS ranges in an external file / database. The Redump filename or crc/md5/sha1 hash of the ISO could be the identifier to look up the values.

21 (edited by Jackal 2019-07-11 19:28:31)

About the timestamps: https://www.epochconverter.com/ldap

DiscImageCreator also decodes the filestamps from DMI and SS, see outputScsiCmdLogforDVD.cpp
and from XDVDFS: execScsiCmdforFileSystem.cpp

Example from http://redump.org/disc/6024/:

DMI:
2005/03/30 20:54:46

SS:
Timestamp of authoring: 2005/03/29 21:07:14
Timestamp of unknown: 2005/03/29 18:38:08

volDesc:
Image creation time: 2005/03/30 02:38:16


I guess it's strange that the image creation date is later than the other timestamps. But this would give a time range of ~8 hours that we could try for brute forcing (if this is even possible).

22 (edited by Jackal 2019-07-15 16:53:31)

@LedZeppelin68 are you still planning to investigate the second seed type?

first type seed is only 32bit, 256^4 combinations

second type seed is 128bit, 256^16 combinations, impossible to bruteforce, forever

24 (edited by Jackal 2019-07-15 19:26:25)

Ok but my questions about the timestamp? Is there no way to predict the seed?

Jackal wrote:

Ok but my questions about the timestamp? Is there no way to predict the seed?

i can say - no

the seed value for second type is 128 bit, which consists of two 64 values:
1: system time and date (we can reduce this value to year, or maybe a month) +
2: and time from last reset (2^64 combos)

it is a very huge number to bruteforce