[DiscordArchive] isn't there another internal listfile ?
[DiscordArchive] isn't there another internal listfile ?
Original source
rektbyfaith
01-17-2025, 05:35 PM #11Archived author: Titi • Posted: 2025-01-17T17:35:10.152000+00:00
Original source
Original source
blizzard rolls it's own custom hashing algorithm.
1 hash is used for indexing into the hash list (hash type is an argument for their hashing function) -> this hash is then masked (logical &) with $(hash table size - 1), which results in the index for starting the search.
2 hashes (a and b) are used for determining if you found the correct entry in the hash table (2 * 32 bytes) -> this is in essence a content similarity check.
the client searches the file not by it's real name, but by it's a+b hashes.
all official 3.3.5.12340 MPQs contain the (listfile).
hashing can be done easily in one direction (string into hash values), but the reverse is basically impossible.
but since blizzard hashing does NOT involve salting (unique data added to input, which causes different hashes for the same input, if a different salt is given), all files always hsah to the same a+b values.
if you compute the hashes of well known files externally and then give them as input when looking through the hash table, you can determine the contents.
MPQ editor had a name-breaking feature in the past, which was intended for such things.
and robinsch wrote a hook in order to write down file names, when the client tried to look them up in MPQ files -> prb the easiest and fastest way to know the file names
rektbyfaith
01-17-2025, 05:50 PM #12Archived author: 4bhorrent • Posted: 2025-01-17T17:50:57.934000+00:00
Original source
blizzard rolls it's own custom hashing algorithm.
1 hash is used for indexing into the hash list (hash type is an argument for their hashing function) -> this hash is then masked (logical &) with $(hash table size - 1), which results in the index for starting the search.
2 hashes (a and b) are used for determining if you found the correct entry in the hash table (2 * 32 bytes) -> this is in essence a content similarity check.
the client searches the file not by it's real name, but by it's a+b hashes.
all official 3.3.5.12340 MPQs contain the (listfile).
hashing can be done easily in one direction (string into hash values), but the reverse is basically impossible.
but since blizzard hashing does NOT involve salting (unique data added to input, which causes different hashes for the same input, if a different salt is given), all files always hsah to the same a+b values.
if you compute the hashes of well known files externally and then give them as input when looking through the hash table, you can determine the contents.
MPQ editor had a name-breaking feature in the past, which was intended for such things.
and robinsch wrote a hook in order to write down file names, when the client tried to look them up in MPQ files -> prb the easiest and fastest way to know the file names
Original source
btw. it is possible that the MPQ archives contain files, which are not written down in the listfile.
see them as some lost relics
this can be done by comparing the number of entries in the hashtable and blocktable + some sanity checks with data positions.
wouldn't be surprised if there are some unknown assets in there ^^
rektbyfaith
01-17-2025, 05:59 PM #13Archived author: 4bhorrent • Posted: 2025-01-17T17:59:26.429000+00:00
Original source
btw. it is possible that the MPQ archives contain files, which are not written down in the listfile.
see them as some lost relics
this can be done by comparing the number of entries in the hashtable and blocktable + some sanity checks with data positions.
wouldn't be surprised if there are some unknown assets in there ^^
Original source
should also be possible to write a proper multithreaded app for bruteforcing the filename for an a+b hash pair, most of all with SIMD optimization, this should be crackable in a sane amount of time on todays machines.
could even do so with GPU accelleration -> writing down on my huge pile of ideas lol
(also one can optimize the brute forcer via contstraints -> filename can only contain bytes which correspond to ascii characters allowed in OS filenames + directory character)
rektbyfaith
01-17-2025, 06:00 PM #14Archived author: 4bhorrent • Posted: 2025-01-17T18:00:51.279000+00:00
Original source
should also be possible to write a proper multithreaded app for bruteforcing the filename for an a+b hash pair, most of all with SIMD optimization, this should be crackable in a sane amount of time on todays machines.
could even do so with GPU accelleration -> writing down on my huge pile of ideas lol
(also one can optimize the brute forcer via contstraints -> filename can only contain bytes which correspond to ascii characters allowed in OS filenames + directory character)
Original source
doesn't MPQ editor just still show those without listfile entries, as unknown files ?
rektbyfaith
01-17-2025, 06:14 PM #15Archived author: Titi • Posted: 2025-01-17T18:14:19.907000+00:00
Original source
doesn't MPQ editor just still show those without listfile entries, as unknown files ?
Original source
never tried, but it is technically possible to extract files without knowing their real names, you have to make up a name for them in that case
rektbyfaith
01-17-2025, 06:15 PM #16Archived author: 4bhorrent • Posted: 2025-01-17T18:15:12.757000+00:00
Original source
never tried, but it is technically possible to extract files without knowing their real names, you have to make up a name for them in that case
Original source
it shows them like this when they're not in the listfile, it can sometimes detect the expansion somehow
![[Image: image.png?ex=690c3d64&is=690aebe4&hm=759...c4d32f4ed&]](https://cdn.discordapp.com/attachments/1086807686571642900/1329877184344490044/image.png?ex=690c3d64&is=690aebe4&hm=7599089323ba47b610f9d02c85acc3c6daf7e1f2033a7262ba6a5b7c4d32f4ed&)
rektbyfaith
01-17-2025, 06:17 PM #17Archived author: Titi • Posted: 2025-01-17T18:17:08.744000+00:00
Original source
it shows them like this when they're not in the listfile, it can sometimes detect the expansion somehow
Original source
I think it detects non WoW expansion types (probably uses some library to detect common file types headers)
rektbyfaith
01-17-2025, 06:17 PM #18Archived author: Titi • Posted: 2025-01-17T18:17:26.722000+00:00
Original source
I think it detects non WoW expansion types (probably uses some library to detect common file types headers)
Original source
it also lacks the folder structure
rektbyfaith
01-17-2025, 06:17 PM #19Archived author: Titi • Posted: 2025-01-17T18:17:53.490000+00:00
Original source
it also lacks the folder structure
Original source
so, if there are unlisted files, we would know
rektbyfaith
01-17-2025, 06:19 PM #20Archived author: Titi • Posted: 2025-01-17T18:19:44.173000+00:00
Original source
so, if there are unlisted files, we would know
1 Guest(s)
1 Guest(s)