 |
SL2 Analyzer
Read a FromSoftware .sl2 save and report what is in it
|
|
SL2 Analyzer
Read a FromSoftware .sl2 save and report what is in it
|
Classes | |
| class | Reader |
| A little-endian struct reader that exits rather than reading past its buffer. More... | |
Functions | |
| dict[str, str] | load_keys (Path path, str dict_name) |
One game's public keys out of UXM's ArchiveKeys.cs. | |
| bytes | rsa_decrypt (bytes data, str pem) |
Decrypt a .bhd with a PKCS#1 public key. | |
| dict[int, dict] | parse_bhd5 (bytes buf, str variant="ds3") |
Parse a decrypted BHD5 header into {file name hash: entry}. | |
| dict | read_aes_key (bytes buf, int off) |
| The per-file AES key and the ranges it covers, or None where there is none. | |
| int | path_hash (str path, str variant="ds3") |
SFUtil.FromPathHash: the only way to find a file whose name was thrown away. | |
| bytes | read_entry (bdt, dict entry) |
Read one entry out of a .bdt, decrypting the ranges its header marks. | |
| list[Path] | ooz_paths (str|None explicit) |
| Where to find the Kraken decoder. | |
| ooz_load (str|None explicit=None) | |
Load libooz.so once, from the first candidate path that exists. | |
| bytes | ooz_decompress (bytes src, int dst_len) |
| bytes | ooz_decompress_chunked (bytes src, int dst_len) |
| Kraken decode a stream FROM's way: one 256 KiB chunk at a time. | |
| bytes | dcx_decompress (bytes data) |
Undo a DCX\0 wrapper, KRAK or DFLT. | |
| int | _i32 (bytes b, int o) |
| int | _u32 (bytes b, int o) |
| int | _i64 (bytes b, int o) |
| str | _utf16 (bytes b, int off) |
A NUL-terminated UTF-16LE string at off. | |
| dict[str, list[int]] | msb_sections (bytes buf) |
Every section of an MSB as {name: [entry offsets]}. | |
| int | msb_section_pos (bytes buf, str name) |
| Byte offset of one section's header, for reading its version word. | |
| list[str] | msb_models (bytes buf, list[int] offsets) |
Model names in index order, so a part's ModelIndex resolves to c1470_0000. | |
| list[dict] | msb_parts (bytes buf, list[int] offsets, list[str] models) |
| Every part in one MSB as a dict, enemies carrying their param joins. | |
| tuple[str, list[dict]]|None | read_msb (Path path) |
Parse one .msb into | |
| list[tuple[str, list[dict]]] | read_msbs (str root) |
Every .msb under a directory, parsed, in map order; unknown layouts skipped. | |
| str | entity_map (int eid) |
| The map file an entity id belongs to, by the same decomposition the flags use. | |
| list|None | unpack_args (bytes data, list[int] types) |
| Unpack one instruction's argument blob against a type list. | |
| list[int]|None | common_init_args (bytes data) |
Initialize Common Event's variadic arguments: the event id, then its parameters. | |
| list[tuple[int, int, int, bytes]]|None | read_emevd (Path path) |
Every instruction in one .emevd as | |
| dict[str, list] | read_emevds (str root) |
Every .emevd under a directory, as {map name: [instructions]}. | |
| list[tuple] | roster_calls (dict[str, list] scripts) |
Every boss/miniboss call in the scripts, as | |
| dict[int, dict] | build_roster (dict[str, list] scripts) |
| The boss/miniboss roster keyed by entity id. | |
| dict[int, tuple[str, int]] | ds3_death_flags (dict[str, list] scripts) |
| Every death flag the DS3 one-time-enemy templates are initialised with. | |
| dict[str, dict[int, str]] | load_fmg_tables (str msg_dir) |
| Every FMG table in the English message binders, keyed by table name. | |
| tuple[str, int] | pick_table (dict[str, dict[int, str]] tables, set[int] ids) |
The FMG table covering the most of ids, and how many it covered. | |
| dict[int, str] | npc_dev_names (str paramdex) |
Paramdex's NpcParam annotations as {row id: english text}. | |
| list[str] | wanted (str dict_path, list[str] prefixes, list[str] paths) |
| Which dictionary paths to extract, from the prefixes and exact paths given. | |
| int | cmd_unpack (args) |
| int | cmd_msb (args) |
| int | cmd_emevd (args) |
| er_read_names (msg_dir) | |
Read Elden Ring's own English item names out of an unpacked msg/engus. | |
| int | cmd_ernames (args) |
Regenerate db_er/'s item-name tables from the game's own FMGs. | |
| int | cmd_roster (args) |
| Print the roster, compare it with the shipped table, optionally rewrite it. | |
| int | main () |
Variables | |
| BASE = Path(__file__).resolve().parent.parent | |
| dict | GAMES |
| Per-game archive layout: which pairs exist, whose keys open them, which BHD5 shape. | |
| dict | HASH_PRIME = {"ds2": (37, 32), "ds3": (37, 32), "er": (0x85, 64)} |
| The filename hash, per era. | |
| int | AES_BLOCK = 16 |
| AES block size, and so the granularity an encrypted range can be decrypted at. | |
| str | KEY_HELP |
| int | OODLE_BLOCK = 0x40000 |
| Oodle's block length. | |
| int | OOZ_SLACK = 0x10000 |
| Slack on the Kraken output buffer. | |
| _OOZ = None | |
| MODEL_SECTION | |
Section names, from MSBS.cs. | |
| PARTS_SECTION | |
Section names, from MSBS.cs. | |
| PART_ENEMY | |
Part type ids from PartsParam.PartType. | |
| PART_DUMMY_ENEMY | |
Part type ids from PartsParam.PartType. | |
| dict | PART_NAMES |
| P_NAME_OFF | |
| Offsets inside a Part's base struct, and inside an enemy's type data. | |
| P_TYPE | |
| Offsets inside a Part's base struct, and inside an enemy's type data. | |
| P_MODEL_INDEX | |
| Offsets inside a Part's base struct, and inside an enemy's type data. | |
| P_ENTITY_DATA_OFF | |
| P_TYPE_DATA_OFF | |
| E_THINK_PARAM | |
| E_NPC_PARAM | |
| E_CHARA_INIT | |
| E_EVENT_FLAG | |
| E_EVENT_FLAG_STATE | |
| dict | MSBS_PARTS_VERSIONS = {0x21, 0x23} |
PARTS_PARAM_ST versions this reader's part struct is known to match (Sekiro). | |
| MINIBOSS_BAR | |
| The instructions that define a roster, and what each contributes. | |
| MINIBOSS_DEFEAT | |
| The instructions that define a roster, and what each contributes. | |
| BOSS_BAR | |
| BOSS_DEFEAT | |
| BOSS_BANNER | |
| tuple | AWARD_ITEM_LOT = (2003, 4) |
| tuple | INIT_COMMON_EVENT = (2000, 6) |
Initialize Common Event — how a parameterised handler gets its real arguments. | |
| dict | ARG_TYPES |
| EMEDF numeric type code -> (struct format, size). | |
| dict | LAYOUTS |
| Argument layouts for the instructions this file reads, from the EMEDF. | |
| tuple | DS3_DEATH_TEMPLATES = (20005340, 20005341, 20005342, 20000343, 20005416, 20005061, 20005760) |
The Dark Souls III common_func templates that flag a one-time enemy death. | |
| dict | SDT_AREA_BY_MAP |
| The travel-menu area a Sekiro entity id belongs to, for the shipped table's keys. | |
| list | SDT_AREA_ORDER |
| The order areas are printed in: the game's own travel-menu order. | |
| int | DB_INDENT = 1 |
Indent for a generated db_* table. | |
| dict | ER_NAME_FMGS |
The five item.msgbnd name tables that back a db_er/ file, and the category nibble Elden Ring's save ids carry on top of the bare param row id. | |
| str | ER_FMG_PLACEHOLDER = "[ERROR]" |
| FromSoft's marker for a row that exists but carries no player-facing name. | |
gamefiles.py — read a FromSoftware install directly: archives, maps, event scripts.
WHAT THIS IS FOR
Every id table in `db_*/` was transcribed from a community source. That was the only
option while the games' own files were sealed, and it costs accuracy: a randomizer's
enemy list can be missing five entries and mislabel fourteen more, and nothing in a
save can tell you so. This reads the installed game instead — the same four steps a
modding tool takes, in one place, with no Windows dependency:
unpack open the `.bhd`/`.bdt` archives and extract files by path
msb read a map layout: every enemy placement, its entity id and model
emevd read the event scripts: instruction by instruction, arguments unpacked
roster the payoff — check db_sdt/minibosses.json against the scripts themselves
ernames the other payoff — write db_er's item names from Elden Ring's own FMGs
It is a READER. It never writes into a game folder, never patches an executable, and
the one subcommand that can write a `db_*` table only does so when asked (`--write`).
WHY ONE FILE
These began as four throwaway scripts, and four scripts is how the BHD5 reader gets
copied three times and then fixed once. The formats are shared — a `.msb` and an
`.emevd` both come out of a `.bhd`, and both are DCX-wrapped — so the code is too.
WHAT COMES FROM WHERE — none of the layouts here are invented
* Archive container, DCX, MSB and EMEVD layouts: `JKAnderson/SoulsFormats`
(`Formats/BHD5.cs`, `Formats/DCX.cs`, `Formats/MSB/MSBS/*`, `Formats/EMEVD/*`).
* The RSA step: `Nordgaren/UXM-Selective-Unpack` `CryptographyUtility.DecryptRsa` —
raw RSA, no padding, 256-byte blocks in and 255-byte blocks out, left-zero-padded.
* The filename hash: `SFUtil.FromPathHash` — lowercase, backslashes to slashes, a
leading slash, then `h = h * 37 + c` over a uint32.
* Instruction names and argument types: `AinTunez/DarkScript3`'s
`sekiro-common.emedf.json`. Only the handful of instructions this file names are
hardcoded (@ref LAYOUTS), so no EMEDF file is needed at run time.
* Keys and name dictionaries: UXM's `ArchiveKeys.cs` and `res/<game>Dictionary.txt`.
NEITHER IS VENDORED HERE — pass them in. Dark Souls II needs neither: it ships its
own `*KeyCode.pem` beside each archive.
* BND4 and FMG parsing is imported from `gen_sdt_from_regulation.py` rather than
written twice.
THE TWO TRAPS, BOTH FOUND THE HARD WAY
* **FromSoft's Oodle streams decode ONE 256 KiB CHUNK AT A TIME, each against its own
window.** Hand a Kraken decoder the whole stream and chunk 1 comes out perfect and
chunk 2 fails — in `powzix/ooz` and in the unrelated Rust `oozextract` alike. See
@ref ooz_decompress_chunked.
* **A boss/miniboss handler is usually PARAMETERISED.** Its own instruction args are
zeroes; the real values arrive through `2000[6] Initialize Common Event`. Scan only
the inline calls and Sekiro reports one miniboss instead of thirty-seven.
ELDEN RING: WHAT WORKS AND WHAT IS NOT WIRED
Target is **Shadow of the Erdtree Deluxe, v1.16**. This was written blind from UXM's key
list and SoulsFormats' `Game.EldenRing`, and it has now been RUN: the first attempt
against a real install indexed **120,332 entries** and pulled `/msg/` without a change
to any of it. Both ER-only differences below were right the first time.
* `unpack --game er` — works. Archives `Data0..Data3` + `DLC`; keys `EldenRingKeys`;
dictionary `EldenRingDictionary.txt` (9 MB, and the RSA header pass over ~14 MB of
`.bhd` is the slow step — minutes, not seconds).
* `ernames` — works, and `db_er/`'s item-name tables are its output now. It reads the
unpacked `msg/engus` rather than params, so `regulation.bin` stays untouched.
* **Two ER-only differences are already handled, and both fail SILENTLY if missed.**
The filename hash widened to **uint64 with multiplier 0x85** (@ref HASH_PRIME), and
the archive entry keeps DS3's 40-byte stride while laying its fields out differently
— 64-bit hash, then two 32-bit sizes (@ref parse_bhd5). Either one wrong produces an
archive that simply contains nothing you asked for.
* `DCX/ZSTD` is handled as well as `KRAK`, because ER's later patches use it.
* `regulation.bin` is loose at the install root and is ER-encrypted; nothing here
decrypts it, and nothing needs to — `db_er/`'s names come from the `msg/engus`
FMGs, which are plain files once the archive is open.
* **`msb` will NOT read ER maps.** ER is `MSBE`, whose part struct differs from
Sekiro's `MSBS`; the reader asserts rather than guesses (@ref read_msb).
* **`roster` is Sekiro-specific and stays that way.** The instruction ids in
@ref LAYOUTS are Sekiro's. ER's own convention is already known from the flag
research — defeat flag == entity id for 156 of 176 bosses — so the ER equivalent is
an ER EMEDF plus its `Handle Boss Defeat` id, not new machinery.
* `emevd` may work as-is: ER is expected to be the same version `0xCD` container as
Sekiro. If the header check rejects it, the flags it prints are the thing to look at
first — do not widen the check without knowing which permutation it is.
The real ER blocker is elsewhere and this tool does not touch it: the SAVE-side flag
region is still unsolved, so ER ids remain unreadable however many of them get extracted.
THE OODLE DEPENDENCY
Sekiro and later compress with Oodle Kraken, which has no pure-Python decoder. Build
`powzix/ooz` as a shared object once:
git clone --depth 1 https://github.com/powzix/ooz && cd ooz
head -4286 kraken.cpp > kraken_lib.cpp
printf '\\nextern "C" int ooz_decompress(const byte *s, size_t sl, byte *d, size_t dl)'\\
'{ return Kraken_Decompress(s, sl, d, dl); }\\n' >> kraken_lib.cpp
# plus a compat/ shim supplying tchar.h, intrin.h and Windows.h on Linux
g++ -O2 -DNDEBUG -fPIC -shared -Icompat kraken_lib.cpp bitknit.cpp lzna.cpp \\
-o libooz.so
Point `--ooz` (or `$SL2_OOZ_LIB`) at the result. Games that use Deflate — Dark Souls
II, Dark Souls III, Dark Souls Remastered — need none of this.
USAGE
python3 tools/gamefiles.py unpack --game sekiro --game-root ~/Games/Sekiro \\
--keys ArchiveKeys.cs --dict SekiroDictionary.txt --out ~/Games/Sekiro-unpacked \\
--prefix /event/ --prefix /param/ --prefix /msg/engus/ --prefix /map/mapstudio/
python3 tools/gamefiles.py msb <unpacked>/map/mapstudio --entity 1120450
python3 tools/gamefiles.py emevd <unpacked>/event --instr 2003:87
python3 tools/gamefiles.py roster <unpacked>/event --msg <unpacked>/msg/engus \\
[--maps <unpacked>/map/mapstudio] [--paramdex <Paramdex>] [--write db_sdt/minibosses.json]
EXIT CODES
0 ok · 2 bad input/paths · 3 parse failure
| dict[str, str] gamefiles.load_keys | ( | Path | path, |
| str | dict_name | ||
| ) |
One game's public keys out of UXM's ArchiveKeys.cs.
A C# source file rather than anything structured, so this is a text scrape: the named dictionary, then each ["DataN"] = @"...PEM..." inside it. Scoping to one dictionary body matters — the same file carries DS3's, Sekiro's, Elden Ring's and AC6's, and the wrong game's key fails as plausible garbage rather than as an error.
No completeness check on purpose: UXM publishes no key for DS3's Data0 because that header is not encrypted, so an archive missing here is a fact, not a failure.
| path | ArchiveKeys.cs. |
| dict_name | e.g. SekiroKeys. |
{archive name: PEM text}. Definition at line 193 of file gamefiles.py.
Referenced by cmd_unpack().
| bytes gamefiles.rsa_decrypt | ( | bytes | data, |
| str | pem | ||
| ) |
Decrypt a .bhd with a PKCS#1 public key.
Raw RSA, deliberately: UXM runs BouncyCastle's RsaEngine with no padding scheme, so every 256-byte ciphertext block becomes a 255-byte plaintext block — the engine's own input and output block sizes for a 2048-bit modulus. The left-zero pad is not cosmetic: a block whose plaintext happens to start with a zero byte would otherwise shift the whole rest of the header.
Definition at line 214 of file gamefiles.py.
Referenced by cmd_unpack().
| dict[int, dict] gamefiles.parse_bhd5 | ( | bytes | buf, |
| str | variant = "ds3" |
||
| ) |
Parse a decrypted BHD5 header into {file name hash: entry}.
Game.DarkSouls3 shape is a 32-bit name hash, a padded size, a 64-bit offset, the SHA and AES side-table offsets, then the unpadded size. DS2 stops before that last field, so its entries are 32 bytes and not 40 — read a DS2 header on the DS3 stride and it does not error, it walks off by eight bytes per entry and reports an empty archive, which is exactly what a wrong key looks like.
Definition at line 270 of file gamefiles.py.
References read_aes_key().
Referenced by cmd_unpack().
| dict gamefiles.read_aes_key | ( | bytes | buf, |
| int | off | ||
| ) |
The per-file AES key and the ranges it covers, or None where there is none.
Most entries carry no key at all, and the ones that do encrypt only slices — so the ranges have to be honoured rather than decrypting the whole file. A -1..-1 range is the unused-slot marker SoulsFormats filters out.
Definition at line 321 of file gamefiles.py.
Referenced by parse_bhd5().
| int gamefiles.path_hash | ( | str | path, |
| str | variant = "ds3" |
||
| ) |
SFUtil.FromPathHash: the only way to find a file whose name was thrown away.
The archives store no names. Knowing the path is enough — but only exactly: case and the leading slash both feed the hash, so /Event/x, /event/x and event/x are three different numbers and two of them are in no archive.
Definition at line 337 of file gamefiles.py.
Referenced by cmd_unpack().
| bytes gamefiles.read_entry | ( | bdt, | |
| dict | entry | ||
| ) |
Read one entry out of a .bdt, decrypting the ranges its header marks.
The read is padded-size, and the entry's own unpadded size is only trusted when POSITIVE: Sekiro writes 0 there on every entry checked, so honouring it blindly truncates every file to nothing. The real length comes from the DCX header, which is verified anyway. Ranges are clamped to whole AES blocks for the same reason the read is padded.
Definition at line 355 of file gamefiles.py.
Referenced by cmd_unpack().
| list[Path] gamefiles.ooz_paths | ( | str | None | explicit | ) |
Where to find the Kraken decoder.
See the module docstring for the build recipe.
Definition at line 392 of file gamefiles.py.
Referenced by ooz_load().
| gamefiles.ooz_load | ( | str | None | explicit = None | ) |
Load libooz.so once, from the first candidate path that exists.
Definition at line 400 of file gamefiles.py.
References ooz_paths().
Referenced by main(), and ooz_decompress().
| bytes gamefiles.ooz_decompress | ( | bytes | src, |
| int | dst_len | ||
| ) |
Definition at line 422 of file gamefiles.py.
References ooz_load().
Referenced by ooz_decompress_chunked().
| bytes gamefiles.ooz_decompress_chunked | ( | bytes | src, |
| int | dst_len | ||
| ) |
Kraken decode a stream FROM's way: one 256 KiB chunk at a time.
THE SINGLE MOST EXPENSIVE THING IN THIS FILE TO LEARN. Handing the whole stream to a Kraken decoder decodes chunk 1 and then fails on chunk 2, in powzix/ooz AND in the unrelated Rust oozextract — two independent implementations, the same failure, which is what ruled out the decoders; libooz decoding ooz's own 21-chunk xml.kraken vector cleanly ruled out the build. The difference is the WINDOW: FromSoft compresses each chunk against its own base, and a whole-stream decode hands chunk 2 the buffer's start instead. Decode chunk by chunk and every file decodes.
The framing has to be walked to do that, since a chunk's compressed size is only in its own header: two bytes of block header (magic nibble 0xC, then decoder type and a checksum flag), a three-byte quantum header holding size - 1 in its low 18 bits, then three more bytes of checksum if the flag is set. An uncompressed block carries raw bytes instead, and a zero compressed size is a memset/whole-match chunk with no payload at all.
Definition at line 446 of file gamefiles.py.
References ooz_decompress().
Referenced by dcx_decompress().
| bytes gamefiles.dcx_decompress | ( | bytes | data | ) |
Undo a DCX\0 wrapper, KRAK or DFLT.
Returns the input unchanged if it is not one.
The header is BIG-endian inside a format whose every other number is little, which is the one thing here that produces nonsense rather than an error. The declared uncompressed size is checked against what came out, so a wrong decoder fails loudly instead of writing a plausible short file.
Definition at line 476 of file gamefiles.py.
References ooz_decompress_chunked().
Referenced by cmd_unpack().
|
protected |
Definition at line 532 of file gamefiles.py.
Referenced by msb_parts(), msb_section_pos(), msb_sections(), read_emevd(), and read_msb().
|
protected |
Definition at line 536 of file gamefiles.py.
Referenced by msb_parts().
|
protected |
Definition at line 540 of file gamefiles.py.
Referenced by msb_models(), msb_parts(), msb_section_pos(), msb_sections(), and read_emevd().
|
protected |
A NUL-terminated UTF-16LE string at off.
Definition at line 545 of file gamefiles.py.
Referenced by msb_models(), msb_parts(), msb_section_pos(), and msb_sections().
| dict[str, list[int]] gamefiles.msb_sections | ( | bytes | buf | ) |
Every section of an MSB as {name: [entry offsets]}.
Walks the chain by each section's own nextParamOffset, which is what makes this robust: the section ORDER is a fact about the game version, the chain is a fact about the file.
Definition at line 557 of file gamefiles.py.
References _i32(), _i64(), and _utf16().
Referenced by read_msb().
| int gamefiles.msb_section_pos | ( | bytes | buf, |
| str | name | ||
| ) |
Byte offset of one section's header, for reading its version word.
Walks the same chain msb_sections does. Kept separate rather than folded into that function's return shape, because every other caller wants the entry list and nothing else, and the version is only consulted to refuse a layout this reader does not know.
Definition at line 576 of file gamefiles.py.
References _i32(), _i64(), and _utf16().
Referenced by read_msb().
| list[str] gamefiles.msb_models | ( | bytes | buf, |
| list[int] | offsets | ||
| ) |
Model names in index order, so a part's ModelIndex resolves to c1470_0000.
Definition at line 587 of file gamefiles.py.
References _i64(), and _utf16().
Referenced by read_msb().
| list[dict] gamefiles.msb_parts | ( | bytes | buf, |
| list[int] | offsets, | ||
| list[str] | models | ||
| ) |
Every part in one MSB as a dict, enemies carrying their param joins.
The entity id is read for every part type, because objects carry them too. The NPC and think ids exist only on the enemy types and are None elsewhere, rather than a misread of some other struct's bytes.
Definition at line 596 of file gamefiles.py.
References _i32(), _i64(), _u32(), and _utf16().
Referenced by read_msb().
| tuple[str, list[dict]] | None gamefiles.read_msb | ( | Path | path | ) |
Parse one .msb into (map name, [part rows]).
SEKIRO's MSBS layout only.
Elden Ring's MSBE shares the section names and the chain, and its Part base struct is NOT the same — extra offsets before the type data. So an ER map would walk this reader without erroring and report entity ids read out of the wrong words. The version word is what separates them, and it is asserted rather than assumed: a file this reader does not know is refused, because a wrong entity id here becomes a wrong flag downstream.
Definition at line 633 of file gamefiles.py.
References _i32(), msb_models(), msb_parts(), msb_section_pos(), and msb_sections().
Referenced by read_msbs().
| list[tuple[str, list[dict]]] gamefiles.read_msbs | ( | str | root | ) |
Every .msb under a directory, parsed, in map order; unknown layouts skipped.
Definition at line 657 of file gamefiles.py.
References read_msb().
Referenced by cmd_msb(), and cmd_roster().
| str gamefiles.entity_map | ( | int | eid | ) |
The map file an entity id belongs to, by the same decomposition the flags use.
A Sekiro entity id is AAB nnnn — area, sub-area, placement — so 1120450 is m11_02. That identical decomposition is the whole reason a defeat flag and an entity id can be the same number.
Definition at line 679 of file gamefiles.py.
Referenced by cmd_roster().
| list | None gamefiles.unpack_args | ( | bytes | data, |
| list[int] | types | ||
| ) |
Unpack one instruction's argument blob against a type list.
Each field is aligned to min(its size, 4) before it is read — FromSoft's own packing, and the thing that silently shifts every later argument if you skip it. A blob shorter than the layout demands returns None rather than a wrong number.
Definition at line 727 of file gamefiles.py.
Referenced by cmd_emevd(), and roster_calls().
| list[int] | None gamefiles.common_init_args | ( | bytes | data | ) |
Initialize Common Event's variadic arguments: the event id, then its parameters.
The EMEDF entry is [Event ID, Parameters] with Parameters repeating, so the blob is a plain run of u32 with no alignment to honour.
Definition at line 743 of file gamefiles.py.
Referenced by ds3_death_flags(), and roster_calls().
| list[tuple[int, int, int, bytes]] | None gamefiles.read_emevd | ( | Path | path | ) |
Every instruction in one .emevd as (event id, bank, instruction id, argdata), or None if the file is not the DS3/Sekiro shape.
Sekiro sets unk07 and DS3 does not; both are version 0xCD with the same tables, so both are read. An older permutation is SKIPPED rather than guessed at — DS3 ships eight stray m20_*/m21_00/m29_* scripts in the Bloodborne shape (version 0xCC, no unicode flag) whose varint width differs, and parsing those on this layout would invent events.
Definition at line 757 of file gamefiles.py.
References _i32(), and _i64().
Referenced by read_emevds().
| dict[str, list] gamefiles.read_emevds | ( | str | root | ) |
Every .emevd under a directory, as {map name: [instructions]}.
Definition at line 786 of file gamefiles.py.
References read_emevd().
Referenced by cmd_emevd(), and cmd_roster().
| list[tuple] gamefiles.roster_calls | ( | dict[str, list] | scripts | ) |
Every boss/miniboss call in the scripts, as (map, event, kind, entity, name id).
TWO MECHANISMS, and missing the second is why a first pass finds ONE miniboss in the whole of Sekiro. Bosses are handled inline, so their entity id is a literal in the instruction. Minibosses go through a single PARAMETERISED event in common_func whose own copy of Display Miniboss Health Bar has zeroes where the entity and name belong; the real values arrive per placement through Initialize Common Event.
The handler set is DISCOVERED, not hardcoded — any common event carrying one of the five instructions counts — so a patch adding a second handler cannot silently halve the roster. In Sekiro 1.06 there is exactly one, 20005330, called 41 times with (entity, name id, 0, kind).
Definition at line 811 of file gamefiles.py.
References common_init_args(), and unpack_args().
Referenced by build_roster().
| dict[int, dict] gamefiles.build_roster | ( | dict[str, list] | scripts | ) |
The boss/miniboss roster keyed by entity id.
An entity can be named by its health bar in one event and defeated in another, so the two halves meet on the entity. miniboss and boss are not exclusive by construction — a placement called by both is reported as such rather than silently filed under one.
Definition at line 854 of file gamefiles.py.
References roster_calls().
Referenced by cmd_roster().
| dict[int, tuple[str, int]] gamefiles.ds3_death_flags | ( | dict[str, list] | scripts | ) |
Every death flag the DS3 one-time-enemy templates are initialised with.
The first template argument is the flag; the rest are entity ids and extras that differ per template. Returns {flag: (map, template)}.
Definition at line 885 of file gamefiles.py.
References common_init_args().
Referenced by cmd_emevd().
| dict[str, dict[int, str]] gamefiles.load_fmg_tables | ( | str | msg_dir | ) |
Every FMG table in the English message binders, keyed by table name.
Which table holds an NPC name is not assumed: every table is loaded and the caller picks by coverage of the ids it is resolving. Same probe-then-report discipline the param extractor uses. An unpacked tree has no .dcx suffix; a raw one does; both work.
Definition at line 907 of file gamefiles.py.
Referenced by cmd_roster().
| tuple[str, int] gamefiles.pick_table | ( | dict[str, dict[int, str]] | tables, |
| set[int] | ids | ||
| ) |
The FMG table covering the most of ids, and how many it covered.
Definition at line 925 of file gamefiles.py.
Referenced by cmd_roster().
| dict[int, str] gamefiles.npc_dev_names | ( | str | paramdex | ) |
Paramdex's NpcParam annotations as {row id: english text}.
Definition at line 935 of file gamefiles.py.
Referenced by cmd_roster().
| list[str] gamefiles.wanted | ( | str | dict_path, |
| list[str] | prefixes, | ||
| list[str] | paths | ||
| ) |
Which dictionary paths to extract, from the prefixes and exact paths given.
Definition at line 991 of file gamefiles.py.
Referenced by cmd_unpack().
| int gamefiles.cmd_unpack | ( | args | ) |
Definition at line 1004 of file gamefiles.py.
References dcx_decompress(), load_keys(), parse_bhd5(), path_hash(), read_entry(), rsa_decrypt(), and wanted().
| int gamefiles.cmd_msb | ( | args | ) |
Definition at line 1094 of file gamefiles.py.
References read_msbs().
| int gamefiles.cmd_emevd | ( | args | ) |
Definition at line 1125 of file gamefiles.py.
References ds3_death_flags(), read_emevds(), and unpack_args().
| gamefiles.er_read_names | ( | msg_dir | ) |
Read Elden Ring's own English item names out of an unpacked msg/engus.
Three binders are merged in patch order — base, then each DLC layer — and a real name never loses to a placeholder or a blank, whichever binder it came from. The result is split into named (what the game calls the row) and blank (rows the game lists and deliberately leaves nameless), because the difference between those two is what decides whether a db_er row that the game does not name is worth keeping.
| msg_dir | The unpacked msg/engus directory. |
(named, blank), both keyed by category then bare row id. Definition at line 1179 of file gamefiles.py.
Referenced by cmd_ernames().
| int gamefiles.cmd_ernames | ( | args | ) |
Regenerate db_er/'s item-name tables from the game's own FMGs.
The tables were transcribed from a community id list, and this is what checks them against the game that owns them. The merge rule turns on a distinction the FMGs make and a flat id list cannot:
db_er/ashes.json's forward-filled junk: 123 rows the game leaves blank, which the old table had filled by repeating the previous name — sixteen separate ids all reading "Ash of War: Lion's Claw", and seven reading "Ash of War:";[NPC]-prefixed weapons and the NPC flask variants, real rows the shipped FMGs simply do not name.The split is clean rather than convenient: across db_er every db-only ash falls in the second case and every db-only weapon and good in the third, with nothing straddling.
| args | Parsed CLI arguments. |
Definition at line 1226 of file gamefiles.py.
References er_read_names().
| int gamefiles.cmd_roster | ( | args | ) |
Print the roster, compare it with the shipped table, optionally rewrite it.
The comparison is the point. It reports three kinds of disagreement separately, because they need different judgements: an id the scripts do not call a miniboss (a row that should go), a miniboss the table lacks (a row that should arrive), and a row whose id is right but whose name is a model family rather than a character.
Definition at line 1270 of file gamefiles.py.
References build_roster(), entity_map(), load_fmg_tables(), npc_dev_names(), pick_table(), read_emevds(), and read_msbs().
| int gamefiles.main | ( | ) |
Definition at line 1392 of file gamefiles.py.
References main(), and ooz_load().
Referenced by main().
| gamefiles.BASE = Path(__file__).resolve().parent.parent |
Definition at line 127 of file gamefiles.py.
| dict gamefiles.GAMES |
Per-game archive layout: which pairs exist, whose keys open them, which BHD5 shape.
keys names the dictionary inside UXM's ArchiveKeys.cs, or None for a game that ships its own key file — DS2 puts GameDataKeyCode.pem next to GameDataEbl.bhd, so nothing external is needed there. bhd5 picks the entry stride. Sekiro having no Data0 is a fact about Sekiro, not a gap in this table.
fmt: off here and on the other hand-laid-out tables in this file: ruff formats a dict literal one key per line, which is right for code and wrong for a table — this one goes from four rows you can compare at a glance to thirty-five you cannot. The code between the tables is formatted normally. fmt: off
Definition at line 145 of file gamefiles.py.
| dict gamefiles.HASH_PRIME = {"ds2": (37, 32), "ds3": (37, 32), "er": (0x85, 64)} |
The filename hash, per era.
Same rolling shape, different width and multiplier.
Through Sekiro it is SFUtil.FromPathHash: uint32, multiplier 37. Elden Ring widened it to uint64 with multiplier 0x85 (133) — UXM's ArchiveDictionary.ComputeHash switches on game >= EldenRing. Using the old one against an ER archive finds nothing at all, which reads as a bad dictionary rather than as a wrong hash.
Definition at line 169 of file gamefiles.py.
| int gamefiles.AES_BLOCK = 16 |
AES block size, and so the granularity an encrypted range can be decrypted at.
Definition at line 171 of file gamefiles.py.
| str gamefiles.KEY_HELP |
Definition at line 173 of file gamefiles.py.
| int gamefiles.OODLE_BLOCK = 0x40000 |
Oodle's block length.
Every chunk past the first starts a fresh match window.
Definition at line 382 of file gamefiles.py.
| int gamefiles.OOZ_SLACK = 0x10000 |
Slack on the Kraken output buffer.
ooz's decoder writes in quantums and can overshoot the exact declared size while finishing the last one. Sizing to the declared length alone corrupts the heap.
Definition at line 387 of file gamefiles.py.
|
protected |
Definition at line 388 of file gamefiles.py.
| gamefiles.MODEL_SECTION |
Section names, from MSBS.cs.
Only the two a placement needs are named.
Definition at line 513 of file gamefiles.py.
| gamefiles.PARTS_SECTION |
Section names, from MSBS.cs.
Only the two a placement needs are named.
Definition at line 513 of file gamefiles.py.
| gamefiles.PART_ENEMY |
Part type ids from PartsParam.PartType.
Both enemy types matter: DummyEnemy placements are cutscene or unused copies and they still carry entity ids, so filtering on "has an id" does not separate them.
Definition at line 519 of file gamefiles.py.
| gamefiles.PART_DUMMY_ENEMY |
Part type ids from PartsParam.PartType.
Both enemy types matter: DummyEnemy placements are cutscene or unused copies and they still carry entity ids, so filtering on "has an id" does not separate them.
Definition at line 519 of file gamefiles.py.
| dict gamefiles.PART_NAMES |
Definition at line 521 of file gamefiles.py.
| gamefiles.P_NAME_OFF |
Offsets inside a Part's base struct, and inside an enemy's type data.
Definition at line 526 of file gamefiles.py.
| gamefiles.P_TYPE |
Offsets inside a Part's base struct, and inside an enemy's type data.
Definition at line 526 of file gamefiles.py.
| gamefiles.P_MODEL_INDEX |
Offsets inside a Part's base struct, and inside an enemy's type data.
Definition at line 526 of file gamefiles.py.
| gamefiles.P_ENTITY_DATA_OFF |
Definition at line 527 of file gamefiles.py.
| gamefiles.P_TYPE_DATA_OFF |
Definition at line 527 of file gamefiles.py.
| gamefiles.E_THINK_PARAM |
Definition at line 528 of file gamefiles.py.
| gamefiles.E_NPC_PARAM |
Definition at line 528 of file gamefiles.py.
| gamefiles.E_CHARA_INIT |
Definition at line 528 of file gamefiles.py.
| gamefiles.E_EVENT_FLAG |
Definition at line 529 of file gamefiles.py.
| gamefiles.E_EVENT_FLAG_STATE |
Definition at line 529 of file gamefiles.py.
| dict gamefiles.MSBS_PARTS_VERSIONS = {0x21, 0x23} |
PARTS_PARAM_ST versions this reader's part struct is known to match (Sekiro).
Definition at line 653 of file gamefiles.py.
| gamefiles.MINIBOSS_BAR |
The instructions that define a roster, and what each contributes.
Definition at line 688 of file gamefiles.py.
| gamefiles.MINIBOSS_DEFEAT |
The instructions that define a roster, and what each contributes.
Definition at line 688 of file gamefiles.py.
| gamefiles.BOSS_BAR |
Definition at line 689 of file gamefiles.py.
| gamefiles.BOSS_DEFEAT |
Definition at line 689 of file gamefiles.py.
| gamefiles.BOSS_BANNER |
Definition at line 689 of file gamefiles.py.
| tuple gamefiles.AWARD_ITEM_LOT = (2003, 4) |
Definition at line 690 of file gamefiles.py.
| tuple gamefiles.INIT_COMMON_EVENT = (2000, 6) |
Initialize Common Event — how a parameterised handler gets its real arguments.
Definition at line 692 of file gamefiles.py.
| dict gamefiles.ARG_TYPES |
EMEDF numeric type code -> (struct format, size).
fmt: off
Definition at line 696 of file gamefiles.py.
| dict gamefiles.LAYOUTS |
Argument layouts for the instructions this file reads, from the EMEDF.
fmt: off
Definition at line 702 of file gamefiles.py.
| tuple gamefiles.DS3_DEATH_TEMPLATES = (20005340, 20005341, 20005342, 20000343, 20005416, 20005061, 20005760) |
The Dark Souls III common_func templates that flag a one-time enemy death.
Each takes a death flag and an entity id (some take more). Reading these out of an install is how db_ds3/enemies.json was confirmed: 125 of the 130 flags they carry land on a byte and bit that table already has, and none contradict it. fmt: off
Definition at line 718 of file gamefiles.py.
| dict gamefiles.SDT_AREA_BY_MAP |
The travel-menu area a Sekiro entity id belongs to, for the shipped table's keys.
db_sdt/minibosses.json is keyed by the game's TRAVEL AREAS and the maps are the game's FILES — two different partitions, the same split the idol table lives with. This mapping is the one the shipped table already uses, so a regenerated table keeps its shape; the two Ashina Castle map files both fall under one menu area, and m11_02 is the Reservoir, which the menu files under its own name. fmt: off
Definition at line 955 of file gamefiles.py.
| list gamefiles.SDT_AREA_ORDER |
The order areas are printed in: the game's own travel-menu order.
db_sdt/idols.json uses exactly this sequence, so the two Sekiro progress sections read in the same order, and a regenerated table does not reshuffle the output. Sorting by map id instead would put Hirata before the Outskirts, which is neither the menu's order nor the order the player sees them in. The Reservoir has no key of its own in idols.json — its idol is filed under Ashina Castle — but it does have minibosses, so it is inserted where the menu puts it rather than appended. fmt: off
Definition at line 972 of file gamefiles.py.
| int gamefiles.DB_INDENT = 1 |
Indent for a generated db_* table.
ONE space, matching every other table there.
.prettierignore excludes db_*/ on purpose — these are data, not code — so nothing reformats them afterwards and the house style has to be produced here. Writing prettier's two-space default instead reflows the whole file and buries the real diff.
Definition at line 982 of file gamefiles.py.
| dict gamefiles.ER_NAME_FMGS |
The five item.msgbnd name tables that back a db_er/ file, and the category nibble Elden Ring's save ids carry on top of the bare param row id.
db_er/ is keyed by the id as the SAVE stores it, so the nibble goes back on before writing. Weapons are nibble 0, which is why they are the one category whose db keys already match the game's row ids one for one.
Definition at line 1154 of file gamefiles.py.
| str gamefiles.ER_FMG_PLACEHOLDER = "[ERROR]" |
FromSoft's marker for a row that exists but carries no player-facing name.
It appears either alone or in front of a real string. A bare one means the row is not an item the player ever sees named; a prefixed one is a real name with a development marker still on it, so the marker comes off and the name stays.
Definition at line 1167 of file gamefiles.py.
1.9.8