Fix bip39 french bom - #88
Merged
Merged
Conversation
The embedded French BIP39 wordlist in src/key/wordlists/french.h began with a UTF-8 BOM (bytes EF BB BF, U+FEFF). That marker is not part of BIP-0039; it was almost certainly introduced when french.txt was saved with a BOM and then hex-embedded. As a result, word index 0 was the string "\ufeffabaisser" instead of "abaisser". Indices 1..2047 were unaffected. Encode() therefore emitted a non-interoperable token whenever entropy selected index 0, so PBKDF2-HMAC-SHA512 seeds diverged from the Trezor/BIP39 reference vectors. Decode() of a correct French mnemonic containing "abaisser" failed word lookup for that index. french_txt_len was 16780; the correct length after removing the three BOM bytes is 16777. Regenerate french.h from the official BIP-0039 french.txt without a BOM. Add a unit test that asserts GetWord(0) is "abaisser" and that all-zero 128-bit entropy encodes/seeds to the published Trezor French vector (passphrase TREZOR). Default builds become BIP39/Trezor-compatible. Wallets whose French mnemonics were generated under the buggy list and used word index 0 will not open on a fixed binary; a follow-up commit adds an opt-in legacy build switch for recovery of those wallets. Co-authored-by: Cursor <cursoragent@cursor.com>
After removing the UTF-8 BOM from the French BIP39 wordlist, Particl matches Trezor/BIP-0039. That breaks restore for any wallet whose mnemonic was generated with the historical buggy list and selected word index 0 (Encode/Decode used "\ufeffabaisser" instead of "abaisser"). Seeds are derived from the mnemonic string itself, so those users cannot recover on a fixed-only binary. Preserve the exact pre-fix french blob as key/wordlists/french_legacy_bom.h and add --enable-bip39-french-legacy-bom (default no). When enabled, mnemonic.cpp includes the legacy header so Encode/Decode/ToSeed reproduce the old behavior for recovery builds only. Default and release builds remain standards-compliant. Document that the flag is for one-off recovery of affected French wallets, not for generating new seeds. Unit tests assert correct vs legacy index-0 / seed behavior under each compile mode. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes particl/particl-core #87
I was really aiming to just point the problem out. It seems to have cascaded into several forked projects as well. Since it was documented in the BIP repo several years ago I guess it might be that nobody cares? or most people using this and the derivations aren't using french with other wallet sources.?