fix(html2text): thread escape_backslash into escape_md_section() call#2082
Open
chuenchen309 wants to merge 1 commit into
Open
fix(html2text): thread escape_backslash into escape_md_section() call#2082chuenchen309 wants to merge 1 commit into
chuenchen309 wants to merge 1 commit into
Conversation
Sibling flags (escape_dot/escape_plus/escape_dash) are all forwarded from the HTML2Text instance into escape_md_section() at its call site in handle_data(), but escape_backslash was never passed. As a result, escape_md_section()'s own hardcoded default of True always wins, so backslashes are escaped/doubled in generated markdown regardless of what the instance (or CustomHTML2Text's own escape_backslash=False default, used by the default markdown-generation pipeline) configures. Fix: pass escape_backslash=self.escape_backslash alongside the other escape_* flags, the same way the siblings are threaded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
escape_backslashis never forwarded intoescape_md_section()at its call site inHTML2Text.handle_data()(crawl4ai/html2text/__init__.py), unlike its sibling flagsescape_dot/escape_plus/escape_dash, which are all threaded through correctly. Becauseescape_md_section()has its own hardcoded default ofescape_backslash=True, that default always wins — backslashes get escaped/doubled in the generated markdown regardless of what the instance (orCustomHTML2Text's ownescape_backslash=Falsedefault, which is what the default markdown-generation pipeline actually uses) configures.Fixes: n/a (no existing issue filed for this)
List of files changed and why
crawl4ai/html2text/__init__.py— passescape_backslash=self.escape_backslashinto theescape_md_section()call inhandle_data(), the same wayescape_dot/escape_plus/escape_dashare already passed.tests/unit/test_html2text_escape_backslash.py— new unit test covering bothescape_backslash=False(must not double backslashes) andescape_backslash=True(must still escape), usingCustomHTML2Textdirectly.How Has This Been Tested?
escape_backslash=Falseexplicitly set, converting<p>Use \* for multiplication</p>producedUse \\* for multiplication(doubled) instead of the expectedUse \* for multiplication— confirmed red against current code.escape_backslash=Truestill escapes as before (no default-behavior flip).tests/unit/,tests/deep_crawling/,tests/regression/test_reg_extraction.py -m "not network",tests/test_source_sibling_selector.py,tests/general/test_strip_markdown_fences.py,tests/general/test_async_markdown_generator.py): same pass/fail counts before and after the change (verified viagit stash/git stash popA-B comparison) — the only failures present are pre-existing environment failures unrelated to this diff (missing local Playwright browser binary,BrowserType.launch: Executable doesn't exist), not caused by this change.python -m py_compileon both changed files: clean.Checklist:
AI disclosure
This fix was found, implemented, and verified with Claude (Anthropic) assistance in a human-in-the-loop workflow: I reviewed the exact code path, wrote/ran the failing test myself before the fix, confirmed the fix goes green, and ran the broader test subset before submitting. Commit includes a
Co-Authored-By: Claude Opus 4.8trailer for transparency.