Skip to content

Match all HTML case-insensitive attribute values without case sensitivity - #329

Merged
facelessuser merged 5 commits into
facelessuser:mainfrom
lprnmns:html-case-insensitive-attributes
Oct 27, 2026
Merged

facelessuser merged 5 commits into
facelessuser:mainfrom
lprnmns:html-case-insensitive-attributes

Conversation

@lprnmns

@lprnmns lprnmns commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Only type gets case-insensitive attribute value matching in HTML today (from #7, which assumed it was the only attribute with that convention). The HTML specification lists 46 attributes whose values are matched ASCII case-insensitively by attribute selectors on HTML elements in HTML documents, type being one of them (Case-sensitivity of selectors): rel, method, lang, dir, align, target, media, and so on.

soup = bs4.BeautifulSoup('<a rel="NoFollow" href="#">x</a><form method="POST"></form>', 'html5lib')
sv.select('[rel="nofollow"]', soup)   # [] (Chrome: the <a>)
sv.select('[method="post"]', soup)    # [] (Chrome: the <form>)

The same section limits this to HTML elements. In Chrome, <svg><feTurbulence type="fractalNoise"> is not matched by [type="fractalnoise"] and <svg><a rel="NoFollow"> is not matched by [rel="nofollow"]. Soup Sieve currently applies the type rule to SVG elements too.

Changes:

  • css_parser.py: the type check becomes a lookup in HTML_CASE_INSENSITIVE_ATTRIBUTES, which is the list from the spec. The compiled patterns are unchanged: a case-insensitive one, plus the case-sensitive xml_type_pattern as before.
  • css_match.py: match_attributes uses the case-sensitive pattern for XML documents (as before) and also for non-HTML elements (not self.is_html_tag(el)). With html.parser and lxml, which don't provide namespaces, every element still counts as HTML, so their behavior only changes by the added attribute names.

I compared the results with Chrome's querySelectorAll on the same markup. With html5lib they match for [type="checkbox"], [type="fractalnoise"], [rel="nofollow"], [method="post"], [lang="en-US"], [dir="rtl"] and [title="abc"] (not in the list, so it stays case-sensitive).

Tests: test_attribute_html_case_insensitive_values and test_attribute_html_case_insensitive_values_foreign in tests/test_level2/test_attribute.py fail without the change. The existing type tests for HTML, XML and XHTML pass unchanged. The full suite passes (434 passed, 1 skipped), along with ruff check . and mypy. I added a changelog entry under Unreleased.

@gir-bot gir-bot added S: needs-review Needs to be reviewed and/or approved. C: css-level-2 CSS level 2 selectors. C: css-matching Related to CSS matching. C: css-parsing Related to CSS parsing. C: docs Related to documentation. C: source Related to source code. C: tests Related to testing. labels Oct 23, 2026
Comment thread docs/src/markdown/about/changelog.md Outdated
`type` values that contain a newline.
- **FIX**: Match the values of every attribute that the HTML specification lists as case insensitive (`rel`,
`method`, `lang`, `dir`, ...) without case sensitivity on HTML elements in HTML documents, not just `type`. SVG
and MathML elements are matched case sensitively.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add "MathML" to en-custom.txt in sorted order so that the spell checker will not fail.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changelog entry no longer mentions MathML: in c694e30 it says "Non-HTML elements are matched case sensitively" instead, so there is nothing to add to en-custom.txt. The documents spelling job passes on the latest commit. The two regressions @MaciejZet reported are fixed in 5da4f5e, with tests.

@MaciejZet

Copy link
Copy Markdown

Two regressions on c10b4da, both confirmed against Chrome 155:

from bs4 import BeautifulSoup
import soupsieve as sv

soup = BeautifulSoup('<a id="x" rel="ſtylesheet"></a>', 'html5lib')
print(bool(sv.select('[rel~="stylesheet"]', soup)))
# PR: True; expected: False

soup = BeautifulSoup('<r><a id="x" rel="pre&#10;match"/></r>', 'xml')
print(bool(sv.select('[rel*="match"]', soup)))
# PR: False; expected: True

Both cases return the expected result on base b4f750c and main 72b2600. re.I matches Unicode pairs such as ſ/s, but the new HTML attribute matching should be ASCII-only. The case-sensitive fallback also loses re.DOTALL, reintroducing the newline problem discussed in #233 for additional attributes.

All 18 local functional test runs passed on macOS across Python 3.11–3.15.0rc1 and the stdlib-only configuration; Ruff and mypy also passed. These two cases need regression coverage.

@facelessuser

Copy link
Copy Markdown
Owner

@MaciejZet these are good points and they'll need to be resolved before we can merge.

@lprnmns

lprnmns commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Addressed both reported regressions in 5da4f5e. The HTML-only default now uses ASCII case-insensitive matching, so ſ no longer matches s; the case-sensitive XML/foreign-element fallback retains DOTALL for multiline values. Both new tests failed on the prior PR commit and pass now. Local validation: 436 passed, 1 skipped; Ruff and mypy passed. The updated CI run is pending.

@facelessuser

Copy link
Copy Markdown
Owner

Thanks! I'll take a look at this sometime today.

@facelessuser

Copy link
Copy Markdown
Owner

@gir-bot lgtm

@gir-bot gir-bot added S: approved The pull request is ready to be merged. and removed S: needs-review Needs to be reviewed and/or approved. labels Oct 27, 2026
@facelessuser
facelessuser merged commit 63a69f4 into facelessuser:main Oct 27, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C: css-level-2 CSS level 2 selectors. C: css-matching Related to CSS matching. C: css-parsing Related to CSS parsing. C: docs Related to documentation. C: source Related to source code. C: tests Related to testing. S: approved The pull request is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants