[bug] Upgrading from 1.16.7 causes encoding error downstream

[goulvench]

Please describe the bug

In a Rails app generating (Prawn) PDFs using ActionText, I noticed that ActionText HTML came out garbled after upgrading from Nokogiri 1.16.7 to latest. Version 1.17.1 has the bug, version 1.17.0 segfaults, so I can't tell.

By "garbled", I mean that clean and proper UTF8 characters like é come out as é in the final document, even though its encoding is UTF8.

Segfault location:

ruby/gems/3.3.0/gems/nokogiri-1.17.0-arm64-darwin/lib/nokogiri/xml/searchable.rb:215: [BUG] Segmentation fault at 0x0000000000000000

I've stored the Crash Report log, let me know if you want to have a look at it, but I see there are some issues like #3112 mentioning segfaults.

Help us reproduce what you're seeing

I'll try creating a reproduction script, but I wanted to write this down while it's hot. There's quite a lot of unwrapping to do to see where this comes from :-)

Expected behavior

Strings containing é appear as is, instead of é.

Environment

# Nokogiri (1.18.1)
    ---
    warnings: []
    nokogiri:
      version: 1.18.1
      cppflags:
      - "-I/Users/goulven/.rbenv/versions/3.3.4/lib/ruby/gems/3.3.0/gems/nokogiri-1.18.1-arm64-darwin/ext/nokogiri"
      - "-I/Users/goulven/.rbenv/versions/3.3.4/lib/ruby/gems/3.3.0/gems/nokogiri-1.18.1-arm64-darwin/ext/nokogiri/include"
      - "-I/Users/goulven/.rbenv/versions/3.3.4/lib/ruby/gems/3.3.0/gems/nokogiri-1.18.1-arm64-darwin/ext/nokogiri/include/libxml2"
      ldflags: []
    ruby:
      version: 3.3.4
      platform: arm64-darwin24
      gem_platform: arm64-darwin-24
      description: ruby 3.3.4 (2024-07-09 revision be1089c8ec) [arm64-darwin24]
      engine: ruby
    libxml:
      source: packaged
      precompiled: true
      patches:
      - 0001-Remove-script-macro-support.patch
      - 0002-Update-entities-to-remove-handling-of-ssi.patch
      - '0009-allow-wildcard-namespaces.patch'
      - 0010-update-config.guess-and-config.sub-for-libxml2.patch
      - 0011-rip-out-libxml2-s-libc_single_threaded-support.patch
      - '0019-xpath-Use-separate-static-hash-table-for-standard-fu.patch'
      memory_management: ruby
      iconv_enabled: true
      compiled: 2.13.5
      loaded: 2.13.5
    libxslt:
      source: packaged
      precompiled: true
      patches:
      - 0001-update-config.guess-and-config.sub-for-libxslt.patch
      datetime_enabled: true
      compiled: 1.1.42
      loaded: 1.1.42
    other_libraries:
      zlib: 1.3.1
      libiconv: '1.17'
      libgumbo: 1.0.0-nokogiri

[flavorjones]

Sorry you're having this problem. I will really need more information before I can start investigating, though.