Files
Ichimura Tomoo 1bc3fa8154 Correct UTF-16 saving and add heuristic encoding detection (#45243)
This commit fixes an issue where saving UTF-16 files resulted in UTF-8
bytes due to `encoding_rs` default behavior. It also introduces a
heuristic to detect BOM-less UTF-16 and binary files.

Changes:
- Manually implement UTF-16LE/BE encoding during file save to avoid
implicit UTF-8 conversion.
- Add `analyze_byte_content` to guess UTF-16LE/BE or Binary based on
null byte distribution.
- Prevent loading binary files as text by returning an error when binary
content is detected.

Special thanks to @CrazyboyQCD for pointing out the `encoding_rs`
behavior and providing the fix, and to @ConradIrwin for the suggestion
on the detection heuristic.

Closes #14654

Release Notes:

- (nightly only) Fixed an issue where saving files with UTF-16 encoding
incorrectly wrote them as UTF-8. Also improved detection for binary
files and BOM-less UTF-16.
2025-12-19 18:18:20 +00:00
..
2025-10-17 18:58:14 +00:00
2025-12-16 05:10:33 +00:00
2025-12-15 20:53:50 -07:00
2025-10-17 18:58:14 +00:00
2025-11-19 12:34:41 +01:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-12-15 10:14:15 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-11-20 22:17:50 -07:00
2025-12-17 17:31:12 -05:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-11-11 16:10:47 +01:00
2025-12-11 15:49:10 -05:00
2025-10-17 18:58:14 +00:00
2025-12-12 06:32:30 -08:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-11-07 11:36:05 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-10-17 18:58:14 +00:00
2025-12-17 16:14:57 +00:00