81463223d5
## Summary Addresses #16965 This PR adds support for **opening and saving** files with legacy encodings (non-UTF-8). Previously, Zed failed to open files encoded in Shift-JIS, EUC-JP, Big5, etc., displaying a "Could not open file" error screen. This PR implements automatic encoding detection upon opening and ensures the original encoding is preserved when saving. ## Implementation Details 1. **Worktree (Loading)**: * Updated `load_file` to use `chardetng` for automatic encoding detection. * Files are decoded to UTF-8 internal strings for editing, while preserving the detected `Encoding` metadata. 2. **Language / Buffer**: * Added an `encoding` field to the `Buffer` struct to store the detected encoding. 3. **Worktree (Saving)**: * Updated `write_file` to accept the stored encoding. * **Performance Optimization**: * **UTF-8 Path**: Uses the existing optimized `fs.save` (streaming chunks directly from Rope), ensuring no performance regression for the vast majority of files. * **Legacy Encoding Path**: Implemented a fallback that converts the Rope to a contiguous `String/Bytes` in memory, re-encodes it to the target format (e.g., Shift-JIS), and writes it to disk. * *Note*: This fallback involves memory allocation, but it is necessary to support legacy encodings without refactoring the `fs` crate's streaming interfaces. ## Changes - `crates/worktree`: - Add dependencies: `encoding_rs`, `chardetng`. - Update `load_file` to detect encoding and decode content. - Update `write_file` to handle re-encoding on save. - `crates/language`: Add `encoding` field and accessors to `Buffer`. - `crates/project`: Pass encoding information between Worktree and Buffer. - `crates/vim`: Update `:w` command to use the new `write_file` signature. ## Verification I validated this manually using a Rust script to generate test files with various encodings. **Results:** * ✅ **Success (Opened & Saved correctly):** * **Japanese:** `Shift-JIS` (CP932), `EUC-JP`, `ISO-2022-JP` * **Chinese:** `Big5` (Traditional), `GBK/GB2312` (Simplified) * **Western/Unicode:** `Windows-1252` (CP1252), `UTF-16LE`, `UTF-16BE` * ⚠️ **limitations (Detection accuracy):** * Some specific encodings like `KOI8-R` or generic `Latin1` (ISO-8859-1) may partially display replacement characters (`?`) depending on the file content length. This is a known limitation of the heuristic detection library (`chardetng`) rather than the saving logic. Release Notes: - Added support for opening and saving files with legacy encodings (Shift-JIS, Big5, etc.) --------- Co-authored-by: CrazyboyQCD <53971641+CrazyboyQCD@users.noreply.github.com> Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
This contains the code for Zed's Vim emulation mode.
Vim mode in Zed is supposed to primarily "do what you expect": it mostly tries to copy vim exactly, but will use Zed-specific functionality when available to make things smoother. This means Zed will never be 100% vim compatible, but should be 100% vim familiar!
The backlog is maintained in the #vim channel notes.
Testing against Neovim
If you are making a change to make Zed's behavior more closely match vim/nvim, you can create a test using the NeovimBackedTestContext.
For example, the following test checks that Zed and Neovim have the same behavior when running * in visual mode:
#[gpui::test]
async fn test_visual_star_hash(cx: &mut gpui::TestAppContext) {
let mut cx = NeovimBackedTestContext::new(cx).await;
cx.set_shared_state("ˇa.c. abcd a.c. abcd").await;
cx.simulate_shared_keystrokes(["v", "3", "l", "*"]).await;
cx.assert_shared_state("a.c. abcd ˇa.c. abcd").await;
}
To keep CI runs fast, by default the neovim tests use a cached JSON file that records what neovim did (see crates/vim/test_data), but while developing this test you'll need to run it with the neovim flag enabled:
cargo test -p vim --features neovim test_visual_star_hash
This will run your keystrokes against a headless neovim and cache the results in the test_data directory. Note that neovim must be installed and reachable on your $PATH in order to run the feature.
Testing zed-only behavior
Zed does more than vim/neovim in their default modes. The VimTestContext can be used instead. This lets you test integration with the language server and other parts of zed's UI that don't have a NeoVim equivalent.