## Summary
Addresses #16965
This PR adds support for **opening and saving** files with legacy
encodings (non-UTF-8).
Previously, Zed failed to open files encoded in Shift-JIS, EUC-JP, Big5,
etc., displaying a "Could not open file" error screen. This PR
implements automatic encoding detection upon opening and ensures the
original encoding is preserved when saving.
## Implementation Details
1. **Worktree (Loading)**:
* Updated `load_file` to use `chardetng` for automatic encoding
detection.
* Files are decoded to UTF-8 internal strings for editing, while
preserving the detected `Encoding` metadata.
2. **Language / Buffer**:
* Added an `encoding` field to the `Buffer` struct to store the detected
encoding.
3. **Worktree (Saving)**:
* Updated `write_file` to accept the stored encoding.
* **Performance Optimization**:
* **UTF-8 Path**: Uses the existing optimized `fs.save` (streaming
chunks directly from Rope), ensuring no performance regression for the
vast majority of files.
* **Legacy Encoding Path**: Implemented a fallback that converts the
Rope to a contiguous `String/Bytes` in memory, re-encodes it to the
target format (e.g., Shift-JIS), and writes it to disk.
* *Note*: This fallback involves memory allocation, but it is necessary
to support legacy encodings without refactoring the `fs` crate's
streaming interfaces.
## Changes
- `crates/worktree`:
- Add dependencies: `encoding_rs`, `chardetng`.
- Update `load_file` to detect encoding and decode content.
- Update `write_file` to handle re-encoding on save.
- `crates/language`: Add `encoding` field and accessors to `Buffer`.
- `crates/project`: Pass encoding information between Worktree and
Buffer.
- `crates/vim`: Update `:w` command to use the new `write_file`
signature.
## Verification
I validated this manually using a Rust script to generate test files
with various encodings.
**Results:**
* ✅ **Success (Opened & Saved correctly):**
* **Japanese:** `Shift-JIS` (CP932), `EUC-JP`, `ISO-2022-JP`
* **Chinese:** `Big5` (Traditional), `GBK/GB2312` (Simplified)
* **Western/Unicode:** `Windows-1252` (CP1252), `UTF-16LE`, `UTF-16BE`
* ⚠️ **limitations (Detection accuracy):**
* Some specific encodings like `KOI8-R` or generic `Latin1` (ISO-8859-1)
may partially display replacement characters (`?`) depending on the file
content length. This is a known limitation of the heuristic detection
library (`chardetng`) rather than the saving logic.
Release Notes:
- Added support for opening and saving files with legacy encodings
(Shift-JIS, Big5, etc.)
---------
Co-authored-by: CrazyboyQCD <53971641+CrazyboyQCD@users.noreply.github.com>
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
Both `test_miniquotes_object` and `test_minibrackets_object` rely on
tree-sitter parsing for `MultiBufferSnapshot.bracket_ranges` to find
quote/bracket pairs. The `VimTestContext.set_state` call eventually
triggers async tree-sitter parsing, but `run_until_parked` doesn't
guarantee parsing completion.
We suspect this is what might be causing the flakiness on Windows, as
the syntax might not yet be parsed when the
`VimTestContext.simulate_keystrokes` call is made, so there's no bracket
pairs returned.
This commit adds an explicit await call on `Bufffer.parsing_idle` after
each `VimTestContext.set_state` call, to ensure tree-sitter parsing
completes before simulating keystrokes.
Release Notes:
- N/A
In a previous Pull Request, a new field was added to `editor::Editor`,
namely `cursor_offset_on_selection`, in order to control whether the
cursor representing the head of a selection should be positioned in the
last selected character, as we have on Vim mode, or after, like we have
when Vim mode is disabled.
This field would then be set by the `vim` crate, depending on the
current vim mode. However, it was noted that
`vim_mode_setting::VimModeSetting` already exsits and allows other
crates to determine whether Vim mode is enabled or not. Since we're
already checking `!range.is_empty()` in
`editor::element::SelectionLayout::new` we can then rely on simply
determining whether Vim mode is enabled to decide whether tho shift the
cursor one position to the left when making a selection.
As such, this commit removes the `cursor_offset_on_selection` field, as
well as any related methods in favor of a new `Editor.vim_mode_enabled`
method, which can be used to achieve the same behavior.
Relates to #42837
Release Notes:
- N/A
Vim visual mode and Helix selection mode both require the cursor to be
on the last character of the selection. Until now, this was implemented
by offsetting the cursor one character to the left whenever a block
cursor is used. (Since the visual modes use a block cursor.)
However, this oversees the problem that **some users might want to use
the block cursor without being in visual mode**. Meaning that the cursor
is offset by one character to the left even though Vim/Helix mode isn't
even activated.
Since the Vim mode implementation is separate from the `editor` crate
the solution is not as straightforward as just checking the current vim
mode. Therefore this PR introduces a new `Editor` struct field called
`cursor_offset_on_selection`. This field replaces the previous check
condition and is set to `true` whenever the Vim mode is changed to a
visual mode, and `false` otherwise.
Closes#36677 and #20121
Release Notes:
- Fixes block and hollow cursor being offset when selecting text
---------
Co-authored-by: dino <dinojoaocosta@gmail.com>
Closes#21324
Adds four new commands:
- `markdown::MoveUp`, `markdown::MoveDown` - these scroll up and down in
markdown preview mode, by no more than the height of a large headline.
- `markdown::MoveUpByItem`, and `markdown::MoveDownByItem` - these
scroll up and down by the height of the item at the top of the markdown
preview window. So headlines and large codeblocks, for instance, scroll
further than individual paragraph lines.
Also attempts to create sensible defaults:
`down` -> `markdown::ScrollDown`
`up` -> `markdown::ScrollUp`
`alt-down` -> `markdown::ScrollDownByItem`
`alt-up` -> `markdown::ScrollUpByItem`
And in Vim:
`ctrl-u` -> `markdown::ScrollPageUp`
`ctrl-d` -> `markdown::ScrollPageDown`
`ctrl-e` -> `markdown::ScrollDown`
`ctrl-y` -> `markdown::ScrollUp`
Release Notes:
- Added commands `markdown::ScrollUp`, `markdown::ScrollDown`,
`markdown::ScrollUpByItem`, and `markdown::ScrollDownByItem`
- Changed commands `markdown::MovePageUp` to `markdown::ScrollPageUp`
and `markdown::MovePageDown` to `markdown::ScrollPageDown`
Fixes a few cases where the commit view would layout shift as the diff
loaded. This was caused by:
- Adding the commit message buffer after all the diff files
- Using the gutter dimensions from the last frame for the avatar spacing
Release Notes:
- commit view: Fix layout shift while loading commit
---------
Co-authored-by: MrSubidubi <dev@bahn.sh>
Implements a specialized constructor `LanguageName::new_static` for
`&'static str` which reduces allocations.
`LanguageName::new` always backs the underlying `SharedString` with an
owned `Arc<str>` even when a `&'static str` is passed. This makes us
allocate each time we create a new `LanguageName` no matter what.
Creating a specialized constructor for `&'static str` allows us to
essentially construct them for free.
Additional change:
Encourages using explicit constructors to avoid needless allocations.
Currently there were no instances of this trait being called where the
lifetime was not `'static` saving another 48 locations of allocation.
```rust
impl<'a> From<&'a str> for LanguageName {
fn from(str: &'a str) -> Self {
Self(SharedString::new(str))
}
}
// to
impl From<&'static str> for LanguageName {
fn from(str: &'static str) -> Self {
Self(SharedString::new_static(str))
}
}
```
Release Notes:
- N/A
Use `MotionKind::LineWise` in both
`vim::normal::change::Vim.change_object` and
`vim::normal::yank::Vim.yank_object` when dealing with objects that
target `Mode::VisualLine`, for example, paragraphs. This fixes an issue
where yanking and changing paragraphs would not include the trailing
newline character.
Closes#28804
Release Notes:
- Fixed linewise text object operations (`yap`, `cap`, etc.) omitting
trailing blank line in vim mode
---------
Co-authored-by: dino <dinojoaocosta@gmail.com>
Closes#41582
Release Notes:
- Improves the '%' vim motion for html by moving the cursor to the
opening tag when its positioned on the `/` ( slash ) of the closing tag
Gist is we only need to block the foreground thread for reparsing if
immediate language changes are useful to the user. That is usually only
the case when they edit the buffer
Release Notes:
- Improved performance of large project searches and project diffs
Co-authored by: David Kleingeld <david@zed.dev>
We were using `std::path::Path::strip_prefix` to determine which
repository an absolute path belongs to, which doesn't work when the
paths are Windows-style but the code is running on unix. Replace it with
a platform-agnostic implementation of `strip_prefix`.
Release Notes:
- Fixed git features not working when a Windows host collaborates with a
unix guest
It is easy for us to get the two fields out of sync causing weird
problems, there is no reason to have both here so.
Release Notes:
- N/A *or* Added/Fixed/Improved ...
Co-authored by: Antonio Scandurra <antonio@zed.dev>
Closes#43209Closes#38121
Starting on the first character.
Running `v e` before changes:
<img width="410" height="162" alt="image"
src="https://github.com/user-attachments/assets/ee13fa29-826c-45c0-9ea0-a598cc8e781a"
/>
Running `v e` after changes:
<img width="483" height="166" alt="image"
src="https://github.com/user-attachments/assets/24791a07-97df-47cd-9ef2-171522adb796"
/>
Change Notes:
- Added helix selection sanitation code that directly mirrors the code
in the Vim
[`visual_motion`](b6728c080c/crates/vim/src/visual.rs (L237))
method. I kept the comments from the Vim section that explains its
purpose.
- The above change converted the problem from fixing `v e` to fixing `v
w`. Since `w` is treated differently in Helix than in Vim (i.e. `w` in
Vim goes to the first character of a word and `w` in Helix goes to the
character before a word. Commented
[here](b6728c080c/crates/vim/src/helix.rs (L132))),
the code treats `w` in `HelixSelect` as a motion that differs from the
Vim motion in the same way that the function
[`helix_move_cursor`](b6728c080c/crates/vim/src/helix.rs (L353))
separates these behaviors.
- Added a regression test
Release Notes:
- Fixes bug where `Vim::NextWordEnd` in `HelixSelect` would not select
whole word.
Update the `Vim.deactivate` method to ensure that the cursor shape is
reset to the one available in the user's settings, in the `cursor_shape`
setting, instead of simply defaulting to `CursorShape::Bar`.
In order to test this behavior, the `Editor.cursor_shape` method was
also introduced.
Release Notes:
- Fixed the cursor shape reset in vim mode deactivation, ensuring that
the user's `cursor_shape` setting is used
---------
Co-authored-by: dino <dinojoaocosta@gmail.com>
This bug seems to be caused by pushing an operator (i.e. `d`) followed
by a repeat (i.e. `.`) so the recording includes the push operator and
the repeat. When this is repeated (i.e. `.`) it causes an infinite loop.
This change fixes this bug by pushing a ClearOperator action if there is
an ongoing recording when repeat is called.
Release Notes:
- Fixed bug where pressing `d . .` in Vim mode would freeze the editor.
---------
Co-authored-by: dino <dinojoaocosta@gmail.com>
Closes#41550
Release Notes:
- Fixed `<g-l>` behavior in helix mode which will now correctly go to the last charactor of the line.
- Fixed not switching to helix normal mode when in default vim context and pressing escape.
---------
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
This PR introduces a new `MultiBufferOffset` new type wrapping size. The
goal of this is to make it clear at the type level when we are
interacting with offsets of a multi buffer versus offsets of a language
/ text buffer. This improves readability of things quite a bit by making
it clear what kind of offsets one is working with while also reducing
accidental bugs by using the wrong kin of offset for the wrong API.
This PR also uncovered two minor bugs due to that.
Does not yet introduce the MultiBufferPoint equivalent, that is for a
follow up PR.
Release Notes:
- N/A *or* Added/Fixed/Improved ...
This PR redoes the desired behavior changes of #41583 (reverted in
#42892) but less invasively
Closes#41125Closes#41164
Release Notes:
- N/A
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
The `Vim.exit_temporary_normal` method had been updated
(https://github.com/zed-industries/zed/pull/42742) to expect and
`Option<&Motion>` that would then be used to determine whether to move
the cursor right in case the motion was `Some(EndOfLine { ..})`.
Unfortunately this meant that all callers now had to provide this
argument, even if just `None`.
After merging those changes I remember that we could probably play
around with `clip_at_line_ends` so this commit removes those intial
changes in favor of updating the `vim::normal::Vim.move_cursor` method
so that, if vim is in temporary mode and `EndOfLine` is used, it
disables clipping at line ends so that the newline character can be
selected.
Closes [#42278](https://github.com/zed-industries/zed/issues/42278)
Release Notes:
- N/A
When using the end of line motion ($) while in temporary mode, the
cursor would be placed in insert mode just before the last character
instead of after, just like in NeoVim.
This happens because `EndOfLine` kind of assumes that we're in `Normal`
mode and simply places the cursor in the last character instead of the
newline character.
This commit moves the cursor one position to the right when exiting
temporary mode and the motion used was `Motion::EndOfLine`
- Update `vim::normal::Vim.exit_temporary_normal` to now accept a
`Option<&Motion>` argument, in case callers want this new logic to
potentially be applied
Closes#42278
Release Notes:
- Fixed temporary mode exit when using `$` to move to the end of the
line
We now correctly use bracket ranges from the deepest syntax layer when
finding enclosing brackets.
Release Notes:
- Fixed an issue where `MoveToEnclosingBracket` didn’t work correctly
inside Markdown code blocks.
- Fixed an issue where unmatched forward/backward Vim motions didn’t
work correctly inside Markdown code blocks.
---------
Co-authored-by: MuskanPaliwal <muskan10112002@gmail.com>
Release Notes:
- Adds support for `:[range]w {file}`
- This writes the lines in the range to the specified
- Adds support for `:[range]w`
- This replaces the current file with the selected lines
Release Notes:
- Pressing `u` after multiple `.` in rapid succession will now only undo
the latest repeat instead of all repeats.
---------
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
Update `Vim.change_surround` in order to ensure that there's no
overlapping edits by keeping track of where the open string range ends
and ensuring that the closing string range start does not go lower than
the open string range end.
Closes#42316
Release Notes:
- Fix vim's change surrounds `cs` inserting spaces with quotes by
preventing overlapping edits
---------
Co-authored-by: dino <dinojoaocosta@gmail.com>
This reverts commit 082b80ec89.
This broke clicking, e.g. in snippets like
```rs
let x = vec![
1, 2, //
3,
];
```
clicking between `2` and `,` is quite off now.
Release Notes:
- N/A
- Introduce a `default_range` field to `VimCommand`, to be optionally
used when no range is specified for the command
- Update `VimCommand.parse` to take into consideration the
`default_range`
- Introduce `CommandRange::buffer` to obtain the `CommandRange` which
corresponds to the whole buffer
- Update the `VimCommand` definitions for both `sort` and `sort i` to
default to the whole buffer when no range is specified
Closes#41750
Release Notes:
- Improved vim's `:sort` command to sort the buffer's content when no
selection is used
