We've been considering removing workspace-hack for a couple reasons:
- Lukas ran into a situation where its build script seemed to be causing
spurious rebuilds. This seems more likely to be a cargo bug than an
issue with workspace-hack itself (given that it has an empty build
script), but we don't necessarily want to take the time to hunt that
down right now.
- Marshall mentioned hakari interacts poorly with automated crate
updates (in our case provided by rennovate) because you'd need to have
`cargo hakari generate && cargo hakari manage-deps` after their changes
and we prefer to not have actions that make commits.
Currently removing workspace-hack causes our workspace to grow from
~1700 to ~2000 crates being built (depending on platform), which is
mainly a problem when you're building the whole workspace or running
tests across the the normal and remote binaries (which is where
feature-unification nets us the most sharing). It doesn't impact
incremental times noticeably when you're just iterating on `-p zed`, and
we'll hopefully get these savings back in the future when
rust-lang/cargo#14774 (which re-implements the functionality of hakari)
is finished.
Release Notes:
- N/A
Follow-up after #40417, which should've fixed hangs.
smol::fs uses a separate threadpool, which is a bit yuck.
This PR also added a benchmark you can use to run a full worktree scan
(initial one, that is) for arbitrary worktree.. and refactored worktree
scanner to use async locks, as otherwise tests were deadlocking. :)
I've benchmarked it against Zed, Linux and Chromium and saw a ~60% drop
in initial worktree scan times across the board.
Release Notes:
- Significantly (3.3x speedup over the old implementation) improved
speed of Zed's worktree scanner, that's responsible for synchronizing
the state of your project with the state of files on hard drive.
---------
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
smol::fs uses a separate threadpool, which is a bit yuck.
This PR also added a benchmark you can use to run a full worktree scan
(initial one, that is) for arbitrary worktree.. and refactored worktree
scanner to use async locks, as otherwise tests were deadlocking. :)
I've benchmarked it against Zed, Linux and Chromium and saw a ~60% drop
in initial worktree scan times across the board.
Release Notes:
- Significantly (3.3x speedup over the old implementation) improved
speed of Zed's worktree scanner, that's responsible for synchronizing
the state of your project with the state of files on hard drive.
---------
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Co-Authored-By: Ben K <ben@zed.dev>
Co-Authored-By: Anthony <anthony@zed.dev>
Co-Authored-By: Mikayla <mikayla@zed.dev>
Release Notes:
- settings: Major internal changes to settings. The primary user-facing
effect is that some settings which did not make sense in project
settings files are no-longer read from there. (For example the inline
blame settings)
---------
Co-authored-by: Ben Kunkle <ben@zed.dev>
Co-authored-by: Mikayla Maki <mikayla.c.maki@gmail.com>
Co-authored-by: Anthony <anthony@zed.dev>
This adds a "workspace-hack" crate, see
[mozilla's](https://hg.mozilla.org/mozilla-central/file/3a265fdc9f33e5946f0ca0a04af73acd7e6d1a39/build/workspace-hack/Cargo.toml#l7)
for a concise explanation of why this is useful. For us in practice this
means that if I were to run all the tests (`cargo nextest r
--workspace`) and then `cargo r`, all the deps from the previous cargo
command will be reused. Before this PR it would rebuild many deps due to
resolving different sets of features for them. For me this frequently
caused long rebuilds when things "should" already be cached.
To avoid manually maintaining our workspace-hack crate, we will use
[cargo hakari](https://docs.rs/cargo-hakari) to update the build files
when there's a necessary change. I've added a step to CI that checks
whether the workspace-hack crate is up to date, and instructs you to
re-run `script/update-workspace-hack` when it fails.
Finally, to make sure that people can still depend on crates in our
workspace without pulling in all the workspace deps, we use a `[patch]`
section following [hakari's
instructions](https://docs.rs/cargo-hakari/0.9.36/cargo_hakari/patch_directive/index.html)
One possible followup task would be making guppy use our
`rust-toolchain.toml` instead of having to duplicate that list in its
config, I opened an issue for that upstream: guppy-rs/guppy#481.
TODO:
- [x] Fix the extension test failure
- [x] Ensure the dev dependencies aren't being unified by Hakari into
the main dependencies
- [x] Ensure that the remote-server binary continues to not depend on
LibSSL
Release Notes:
- N/A
---------
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Mikayla Maki <mikayla.c.maki@gmail.com>
This PR completes the process of moving git repository state storage and
scanning logic from the worktree crate to `project::git_store`.
Release Notes:
- N/A
---------
Co-authored-by: Max Brunsfeld <maxbrunsfeld@gmail.com>
Co-authored-by: Conrad <conrad@zed.dev>
This PR adds the branch selector to the git panel and fixes a few bugs
in the repository selector.
Release Notes:
- N/A
---------
Co-authored-by: ConradIrwin <conrad.irwin@gmail.com>
Co-authored-by: Conrad <conrad@zed.dev>
Closes#4745
Release Notes:
- Added a new `file_scan_inclusions` setting to force Zed to index files
that match the provided globs, even if they're gitignored.
---------
Co-authored-by: Mikayla Maki <mikayla@zed.dev>
This PR adds support for self-hosted GitLab instances when generating
Git permalinks.
If the `origin` Git remote contains `gitlab` in the URL hostname we will
then attempt to register it as a self-hosted GitLab instance.
A note on this: I don't think relying on specific keywords is going to
be a suitable long-term solution to detection. In reality the
self-hosted instance could be hosted anywhere (e.g.,
`vcs.my-company.com`), so we will ultimately need a way to have the user
indicate which Git provider they are using (perhaps via a setting).
Closes https://github.com/zed-industries/zed/issues/18012.
Release Notes:
- Added support for self-hosted GitLab instances when generating Git
permalinks.
- The instance URL must have `gitlab` somewhere in the host in order to
be recognized.
This PR updates the `worktree` crate to depend on `rpc` with the
`test-support` feature flag when running tests.
This fixes an issue I was seeing locally when trying to run tests in the
`worktree` crate:
```
λ cargo test -p worktree -- test_repository_subfolder_git_status
Compiling worktree v0.1.0 (/Users/maxdeviant/projects/zed/crates/worktree)
error[E0432]: unresolved import `rpc::AnyProtoClient`
--> crates/worktree/src/worktree.rs:39:18
|
39 | use rpc::{proto, AnyProtoClient};
| ^^^^^^^^^^^^^^ no `AnyProtoClient` in the root
For more information about this error, try `rustc --explain E0432`.
error: could not compile `worktree` (lib test) due to 1 previous error
```
Release Notes:
- N/A
***Update**: after rebasing on top of
https://github.com/zed-industries/zed/pull/16915/ (that also changed how
search worked), the results are still good, but not 10x. Instead of
going from 10s to 500ms, it goes from 10s to 3s.*
This improves the performance of project-wide search by an order of
magnitude.
After digging in, @as-cii and I found that opening buffers was the
bottleneck for project-wide search (since Zed opens, parses, ... buffers
when finding them, which is something VS Code doesn't do, for example).
So this PR improves the performance of opening multiple buffers at once.
It does this by doing two things:
- It batches scan-requests in the worktree. When we search, we search
files in chunks of 64. Previously we'd handle all 64 scan requests
separately. The new code checks if the scan requests can be batched and
if so it, it does that.
- It batches `git status` calls when reloading the project entries for
the opened buffers. Instead of calling `git status` for each file, it
calls `git status` for a batch of files, and then extracts the status
for each file.
(It has to be said that I think the slow performance on `main` has been
a regression introduced over the last few months with the changes made
to project/worktree/git. I don't think it was this slow ~5 months ago.
But I also don't think it was this fast ~5 months ago.)
## Benchmarks
| Search | Before | After (without
https://github.com/zed-industries/zed/pull/16915) | After (with
https://github.com/zed-industries/zed/pull/16915)
|--------|--------|-------|------|
| `zed.dev` at `2b2a501192e78e`, searching for `<` (`4484` results) |
3.0s<br>2.9s<br>2.89s | 489ms<br>517ms<br>476ms | n/a |
| `zed.dev` at `2b2a501192e78e`, searching for `:` (`25886+` results) |
3.9s<br>3.9s<br>3.8s | 70ms<br>66ms<br>72ms | n/a |
| `zed` at `55dda0e6af`, searching for `<` (`10937+` results) |
10s<br>11s<br>12s | 500m<br>499ms<br>543ms | 3.4s<br>3.1s<br> |
(All results recorded after doing a warm-up run that would start
language servers etc.)
Release Notes:
- Performance of project-wide search has been improved by up to 10x.
---------
Co-authored-by: Antonio <antonio@zed.dev>
This fixes#4432 by ensuring that we scan & watch the `.zed` folder,
just like we watch the `.git`, for changes.
Release Notes:
- Settings are now loaded from local `.zed/settings.json` files even if
they are `.gitignore`d.
([#4432](https://github.com/zed-industries/zed/issues/4432)).
Co-authored-by: Bennet <bennet@zed.dev>
This PR is an internal refactor in preparation for remote editing. It
restructures the public interface of `Worktree`, reducing the number of
call sites that assume that a worktree is local or remote.
* The Project no longer calls `worktree.as_local_mut().unwrap()` in code
paths related to basic file operations
* Fewer code paths in the app rely on the worktree's `LocalSnapshot`
* Worktree-related RPC message handling is more fully encapsulated by
the `Worktree` type.
to do:
* [x] file manipulation operations
* [x] sending worktree updates when sharing
for later
* opening buffers
* updating open buffers upon worktree changes
Release Notes:
- N/A
The `worktree` crate mainly provides an in-memory model of a directory
and its git repositories. But because it was originally extracted from
the Project crate, it also contained lingering bits of code that were
outside of that area:
* it had a little bit of logic related to buffers (though most buffer
management lives in `project`)
* it had a *little* bit of logic for storing diagnostics (though the
vast majority of LSP and diagnostic logic lives in `project`)
* it had a little bit of logic for sending RPC message (though the
*receiving* logic for those RPC messages lived in `project`)
In this PR, I've moved those concerns entirely to the project crate
(where they were already dealt with for the most part), so that the
worktree crate can be more focused on its main job, and have fewer
dependencies.
Worktree no longer depends on `client` or `lsp`. It still depends on
`language`, but only because of `impl language::File for
worktree::File`.
Release Notes:
- N/A
Fixes https://github.com/zed-industries/zed/issues/9575
Fixes https://github.com/zed-industries/zed/issues/4294
### Problem
When a large git repository's `.git` folder changes (due to a `git
commit`, `git reset` etc), Zed needs to recompute the git status for
every file in that git repository. Part of computing the git status is
the *unstaged* part - the comparison between the content of the file and
the version in the git index. In a large git repository like `chromium`
or `linux`, this is inherently pretty slow.
Previously, we performed this git status all at once, and held a lock on
our `BackgroundScanner`'s state for the entire time. On my laptop, in
the `linux` repo, this would often take around 13 seconds.
When opening a file, Zed always refreshes the metadata for that file in
its in-memory snapshot of worktree. This is normally very fast, but if
another task is holding a lock on the `BackgroundScanner`, it blocks.
### Solution
I've restructured how Zed handles Git statuses, so that when a git
repository is updated, we recompute files' git statuses in fixed-sized
batches. In between these batches, the `BackgroundScanner` is free to
perform other work, so that file operations coming from the main thread
will still be responsive.
Release Notes:
- Fixed a bug that caused long delays in opening files right after
performing a commit in very large git repositories.
This is just a refactor. I noticed that we now have a `project_core`
crate, which mainly contains the `Worktree` type and its private
helpers, plus the project's settings.
In this PR, I've renamed that crate to `worktree` and did some minor
simplification to its module structure. I also extracted a new
`WorktreeSettings` settings type from the `ProjectSettings`, so that the
worktree settings could live in the worktree crate. This way, the crate
is now exclusively about worktree logic.
Release Notes:
- N/A
