We've been considering removing workspace-hack for a couple reasons:
- Lukas ran into a situation where its build script seemed to be causing
spurious rebuilds. This seems more likely to be a cargo bug than an
issue with workspace-hack itself (given that it has an empty build
script), but we don't necessarily want to take the time to hunt that
down right now.
- Marshall mentioned hakari interacts poorly with automated crate
updates (in our case provided by rennovate) because you'd need to have
`cargo hakari generate && cargo hakari manage-deps` after their changes
and we prefer to not have actions that make commits.
Currently removing workspace-hack causes our workspace to grow from
~1700 to ~2000 crates being built (depending on platform), which is
mainly a problem when you're building the whole workspace or running
tests across the the normal and remote binaries (which is where
feature-unification nets us the most sharing). It doesn't impact
incremental times noticeably when you're just iterating on `-p zed`, and
we'll hopefully get these savings back in the future when
rust-lang/cargo#14774 (which re-implements the functionality of hakari)
is finished.
Release Notes:
- N/A
### Background
Zed extensions use WASI to access the file-system. They only have
read-write access to one specific folder called their work dir. But
extensions do need to be able to *refer* to other arbitrary files on the
user's machine. For instance, extensions need to be able to look up
existing binaries on the user's `PATH`, and request that Zed invoke them
as language servers. Similarly, extensions can create paths to files in
the user's project, and use them as arguments in commands that Zed
should run. For these reasons, we pass *real* paths back and forth
between the host and extensions; we don't try to abstract over the
file-system with some virtualization scheme.
On Windows, this results in a bit of mismatch, because `wasi-libc` uses
*unix-like* path conventions (and thus, so does the Rust standard
library when compiling to WASI).
### Change 1 - Fixing `current_dir`
In order to keep the extension API minimal, extensions use the standard
library function`env::current_dir()` to query the location of their
"work" directory. Previously, when initializing extensions, we used the
`env::set_current_dir` function to set their work directory, but on
Windows, where absolute paths typically begin with a drive letter, like
`C:`, the [`wasi-libc` implementation of
`chdir`](d1793637d8/libc-bottom-half/sources/chdir.c (L21))
was prepending an extra forward slash to the path, which caused
`current_dir()` to return an invalid path.
See https://github.com/bytecodealliance/wasmtime/issues/10415
In this PR, I've switched our extension initialization function to
*bypass* wasi-libc's `chdir` function, and instead write directly to
wasi-libc's private, internal state. This is a bit of a hack, but it
causes the `current_dir()` function to do what we want on Windows
without any changes to extensions' source code.
### Change 2 - Working around WASI's relative path handling
Once `current_dir` was fixed (giving us correct absolute paths on
Windows), @kubkon and I discovered that without the spurious leading `/`
character, windows absolute paths were no longer accepted by Rust's
`std::fs` APIs, because they were now recognized as relative paths, and
were being appended to the working directory.
We first tried to override the `__wasilibc_find_abspath` function in
`wasi-libc` to make it recognize windows absolute paths as being
absolute, but that functionality is difficult to override. Eventually
@kubkon realized that we could prevent WASI-libc's CWD handling from
being linked into the WASM file by overriding the `chdir` function.
wasi-libc is designed so that if you don't use their `chdir` function,
then all paths will be interpreted as relative to `/`. This makes
absolute paths behave correctly. Then, in order to make *relative* paths
work again, we simply add a preopen for `.`. Relative paths will match
that.
### Next Steps
This is a change to `zed-extension-api`, so we do need to update every
Zed extension to use the new version, in order for them to work on
windows.
Release Notes:
- N/A
---------
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
Fixes an issue that caused Windows to fail when removing extension's
directories, as Zed had never stop any related processes.
Now:
* Zed shuts down and waits until the end when the language servers are
shut down
* Adds `impl Drop for WasmExtension` where does
`self.tx.close_channel();` to stop a receiver loop that holds the "lock"
on the extension's work dir.
The extension was dropped, but the channel was not closed for some
reason.
* Does more unregistration to ensure `Arc<WasmExtension>` with the `tx`
does not leak further
* Tidies up the related errors which had never reported a problematic
path before
Release Notes:
- N/A
---------
Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
Co-authored-by: Smit <smit@zed.dev>
This adds a "workspace-hack" crate, see
[mozilla's](https://hg.mozilla.org/mozilla-central/file/3a265fdc9f33e5946f0ca0a04af73acd7e6d1a39/build/workspace-hack/Cargo.toml#l7)
for a concise explanation of why this is useful. For us in practice this
means that if I were to run all the tests (`cargo nextest r
--workspace`) and then `cargo r`, all the deps from the previous cargo
command will be reused. Before this PR it would rebuild many deps due to
resolving different sets of features for them. For me this frequently
caused long rebuilds when things "should" already be cached.
To avoid manually maintaining our workspace-hack crate, we will use
[cargo hakari](https://docs.rs/cargo-hakari) to update the build files
when there's a necessary change. I've added a step to CI that checks
whether the workspace-hack crate is up to date, and instructs you to
re-run `script/update-workspace-hack` when it fails.
Finally, to make sure that people can still depend on crates in our
workspace without pulling in all the workspace deps, we use a `[patch]`
section following [hakari's
instructions](https://docs.rs/cargo-hakari/0.9.36/cargo_hakari/patch_directive/index.html)
One possible followup task would be making guppy use our
`rust-toolchain.toml` instead of having to duplicate that list in its
config, I opened an issue for that upstream: guppy-rs/guppy#481.
TODO:
- [x] Fix the extension test failure
- [x] Ensure the dev dependencies aren't being unified by Hakari into
the main dependencies
- [x] Ensure that the remote-server binary continues to not depend on
LibSSL
Release Notes:
- N/A
---------
Co-authored-by: Mikayla <mikayla@zed.dev>
Co-authored-by: Mikayla Maki <mikayla.c.maki@gmail.com>
Part of https://github.com/zed-industries/zed/issues/22606
Before, `tsdk_path` for vtsls and typescript-language-server
unconditionally set a `tsdk`/`tsserver` property for the corresponding
language server, even if there were no such directory at all.
Instead, make the corresponding code to omit such property if it was not
found on the FS.
Release Notes:
- Fixed "The path /.../tsserver.js doesn't point to a valid tsserver
install. Falling back to bundled TypeScript version." pop-up appearing
This PR overhauls extension registration in order to make it more
modular.
The `extension` crate now contains an `ExtensionHostProxy` that can be
used to register various proxies that the extension host can use to
interact with the rest of the system.
There are now a number of different proxy traits representing the
various pieces of functionality that can be provided by an extension.
The respective crates that provide this functionality can implement
their corresponding proxy trait in order to register a proxy that the
extension host will use to register the bits of functionality provided
by the extension.
Release Notes:
- N/A
