We currently attempt to flush all rejected predictions at once even if
we have accumulated more than
`MAX_EDIT_PREDICTION_REJECTIONS_PER_REQUEST`. Instead, we will now flush
as many as possible, and then keep the rest for the next batch.
Release Notes:
- N/A
Adds a `trigger` field to the zeta1/zeta2 prediction requests so that we
can distinguish between editor, diagnostic, and zeta-cli requests.
Release Notes:
- N/A
Many prediction requests end up being rejected early without ever being
set as the current prediction. Before this change, those cases weren’t
reported as rejections because the `request_prediction_with_*` functions
simply returned `Ok(None)`.
With this update, whenever we get a successful response from the
provider, we will return at least the `id`, allowing it to be properly
reported. The request now also includes a “reject reason,” since the
different variants carry distinct implications for prediction quality.
All of these scenarios are now covered by tests. While adding them, I
also found and fixed a bug where some cancelled predictions were
incorrectly being set as the current one.
Release Notes:
- N/A
---------
Co-authored-by: MrSubidubi <dev@bahn.sh>
We've realized that a lot of the logic within an
`EditPredictionProvider` is not specific to a particular edit prediction
model / service. Rather, it is just the generic state management
required to perform edit predictions at all in Zed. We want to move to a
setup where there's one "built-in" edit prediction provider in Zed,
which can be pointed at different edit prediction models. The only logic
that is different for different models is how we construct the prompt,
send the request, and parse the output.
This PR also changes the behavior of the staff-only `zeta2` feature flag
so that in only gates your *ability* to use Zeta2, but you can still use
your local settings file to choose between different edit prediction
models/services: zeta1, zeta2, and sweep.
This PR also makes zeta1's outcome reporting and prediction-rating
features work with all prediction models, not just zeta1.
To do:
* [x] remove duplicated logic around sending cloud requests between
zeta1 and zeta2
* [x] port the outcome reporting logic from zeta to zeta2.
* [x] get the "rate completions" modal working with all EP models
* [x] display edit prediction diff
* [x] show edit history events
* [x] remove the original `zeta` crate.
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Ben Kunkle <ben@zed.dev>
1. Introduce a common `PromptFormatter` trait
2. Let models define their generation params.
3. Add support for the experimental 1120-seedcoder prompt format
Release Notes:
- N/A
This prompt is for a fine-tuned model. It has the following changes,
compared to `minimal`:
- No instructions at all, except for one sentence at the beginning of
the prompt.
- Output is a simplified unified diff -- hunk headers have no line
counts (e.g., `@@ -20 +20 @@`)
- Qwen's FIM tokens are used where possible (`<|file_sep|>`,
`<|fim_prefix|>`, `<|fim_suffix|>`, etc.)
To evaluate this model:
```
ZED_ZETA2_MODEL=zeta2-exp [usual zeta-cli eval params ...] --prompt-format minimal-qwen
```
This will point to the most recent Baseten deployment of zeta2-exp
(which may change in the future, so the prompt-format may get out of
sync).
Release Notes:
- N/A
1. Add `--prompt-format=minimal` that matches single-sentence
instructions used in fine-tuned models (specifically, in `1028-*` and
`1029-*` models)
2. Use separate configs for agentic context search model and edit
prediction model. This is useful when running a fine-tuned EP model, but
we still want to run vanilla model for context retrieval.
3. `zeta2-exp` is a symlink to the same-named Baseten deployment. This
model can be redeployed and updated without having to update the
deployment id.
4. Print scores as a compact table
Release Notes:
- N/A
---------
Co-authored-by: Piotr Osiewicz <piotr@zed.dev>
I wanted to be able to work offline, so I made it a little bit more
convenient to point zeta2 at ollama.
* For zeta2, don't require that request ids be UUIDs
* Add an env var `ZED_ZETA2_OLLAMA` that sets the edit prediction URL
and model id to work w/ ollama.
Release Notes:
- N/A
This PR implements the `zeta-cli eval` command. It will:
- Run the edit prediction model if there are no cached results
- Compute precision/recall/F1 for context retrieval at the line level:
every retrieved line of context is counted as a true positive (correct
retrieval), false positive (retrieved something that was not expected),
or false negative (didn't retrieve an expected line)
- Compute similar metrics for edit predictions
- Pretty-print results, highlighting the difference between actual and
expected when printing to tty
Other changes:
- `zeta-cli predict` accepts a `--format` argument with options `md`,
`json`, `diff`
- Code restructure
Release Notes:
- N/A
---------
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Agus Zubiaga <agus@zed.dev>
This PR adds a `zeta zeta2 predict` subcommand that takes an edit
prediction example markdown file as an argument, and performs zeta2's
prediction, showing the retrieved context and the predicted edit.
* [x] Apply uncommitted diff to get repo into the right state.
* [x] Apply edits in edit history
* [x] Display predicted edits as unified diff, regardless of model
output format
Release Notes:
- N/A
---------
Co-authored-by: Agus Zubiaga <agus@zed.dev>
Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com>
Co-authored-by: Ben Kunkle <ben.kunkle@gmail.com>
Instead of producing multiple code blocks for each edit history event,
we now produce a continuous unified diff.
Release Notes:
- N/A
---------
Co-authored-by: Oleksiy Syvokon <oleksiy.syvokon@gmail.com>
Adds a new `NumberedLines` format which is similar to `MarkedExcerpt`
but each line is prefixed with its line number.
Also fixes a bug where contagious snippets wouldn't get merged.
Release Notes:
- N/A
---------
Co-authored-by: Michael Sloan <mgsloan@gmail.com>
Co-authored-by: Michael <michael@zed.dev>
This PR adds an `x-zed-client-supports-x-ai` header to the `GET /models`
request sent to Cloud to indicate that the client supports xAI models.
Release Notes:
- N/A
Creates a new `EditPredictionProvider` for zeta2, that requests
completions from a new cloud endpoint including context from the new
`edit_prediction_context` crate. This is not ready for use, but it
allows us to iterate.
Release Notes:
- N/A
---------
Co-authored-by: Michael Sloan <michael@zed.dev>
Co-authored-by: Bennet <bennet@zed.dev>
Co-authored-by: Bennet Bo Fenner <bennetbo@gmx.de>
This PR fixes the backwards compatibility of the new `Plan` variants.
We can't add new variants to the wire representation, as old clients
won't be able to understand them.
Release Notes:
- N/A
Release Notes:
- Edit Prediction: Added Git info to edit predictions requests (only
sent for opensource projects when data collection is enabled). The sent
Git info is the SHA of the current commit and the URLs for the `origin`
and `upstream` remotes.
This PR adds more data to the `GetAuthenticatedUserResponse`.
We now return more information about the authenticated user, as well as
their plan information.
Release Notes:
- N/A
