oak-team/oak-gpui
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,734 Commits 1 Branch 0 Tags
4e68b588be4bf2107d03749bbf60b0af9ca80da2
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kyle Caverly fea6d70d4d return code inside a markdown block during inline assist (#3137) 
Reverted prior small change in inline prompting.
We should now only return code in a markdown block during inline
assists.
 2023-10-18 10:40:53 -04:00
KCaverly ed8a2c8793 revert change to return only the text and inside return all text inside markdown blocks 2023-10-18 10:35:11 -04:00
Kyle Caverly 2795091f0c Introduce Context Retrieval in Inline Assistant (#3097) 
This PR introduces a new Inline Assistant feature "Retrieve Context", to
dynamically fill the content in your generation prompt based on relevant
results returned from the Semantic Search for the Prompt.

Release Notes:

- Introduce "Retrieve Context" button in Inline Assistant
 2023-10-17 15:04:36 -04:00
KCaverly 29f45a2e38 clean up warnings 2023-10-16 10:02:11 -04:00
KCaverly 5e1e0b4759 remove print from prompts 2023-10-16 09:55:45 -04:00
KCaverly d2e769027a catchup with main 2023-10-16 09:47:07 -04:00
KCaverly 391179657c clean up redundancies in prompts and ensure tokens are being reserved for generation when filling semantic context 2023-10-06 16:43:19 +02:00
KCaverly ecfece3ac4 catchup with main 2023-10-06 16:30:31 +02:00
KCaverly ed548a0de2 ensure indexing is only done when permissioned 2023-10-06 16:08:36 +02:00
KCaverly 84553899f6 updated spacing for assistant context status icon 2023-10-06 15:43:28 +02:00
KCaverly 38ccf23567 add indexing on inline assistant opening 2023-10-06 08:46:40 +03:00
KCaverly c0a1328532 fix spawn bug from calling 2023-10-06 08:30:54 +03:00
KCaverly 0666fa80ac moved status to icon with additional information in tooltip 2023-10-05 16:49:25 +03:00
KCaverly ec1b4e6f85 added initial working status in inline assistant prompt 2023-10-05 13:01:11 +03:00
KCaverly 933c21f3d3 add initial (non updating status) toast 2023-10-03 16:53:57 +03:00
KCaverly f40d3e82c0 add user prompt for permission to index the project, for context retrieval 2023-10-03 16:26:08 +03:00
KCaverly 1a2756a232 start greedily indexing when inline assistant is started, if project has been previously indexed 2023-10-03 14:07:42 +03:00
KCaverly ed894cc06f only render retrieve context button if semantic index is enabled 2023-10-03 12:09:35 +03:00
KCaverly 166ca2a227 catching up with main 2023-10-03 12:05:00 +03:00
Kyle Caverly cf5d89d13c Leverage embeddings query to collapse syntax nodes if not selected (#3067) 
Reverts zed-industries/zed#3049
 2023-10-03 12:02:47 +03:00
KCaverly 9f160537ef move collapsed only matches outside item parent in embedding.scm 2023-10-03 11:56:45 +03:00
KCaverly bfe76467b0 add retrieve context button to inline assistant 2023-10-03 11:19:54 +03:00
KCaverly e9637267ef add placeholder button for retrieving additional context 2023-10-02 19:50:57 +03:00
KCaverly f20f096a30 searching the semantic index, and passing returned snippets to prompt generation 2023-10-02 19:15:59 +03:00
Kyle Caverly dbaaf4216d add scheme for full parseable files in semantic index (#3069) 
add scheme as a parseable file type in semantic index.
Each file will operate as a single embedding, in which no real scheme
syntax or tree-sitter level data is stored.

Release Notes:

- Added scheme to Semantic Index
 2023-09-29 14:42:15 -04:00
KCaverly 3c12e711a4 add scheme for full parseable files in semantic index 2023-09-29 14:35:02 -04:00
Kyle Caverly 2c0547079a Revert "leverage file outline and selection as opposed to entire file" (#3049) 
Reverts zed-industries/zed#3040
 2023-09-27 12:21:11 -04:00
Kyle Caverly 4242b45646 Revert "removed stale dbg in assistant from main" (#3048) 
Reverts zed-industries/zed#3046
 2023-09-27 12:19:54 -04:00
Kyle Caverly edf29aa67d implement new search strategy (#3029) 
Augment current search strategy in semantic search, reducing search
times by ~60%

Release Notes:

- Implemented minimum batch sizes for concurrent database reads.
- Batch embedding matrix multiplication.
- Calculate matmul with ndarray
 2023-09-27 10:37:48 -04:00
KCaverly 0e6fd645fd leverage embeddings len returned in construction matrix multiplication 2023-09-27 10:33:04 -04:00
KCaverly 3682751455 Merge branch 'main' of github.com:zed-industries/zed into faster_semantic_search 2023-09-27 09:43:39 -04:00
KCaverly abefa2738b removed blas and increase batch size for vector search 2023-09-27 09:43:23 -04:00
Kyle Caverly 4ccd69350b removed stale dbg in assistant from main (#3046) 
remove small dbg! statement in main
 2023-09-27 09:13:41 -04:00
KCaverly 0d6880adb3 removed stale dbg in assistant from main 2023-09-27 09:13:00 -04:00
Kyle Caverly 2f368de397 leverage file outline and selection as opposed to entire file (#3040) 
Transition generate prompt for inline assist to leverage outline as
opposed to full file.
This enables, us to leverage the inline assist for large files.

Release Notes:

- Change inline assist to use tree-sitter based outlines for code
generation instead of full files
 2023-09-27 09:10:18 -04:00
KCaverly 650a160f04 update test outline for prompt tests for new cursor span 2023-09-27 09:06:53 -04:00
KCaverly 90f17d4a28 updated codegen match to leverage unused values 2023-09-26 17:11:20 -04:00
KCaverly e8dd412ac1 update inline generate prompt to leverage more explicit <|START| and |END|> spans 2023-09-26 17:10:31 -04:00
KCaverlyandAntonio 54c63063e4 changed inline assist generate prompt to leverage outline as opposed to entire prior file 
Co-Authored-by: Antonio <antonio@zed.dev>
 2023-09-26 16:23:48 -04:00
KCaverly e75f56a0f2 move to system blas 2023-09-26 12:39:22 -04:00
KCaverly 330a71d28b fixed bug limiting number of results returned 2023-09-26 10:29:55 -04:00
KCaverly ea278b5b12 ensure desc sort and cleanup unused imports 2023-09-26 09:53:49 -04:00
KCaverly 86ec0b1d9f implement new search strategy 2023-09-25 13:44:19 -04:00
Kyle Caverly e84339ef4a reorganize AI crates to structure future development (#3015) 
Reorganized assistant/semantic_index crates and introduced AI crate to
include shared functionality.

Release Notes:

- Moved most of the Assistant functionality from ai crate to assistant
crate
- Moved interaction with embedding providers from semantic_index to ai
crate
 2023-09-22 09:54:46 -04:00
KCaverly fbd6b5b434 cargo fmt 2023-09-22 09:46:06 -04:00
KCaverly dc49dec4f0 catchup with main 2023-09-22 09:43:39 -04:00
KCaverly 68c37ca2a4 move embedding provider to ai crate 2023-09-22 09:33:59 -04:00
KCaverly 48e151495f introduce ai crate with completion providers 2023-09-21 22:44:56 -04:00
KCaverly 5f6334696a rename ai crate to assistant crate 2023-09-21 21:54:59 -04:00
Kyle Caverly 3c2b05be90 add semantic index status, for non authenticated users (#3005) 
Update project search semantic ui to accommodate for users who have not
set the OPENAI_API_KEY in their environment variables.

Release Notes:

- Expand Semantic Index status to include status for non authenticated
users
- Update Search UI to illustrate this status.
 2023-09-21 14:18:58 -04:00
KCaverly 7b63369df2 move api authentication to embedding provider 2023-09-21 14:00:00 -04:00
KCaverly 997f362cc2 add semantic index status, for non authenticated users 2023-09-21 13:40:01 -04:00
Kyle Caverly 5fe8aa064f Keyboard navigation and setting persistence for project search (#2996) 
Enable keyboard shortcuts for Project Search modes, and ensure project
search settings are persisted search to search.

Release Notes:

- Added alt-cmd-s to Toggle Semantic Search Mode
- Added alt-cmd-g to Toggle Regex Search Mode
- Added alt-cmd-x to Toggle Text Search Mode
- Defaulted new project searches to using last used search mode and
settings.
 2023-09-20 12:44:30 -04:00
KCaverly 37a0c6615f remove conditional action registration for activate semantic search mode 2023-09-20 12:38:29 -04:00
KCaverly 22d095bd35 enable project search states to persist between workspace project searches 2023-09-20 12:12:01 -04:00
KCaverly 28ed406d54 add keymaps for activating text, regex and semantic modes 2023-09-20 10:04:43 -04:00
KCaverly 912e6e8091 create initial action for Semantic search mode 2023-09-20 09:48:27 -04:00
Kyle Caverly 2f44055079 Semantic index eval (#2988) 
v0 of the Semantic Index evaluate test suite

Release Notes:

- Added eval.rs as an example to the semantic-index crates
- Generates test metrics for two small projects, as a starting point to
systematically evaluate retrieval quality
 2023-09-19 19:17:06 -04:00
KCaverly 11b3bfdc99 fix warnings 2023-09-19 19:05:26 -04:00
KCaverly 25cb79e475 remove git2 dependency for repository cloning in semantic_index eval 2023-09-19 18:55:15 -04:00
KCaverly b57b5c0b33 updated git2 to use ssl 2023-09-19 16:36:51 -04:00
KCaverly d85acceeec move git2 to workspace dependency globally 2023-09-19 16:13:47 -04:00
Kyle Caverly ae019fb7d3 remove release channel flags in semantic_index (#2989) 
Remove Release channel filters from semantic_index allowing the index to
be released to stable.

Release Notes:

- Remove Release channel filters in enabled and semantic_index
initialization
 2023-09-19 12:39:26 -04:00
KCaverly 4f1a59ebf5 formatting 2023-09-19 12:27:33 -04:00
KCaverly fc8dd8433c remove release channel flags in semantic_index 2023-09-19 12:20:59 -04:00
KCaverly 183758a7c5 fix Cargo.lock for merge 2023-09-19 11:44:51 -04:00
KCaverly 25bd357426 add recall and precision to semantic index 2023-09-18 18:25:02 -04:00
KCaverly 566bb9f71b add map to evaluation suite for semantic_index 2023-09-18 09:57:52 -04:00
Kyle Caverly b9c1f3d558 enable includes filtering for open and modified buffers (#2979) 
enable include file filtering for semantic searching in open and
modified buffers

Release Notes:

- Added include based filtering for modified buffers
 2023-09-15 15:23:24 -04:00
KCaverly 1433160a08 enable include based filtering for search inside open and modified buffers 2023-09-15 15:16:20 -04:00
KCaverly 04bd107ada add ndcg@k to evaluate metrics 2023-09-15 10:36:21 -04:00
KCaverly 3a661c5977 catchup with main 2023-09-15 09:31:33 -04:00
KCaverly 796bdd3da7 update searching in modified buffers to accomodate for excluded paths 2023-09-14 19:42:06 -04:00
Kyle Caverly 9b901df256 small fix to rate status update (#2967) 
Small fix to update code for rate limiting status.

Release Notes (Preview only)
- Fixed update to only stop updating status, when the rate limit is
reset to None
 2023-09-14 15:43:30 -04:00
KCaverly 8ff3e37044 small fix to rate status update 2023-09-14 15:42:21 -04:00
KCaverly c19c8899fe add initial search inside modified buffers 2023-09-14 14:58:34 -04:00
KCaverly 137dda3ee6 wip eval framework for semantic index 2023-09-14 09:30:19 -04:00
KCaverly 0c1b2e5aa6 cleaned up warnings 2023-09-13 20:04:53 -04:00
KCaverly eff44f9aa4 semantic index eval, indexing appropriately 2023-09-13 20:02:15 -04:00
KCaverly 6f29582fb0 progress on eval 2023-09-13 10:32:36 -04:00
KCaverly d4fbe99052 add eval for gpt-engineer 2023-09-12 21:27:35 -04:00
KCaverly 0d14bbbf5b add eval values for tree-sitter 2023-09-12 20:36:06 -04:00
KCaverly 66c967da88 start work on eval script for semantic_index 2023-09-12 16:25:31 -04:00
Kyle Caverly d4da5540ac Rate limiting status (#2955) 
Add a rate limit remaining status to Project Search Semantic Search
minor text

Release Notes (Preview-Only):

- Added tracking functionality within EmbeddingProvider, to track rate
limit expiry
- Update minor text within Project Search to show countdown remaining
before rate limit expiry
 2023-09-11 10:40:11 -04:00
KCaverly 74ab41000b Merge branch 'main' of github.com:zed-industries/zed into rate_limiting_status 2023-09-11 10:26:22 -04:00
KCaverly e678c7d9ee swap 
SystemTime for Instant throughout rate_limit_expiry tracking
 2023-09-11 10:26:14 -04:00
KCaverlyandAntonio 7df21f86dd move cx notify observe for rate_limit_expiry into ProjectState in the semantic index 
Co-authored-by: Antonio <antonio@zed.dev>
 2023-09-11 10:11:40 -04:00
KCaverly 37915ec4f2 updated notify to accomodate for updated countdown 2023-09-08 16:53:16 -04:00
KCaverly bf43f93197 updated semantic_index reset status to leverage target reset system time as opposed to duration 2023-09-08 15:04:50 -04:00
KCaverly a5ee8fc805 initial outline for rate limiting status updates 2023-09-08 12:35:15 -04:00
Kyle Caverly 560d6b1644 update semantic search to show no results if search query is blank (#2942) 
Update semantic search to show no search results if search query is
blank
 2023-09-07 14:58:00 -04:00
KCaverly cf5d1d91a4 update semantic search to go to no results if search query is blank 2023-09-07 14:43:41 -04:00
Kyle Caverly bbeb82f884 Token count fix (#2935) 
Fix token count for OpenAIEmbeddings

Release Notes (Preview Only)

- update token count calculation for truncated OpenAIEmbeddings
- increased request timeout for OpenAI
 2023-09-06 15:15:02 -04:00
KCaverly 265d02a583 update request timeout for open ai embeddings 2023-09-06 15:09:46 -04:00
KCaverly 17237f748c update token_count for OpenAIEmbeddings to accomodate for truncation 2023-09-06 15:09:15 -04:00
Kyle Caverly 49af2874bb Eager background indexing (#2928) 
This PR ships a series of optimizations for the semantic search engine.
Mostly focused on removing invalid states, optimizing requests to
OpenAI, and reducing token usage.

Release Notes (Preview-Only):

- Added eager incremental indexing in the background on a debounce.
- Added a local embeddings cache for reducing redundant calls to OpenAI.
- Moved to an Embeddings Queue model which ensures optimal batch sizes
at the token level, and atomic file & document writes.
- Adjusted OpenAI Embedding API requests to use provided backoff delays
during Rate Limiting.
- Removed flush races between parsing files step and embedding queue
steps.
- Moved truncation to parsing step reducing the probability that OpenAI
encounters bad data.
 2023-09-05 13:15:54 -04:00
KCaverly 8dbc0fe033 update pragma settings for improved database performance 2023-09-01 17:07:20 -04:00
KCaverlyandMax 54235f4fb1 updated embeddings background delay to 5 minutes 
Co-authored-by: Max <max@zed.dev>
 2023-09-01 13:04:09 -04:00
KCaverlyandMax e86964eb5d optimize insert file in vector database 
Co-authored-by: Max <max@zed.dev>
 2023-09-01 13:01:37 -04:00
KCaverlyandAntonio 524533cfb2 flush embeddings queue when no files are parsed for 250 milliseconds 
Co-authored-by: Antonio <antonio@zed.dev>
 2023-09-01 11:24:08 -04:00