oak-team/oak-gpui
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,495 Commits 1 Branch 0 Tags
ff75d1663b4364fc288e76790422477e1895dc49
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Kyle Caverly bbe6d3b261 Semantic index rough edges (#2871) 
Small PR aimed at improving a few edge cases for semantic indexing large
projects.

Release Notes (Preview-only).

- Batched large files with a number of documents greater than
EMBEDDINGS_BATCH_SIZE.
- Ensured that the job handle counting mechanism is consistent with
inner file batching.
- Updated tab content names for semantic search, to match text/regex
searches.
 2023-08-21 13:43:08 +02:00
KCaverlyandPiotr def215af9f update job handle to ensure file count is consistent 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-21 12:47:43 +02:00
KCaverly 1cae4758cc manage for edge case in which file documents are larger than the allowable limit 2023-08-21 11:29:45 +02:00
Kyle Caverly 8451e7eb7e Project search design (#2834) 
TODO before merging: 
- [x] Re-run project search when options (case, word, regex) change

/cc @PixelJanitor 
Release Notes:
- Revamped project & buffer search UI.
- Added "Cycle Mode" command for search
 2023-08-18 14:38:01 +02:00
KCaverly b7dd12e53e ensured search results are cleared appropriately while cycling modes 2023-08-17 11:11:09 +01:00
KCaverlyandPiotr c99b530968 remove kill search, and restart search immediately upon cycle mode 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 16:13:21 +01:00
KCaverlyandPiotr aeda5d9842 fix semantic search panic which is created via incompatible build_search_query path 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 15:50:54 +01:00
KCaverly 6e3e61ec95 removed commented out code 2023-08-16 14:01:53 +01:00
KCaverlyandPiotr 2a7df106e1 adjusted icon sizes downwards 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 14:01:06 +01:00
KCaverlyandPiotr 6f78a1633d fix editor height in buffer search, but the dancing is back 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 13:51:41 +01:00
KCaverlyandPiotr a59535efa1 remove redundant and unneeded styling 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 12:55:32 +01:00
KCaverly d727ba18f2 Merge branch 'main' of github.com:zed-industries/zed into project_search_design 2023-08-16 12:43:04 +01:00
KCaverlyandPiotr 9bf227b884 remove regex keymap, and made spacing consistent between search objects 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 12:17:59 +01:00
KCaverlyandPiotr 11ecb7b604 reorganize search bar, enable filters and disable select all during invalid states 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-16 11:35:09 +01:00
KCaverly 32bec2e401 left align buffer search with new structure 2023-08-15 23:18:03 +01:00
KCaverlyandmaxbrunsfeld 1e8a7c7caa refactor buffer_search to reduce redundancy and simplying height management 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-08-15 20:04:14 +01:00
KCaverly 695e6d2f2e fix spacing on editor magnifying glass icon 2023-08-15 15:05:39 +01:00
KCaverly c14a99d8fa updated project_search text 2023-08-03 17:02:46 -04:00
KCaverlyandPiotr 8dd330a160 add semantic search prompt for the indexing permission on first search 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-03 12:52:20 -04:00
KCaverly 358e4e5ccf update minor text when actively searching 2023-08-03 10:22:31 -04:00
KCaverly b4f6d6eadc update search text for no results 2023-08-02 17:14:15 -04:00
KCaverly a125e318fe catch up with main 2023-08-02 16:48:11 -04:00
KCaverly 71bbd5f2f6 update keymaps for cycle search mode and toggle filters 2023-08-02 15:08:54 -04:00
KCaverly acf78f5fb4 add kill_search function to stop searching on mode change 2023-08-02 14:18:28 -04:00
KCaverlyandPiotr 7b43b0d4f1 refactored search mode to ensure state is consistent 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-02 12:29:19 -04:00
KCaverlyandPiotr caaa4b1618 add theme for search mode option buttons, and adjust padding 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-02 11:23:55 -04:00
KCaverlyandPiotr cf060f0011 added major and minor display text to project search, and fixed icon padding 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-01 13:28:21 -04:00
KCaverlyandPiotr 7d5ff60ff0 added svg right margin in search bar 
Co-authored-by: Piotr <piotr@zed.dev>
 2023-08-01 12:34:02 -04:00
Kyle Caverly ce258dfeb9 Expanded Semantic language support (#2818) 
Expand Language Support within Semantic Search

Release Notes (Preview-only)

- Added semantic search support for lua, ruby, php, svelte, erb, heex,
html and markdown.
- Full details are included here:
https://linear.app/zed-industries/issue/Z-2611/allow-semantic-search-for-all-supported-languages
 2023-08-01 11:51:51 -04:00
KCaverlyandPiotrek c28ba3a11a add cycle mode to project search 
Co-authored-by: Piotrek <piotr@zed.dev>
 2023-08-01 11:47:30 -04:00
KCaverly 300c693d55 catchup with main 2023-08-01 10:40:38 -04:00
KCaverly e221f23018 add support for markdown files to semantic search 2023-08-01 10:30:34 -04:00
KCaverly 9a50b43eaa add templating languages html, erb, heex, svelte as entire parseable file types 2023-07-31 21:03:02 -04:00
KCaverly 599f674827 add php support for semantic search 2023-07-31 16:36:09 -04:00
KCaverly 89edb3d1b5 fix templating bug for parseable entire files 2023-07-31 11:41:18 -04:00
KCaverly ca4e21881e add ruby support for semantic search 2023-07-31 10:54:30 -04:00
KCaverly a5dd8dd0a9 add lua embedding query for semantic search 2023-07-31 10:02:28 -04:00
Kyle Caverly ee66f99ce6 Parallel vector db (#2792) 
Parallelize Vector Database calls for project semantic search.

Release Notes: (Preview-only)

- Parallelize Vector database calls for project semantic search. Cuts
query time by 2/3rds.
- Removed default keymap for old semantic search modal.
 2023-07-26 17:17:59 -04:00
KCaverly fbede4a5a3 removed old code 2023-07-26 17:11:30 -04:00
KCaverly e2b38f7a31 remove unused imports 2023-07-26 17:01:44 -04:00
KCaverly 06167889c3 remove default keymap for semantic search modal 2023-07-26 16:59:19 -04:00
KCaverly 5c48729c7c managed for small batch size edge case in semantic search 2023-07-26 16:56:41 -04:00
KCaverly 89bbcdfa4f remove debug logging for project_search semantic search 2023-07-26 16:51:25 -04:00
KCaverly 98fde36834 batch search queries in the vector database 2023-07-26 16:36:39 -04:00
Kyle Caverly fc1844d684 Semantic search v2 (#2789) 
Move semantic search from navigation modal, to project search option.
This PR is intended to be released in Preview only, and requires an
opt-in semantic_index option to enable. Without this opt-in setting
enable, the user should perceive no differences between previous project
search.

Release Notes: (Preview-only)

- Added Semantic Search as a opt-in feature within Project Search
- Show indexing feedback on indexing process within project search view
 2023-07-26 10:34:58 -04:00
KCaverly 0b61c93a25 ensure semantic search is not enabled on stable 2023-07-26 10:22:33 -04:00
KCaverly 394a105639 fix warnings 2023-07-26 10:03:30 -04:00
KCaverly 0ac919f6e0 catchup with main 2023-07-26 09:50:38 -04:00
KCaverly ca6f7d8a80 add worktree previously indexed functionality to vector db 2023-07-26 09:17:04 -04:00
KCaverly 75999204ad update project search to only show semantic button visible with semantic_index enabled 2023-07-25 16:26:37 -04:00
KCaverly e8210b827d move visible text to just start anchor with context lines for semantic search 2023-07-25 15:24:27 -04:00
KCaverly cdceddd2cc update semantic index tests for elixir 2023-07-25 15:20:35 -04:00
KCaverly 97c3d97792 update semantic index tests for cpp 2023-07-25 13:30:38 -04:00
KCaverly c86096a886 update semantic index tests for javascript 2023-07-25 10:38:37 -04:00
KCaverly e02d6bc0d4 add glob filtering functionality to semantic search 2023-07-20 13:46:27 -04:00
KCaverlyandmaxbrunsfeld efe973ebe2 add embedding query for json with nested arrays and strings 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-19 16:52:44 -04:00
KCaverlyandmaxbrunsfeld 9809ec3d70 update treesitter parsing to accomodate for collapsed nested functions 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-19 15:47:05 -04:00
KCaverly 0e071919a0 parellelize embedding api calls 2023-07-18 16:09:44 -04:00
KCaverly 80ef92a3e1 fix db schema update process to ensure all tables are dropped 2023-07-18 11:14:13 -04:00
KCaverly ed1b1a5ccd update logging for open ai embedding and remove redundant truncation 2023-07-18 11:00:21 -04:00
KCaverly b9fdfd60f0 catch up with main 2023-07-18 10:26:28 -04:00
KCaverly d83c4ffb07 remove debug logging for enabled settings 2023-07-17 17:09:51 -04:00
KCaverly 8b42f5b1b3 rename vector_store crate to semantic_index 2023-07-17 17:06:10 -04:00
KCaverly e630ff38c4 add embedding treesitter queries for elixir 2023-07-17 16:29:25 -04:00
KCaverly f0bf60fded add css as a embeddable file type in which the entire file is embedded individually 2023-07-17 14:53:57 -04:00
KCaverly 1362c5a3d9 add embedding treesitter query for cpp 2023-07-17 14:43:29 -04:00
KCaverly cf0dd09b5c update vector_store to accomodate for full file parsing for JSON, TOML and YAML files 2023-07-17 10:04:32 -04:00
KCaverly 4bece54655 update jsx family of languages for preceeding comments and nested exports 2023-07-17 09:22:37 -04:00
KCaverly 2dae42b1ba update embedding query for tsx to accomodate for leading comments 2023-07-14 14:25:08 -04:00
KCaverly d4971e9ead update typescript parsing to manage for leading tsdoc comments 2023-07-14 13:47:10 -04:00
KCaverly 3a625d15d3 update c embedding query for preceding comments 2023-07-14 11:33:49 -04:00
KCaverlyandmaxbrunsfeld b38e3b804c remove reindexing subscription, and add status methods for vector store 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-13 18:14:44 -04:00
KCaverly d8fd0be598 update vector store to remove dummy embeddings 2023-07-13 17:01:56 -04:00
KCaverlyandmaxbrunsfeld 623cb9833c add tests for rust context parsing, and update rust embedding query 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-13 16:58:42 -04:00
KCaverlyandmaxbrunsfeld 0a0e40fb24 refactored code context retrieval and standardized database migration 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-13 16:34:32 -04:00
KCaverlyAlexmaxbrunsfeld
 		5eab628580 Added go parsing for semantic search, and added preceeding comments on go and rust. 
Co-authored-by: Alex <alexviscreanu@gmail.com>
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-13 14:33:31 -04:00
KCaverly a56d454a07 added semantic search support for c 2023-07-13 10:10:24 -04:00
Kyle Caverly 37568ccbf0 Vector store (#2658) 
This PR includes a new crate, aimed at maintaining a consistent semantic
embedding database, for any project opened with Zed. At a high level,
for each file in a project, we parse the file with treesitter, embed the
symbol "document" objects with OpenAI, and maintain a consistent
database of these embeddings and offset locations in a sqlite database.
Once stored, we have built a simple modal interface for querying on
these symbols embeddings using natural language, offering the
opportunity to navigate to the selected symbol.

This initial PR is intended to provide this functionality only in preview,
as we explore, evaluate and iterate on the vector store.

- Full task details are provided in the [Semantic Search Linear
Project](https://linear.app/zed-industries/project/semantic-search-7c787d198ebe/Z)
 2023-07-12 13:26:17 -04:00
KCaverly c141519dba merged with main 2023-07-12 13:15:23 -04:00
KCaverly 33e2b52a01 added test registration for project settings 2023-07-11 20:12:43 -04:00
KCaverly 297fa029e3 Merge branch 'main' of github.com:zed-industries/zed into vector_store 2023-07-11 20:05:29 -04:00
KCaverly b68cd58a3b updated vector store settings to remove batch embeddings size 2023-07-11 19:54:03 -04:00
KCaverlyandmaxbrunsfeld af7b2f17ae added initial keymap for toggle semantic search 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-11 17:13:58 -04:00
KCaverly 2ca4b3f4cc cleaned up warnings and added javascript 2023-07-11 16:41:08 -04:00
KCaverly debe6f107e updated embedding queries for tsx and typescript 2023-07-11 16:22:40 -04:00
KCaverly 02f523094b expanded embeddable context to accomodate for struct context and file paths 2023-07-11 15:58:33 -04:00
KCaverly dd0dbdc5bd brought up to speed with main 2023-07-11 14:50:48 -04:00
KCaverly 1649cf81de added versioning to files table 2023-07-11 14:42:03 -04:00
KCaverly 298c2213a0 added opt-in default settings for vector store 2023-07-11 12:03:56 -04:00
KCaverly f5fec55930 updated vector_store to handle for removed files 2023-07-11 10:03:53 -04:00
KCaverly dce72a1ce7 updated tests to accomodate for new dot location 2023-07-10 18:19:29 -04:00
KCaverlyandmaxbrunsfeld 307d8d9c8d Reduced redundant database connections on each worktree change. 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-10 17:50:19 -04:00
KCaverlyandmaxbrunsfeld 82079dd422 Updated batching to accomodate for full flushes, and cleaned up reindexing. 
Co-authored-by: maxbrunsfeld <max@zed.dev>
 2023-07-10 16:33:14 -04:00
KCaverly 0189742497 pulled treesitter parsing to own file for ease of testing and management 2023-07-10 10:06:07 -04:00
KCaverly 3f5667b101 merged main 2023-07-07 14:24:29 -04:00
KCaverly c03dda1a0c fixed bug on absolute vs relative path 2023-07-06 17:15:41 -04:00
KCaverly 6f1e988cb9 updated embedding treesitter query for python 2023-07-06 16:36:28 -04:00
KCaverly 7d634f66e2 updated vector_store to include extra context for semantic search modal 2023-07-06 16:33:54 -04:00
KCaverly e57f6f21fe reindexing update to appropriately accomodate for buffer delay and persistent pending files list 2023-07-06 15:26:43 -04:00
KCaverly a86b6c42c7 corrected batching order and managed for open ai embedding errors 2023-07-06 11:11:39 -04:00