oak-team/oak-gpui
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,765 Commits 1 Branch 0 Tags
2b780ee7b2e63d990400c19c002eb1fc2f7bdfd7
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
KCaverly 2b780ee7b2 add base model to EmbeddingProvider, not yet leveraged for truncation 2023-10-22 15:00:09 +02:00
KCaverly d1dec8314a move OpenAIEmbeddings to OpenAIEmbeddingProvider in providers folder 2023-10-22 14:46:22 +02:00
KCaverly d813ae8845 replace OpenAIRequest with more generalized Box<dyn CompletionRequest> 2023-10-22 14:33:19 +02:00
KCaverly 05ae978cb7 move OpenAICompletionProvider to providers location 2023-10-22 13:57:13 +02:00
KCaverly 3712794e56 move OpenAILanguageModel to providers folder 2023-10-22 13:47:28 +02:00
KCaverly a62baf34f2 rename templates to prompts in ai crate 2023-10-22 13:46:49 +02:00
KCaverly 0b57ab7303 cleaned up truncate vs truncate start 2023-10-22 13:34:22 +02:00
Kyle Caverly 9c49191031 decrease temperature for inline assist on code content (#3145) 
"Temperature" is a parameter in OpenAI GPT models, to control for
randomess in the generated content. To decrease the probability of
either escaping the markdown blocks and creating invalid code, we
decreased temperature for all Non-Prose files. For Markdown or Plain
Text, in which more creativity may be a good thing, we increase the
temperature to allow for more randomness. Along with this, we ask the
generate inline prompt to include only the code and not markdown blocks,
as it appears that lower temperature may decrease the probability of
introducing random markdown blocks.

Release Notes (Internal Only):
- Decrease temperature for inline assist on code content.
 2023-10-19 16:11:29 -04:00
KCaverly 71fb23f769 decrease temperature for inline assist on code content 2023-10-19 15:44:49 -04:00
Kyle Caverly e45491d2f8 Introduction of PromptTemplate and PromptChains (#3139) 
(This PR was written 100% by the Inline Assistant)

This PR brings in new components into our ai and assistant crates namely
PromptTemplate and PromptChains. They offer a new way to generate
prompts that allow for a more flexible and dynamic approach than before.

Release Notes:
- Introduced PromptTemplate: an abstract base for individual parts of
the prompt.
- Added PromptChains: manage multiple PromptTemplates, sort them based
on priority and regulate the output size based on tokens.
- Provided new PromptArguments structure to encapsulate arguments needed
for PromptTemplate.
    - Extended repository_context to include PromptCodeSnippet.
 2023-10-19 14:44:43 -04:00
KCaverly 19c2df4822 outlined when truncation is taking place in the prompt 2023-10-19 14:33:52 -04:00
KCaverly 178a84bcf6 progress on smarter truncation strategy for file context 2023-10-18 17:56:59 -04:00
KCaverly 587fd707ba added smarter error handling for file_context prompts without provided buffers 2023-10-18 16:40:09 -04:00
KCaverly f59f2eccd5 added dumb truncation strategies to file_context and generate 2023-10-18 16:32:14 -04:00
KCaverly a0e01e075d fix for error when truncating a length less than the string length 2023-10-18 16:31:29 -04:00
KCaverly 32853c2044 added initial placeholder for truncation without a valid strategy 2023-10-18 16:23:53 -04:00
KCaverly 473067db31 update PromptPriority to accomodate for both Mandatory and Ordered prompts 2023-10-18 15:56:39 -04:00
KCaverly aa1825681c update the assistant panel to use new prompt templates 2023-10-18 14:20:12 -04:00
KCaverly b9bb27512c fix template ordering during prompt chain generation 2023-10-18 13:10:31 -04:00
KCaverly fa61c1b9c1 add prompt template for generate inline content 2023-10-18 13:03:11 -04:00
KCaverly 178a79fc47 added prompt template for file context without truncation 2023-10-18 12:29:10 -04:00
KCaverly 4b8771f3e2 Merge branch 'main' of github.com:zed-industries/zed into prompt_template 2023-10-18 10:42:16 -04:00
Kyle Caverly fea6d70d4d return code inside a markdown block during inline assist (#3137) 
Reverted prior small change in inline prompting.
We should now only return code in a markdown block during inline
assists.
 2023-10-18 10:40:53 -04:00
KCaverly ed8a2c8793 revert change to return only the text and inside return all text inside markdown blocks 2023-10-18 10:35:11 -04:00
KCaverly 02853bbd60 added prompt template for repository context 2023-10-17 17:29:07 -04:00
KCaverly a874a09b7e added openai language model tokenizer and LanguageModel trait 2023-10-17 16:21:03 -04:00
Kyle Caverly 2795091f0c Introduce Context Retrieval in Inline Assistant (#3097) 
This PR introduces a new Inline Assistant feature "Retrieve Context", to
dynamically fill the content in your generation prompt based on relevant
results returned from the Semantic Search for the Prompt.

Release Notes:

- Introduce "Retrieve Context" button in Inline Assistant
 2023-10-17 15:04:36 -04:00
KCaverly ad92fe49c7 implement initial concept of prompt chain 2023-10-17 11:58:45 -04:00
KCaverly 500af6d775 progress on prompt chains 2023-10-16 18:47:10 -04:00
KCaverly 40755961ea added initial template outline 2023-10-16 11:54:32 -04:00
KCaverly 29f45a2e38 clean up warnings 2023-10-16 10:02:11 -04:00
KCaverly 5e1e0b4759 remove print from prompts 2023-10-16 09:55:45 -04:00
KCaverly d2e769027a catchup with main 2023-10-16 09:47:07 -04:00
KCaverly 391179657c clean up redundancies in prompts and ensure tokens are being reserved for generation when filling semantic context 2023-10-06 16:43:19 +02:00
KCaverly ecfece3ac4 catchup with main 2023-10-06 16:30:31 +02:00
KCaverly ed548a0de2 ensure indexing is only done when permissioned 2023-10-06 16:08:36 +02:00
KCaverly 84553899f6 updated spacing for assistant context status icon 2023-10-06 15:43:28 +02:00
KCaverly 38ccf23567 add indexing on inline assistant opening 2023-10-06 08:46:40 +03:00
KCaverly c0a1328532 fix spawn bug from calling 2023-10-06 08:30:54 +03:00
KCaverly 0666fa80ac moved status to icon with additional information in tooltip 2023-10-05 16:49:25 +03:00
KCaverly ec1b4e6f85 added initial working status in inline assistant prompt 2023-10-05 13:01:11 +03:00
KCaverly 933c21f3d3 add initial (non updating status) toast 2023-10-03 16:53:57 +03:00
KCaverly f40d3e82c0 add user prompt for permission to index the project, for context retrieval 2023-10-03 16:26:08 +03:00
KCaverly 1a2756a232 start greedily indexing when inline assistant is started, if project has been previously indexed 2023-10-03 14:07:42 +03:00
KCaverly ed894cc06f only render retrieve context button if semantic index is enabled 2023-10-03 12:09:35 +03:00
KCaverly 166ca2a227 catching up with main 2023-10-03 12:05:00 +03:00
Kyle Caverly cf5d89d13c Leverage embeddings query to collapse syntax nodes if not selected (#3067) 
Reverts zed-industries/zed#3049
 2023-10-03 12:02:47 +03:00
KCaverly 9f160537ef move collapsed only matches outside item parent in embedding.scm 2023-10-03 11:56:45 +03:00
KCaverly bfe76467b0 add retrieve context button to inline assistant 2023-10-03 11:19:54 +03:00
KCaverly e9637267ef add placeholder button for retrieving additional context 2023-10-02 19:50:57 +03:00
KCaverly f20f096a30 searching the semantic index, and passing returned snippets to prompt generation 2023-10-02 19:15:59 +03:00
Kyle Caverly dbaaf4216d add scheme for full parseable files in semantic index (#3069) 
add scheme as a parseable file type in semantic index.
Each file will operate as a single embedding, in which no real scheme
syntax or tree-sitter level data is stored.

Release Notes:

- Added scheme to Semantic Index
 2023-09-29 14:42:15 -04:00
KCaverly 3c12e711a4 add scheme for full parseable files in semantic index 2023-09-29 14:35:02 -04:00
Kyle Caverly 2c0547079a Revert "leverage file outline and selection as opposed to entire file" (#3049) 
Reverts zed-industries/zed#3040
 2023-09-27 12:21:11 -04:00
Kyle Caverly 4242b45646 Revert "removed stale dbg in assistant from main" (#3048) 
Reverts zed-industries/zed#3046
 2023-09-27 12:19:54 -04:00
Kyle Caverly edf29aa67d implement new search strategy (#3029) 
Augment current search strategy in semantic search, reducing search
times by ~60%

Release Notes:

- Implemented minimum batch sizes for concurrent database reads.
- Batch embedding matrix multiplication.
- Calculate matmul with ndarray
 2023-09-27 10:37:48 -04:00
KCaverly 0e6fd645fd leverage embeddings len returned in construction matrix multiplication 2023-09-27 10:33:04 -04:00
KCaverly 3682751455 Merge branch 'main' of github.com:zed-industries/zed into faster_semantic_search 2023-09-27 09:43:39 -04:00
KCaverly abefa2738b removed blas and increase batch size for vector search 2023-09-27 09:43:23 -04:00
Kyle Caverly 4ccd69350b removed stale dbg in assistant from main (#3046) 
remove small dbg! statement in main
 2023-09-27 09:13:41 -04:00
KCaverly 0d6880adb3 removed stale dbg in assistant from main 2023-09-27 09:13:00 -04:00
Kyle Caverly 2f368de397 leverage file outline and selection as opposed to entire file (#3040) 
Transition generate prompt for inline assist to leverage outline as
opposed to full file.
This enables, us to leverage the inline assist for large files.

Release Notes:

- Change inline assist to use tree-sitter based outlines for code
generation instead of full files
 2023-09-27 09:10:18 -04:00
KCaverly 650a160f04 update test outline for prompt tests for new cursor span 2023-09-27 09:06:53 -04:00
KCaverly 90f17d4a28 updated codegen match to leverage unused values 2023-09-26 17:11:20 -04:00
KCaverly e8dd412ac1 update inline generate prompt to leverage more explicit <|START| and |END|> spans 2023-09-26 17:10:31 -04:00
KCaverlyandAntonio 54c63063e4 changed inline assist generate prompt to leverage outline as opposed to entire prior file 
Co-Authored-by: Antonio <antonio@zed.dev>
 2023-09-26 16:23:48 -04:00
KCaverly e75f56a0f2 move to system blas 2023-09-26 12:39:22 -04:00
KCaverly 330a71d28b fixed bug limiting number of results returned 2023-09-26 10:29:55 -04:00
KCaverly ea278b5b12 ensure desc sort and cleanup unused imports 2023-09-26 09:53:49 -04:00
KCaverly 86ec0b1d9f implement new search strategy 2023-09-25 13:44:19 -04:00
Kyle Caverly e84339ef4a reorganize AI crates to structure future development (#3015) 
Reorganized assistant/semantic_index crates and introduced AI crate to
include shared functionality.

Release Notes:

- Moved most of the Assistant functionality from ai crate to assistant
crate
- Moved interaction with embedding providers from semantic_index to ai
crate
 2023-09-22 09:54:46 -04:00
KCaverly fbd6b5b434 cargo fmt 2023-09-22 09:46:06 -04:00
KCaverly dc49dec4f0 catchup with main 2023-09-22 09:43:39 -04:00
KCaverly 68c37ca2a4 move embedding provider to ai crate 2023-09-22 09:33:59 -04:00
KCaverly 48e151495f introduce ai crate with completion providers 2023-09-21 22:44:56 -04:00
KCaverly 5f6334696a rename ai crate to assistant crate 2023-09-21 21:54:59 -04:00
Kyle Caverly 3c2b05be90 add semantic index status, for non authenticated users (#3005) 
Update project search semantic ui to accommodate for users who have not
set the OPENAI_API_KEY in their environment variables.

Release Notes:

- Expand Semantic Index status to include status for non authenticated
users
- Update Search UI to illustrate this status.
 2023-09-21 14:18:58 -04:00
KCaverly 7b63369df2 move api authentication to embedding provider 2023-09-21 14:00:00 -04:00
KCaverly 997f362cc2 add semantic index status, for non authenticated users 2023-09-21 13:40:01 -04:00
Kyle Caverly 5fe8aa064f Keyboard navigation and setting persistence for project search (#2996) 
Enable keyboard shortcuts for Project Search modes, and ensure project
search settings are persisted search to search.

Release Notes:

- Added alt-cmd-s to Toggle Semantic Search Mode
- Added alt-cmd-g to Toggle Regex Search Mode
- Added alt-cmd-x to Toggle Text Search Mode
- Defaulted new project searches to using last used search mode and
settings.
 2023-09-20 12:44:30 -04:00
KCaverly 37a0c6615f remove conditional action registration for activate semantic search mode 2023-09-20 12:38:29 -04:00
KCaverly 22d095bd35 enable project search states to persist between workspace project searches 2023-09-20 12:12:01 -04:00
KCaverly 28ed406d54 add keymaps for activating text, regex and semantic modes 2023-09-20 10:04:43 -04:00
KCaverly 912e6e8091 create initial action for Semantic search mode 2023-09-20 09:48:27 -04:00
Kyle Caverly 2f44055079 Semantic index eval (#2988) 
v0 of the Semantic Index evaluate test suite

Release Notes:

- Added eval.rs as an example to the semantic-index crates
- Generates test metrics for two small projects, as a starting point to
systematically evaluate retrieval quality
 2023-09-19 19:17:06 -04:00
KCaverly 11b3bfdc99 fix warnings 2023-09-19 19:05:26 -04:00
KCaverly 25cb79e475 remove git2 dependency for repository cloning in semantic_index eval 2023-09-19 18:55:15 -04:00
KCaverly b57b5c0b33 updated git2 to use ssl 2023-09-19 16:36:51 -04:00
KCaverly d85acceeec move git2 to workspace dependency globally 2023-09-19 16:13:47 -04:00
Kyle Caverly ae019fb7d3 remove release channel flags in semantic_index (#2989) 
Remove Release channel filters from semantic_index allowing the index to
be released to stable.

Release Notes:

- Remove Release channel filters in enabled and semantic_index
initialization
 2023-09-19 12:39:26 -04:00
KCaverly 4f1a59ebf5 formatting 2023-09-19 12:27:33 -04:00
KCaverly fc8dd8433c remove release channel flags in semantic_index 2023-09-19 12:20:59 -04:00
KCaverly 183758a7c5 fix Cargo.lock for merge 2023-09-19 11:44:51 -04:00
KCaverly 25bd357426 add recall and precision to semantic index 2023-09-18 18:25:02 -04:00
KCaverly 566bb9f71b add map to evaluation suite for semantic_index 2023-09-18 09:57:52 -04:00
Kyle Caverly b9c1f3d558 enable includes filtering for open and modified buffers (#2979) 
enable include file filtering for semantic searching in open and
modified buffers

Release Notes:

- Added include based filtering for modified buffers
 2023-09-15 15:23:24 -04:00
KCaverly 1433160a08 enable include based filtering for search inside open and modified buffers 2023-09-15 15:16:20 -04:00
KCaverly 04bd107ada add ndcg@k to evaluate metrics 2023-09-15 10:36:21 -04:00
KCaverly 3a661c5977 catchup with main 2023-09-15 09:31:33 -04:00
KCaverly 796bdd3da7 update searching in modified buffers to accomodate for excluded paths 2023-09-14 19:42:06 -04:00