agent: Don't track large and common binary files (#31352)
## Issue
The agent may run very slowly on projects that contain many or large
binary files not listed in `.gitignore`.
## Solution
Temporarily rewrite `.git/info/exludes` to ignore:
- Common binary files based on the extension
- Files larger than 2 MB
## Benchmark
I measure the time between sending an agent message in UI ("hitting
Enter") and actually sending it to an LLM. Ideally, it should be
instant. Numbers for a 7.7 GB Rust project with no .gitignore.
Filter | Time
----------------------------------|-----
No filter (= before this change) | 62 s
Exclude common file types only | 1.46 s
Exclude files >2MB only | 1.16 s
Exclude both | 0.10 s
## Planned changes:
- [x] Exclude common binary file types
- [x] Exclude large files
- [ ] Track files added by agent so we could delete them (we can't rely
on git for that anymore)
- [ ] Don't block on waiting for a checkpoint to complete until we
really need it
- [ ] Only `git add` files that are about to change
Closes #ISSUE
Release Notes:
- Improved agent latency on repositories containing many files or large
files
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
# This lists files that we don't track in checkpoints
|
||||
|
||||
# Compiled source and executables
|
||||
*.exe
|
||||
*.dll
|
||||
*.so
|
||||
*.dylib
|
||||
*.a
|
||||
*.lib
|
||||
*.o
|
||||
*.obj
|
||||
*.elf
|
||||
*.out
|
||||
*.app
|
||||
*.deb
|
||||
*.rpm
|
||||
*.dmg
|
||||
*.pkg
|
||||
*.msi
|
||||
|
||||
# Archives and compressed files
|
||||
*.7z
|
||||
*.zip
|
||||
*.tar
|
||||
*.tar.gz
|
||||
*.tgz
|
||||
*.tar.bz2
|
||||
*.tbz2
|
||||
*.tar.xz
|
||||
*.txz
|
||||
*.rar
|
||||
*.jar
|
||||
*.war
|
||||
*.ear
|
||||
|
||||
# Media files
|
||||
*.jpg
|
||||
*.jpeg
|
||||
*.png
|
||||
*.gif
|
||||
*.ico
|
||||
*.svg
|
||||
*.webp
|
||||
*.bmp
|
||||
*.tiff
|
||||
*.mp3
|
||||
*.mp4
|
||||
*.avi
|
||||
*.mov
|
||||
*.wmv
|
||||
*.flv
|
||||
*.mkv
|
||||
*.webm
|
||||
*.wav
|
||||
*.flac
|
||||
*.aac
|
||||
|
||||
# Database files
|
||||
*.db
|
||||
*.sqlite
|
||||
*.sqlite3
|
||||
*.mdb
|
||||
|
||||
# Documents (often binary)
|
||||
*.pdf
|
||||
*.doc
|
||||
*.docx
|
||||
*.xls
|
||||
*.xlsx
|
||||
*.ppt
|
||||
*.pptx
|
||||
|
||||
# IDE and editor files
|
||||
.idea/
|
||||
.vscode/
|
||||
*.swp
|
||||
*.swo
|
||||
*~
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Language-specific files
|
||||
*.rlib
|
||||
*.rmeta
|
||||
*.pdb
|
||||
*.class
|
||||
*.egg
|
||||
*.egg-info/
|
||||
*.pyc
|
||||
*.pto
|
||||
__pycache__
|
||||
Reference in New Issue
Block a user