Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support adding files in directories to context #438

Merged
merged 18 commits into from
Dec 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
6b2c0b9
gptel-context: Support adding files in directories to context
benthamite Oct 26, 2024
c99b2cc
gptel-context: Support add and remove files in directories to context
benthamite Dec 5, 2024
147fe6e
Revert "gptel-context: Support adding files in directories to context"
benthamite Dec 5, 2024
c45ab15
gptel-context: Define logic correctly
benthamite Dec 5, 2024
8bee48a
gptel-context: Make docstring clearer
benthamite Dec 13, 2024
bc9138a
gptel-context: Convert nested ifs to cond clauses
benthamite Dec 15, 2024
6c66982
gptel-context: Replace pcase with pcase-exhaustive
benthamite Dec 15, 2024
2a40dae
gptel-context: Restore if clause deleted accidentally
benthamite Dec 15, 2024
8e5e3c8
gptel-context: Prompt for confirmation when recursively adding dirs
benthamite Dec 15, 2024
92dbe06
gptel-context: Move interactive y-or-n-p out of non-interactive function
benthamite Dec 24, 2024
c755d16
gptel-context: Add private function to add text files
benthamite Dec 24, 2024
5af68b8
gptel-context: Add private function to remove files
benthamite Dec 24, 2024
2df8dff
gptel-context: Name functions to add/handle files/dirs consistently
benthamite Dec 24, 2024
d59aaf6
gptel-context: Handle confirmation prompt via extra optional argument
benthamite Dec 25, 2024
5c4834e
gptel-context: Revert accidental sentence transposition
benthamite Dec 25, 2024
57428e4
gptel-context: Formatting
karthink Dec 25, 2024
f73584e
gptel-context: Handle messaging of removed files properly
benthamite Dec 25, 2024
4bef4ff
README: Mention adding directories to context
karthink Dec 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.org
Original file line number Diff line number Diff line change
Expand Up @@ -876,7 +876,7 @@ By default, gptel will query the LLM with the active region or the buffer conten

You can include additional text regions, buffers or files with gptel's queries. This additional context is "live" and not a snapshot. Once added, the regions, buffers or files are scanned and included at the time of each query. When using multi-modal models, added files can be of any supported type -- typically images.

You can add a selected region, buffer or file to gptel's context from the menu, or call =gptel-add=. (To add a file use =gptel-add= in Dired or use the dedicated =gptel-add-file= command.)
You can add a selected region, buffer or file to gptel's context from the menu, or call =gptel-add=. To add a file use =gptel-add= in Dired, or use the dedicated =gptel-add-file= command. Directories will have their files added recursively after prompting for confirmation.

You can examine the active context from the menu:
#+html: <img src="https://github.com/karthink/gptel/assets/8607532/63cd7fc8-6b3e-42ae-b6ca-06ff935bae9c" align="center" alt="Image showing gptel's menu with the "inspect context" command.">
Expand Down
105 changes: 75 additions & 30 deletions gptel-context.el
Original file line number Diff line number Diff line change
Expand Up @@ -84,22 +84,26 @@ context chunk. This is accessible as, for example:
:group 'gptel
:type 'function)

(defun gptel-context-add (&optional arg)
(defun gptel-context-add (&optional arg confirm)
"Add context to gptel in a DWIM fashion.

- If a region is selected, add the selected region to the
context. If there is already a gptel context at point, remove it
instead.

- If in Dired, add marked files or file at point to the context.
With negative prefix ARG, remove them from the context instead.
- If in Dired, add marked files or file at point to the context. If
karthink marked this conversation as resolved.
Show resolved Hide resolved
the selection includes directories, add all their files recursively,
prompting the user for confirmation if called interactively or
CONFIRM is non-nil. With negative prefix ARG, remove all files from
the context instead.

- Otherwise add the current buffer to the context. With positive
prefix ARG, prompt for a buffer name and add it to the context.

- With negative prefix ARG, remove all gptel contexts from the
current buffer."
(interactive "P")
- With negative prefix ARG, remove all gptel contexts from the current
buffer, prompting the user for confirmation if called interactively
or CONFIRM is non-nil."
(interactive "P\np")
(cond
;; A region is selected.
((use-region-p)
Expand All @@ -110,18 +114,25 @@ context chunk. This is accessible as, for example:
(message "Current region added as context."))
;; If in dired
((derived-mode-p 'dired-mode)
(mapc (if (and arg (< (prefix-numeric-value arg) 0))
#'gptel-context-remove
#'gptel-context-add-file)
(dired-get-marked-files)))
(let* ((files (dired-get-marked-files))
(dirs (cl-remove-if-not #'file-directory-p files))
(remove-p (< (prefix-numeric-value arg) 0))
(action-fn (if remove-p
#'gptel-context-remove
#'gptel-context-add-file)))
(when (or remove-p (null dirs) (null confirm)
(y-or-n-p (format "Recursively add files from %d director%s? "
(length dirs)
(if (= (length dirs) 1) "y" "ies"))))
(mapc action-fn files))))
;; If in an image buffer
((and (derived-mode-p 'image-mode)
(gptel--model-capable-p 'media;)
(gptel--model-capable-p 'media)
(buffer-file-name))
(funcall (if (and arg (< (prefix-numeric-value arg) 0))
#'gptel-context-remove
#'gptel-context-add-file)
(buffer-file-name))))
(buffer-file-name)))
;; No region is selected, and ARG is positive.
((and arg (> (prefix-numeric-value arg) 0))
(let* ((buffer-name (read-buffer "Choose buffer to add as context: " nil t))
Expand All @@ -132,7 +143,8 @@ context chunk. This is accessible as, for example:
(message "Buffer '%s' added as context." buffer-name)))
;; No region is selected, and ARG is negative.
((and arg (< (prefix-numeric-value arg) 0))
(when (y-or-n-p "Remove all contexts from this buffer? ")
(when (or (null confirm)
(y-or-n-p "Remove all contexts from this buffer? "))
(let ((removed-contexts 0))
(cl-loop for cov in
(gptel-context--in-region (current-buffer) (point-min) (point-max))
Expand Down Expand Up @@ -163,32 +175,63 @@ context chunk. This is accessible as, for example:
(eq buffer-file-coding-system 'no-conversion))
(file-missing (message "File \"%s\" is not readable." path))))

(defun gptel-context--add-text-file (path)
benthamite marked this conversation as resolved.
Show resolved Hide resolved
"Add text file at PATH to context."
(cl-pushnew (list path) gptel-context--alist :test #'equal)
(message "File \"%s\" added to context." path)
path)

(defun gptel-context--add-binary-file (path)
"Add binary file at PATH to context if supported.
Return PATH if added, nil if ignored."
(if-let* (((gptel--model-capable-p 'media))
(mime (mailcap-file-name-to-mime-type path))
((gptel--model-mime-capable-p mime)))
(prog1 path
(cl-pushnew (list path :mime mime)
gptel-context--alist :test #'equal)
(message "File \"%s\" added to context." path))
(message "Ignoring unsupported binary file \"%s\"." path)
nil))

(defun gptel-context--add-directory (path action)
"Process all files in directory at PATH according to ACTION.
ACTION should be either `add' or `remove'."
(let ((files (directory-files-recursively path "." t)))
(mapc (lambda (file)
(unless (file-directory-p file)
(pcase-exhaustive action
('add
(if (gptel--file-binary-p file)
(gptel-context--add-binary-file file)
(gptel-context--add-text-file file)))
('remove
(setf (alist-get file gptel-context--alist nil 'remove #'equal) nil)))))
files)
(when (eq action 'remove)
(message "Directory \"%s\" removed from context." path))))

(defun gptel-context-add-file (path)
benthamite marked this conversation as resolved.
Show resolved Hide resolved
"Add the file at PATH to the gptel context.

If PATH is a directory, recursively add all files in it.
PATH should be readable as text."
(interactive "fChoose file to add to context: ")
(if (gptel--file-binary-p path) ;Attach if supported
(if-let* (((gptel--model-capable-p 'media))
(mime (mailcap-file-name-to-mime-type path))
((gptel--model-mime-capable-p mime)))
(prog1 path
(cl-pushnew (list path :mime mime)
gptel-context--alist :test #'equal)
(message "File \"%s\" added to context." path))
(message "Ignoring unsupported binary file \"%s\"." path))
;; Add text file
(cl-pushnew (list path) gptel-context--alist :test #'equal)
(message "File \"%s\" added to context." path)
path))
(cond ((file-directory-p path)
(gptel-context--add-directory path 'add))
((gptel--file-binary-p path)
(gptel-context--add-binary-file path))
((gptel-context--add-text-file path))))

;;;###autoload (autoload 'gptel-add-file "gptel-context" "Add files to gptel's context." t)
(defalias 'gptel-add-file #'gptel-context-add-file)

(defun gptel-context-remove (&optional context)
"Remove the CONTEXT overlay from the contexts list.

If CONTEXT is nil, removes the context at point.
If selection is active, removes all contexts within selection."
If selection is active, removes all contexts within selection.
If CONTEXT is a directory, recursively removes all files in it."
(cond
((overlayp context)
(delete-overlay context)
Expand All @@ -198,9 +241,11 @@ If selection is active, removes all contexts within selection."
for ov in (alist-get (current-buffer) gptel-context--alist)
thereis (overlay-start ov))
(setf (alist-get (current-buffer) gptel-context--alist nil 'remove) nil)))
((stringp context) ;file
(setf (alist-get context gptel-context--alist nil 'remove #'equal)
nil))
((stringp context) ;file or directory
(if (file-directory-p context)
(gptel-context--add-directory context 'remove)
(setf (alist-get context gptel-context--alist nil 'remove #'equal) nil)
(message "File \"%s\" removed from context." context)))
((region-active-p)
(when-let ((contexts (gptel-context--in-region (current-buffer)
(region-beginning)
Expand Down