qlot distify error from dependency with new-style name #198

rpgoldman · 2023-11-24T18:32:17Z

Describe the bug

I have a system that has the following dependencies:

(defsystem "plan-sim/rewrite"
    :depends-on ("plan-sim" "plan-repair-rewrite/shop3" (:version "panda-planner" "3.2.1")) ...)

When I run qlot install on a git repo with this set of dependencies, I get an error message because qlot wants "plan-repair-rewrite/shop3" to be a symbol, inside read-asd-form.

The head of this file reads:

(defpackage :plan-sim-asd
    (:use :common-lisp :asdf))

(in-package plan-sim-asd)

If qlot is trying to read the file without interpreting the defpackage and in-package expressions, then the test ((eq (first form) 'asdf:defsystem) will fail, causing execution to proceed incorrectly.

It may be better to use asdf:parse-defsystem or asdf:load-asd to create the appropriate ASDF data structures and then use slot accessors, instead of writing a defsystem code-walker.

As a short-term kludge, I tried replacing

      ((eq (first form) 'asdf:defsystem)

with

(string-equal (symbol-name (first form)) "DEFSYSTEM")

but my error persisted.

Reproducible steps

Unfortunately, my system definition contains proprietary information, so I cannot submit it here. I will try to create a small example of this issue and add it here.

I apologize, it might take some time to assemble such a test case. I will also see if I can instrument this code so that I can find exactly which line is failing.

Error messages

Unhandled QLOT/UTILS/SHELL:SHELL-COMMAND-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                                            {70088404F3}>:
  Error while executing a shell command: "sbcl" "--noinform" "--no-sysinit" "--no-userinit" "--non-interactive" "--eval" "(require 'asdf)" "--eval" "(setf *debugger-hook* (lambda (c parent) (declare (ignore parent)) (format *error-output* \"~&Error: ~A~2%\" c) (uiop/image:print-backtrace :condition c) (uiop/image:quit -1)))" "--eval" "(push #P\"/Users/rpg/quicklisp/dists/quicklisp/software/qlot-20220331-git/\" asdf/system-registry:*central-registry*)" "--eval" "(setf asdf/source-registry:*default-source-registries* (quote (asdf/source-registry:environment-source-registry asdf/source-registry:system-source-registry asdf/source-registry:system-source-registry-directory)))" "--eval" "(load #P\"/Users/rpg/quicklisp/setup.lisp\")" "--eval" "(uiop/package:symbol-call :ql :quickload \"qlot/distify\" :silent t)" "--eval" "(let ((*error-output* (make-broadcast-stream))) (load (merge-pathnames #P\"setup.lisp\" #P\"/Users/rpg/projects/hideho/sim2exps/.qlot/\")))" "--eval" "(uiop/package:symbol-call :qlot/distify :distify (qlot/source/base:defrost-source (make-instance (quote qlot/source/git:source-git) :project-name \"plan-repair-rewrite\" :remote-url \"[email protected]:hi-de-ho/plan-repair.git\")) #P\"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/\" :distinfo-only nil)" "--eval" "(exit)" (Code=1)

  Unhandled TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {7005310573}>: The value "plan-repair-rewrite/shop3" is not of type SYMBOL

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {7005310573}>
0: (QLOT/UTILS/ASDF::READ-ASD-FORM ("plan-repair-rewrite/shop3" "fiveam"))
1: ((LAMBDA NIL :IN QLOT/UTILS/ASDF::READ-ASD-FILE))
2: ((FLET "THUNK" :IN CALL-WITH-SAFE-IO-SYNTAX))
3: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<FUNCTION (FLET "THUNK" :IN CALL-WITH-SAFE-IO-SYNTAX) {102770D3B}>)
4: (CALL-WITH-SAFE-IO-SYNTAX #<FUNCTION (LAMBDA NIL :IN QLOT/UTILS/ASDF::READ-ASD-FILE) {7007F6E30B}> :PACKAGE :CL)
5: ((LAMBDA (QLOT/UTILS/ASDF::IN) :IN QLOT/UTILS/ASDF::READ-ASD-FILE) #<SB-SYS:FD-STREAM for "file /private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-QXI6G7I/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/plan-repair-by-rewrite/plan-repair-rewrite.asd" {7007F6E153}>)
6: (CALL-WITH-INPUT-FILE #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-QXI6G7I/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/plan-repair-by-rewrite/plan-repair-rewrite.asd" #<FUNCTION (LAMBDA (QLOT/UTILS/ASDF::IN) :IN QLOT/UTILS/ASDF::READ-ASD-FILE) {7007F6D61B}> :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL :IF-DOES-NOT-EXIST :ERROR)
7: (QLOT/UTILS/DISTIFY:SYSTEMS.TXT "plan-repair-rewrite" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-QXI6G7I/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/")
8: ((LAMBDA (QLOT/DISTIFY/GIT::OUT) :IN QLOT/DISTIFY/GIT::WRITE-METADATA-FILES) #<SB-SYS:FD-STREAM for "file /private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/systems.txt" {7007F393B3}>)
9: (CALL-WITH-OUTPUT-FILE #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/systems.txt" #<FUNCTION (LAMBDA (QLOT/DISTIFY/GIT::OUT) :IN QLOT/DISTIFY/GIT::WRITE-METADATA-FILES) {7007F38BAB}> :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL :IF-EXISTS :SUPERSEDE :IF-DOES-NOT-EXIST :CREATE)
10: (QLOT/DISTIFY/GIT::WRITE-METADATA-FILES #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7007780EF3}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-QXI6G7I/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/archive.tar.gz")
11: (QLOT/DISTIFY/GIT:DISTIFY-GIT #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7007780EF3}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/" :DISTINFO-ONLY NIL)
12: (QLOT/DISTIFY:DISTIFY #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7007780EF3}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/" :DISTINFO-ONLY NIL)
13: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SYMBOL-CALL :QLOT/DISTIFY :DISTIFY (QLOT/SOURCE/BASE:DEFROST-SOURCE (MAKE-INSTANCE (QUOTE QLOT/SOURCE/GIT:SOURCE-GIT) :PROJECT-NAME "plan-repair-rewrite" :REMOTE-URL "[email protected]:hi-de-ho/plan-repair.git")) #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/" :DISTINFO-ONLY NIL) #<NULL-LEXENV>)
14: (EVAL (SYMBOL-CALL :QLOT/DISTIFY :DISTIFY (QLOT/SOURCE/BASE:DEFROST-SOURCE (MAKE-INSTANCE (QUOTE QLOT/SOURCE/GIT:SOURCE-GIT) :PROJECT-NAME "plan-repair-rewrite" :REMOTE-URL "[email protected]:hi-de-ho/plan-repair.git")) #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-I8I2CUQO/" :DISTINFO-ONLY NIL))
15: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(require 'asdf)") (:EVAL . "(setf *debugger-hook* (lambda (c parent) (declare (ignore parent)) (format *error-output* \"~&Error: ~A~2%\" c) (uiop/image:print-backtrace :condition c) (uiop/image:quit -1)))") (:EVAL . "(push #P\"/Users/rpg/quicklisp/dists/quicklisp/software/qlot-20220331-git/\" asdf/system-registry:*central-registry*)") (:EVAL . #<(SIMPLE-ARRAY CHARACTER (212)) (setf asdf/source-registry:*default-source-registries* (quote (asdf/source-registry:environment-source-registry asdf/source-registry:system-source-registry asdf/source-registry:system-source-registry-... {70053E8FBF}>) (:EVAL . "(load #P\"/Users/rpg/quicklisp/setup.lisp\")") (:EVAL . "(uiop/package:symbol-call :ql :quickload \"qlot/distify\" :silent t)") (:EVAL . "(let ((*error-output* (make-broadcast-stream))) (load (merge-pathnames #P\"setup.lisp\" #P\"/Users/rpg/projects/hideho/sim2exps/.qlot/\")))") (:EVAL . #<(SIMPLE-ARRAY CHARACTER (321)) (uiop/package:symbol-call :qlot/distify :distify (qlot/source/base:defrost-source (make-instance (quote qlot/source/git:source-git) :project-name "plan-repair-rewrite" :remote-url "[email protected]:hi... {70053E960F}>) (:EVAL . "(exit)") (:QUIT)))
16: (SB-IMPL::TOPLEVEL-INIT)
17: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
18: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::START-LISP))
19: (SB-IMPL::%START-LISP)

Versions:

Qlot version: qlot-20220331-git
Where did you get it from: Quicklisp
OS: Mac OS 13.6.2
Lisp: SBCL 2.3.9

The text was updated successfully, but these errors were encountered:

rpgoldman · 2023-11-24T18:37:03Z

@fukamachi I have just updated my Quicklisp dist to qlot-20231021-git but the error persists.

qlot: Unhandled TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {70053103D3}>: The value "plan-repair-rewrite/shop3" is not of type SYMBOL when binding SYMBOL

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {70053103D3}>
0: (MACRO-FUNCTION "plan-repair-rewrite/shop3" #<unavailable argument>) [external]
1: (QLOT/UTILS/ASDF::READ-ASD-FORM ("plan-repair-rewrite/shop3" "fiveam"))
2: ((LAMBDA NIL :IN QLOT/UTILS/ASDF::READ-ASD-FILE))
3: ((FLET "THUNK" :IN CALL-WITH-SAFE-IO-SYNTAX))
4: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<FUNCTION (FLET "THUNK" :IN CALL-WITH-SAFE-IO-SYNTAX) {104FE0D5B}>)
5: (CALL-WITH-SAFE-IO-SYNTAX #<FUNCTION (LAMBDA NIL :IN QLOT/UTILS/ASDF::READ-ASD-FILE) {700930E92B}> :PACKAGE :CL)
6: ((LAMBDA (QLOT/UTILS/ASDF::IN) :IN QLOT/UTILS/ASDF::READ-ASD-FILE) #<SB-SYS:FD-STREAM for "file /private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-BIILT0I1/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/plan-repair-by-rewrite/plan-repair-rewrite.asd" {700930E793}>)
7: (CALL-WITH-INPUT-FILE #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-BIILT0I1/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/plan-repair-by-rewrite/plan-repair-rewrite.asd" #<FUNCTION (LAMBDA (QLOT/UTILS/ASDF::IN) :IN QLOT/UTILS/ASDF::READ-ASD-FILE) {700930DC5B}> :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL :IF-DOES-NOT-EXIST :ERROR)
8: (QLOT/UTILS/DISTIFY:SYSTEMS.TXT "plan-repair-rewrite" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-BIILT0I1/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/")
9: ((LAMBDA (QLOT/DISTIFY/GIT::OUT) :IN QLOT/DISTIFY/GIT::WRITE-METADATA-FILES) #<SB-SYS:FD-STREAM for "file /private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/systems.txt" {70092C9723}>)
10: (CALL-WITH-OUTPUT-FILE #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/systems.txt" #<FUNCTION (LAMBDA (QLOT/DISTIFY/GIT::OUT) :IN QLOT/DISTIFY/GIT::WRITE-METADATA-FILES) {70092C8F1B}> :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL :IF-EXISTS :SUPERSEDE :IF-DOES-NOT-EXIST :CREATE)
11: (QLOT/DISTIFY/GIT::WRITE-METADATA-FILES #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7008ACAD03}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-BIILT0I1/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/archive.tar.gz")
12: (QLOT/DISTIFY/GIT:DISTIFY-GIT #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7008ACAD03}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/" :DISTINFO-ONLY NIL)
13: (QLOT/DISTIFY:DISTIFY #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7008ACAD03}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/" :DISTINFO-ONLY NIL)
14: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SYMBOL-CALL :QLOT/DISTIFY :DISTIFY (QLOT/SOURCE/BASE:DEFROST-SOURCE (MAKE-INSTANCE (QUOTE QLOT/SOURCE/GIT:SOURCE-GIT) :PROJECT-NAME "plan-repair-rewrite" :REMOTE-URL "[email protected]:hi-de-ho/plan-repair.git")) #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/" :DISTINFO-ONLY NIL) #<NULL-LEXENV>)
15: (EVAL (SYMBOL-CALL :QLOT/DISTIFY :DISTIFY (QLOT/SOURCE/BASE:DEFROST-SOURCE (MAKE-INSTANCE (QUOTE QLOT/SOURCE/GIT:SOURCE-GIT) :PROJECT-NAME "plan-repair-rewrite" :REMOTE-URL "[email protected]:hi-de-ho/plan-repair.git")) #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-NY9GM394/" :DISTINFO-ONLY NIL))
16: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(require 'asdf)") (:EVAL . "(setf *debugger-hook* (lambda (c parent) (declare (ignore parent)) (format *error-output* \"
Error:
Unhandled SB-FORMAT:FORMAT-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                              {7005320403}>:
  error in FORMAT: No more arguments

fukamachi · 2023-11-24T18:59:44Z

Thank you for reporting.
Does plan-repair-rewrite.asd contain ("plan-replair-rewirte/shop3" "fiveam") as a toplevel form?

rpgoldman · 2023-11-24T19:03:04Z

No, it does not. That is why I thought perhaps there was some recursive processing caused by failure to match defsystem. Tracing...

rpgoldman · 2023-11-24T19:07:08Z

Here is the file it is reading when it gets the error (not the same as the first):

;;;; plan-repair-rewrite.asd
;;
;;;; Copyright (c) 2023 Robert P. Goldman (SIFT, LLC)

(defpackage :plan-repair-rewrite-asd
  (:use :common-lisp :asdf))

(in-package :plan-repair-rewrite-asd)


(defsystem #:plan-repair-rewrite
    :description "Implementatino of Hoeller et al.'s algorithm for plan repair by
rewriting."
  :author "Robert P. Goldman (SIFT, LLC)"
  :license  "BSD 3-clause"
  :version (:read-file-form "version.lisp-expr")
  :serial t
  :depends-on ("iterate"
               "alexandria"
               (:version "hddl-utils" "3")
               "shop3/unifier")
  :components ((:file "package")
               (:file "plan-repair-rewrite")))

(defsystem #:plan-repair-rewrite/shop3
  :description "Version of the plan-repair-rewrite algorithm adapted for SHOP3."
  :author "Robert P. Goldman (SIFT, LLC)"
  :license  "BSD 3-clause"
  :version (:read-file-form "version.lisp-expr")
  :serial t
  :depends-on ("plan-repair-rewrite"
               "shop3")
  :in-order-to ((test-op (test-op "plan-repair-rewrite/shop3-test")))
  :pathname "shop3/"
  :components ((:file "decls")
               (:file "plan-repair-rewrite")))

(defsystem #:plan-repair-rewrite/shop3-test
  :author "Robert P. Goldman (SIFT, LLC)"
  :license  "BSD 3-clause"
  :version (:read-file-form "version.lisp-expr")
  :serial t
  :depends-on ("plan-repair-rewrite/shop3" "fiveam")
  :perform (test-op (op sys)
                    (let* ((test-results
                            (uiop:symbol-call :fiveam '#:run
                                              (uiop:intern* '#:shop-plan-repair-rewrite-tests
                                                            '#:test-shop-plan-repair-rewrite)))
                          (result (every #'(lambda (x) (typep x (uiop:intern* '#:test-passed '#:fiveam)))
                                         test-results)))
                      (if result
                          (format t "~&TEST-OP for plan-repair-rewrite/shop3 passed.~%")
                          (error "TEST-OP for plan-repair-rewrite/shop3 failed.~%Failing tests:~%~{~T~A~%~}"
                                 (mapcar #'(lambda (failure)
                                             (uiop:symbol-call '#:fiveam '#:name
                                                               (uiop:symbol-call
                                                                '#:fiveam '#:test-case
                                                                failure)))
                                         (remove-if #'(lambda (x) (typep x (uiop:intern* '#:test-passed '#:fiveam)))
                                             test-results))))))
  :pathname "shop3/"
  :components ((:file "tests")))

Error for this case (qlot was loading a different ASD file when I first got this error):

qlot: Unhandled TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {70053103A3}>: The value "plan-repair-rewrite/shop3" is not of type SYMBOL

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {70053103A3}>
0: (QLOT/UTILS/ASDF::READ-ASD-FORM ("plan-repair-rewrite/shop3" "fiveam"))
1: ((LAMBDA NIL :IN QLOT/UTILS/ASDF::READ-ASD-FILE))
2: ((FLET "THUNK" :IN CALL-WITH-SAFE-IO-SYNTAX))
3: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<FUNCTION (FLET "THUNK" :IN CALL-WITH-SAFE-IO-SYNTAX) {100E00D5B}>)
4: (CALL-WITH-SAFE-IO-SYNTAX #<FUNCTION (LAMBDA NIL :IN QLOT/UTILS/ASDF::READ-ASD-FILE) {70092EE92B}> :PACKAGE :CL)
5: ((LAMBDA (QLOT/UTILS/ASDF::IN) :IN QLOT/UTILS/ASDF::READ-ASD-FILE) #<SB-SYS:FD-STREAM for "file /private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-1X9PFPUS/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/plan-repair-by-rewrite/plan-repair-rewrite.asd" {70092EE793}>)
6: (CALL-WITH-INPUT-FILE #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-1X9PFPUS/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/plan-repair-by-rewrite/plan-repair-rewrite.asd" #<FUNCTION (LAMBDA (QLOT/UTILS/ASDF::IN) :IN QLOT/UTILS/ASDF::READ-ASD-FILE) {70092EDC5B}> :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL :IF-DOES-NOT-EXIST :ERROR)
7: (QLOT/UTILS/DISTIFY:SYSTEMS.TXT "plan-repair-rewrite" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-1X9PFPUS/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/")
8: ((LAMBDA (QLOT/DISTIFY/GIT::OUT) :IN QLOT/DISTIFY/GIT::WRITE-METADATA-FILES) #<SB-SYS:FD-STREAM for "file /private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/systems.txt" {70092B9723}>)
9: (CALL-WITH-OUTPUT-FILE #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/systems.txt" #<FUNCTION (LAMBDA (QLOT/DISTIFY/GIT::OUT) :IN QLOT/DISTIFY/GIT::WRITE-METADATA-FILES) {70092B8F1B}> :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL :IF-EXISTS :SUPERSEDE :IF-DOES-NOT-EXIST :CREATE)
10: (QLOT/DISTIFY/GIT::WRITE-METADATA-FILES #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7008AF2C63}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-1X9PFPUS/plan-repair-rewrite-ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/" #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/plan-repair-rewrite/git-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759/archive.tar.gz")
11: (QLOT/DISTIFY/GIT:DISTIFY-GIT #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7008AF2C63}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/" :DISTINFO-ONLY NIL)
12: (QLOT/DISTIFY:DISTIFY #<QLOT/SOURCE/GIT:SOURCE-GIT plan-repair-rewrite [email protected]:hi-de-ho/plan-repair.git ref-4f3f64c0842bd87cb03e9dc9417f21dbfe3cb759 {7008AF2C63}> #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/" :DISTINFO-ONLY NIL)
13: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SYMBOL-CALL :QLOT/DISTIFY :DISTIFY (QLOT/SOURCE/BASE:DEFROST-SOURCE (MAKE-INSTANCE (QUOTE QLOT/SOURCE/GIT:SOURCE-GIT) :PROJECT-NAME "plan-repair-rewrite" :REMOTE-URL "[email protected]:hi-de-ho/plan-repair.git")) #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/" :DISTINFO-ONLY NIL) #<NULL-LEXENV>)
14: (EVAL (SYMBOL-CALL :QLOT/DISTIFY :DISTIFY (QLOT/SOURCE/BASE:DEFROST-SOURCE (MAKE-INSTANCE (QUOTE QLOT/SOURCE/GIT:SOURCE-GIT) :PROJECT-NAME "plan-repair-rewrite" :REMOTE-URL "[email protected]:hi-de-ho/plan-repair.git")) #P"/private/var/folders/5s/cx04gv9j1mq4gvcv2lw70s400000gn/T/qlot-6H9V711X/" :DISTINFO-ONLY NIL))

rpgoldman · 2023-11-24T19:10:54Z

Could you post a line of code I could use to run read-asd-file? I am trying to find one, but it seems that many prerequisites must be established before I can call the function (*load-asd-file* must be bound, etc.). If I could invoke this function stand-alone, then I could more easily trace sub-functions, etc. to find the precise point of failure.

Thank you!

fukamachi · 2023-11-24T19:35:05Z

Sure.

(defvar *file* #P"~/Programs/lib/plan-repair-rewrite/plan-repair-rewrite.asd")

(let ((qlot/utils/asdf::*registry* (make-hash-table :test 'equal))
      (qlot/utils/asdf::*system-class-name* (make-hash-table :test 'equal))
      (qlot/utils/asdf::*load-asd-file* *file*))
  (qlot/utils/asdf::read-asd-file *file*))

rpgoldman · 2023-11-24T20:11:10Z

Oddly, that expression runs successfully. I will test it on all the asd files.

fukamachi · 2023-11-24T20:14:50Z

You may want to test with with-directory, which is a macro to traverse all system files and those dependencies in a directory.

(defparameter *dir* #P"...")

(qlot/utils/asdf:with-directory (system-file system-name dependencies) *dir*
  (print (list system-file system-name dependencies)))

fukamachi · 2023-11-24T23:40:00Z

Also, --debug option shows extra information like qlot install --debug.

As qlot/distify requires some dependencies, Qlot invokes it in a different process to prevent loading them in the Qlot process. Your distify process can be loading different version of Qlot, maybe? --debug option will show the exact sbcl command to invoke the distify process.

fukamachi · 2023-11-25T01:53:08Z

Although it is superfluous to say for solving the issue, I'm leaving a comment to your advice in a description.

It may be better to use asdf:parse-defsystem or asdf:load-asd to create the appropriate ASDF data structures and then use slot accessors, instead of writing a defsystem code-walker.

Yeah, I wanted to use asdf:load-asd, but it's difficult in the case of Qlot.

ASD files can have any Common Lisp forms like (ql:quickload ...) or :defsystem-depends-on to load dependencies before reading system definitions.

Qlot is a tool for using a specific version of libraries, so the external libraries must be loaded from the project-local Quicklisp, while qlot/distify requires its dependencies loaded from the system-wide one.

Therefore, it doesn't use asdf:load-asd and parses the contents of ASD files by itself.

I know you're a specialist in ASDF, so you may have a better solution for it. I'd appreciate it if you could give me some advice. Thanks!

fukamachi · 2023-11-25T13:58:11Z

The error probably won't be raised at HEAD of Qlot since I deleted the macro-function clause at 2b93648.
However, Qlot may not be able to list the dependencies of your system downloaded from a git repo, which leads to the possibility that the qlot bundle won't work as expected.

rpgoldman · 2024-02-19T21:18:46Z

Although it is superfluous to say for solving the issue, I'm leaving a comment to your advice in a description.

It may be better to use asdf:parse-defsystem or asdf:load-asd to create the appropriate ASDF data structures and then use slot accessors, instead of writing a defsystem code-walker.

Yeah, I wanted to use asdf:load-asd, but it's difficult in the case of Qlot.

ASD files can have any Common Lisp forms like (ql:quickload ...) or :defsystem-depends-on to load dependencies before reading system definitions.

Even if they don't, they can have :defsystem-depends-on and, at least in theory, that could mean the containing defsystem can't be correctly evaluated without loading those dependencies.

Therefore, it doesn't use asdf:load-asd and parses the contents of ASD files by itself.

I know you're a specialist in ASDF, so you may have a better solution for it. I'd appreciate it if you could give me some advice. Thanks!

The only answer I can think of is painful: start up a lisp subprocess, load the ASD file in the subprocess, and have the subprocess tell the main process what it needs to know about the system definition.

I wish I had something better.

fukamachi mentioned this issue Nov 24, 2023

Fix read-asd-form to ignore ill-formed forms in ASD files. #199

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

qlot distify error from dependency with new-style name #198

qlot distify error from dependency with new-style name #198

rpgoldman commented Nov 24, 2023

rpgoldman commented Nov 24, 2023

fukamachi commented Nov 24, 2023

rpgoldman commented Nov 24, 2023

rpgoldman commented Nov 24, 2023 •

edited

Loading

rpgoldman commented Nov 24, 2023

fukamachi commented Nov 24, 2023

rpgoldman commented Nov 24, 2023

fukamachi commented Nov 24, 2023

fukamachi commented Nov 24, 2023 •

edited

Loading

fukamachi commented Nov 25, 2023

fukamachi commented Nov 25, 2023

rpgoldman commented Feb 19, 2024

qlot distify error from dependency with new-style name #198

qlot distify error from dependency with new-style name #198

Comments

rpgoldman commented Nov 24, 2023

Describe the bug

Reproducible steps

Error messages

Versions:

rpgoldman commented Nov 24, 2023

fukamachi commented Nov 24, 2023

rpgoldman commented Nov 24, 2023

rpgoldman commented Nov 24, 2023 • edited Loading

rpgoldman commented Nov 24, 2023

fukamachi commented Nov 24, 2023

rpgoldman commented Nov 24, 2023

fukamachi commented Nov 24, 2023

fukamachi commented Nov 24, 2023 • edited Loading

fukamachi commented Nov 25, 2023

fukamachi commented Nov 25, 2023

rpgoldman commented Feb 19, 2024

rpgoldman commented Nov 24, 2023 •

edited

Loading

fukamachi commented Nov 24, 2023 •

edited

Loading