eta-template for QuickCopy struggles with iterating over `relations` content #2779

acsr · 2024-02-04T19:07:59Z

Debug log ID

EKF7WRXW-euc/6.7.150-6

What happened?

I am using an extended eta-template to export Zotero entries as markdown to paste an item into my Logseq.com PKM.

During the processing of related items I am not able to serialize the content of a list of related urls.

I host my current development version eta-template publicly as gist here: https://gist.github.com/acsr/4be9b01cc2c989519c497ee87510f9a4

I am testing the template with an Zotero item with 3 relations urls.
When I export the item via BetterBibTeX as JSON, the relations are included.
(by the way: rights field is missing in the JSON, but that needs to rise a different issue)

Any ideas by users how to fix this, or a better place to ask the community.

My eta-template works in general and does not break.
The included relations code first dumps the whole object and then tries to iterate over the content, but skips it without errors. I also added the toString() method when dumping the whole object (and for the list as well)
I tested various combinations but ran out of ideas.

Template Code:

<% it.items.forEach(function(item) { 
const [ , kind, lib, key ] = item.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/)
const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`
%>- ### <%= item.title %>
<% if (item.creators.length > 0){%>
<%= "\ncreators::" %> <% for (var i = 0; i < item.creators.length; i++) { %>
<%= item.creators[i].creatorType %><%= ": " %>
<% if (item.creators[i].name){%>
<%= item.creators[i].name %>
<% } else { %>
<%= item.creators[i].firstName %> <%= item.creators[i].lastName %>
<% } %>
<% if (i < item.creators.length -1){%>, <%}%>
<%}%><%}%>
<% if (item.url){%><%= "\nurl::" %> <%= item.url %><%}%>
<% if (item.tags.length > 0){%>
<%= "\ntags::" %> <% for (var i = 0; i < item.tags.length; i++) { %>
<%= "#[[" %><%= item.tags[i].tag %><%= "]]" %>
<% if (i < item.tags.length -1){%>, <%}%>
<%}%><%}%>
<%= "\nzotero::" %> [@<%= item.citationKey %>](<%= select %>)
<%= "\n  - #### abstractNote"%>
<%= "\n    "%><%= item.abstractNote %><%= "\r" %>
<% if (item.notes.length > 0){%>
<%= "\n  - #### Notes"%>
<% for (var i = 0; i < item.notes.length; i++) { %>
<%= "\n    - ##### Note "%><%= "[" %><%= item.notes[i].key %><%= "](" %> <%= item.notes[i].uri %><%= ' "URI Link")' %>
<%= "\n      "%><%= item.notes[i].note %>
<%}%><%}%>
<% if (item.relations) {%>
<%= "\n  - #### Relations"%>
<%= "\n    - "%><%= item.relations.toString() %>
<% for (var i = 0; i < item.relations.length; i++) { %>
<%= "\n    - "%><%= item.relations[i].toString() %>
<%}%><%}%>
<% if (item.rights) {%>
<%= "\n  - #### Rights"%>
<%= "\n    - "%><%= item.rights.toString() %>
<%}%>
<% }) %>

retorquere · 2024-02-04T20:30:19Z

I need a debug log with the sample item in it for which this doesn't work -- right-click the item and send a debug log from the popup menu. The ID will have -refs- in it.

acsr · 2024-02-05T10:29:06Z

Debug Log ID with item

Here the requested Debug Log ID: YCGUKYR4-refs-euc/6.7.150-6

Offtopic

getting rights field by eta-template works by the way!
eta-template.org docs are very limited, they expect Javascript cracks. But my Javascript knowledge/experience is low. Your examples in the Zotero forum were very helpful.
to share my template with others I started a Zotero forum entry on this:
https://forums.zotero.org/discussion/111536/eta-template-example-to-paste-as-block-in-logseq-as-markdown-wip#latest

retorquere · 2024-02-05T10:39:26Z

How many items (approximately) were in YCGUKYR4-refs-euc/6.7.150-6? The log shows it timed out before it could send the items.

retorquere · 2024-02-05T10:41:53Z

I can't replicate rights not exporting. In case you were sending your whole library in YCGUKYR4-refs-euc/6.7.150-6 -- a log for a single item for which you can't export rights will do.

retorquere · 2024-02-05T10:44:00Z

eta-template.org docs are very limited, they expect Javascript cracks.

It's the most lightweight templating lib I could find, but any format that I would have chosen was going to be technical. It's always OK to ask. But eta is "I want to do this without @retorquere being involved" and that's dev territory.

retorquere · 2024-02-05T10:51:22Z

During the processing of related items I am not able to serialize the content of a list of related urls.
...
<%= "\n - "%><%= item.relations.toString() %>
<% for (var i = 0; i < item.relations.length; i++) { %>
<%= "\n - "%><%= item.relations[i].toString() %>

I'm a little unclear on what it is you are asking and what is going on in this code. Can you just add or describe the output you want from the item in the (new) debug log rather than the code to produce it?

acsr · 2024-02-05T12:21:51Z

How many items (approximately) were in YCGUKYR4-refs-euc/6.7.150-6? The log shows it timed out before it could send the items.

I selected just one item and issued the log from the context menu of the item. during my first attempt there was a serious delay. The I started a second one but cancelled the sending (I remember it looks alike but was stuck at the end of the id, missing the UUID is at the start!)

acsr · 2024-02-05T12:24:24Z

I can't replicate rights not exporting. In case you were sending your whole library in YCGUKYR4-refs-euc/6.7.150-6 -- a log for a single item for which you can't export rights will do.

Rights is exporting to it.items to be accessed by eta!
But I could not find it in the JSON export. This is not critical (offtopic) but I will file another debug log from a vanilla group I can share with you and i will move a copy over there.

acsr · 2024-02-05T12:26:09Z

eta-template.org docs are very limited, they expect Javascript cracks.

It's the most lightweight templating lib I could find, but any format that I would have chosen was going to be technical. It's always OK to ask. But eta is "I want to do this without @retorquere being involved" and that's dev territory.

It is like/dislike your Jekyll/Hyde avatar. I love/hate this story (both sides)

acsr · 2024-02-05T12:38:46Z

During the processing of related items I am not able to serialize the content of a list of related urls.
...
<%= "\n - "%><%= item.relations.toString() %>
<% for (var i = 0; i < item.relations.length; i++) { %>
<%= "\n - "%><%= item.relations[i].toString() %>

I'm a little unclear on what it is you are asking and what is going on in this code. Can you just add or describe the output you want from the item in the (new) debug log rather than the code to produce it?

OK

In the first line I catch the whole relations object and try to serialize it as a whole list expected from the JSON Export dirctly to a string. (This was addeed later for debugging, my intend is the latter part)
The loop should iterate through the elements of the listed relations urls and display them as seperate markdown list items (and to put mor eye candy on the markup later, e.g. fix the urls to https and /or transform into local select item links

I add here the raw markdown created for me and expected relations as a markdown list (the code in the first line of your quote can be omitted if the listing works.

TODO (for me): Create a vanilla public Zotero group, invite @retorquere, copy over the cited item below, share the link here.

Markdown result from the QuickCopy using my latest version of the eta-template (I added rights export and that works! (Content can be shared)

additional: you see the notes exported as html entities (this is a different challenge (on my Jekyll side!)

- ### #Fundstück: Schreibblockaden mit „The Most Dangerous Writing App“ überwinden
creators:: author: Florian Hagen
url:: https://www.tub.tuhh.de/tubtorials/2023/12/29/fundstuck-the-most-dangerous-writing-app/
tags:: #[[Schreiben]], #[[Writing]], #[[Online Tools]], #[[Tools]], #[[Writing Software]], #[[Writing Blog Posts]], #[[Text Creation]], #[[Schreibblockade]], #[[Manu Ebert]], #[[The Most Dangerous Writing App]]
zotero:: [@hagenFundstueckSchreibblockadenMit2023](zotero://select/groups/3305/items/B6IC7QKD)

  - #### abstractNote
    „The Most Dangerous Writing App“ ist ein Open-Source-Tool, das mit Hilfe von Free Writing helfen kann, Schreibblockaden zu überwinden.
  - #### Notes
    - ##### Note [I5D5Z24J]( http://zotero.org/groups/3305/items/I5D5Z24J &quot;URI Link&quot;)
      &lt;div data-schema-version=&quot;8&quot;&gt;&lt;pre&gt;In der Kategorie ‚Fundstück‘ werden Tools, Services und andere Entdeckungen rund um den &lt;a href=&quot;https://www.tub.tuhh.de/tubtorials/2019/08/10/wissenschaftliche-kommunikation-ein-individueller-kreislauf/&quot; rel=&quot;noopener noreferrer nofollow&quot;&gt;life cycle wissenschaftlicher Kommunikation&lt;/a&gt; in kurzen Texten vorgestellt.&lt;/pre&gt;
&lt;/div&gt;
    - ##### Note [GJ9E72RG]( http://zotero.org/groups/3305/items/GJ9E72RG &quot;URI Link&quot;)
      &lt;div data-schema-version=&quot;8&quot;&gt;&lt;h2&gt;&lt;strong&gt;&lt;span style=&quot;color: rgb(34, 34, 34)&quot;&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;„The Most Dangerous Writing App“ – Was ist das?&lt;/span&gt;&lt;/span&gt;&lt;/strong&gt;&lt;/h2&gt;
&lt;p&gt;&lt;span style=&quot;color: rgb(17, 17, 17)&quot;&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;Die&amp;nbsp;&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;color: rgb(34, 119, 187)&quot;&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;&lt;a href=&quot;https://maebert.github.io/themostdangerouswritingapp&quot; rel=&quot;noopener noreferrer nofollow&quot;&gt;originale Open-Source-Variante&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;&amp;nbsp;von „The Most Dangerous Writing App“ wurde von Manu Ebert veröffentlicht. Zu dieser gibt es auch ein&amp;nbsp;&lt;/span&gt;&lt;span style=&quot;color: rgb(34, 119, 187)&quot;&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;&lt;a href=&quot;https://github.com/maebert/themostdangerouswritingapp&quot; rel=&quot;noopener noreferrer nofollow&quot;&gt;GitHub-Repositorium&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;. Zusätzlich gibt es eine leicht angepasste Variante, die über&amp;nbsp;&lt;/span&gt;&lt;span style=&quot;color: rgb(34, 119, 187)&quot;&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;&lt;a href=&quot;https://www.squibler.io/dangerous-writing-prompt-app&quot; rel=&quot;noopener noreferrer nofollow&quot;&gt;Squibler&lt;/a&gt;&lt;/span&gt;&lt;/span&gt;&lt;span style=&quot;background-color: rgb(255, 255, 255)&quot;&gt;, einer KI-Plattform zur Unterstützung von Schreibenden, angeboten wird. Unabhängig von der gewählten Version setzen beide auf den Free-Writing-Ansatz, um Schreibblockaden zu überwinden.&lt;/span&gt;&lt;/p&gt;
&lt;/div&gt;
  - #### Relations
    - [object Object]
  - #### Rights
    - CC BY 4.0

rendered similar to

#Fundstück: Schreibblockaden mit „The Most Dangerous Writing App“ überwinden

creators:: author: Florian Hagen
url:: https://www.tub.tuhh.de/tubtorials/2023/12/29/fundstuck-the-most-dangerous-writing-app/
tags:: #[[Schreiben]], #[[Writing]], #[[Online Tools]], #[[Tools]], #[[Writing Software]], #[[Writing Blog Posts]], #[[Text Creation]], #[[Schreibblockade]], #[[Manu Ebert]], #[[The Most Dangerous Writing App]]
zotero:: @hagenFundstueckSchreibblockadenMit2023

abstractNote
„The Most Dangerous Writing App“ ist ein Open-Source-Tool, das mit Hilfe von Free Writing helfen kann, Schreibblockaden zu überwinden.
Notes
- Note [I5D5Z24J]( http://zotero.org/groups/3305/items/I5D5Z24J "URI Link")
 <div data-schema-version="8"><pre>In der Kategorie ‚Fundstück‘ werden Tools, Services und andere Entdeckungen rund um den <a href="https://www.tub.tuhh.de/tubtorials/2019/08/10/wissenschaftliche-kommunikation-ein-individueller-kreislauf/" rel="noopener noreferrer nofollow">life cycle wissenschaftlicher Kommunikation</a> in kurzen Texten vorgestellt.</pre>
 </div>
- Note [GJ9E72RG]( http://zotero.org/groups/3305/items/GJ9E72RG "URI Link")
 <div data-schema-version="8"><h2>„The Most Dangerous Writing App“ – Was ist das?</h2>
 Die <a href="https://maebert.github.io/themostdangerouswritingapp" rel="noopener noreferrer nofollow">originale Open-Source-Variante</a> von „The Most Dangerous Writing App“ wurde von Manu Ebert veröffentlicht. Zu dieser gibt es auch ein <a href="https://github.com/maebert/themostdangerouswritingapp" rel="noopener noreferrer nofollow">GitHub-Repositorium</a>. Zusätzlich gibt es eine leicht angepasste Variante, die über <a href="https://www.squibler.io/dangerous-writing-prompt-app" rel="noopener noreferrer nofollow">Squibler</a>, einer KI-Plattform zur Unterstützung von Schreibenden, angeboten wird. Unabhängig von der gewählten Version setzen beide auf den Free-Writing-Ansatz, um Schreibblockaden zu überwinden.
 </div>
Relations
- [object Object]
Rights
- CC BY 4.0

retorquere · 2024-02-05T12:46:06Z

1....
2....

Can you give me a sample of what you actually want as output?

acsr · 2024-02-05T12:46:35Z

Done: I invited emilianoeheyns to my public closed group: BetterBibTeX to Logseq Dev and may make him admin as soon the invitation is accepted.
The group contains the cited item above as a copy.

zotero://select/groups/5387086/collections/TSYLQ2FT
zotero://select/groups/5387086/items/IJ6LI577
https://www.zotero.org/groups/5387086/items/IJ6LI577

retorquere · 2024-02-05T12:47:15Z

Or is the yaml you posted the actual output you want?

acsr · 2024-02-05T12:49:31Z

1....
2....

Can you give me a sample of what you actually want as output?

it is not YAML but the Markdown indentation necessary for my Logseq insert via clipboard.
I provided the actual output not the expected.

just: four space characters, one dash, another space, the url contained in the relation list
repeated for every list item.

for Relations the raw Markdown copied to the clipboard should be for now exactly matching the JSON content payload like:

…
  - #### Relations
    - http://...
    - http://...
    - http://...
…

And of course in the end with proper https

acsr · 2024-02-05T12:58:54Z

@retorquere I invoked a new debug log from the whole test group library BetterBibTeX to Logseq Dev I invited you to.
Debug Log ID: BA72P8EU-refs-euc/6.7.150-6

(right click on the group, Better BibTeX -> Send Better BibTeX debug log …)

retorquere · 2024-02-05T13:02:57Z

You can right-click a single item and send a log for that.

retorquere · 2024-02-05T13:15:20Z

Can you invite emilianoheyns instead?

retorquere · 2024-02-05T13:20:35Z

When I re-export BA72P8EU-refs-euc/6.7.150-6 I get "rights": "CC BY 4.0",. Are you sure perhaps exporting as Better CSL JSON instead of BetterBibTeX JSON?

acsr · 2024-02-05T13:21:50Z

I created a single Item log as well: VE2Q5ULE-refs-euc/6.7.150-6
Sorry for the typo, I corrected the invitation.

retorquere · 2024-02-05T13:22:29Z

It's not a typo, emilianoheyns is my test account, emilianoeheyns is my true account.

acsr · 2024-02-05T13:26:08Z

I am sure exporting as BetterBibTeX JSON!
Reexported this and here is the file (ID obfuscated)
Exportierte Einträge-3.json

I could not find any entries where export fields can be included excluded except the exclude option in Better BibTeX prefs, but it is empty.

retorquere · 2024-02-05T14:08:52Z

(ID obfuscated)

Please don't do this. That ID is the ID of the translator; it is the same for everyone and there are exactly zero privacy aspects to this. It makes my life harder if people corrupt files to debug stuff with. The file won't import with the translatorID altered.

I also don't understand what you mean that "rights is missing". It is right there on line 113.

wrt the eta template, this works for me:

<%- for (const item of it.items) {
  const [ , kind, lib, key ] = item.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/)
  const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`
-%>
- ### <%= item.title %>
  <%- if (item.creators.length) { -%>

creators:: <% item.creators.forEach((creator, i) => { %><%= creator.creatorType %>: <%= creator.name || `${creator.firstName || ''} ${creator.lastName || ''}`.trim() %><% if (i < item.creators.length - 1) { %>, <% } %><% }) %>
  <%- } -%>
  <%- if (item.url) { -%>

url:: <%= item.url %>
  <%- } -%>
  <%- if (item.tags.length) { -%>

tags:: <% item.tags.forEach((tag, i) => { %>#[[<%= tag.tag %>]]<% if (i < item.tags.length - 1){%>, <%}%><% }) %>
  <%- } -%>

zotero::[@<%= item.citationKey %>](<%= select %>)
  - #### abstractNote
    <%= item.abstractNote %><%= "\r" %>
    <%- if (item.notes.length) { -%>
  - #### Notes
      <%- for (const note of item.notes) { -%>
    - ##### Note [<%= note.key %>](<%= note.uri %> "URI Link")
      <%~ note.note %>
      <%- } -%>
    <%- } -%>
  <%- if (item.relations) { -%>

  - #### Relations
    <%- for (const rel of item.relations) { -%>
  - <%= rel %>
    <%- } -%>
  <%- } -%>
  <%- if (item.rights) { -%>

  - #### Rights
    - <%= item.rights %>
  <%- } -%>
<%- } -%>

retorquere · 2024-02-05T14:19:16Z

I could not find any entries where export fields can be included excluded except the exclude option in Better BibTeX prefs, but it is empty.

That is for the Better CSL/YAML/BibTeX/BibLaTeX exports only.

acsr · 2024-02-05T14:49:35Z

Please don't do this. That ID is the ID of the translator; it is the same for everyone and there are exactly zero privacy aspects to this. It makes my life harder if people corrupt files to debug stuff with. The file won't import with the translatorID altered.

The full translator id is:
"id": "36a3b0b5-bad0-4a04-b79b-441c7cef77db"

I also don't understand what you mean that "rights is missing". It is right there on line 113.

I completely apologize. It could be banner blindness and no idea why I did not find it before. I remember that I double checked it. Otherwise I would not have mentioned it here. I just wanted to share the expected additional issue, but is was fata morgana.

this works for me:

Looks much cleaner, template code layout like this was beyond my former skills.

need to test this
understand the approach
add more comments
share it in my post in the forum
thank you at another time again

One big thank you here for now!

Are you interested in providing the final example for your published docs?
Adding at https://retorque.re/zotero-better-bibtex/ may end up as a support pitfall, similar to me ;-) (like you mentioned above)

Or is there a suggested place for a stackoverflow like place to share that (the Zotero forum is not suitable for code markup).

I am looking forward for sharing my workflow in the Logseq ecosphere and then can be linked by you from there without any support expectations for you included.

acsr · 2024-02-05T14:58:36Z

I tested the eta-template code example provided by you above (whole code copied using github button! only snippet cited here for reference):

snippet for relations:

…
  - #### Relations
    <%- for (const rel of item.relations) { -%>
  - <%= rel %>
    <%- } -%>
  <%- } -%>
  <%- if (item.rights) { -%>
…

I got

TypeError: item.relations is not iterable

retorquere · 2024-02-05T15:05:35Z

The full translator id is: "id": "36a3b0b5-bad0-4a04-b79b-441c7cef77db"

Ja I know as it is the same for everyone 😄 but I import these test files (that's what the format is for) and I'd rather not have to make manual fixes if there's no benefit.

One big thank you here for now!

You're welcome

Are you interested in providing the final example for your published docs? Adding at https://retorque.re/zotero-better-bibtex/ may end up as a support pitfall, similar to me ;-) (like you mentioned above)

The pencil at the top right of the site pages let you make suggestions for changes to the site.

I am looking forward for sharing my workflow in the Logseq ecosphere and then can be linked by you from there without any support expectations for you included.

I think the site is fine for that

acsr · 2024-02-05T15:19:36Z

Looks as expected inserted in Logseq with the new code, when I remove the relations.
The notes are now proper plain html and are displayed in Logseq as needed.

Example as screenshot.

The first bullet in test is the insert with the relations code error.
The second one the successful insert with relations code section removed

ToDo:

There are some remaining small glitches visible
- the leading # for this particular headline (it actually starts with the # in the title)
- the linefeed after the last property zotero:: needs to be different.
  I had this in my initial versions as well, and know how to fix it.

20240205_160827-Logseq-insert-Zotero-BetterBibTeX-Quickcopy-via-eta-template-Screenshot

My next steps are enhancing it for attachments and pdf annotation links.
The current workflow in Zotero is a nightmare, but I had no time to look into Zotero 7.
Maybe there is already a better flow.

Here is my latest comment on this topic in a thread targeting PDF links.
https://forums.zotero.org/discussion/comment/454442/#Comment_454442

retorquere · 2024-02-05T15:34:24Z

<%- for (const item of it.items) {
  const [ , kind, lib, key ] = item.uri.match(/^https?:\/\/zotero\.org\/(users|groups)\/((?:local\/)?[^/]+)\/items\/(.+)/)
  const select = (kind === 'users') ? `zotero://select/library/items/${key}` : `zotero://select/groups/${lib}/items/${key}`

  const relations = []
  if (item.relations) {
    for (const [kind, rels] of Object.entries(item.relations)) {
      for (const rel of rels) {
        relations.push(rel)
      }
    }
  }
-%>
- ### <%= item.title %>
  <%- if (item.creators.length) { -%>

creators:: <% item.creators.forEach((creator, i) => { %><%= creator.creatorType %>: <%= creator.name || `${creator.firstName || ''} ${creator.lastName || ''}`.trim() %><% if (i < item.creators.length - 1) { %>, <% } %><% }) %>
  <%- } -%>
  <%- if (item.url) { -%>

url:: <%= item.url %>
  <%- } -%>
  <%- if (item.tags.length) { -%>

tags:: <% item.tags.forEach((tag, i) => { %>#[[<%= tag.tag %>]]<% if (i < item.tags.length - 1){%>, <%}%><% }) %>
  <%- } -%>

zotero::[@<%= item.citationKey %>](<%= select %>)
  - #### abstractNote
    <%= item.abstractNote %><%= "\r" %>
    <%- if (item.notes.length) { -%>
  - #### Notes
      <%- for (const note of item.notes) { -%>
    - ##### Note [<%= note.key %>](<%= note.uri %> "URI Link")
      <%~ note.note %>
      <%- } -%>
    <%- } -%>
  <%- if (relations.length) { -%>

  - #### Relations
    <%- for (const rel of relations) { -%>
  - <%= rel %>
    <%- } -%>
  <%- } -%>
  <%- if (item.rights) { -%>

  - #### Rights
    - <%= item.rights %>
  <%- } -%>
<%- } -%>

You can't see these relations yet in BBT JSON, but they're present. A new BBT is building where BBT JSON will show these relations.

acsr · 2024-02-05T15:47:02Z

The pencil at the top right of the site pages let you make suggestions for changes to the site.
I think the site is fine for that.

OK, I see this leads to forking the repo and create a pull request. I am familar with this and Hugo and hopefully find the time.

One more thing

I have a suggestion for the search in your docs.

Even when AI based searchengines are under way, there is a cool free solution to improve your search using https://docsearch.algolia.com/.

It is free for opensource project docs and does a cool job making your docs search much more helpful.

You can see it in use here, where I implemented it for a Sphinx-doc based project: https://userguide.present4d.com We used it for Plone as well, but we now in transition to an AI based search using https://nuclia.com/

acsr · 2024-02-05T15:49:39Z

You can't see these relations yet in BBT JSON, but they're present. A new BBT is building where BBT JSON will show these relations.

I am looking forward to your new release and you may please wait with closing this issue until I have tested it.

In my JSON (included above) export some Relations were there (see my export), but I failed to serialize the object in my original case using the eta-template. This is maybe of the simple list I had instead of a dictionary.

Now I have something different here:

…
          "relations": {
            "owl:sameAs": [
              "http://zotero.org/groups/3305/items/GJ9E72RG"
            ]
          },
…

in my original export I had relations like this in the JSON Export:

…
      "relations": [
        "http://zotero.org/groups/3305/items/HPSPXE2V",
        "http://zotero.org/groups/3305/items/IUXZMUYM",
        "http://zotero.org/groups/3305/items/K69CGHDJ"
      ],
…

Important: I forgot that relations are gone when copying items to another group, and need to be recreated and: The targets have to be there as well!

I will now recreate the relation in the test group as well.

New insight

Interesting is the owl:sameAs entry in the new JSON, pointing to the entry in the original private group, I copied the item from. This does not show up in the current Zotero 6 UI, but I heard it through the grapevine something is coming for Zotero 7.

retorquere · 2024-02-05T16:07:07Z

in my original export I had relations like this in the JSON Export:

This is the part that is changing in the new build.

acsr · 2024-02-05T16:27:15Z

Update:
I recreated the Relations like in my original group.

here is the snipped from the JSON export

…
      "relations": [
        "http://zotero.org/groups/5387086/items/E6Q3V4RB",
        "http://zotero.org/groups/5387086/items/IQ438J55",
        "http://zotero.org/groups/5387086/items/Y428Z85I"
      ],
…

With your latest code from
#2779 (comment)
and after I recreated the relations I get those (plus another one, see below), but also some bunch of preceeding debugging code as well (correct?)

The Relations part now looks like this in Logseq:

The last one ending with B6IC7QKD is a Relation not coming from the attachment notes I cited above, but from the original entry in https://www.zotero.org/groups/3305/acsr_studio_research/items/B6IC7QKD (but B6IC7QKD is not included in the JSON exported from the test group using your last template code:

retorquere · 2024-02-05T19:29:45Z

6.7.164 will show the relations in the BBT JSON output unchanged from the internal format.

acsr · 2024-02-06T13:20:26Z

Thank You!
Works well, except of a difference to the Zotero UI listing an otherwise invisible relation to the original entry:

Special relation to origin of items copied from other groups/libraries

if the Item was copied from another group, an additional relation is showing up in the list pointing to the original group entry. This relation is not visible in the current groups item in the Zotero 6 UI.
this special relation is not identifyable by a special prefix, like in the JSON

I really like to have this origin link, but to make the relations more useful, we need to enrich the context.

Remark on the special relation: Maybe this is used to deduplicate the copy and original in a copy on write manner and speed up the processes in the background to move a copy to the new group. Since this unofficial, we cannot rely on this.

eta-template gist updated

I updated my eta-template in the gist based on the example privded by @retorquere here to revision 6 and remode the broken wip version.

https://gist.github.com/acsr/7fced5d3f238dc6180fcdb76c9f82803
see the comments of the gist for more.

Unfortunately I could not preserve the nice indentations by @retorquere (for now) due to the fact, that I need more control over linefeed and newline when pasting in Logseq and have to set them explicitly instead of reusing the line delimiters and whitespace of the template defaulting to unicode dec10 in some cases. Otherwise indentation gets corrupted by the Logseq clipboard preprocessor.

result in Logseq looks like this now:

How to debug the preprocessed JSON

To debug the eta-template output you can add

[[[<%~ JSON.stringify(item) %>]]]

after line 13 of the eta template, below the initialization of the items to iterate over them. You then get a perfixed output similar to the screenshot in the comment above.

Status

For me we can

close this issue targeting the title "struggles with iterating over relations" and
open new issue(s) for fixing the below challenges (as long it is BetterBibTeX export related and not eta-template specific and avoid to waste @retorquere time with too individual challenges.)
- to create prefixes for the relations links
- change the urls to https
- replace the http web links with zotero://select item links (can they be part of the JSON export and then end up in the it.items object)

Off this topic:

I started a discussion issue on this in the Zutilo Plugin and the pitfalls of recursive relation (Goal was to set a selection or filter to get related items in the Zotero items list. One challenge: what if multiple items are selected and how to treat recursion depth of relations)

github-actions bot added the awaiting-user-feedback label Feb 4, 2024

retorquere added the needs-debug-log label Feb 4, 2024

github-actions bot removed needs-debug-log awaiting-user-feedback labels Feb 5, 2024