Zotero integration please!!!

User avatar
nontroppo
Posts: 755
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Mon Sep 25, 2017 1:01 pm Post

Good to know it worked well for your PDF structure!

Fr
FranJNS
Posts: 1
Joined: Wed Sep 27, 2017 2:07 pm
Platform: Windows

Wed Sep 27, 2017 4:09 pm Post

I want use zotero or mendeley as this person in this youtube video
https://www.youtube.com/watch?v=s_GlGyzuTMw&t=636s

Watch 10th minute!

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Thu Oct 05, 2017 4:13 am Post

FranJNS wrote:I want use zotero or mendeley as this person in this youtube video
https://www.youtube.com/watch?v=s_GlGyzuTMw&t=636s

Watch 10th minute!


Since I don't understand Spanish, what exactly is it that he is doing that you want to do? Also it is from 2015 and since then Zotero has gone through a major crisis as mentioned in this thread. Zotero no longer plays well with Scrivener.

Ev
EvgenyVK
Posts: 1
Joined: Tue Oct 31, 2017 2:21 pm
Platform: Mac + iOS

Tue Oct 31, 2017 2:59 pm Post

Hm, it looks like this amazing discussion stalled :| .

Come on, let's separate users's concerns: reference manager should manage user's sources and pdfs :P . It's not reference manager's job to reformat user's documents (and let's admit it - all those temporary citations are just ugly :mrgreen: ). On the other side, word-processor is the actual tool that should be using sources from reference manager and format them flawlessly. In the end, now people are forced to do this part in Word! And this is in 21st century :!: If we can have custom spell-checkers and dictionaries, why can't we have a list of references parsed from some web API? :shock:

If there was a kind of interface on Scrivener's side to accept a JSON array :roll: from external sources - like Zotero, it would be relatively easy to make a plugin that would simply request Zotero's API and provide the user's reference list in the format that would be acceptable for Scrivener.
In Javascript it would be a couple of AJAX calls and some tedious :oops: authentication procedure. And actually we aren't limited to Zotero, anything capable of providing user's references in some acceptable format through RESTful calls could work.

:idea: As per Scrivener's implementation scenario, how about autocomplete? No need for new panels or windows. When you want to add a citation press some hotkey to run a query against the list parsed from reference manager. That's how it's expected to work. No nasty "command+tab" carousels, no dragging, copying, pasting, scanning and praying that next time you export your document those citations will miraculously find their way out :roll: .

User avatar
KB
Site Admin
Posts: 19198
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Tue Oct 31, 2017 4:46 pm Post

I'm still not clear on how that would work either in practice or on the implementation side, I'm afraid. What exactly would that JSON array do? Where would it come from and how would Scrivener use it? How would Scrivener know what was in it or what it was for? One problem with Zotero in particular, as has been mentioned up-thread, is that it is currently in a transitional period and the API has not been stable. I'm not sure if that's sorted now, but there's also the problem of a single developer such as myself trying to manage integration with APIs for multiple programs. Even managing integration with the Simplenote API, for instance, became time-consuming because of support issues and API changes.

And I'm also still not clear on exactly what would happen in Scrivener, even from the user angle. I don't use Zotero. I don't use citation managers. I'd love for someone just to describe in simple terms, in a bulleted list, exactly how they see the workflow working. i.e. 1. Click button in Scrivener or choose "Citation" from menu. 2. Zotero opens? 3. ???

All the best,
Keith
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

User avatar
lunk
Posts: 1750
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Tue Oct 31, 2017 5:06 pm Post

The Spanish youtube clip shows integration with Papers 3. Integration is outside Scrivener. You wake the Papers helper with ctrl+ctrl, search for the reference you want and copy the cite key, paste in Scrivener, done!

When you are done writing, you export to Word, run the Papers shortcut, which transforms the cite keys to proper references and enable production of a reference list.

User avatar
nontroppo
Posts: 755
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Wed Nov 01, 2017 2:32 am Post

So, if I was to build this ideally, this is what I would do (S=scrivener, R=ref manager).

  1. S has a button to bring up a requestor, input a search term, this is passed to the applescript/API interface of R[!1]
  2. R returns a list of matching citations[!2]
  3. S presents this list to the user.
  4. User selects one or more refs.
  5. S now has to make a second applescript/API [!3] call for the formatted {{1}} INLINE citation and {{2}} BIBLIOGRAPHY text. These have to be stored in a database somewhere, along with the format that was used in the request.
  6. S inserts {{1}} into the document text, and puts {{2}} into a separate special doc, "References". S stores the location of where {{1}} is in the text, because it needs this for reformatting and citation editing. Mellel and Word make these special "objects", so perhaps S needs to do something similar to track their location (some kind of modified RTF comment?).
  7. There needs to be a reformat references function, in which S must now make a new API call for each reference in the database to request the reformatted text and it must replace each {{1}} and regenerate the references from {{2}}.
  8. The special doc "References" needs to be able to track each reference as a block linked back to the database. If S allows editing of the reference in the database then only that block would be updated.
  9. S also needs a way of letting the user select the format of the bibliography from a list. This requires another API call, or manual entry and a default fallback.

The major problems are that there are many different reference managers, and each has different capabilities and a different API. So [!1-3] would need different interface, and R would return different information (XML, CSV, JSON etc.). There is no standardised API across different reference managers. Keith would have to deal with multiple APIs unless he only supports one reference manager which IMO is untenable (if he supports my preferred manager Bookends, it will upset Endnote, Papers, bibdesk, Zotero users etc.)

The other major issue is the internal database. Mellel, Word and Libreoffice all use this. Mellel has a special link to Bookends so double clicking on the reference object, brings up a requestor where you can edit each reference directly. The Mellel⬄Bookends interface is probably the most advanced of any that I know, there are lots of tools to manage the references between the two: but that is a lot of very specialised code and management functionality built on the specific database interface and not generalisable. Word obviously does something similar with Endnote, built from Word's macro language and IMO horribly kludgy, slow and problematic. There are a whole bunch of choices that need to be made about whether the database is independent of the reference manager (what Endnote does with Word, called a travelling library), and how to translate across difference R's.

And what is the benefit of all this complexity? You can already automatically generate bibliographies from Scrivener with no extra scanning if you use MMD, and you can even visualise the inline citations in the formatted state if you use Marked 2 on your Scrivener project. I really don't think that temporary citations [@doe1998] are difficult to use, and if you use a reference manager like bookends, you can use nice author search interfaces like those I curated for Alfred. Scrivener already works wonderfully for us academic users, and although I would personally prioritise bibliographic functions above tools for fiction writers ( :P ) — Scrivener is a general purpose writers tool, it is not an academic-focussed word processor like Mellel (which doesn't have the research management and structured writing tools that Scrivener does).

User avatar
lunk
Posts: 1750
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Wed Nov 01, 2017 10:24 am Post

I agree with most of Nontroppo's comment, with a few small things were we deviate:

nontroppo wrote:So, if I was to build this ideally, this is what I would do (S=scrivener, R=ref manager).

  1. S has a button to bring up a requestor, input a search term, this is passed to the applescript/API interface of R[!1]
  2. R returns a list of matching citations[!2]
  3. S presents this list to the user.
  4. User selects one or more refs.


  1. With Papers, the keyboard shortcut (default ctrl+ctrl) has nothing to do with the writing software. It is a universal throughout the OS and it brings up a small search window.
  2. Yes, Papers show a search result ...
  3. ... and it is shown under the search window.
  4. User double-click on one reference and gets to choose if it is the {cite key} or the complete bibliographic reference that is wanted, or maybe you want to have a quick look at the article?

    nontroppo wrote:
    1. S now has to make a second applescript/API [!3] call for the formatted {{1}} INLINE citation and {{2}} BIBLIOGRAPHY text. These have to be stored in a database somewhere, along with the format that was used in the request.
    2. S inserts {{1}} into the document text, and puts {{2}} into a separate special doc, "References". S stores the location of where {{1}} is in the text, because it needs this for reformatting and citation editing. Mellel and Word make these special "objects", so perhaps S needs to
      do something similar to track their location (some kind of modified RTF comment?).
    3. There needs to be a reformat references function, in which S must now make a new API call for each reference in the database to request the reformatted text and it must replace each {{1}} and regenerate the references from {{2}}.
    4. The special doc "References" needs to be able to track each reference as a block linked back to the database. If S allows editing of the reference in the database then only that block would be updated.
    5. S also needs a way of letting the user select the format of the bibliography from a list. This requires another API call, or manual entry and a default fallback.


This is roughly what the people behind the Manuscripts app tried to do, i.e develop a very sophisticated reference handling inside a complex writing app. There was massive critique on the now closed user forum (the app was very buggy), and the team has been silent since march. But with Papers, more or less everything that is in the second part of Nontroppo's list can be accomplished after export to Word. One of the alternatives that present itself when hitting ctrl+ctrl is the choice to create a reference list, and this is handled by Papers, not by Word.

According to the team behind Papers, integration with Scrivener is on their list of things they want to do (read at the bottom).

http://support.readcube.com/support/sol ... rs-for-mac

PS. Yes, Papers 3 is originally a pure OS X and iOS app, but they have now released a beta version for Windows.
https://www.readcube.com/papers/windows-beta/

mh
mhalpern
Posts: 1
Joined: Thu Nov 09, 2017 9:44 pm
Platform: Mac

Thu Nov 09, 2017 9:49 pm Post

Ideally, the zotero plug in would work exactly as it does in Word (it is the only think I find remotely useful about word). According to Zotero, Scrivener doesn't really accept plugins, so they can't make one (https://forums.zotero.org/discussion/65 ... er-plug-in) and I know there are some workarounds, but they are pretty awful.

I am not a developer and know little about the back end, but is there a way for you and Zotero to maybe talk to one another about this to see if you can make a way for them to provide a plugin, just as they have for Word?