Zotero integration please!!!

mi
mijalce
Posts: 14
Joined: Fri Aug 14, 2015 9:55 pm
Platform: Mac

Sun Aug 06, 2017 4:42 pm Post

Hello Keith,

Thank you for your interest in enabling better integration between Zotero and Scrivener. I think this will be an excellent feature for academic writers. I will try to make my contribution on how this can be made.

Since two years ago, through BetterBibTex feature Cite As You Write (CAYW), it is possible to use CMD+Y to call Zotero picker and add the citation key in Scrivener. There are different citation key types depending do you write in rtf or markdown.
The current issue for me is that I need to use workarounds (create .md file, use pandoc filters to identify the citations and convert it to .docx) to actually make a .docx that will have the citations (This is the http://davepwsmith.github.io/academic-scrivener-howto/). It is the same situation when you use RTF/ODF-Scan for Zotero (https://zotero-odf-scan.github.io/zotero-odf-scan/).

According to me the real integration will be enabled when you can click compile and the compiled documents (.md, .docx, .pdf) will include the bibliography, based on the citations you added in the documents, and the appropriate citation style.

Now I will explain what is done when you write in Markdown. To a certain extend this is achieved through mmd-xslt (see here https://github.com/retorquere/zotero-be ... -127052688). However, the appropriate solution, which can be also done with Scrivener is what is already done by the BBT developer for integration between Zotero and Atom (https://github.com/retorquere/zotero-citations). According to him this can be easily done with Scrivener (https://github.com/retorquere/zotero-be ... -136747401).

Another option for integration is to provide an option for custom processor (something like in Marked2) where a user can add arguments that can be called when s/he compiles the document. Now there is an option to compile Multimarkdown->RichText (rtf). Thus the integration can be Multimarkdown->Microsoft word (docx) with custom processor.

I hope I was helpful, I am willing to more actively contribute to the development of the integration.

Keep the good work with the excellent Scrivener

All the best,
Mijalche

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Mon Aug 07, 2017 7:13 am Post

KB wrote:Could a Zotero user please explain exactly what it is that you would want from integration with Scrivener? I mean in a practical manner - you can already to use Zotero to insert citations and then scan the exported RTF or ODT file, so what would integration provide over this and what would you it do?

I'm interested in exploring options but I'm not a Zotero user and so it would be very helpful if someone could break down exactly what integration would entail. What would happen in Scrivener? How would it look? What commands would it entail?

As far as I understand it, the problem is that you would still need to scan your final file, wouldn't you? I can't see that there's any auto-magic that would avoid this during Compile. So what exactly is meant by "integration"?

Thanks,
Keith



Well the first thing is that you have to get Zotero both the stand alone part and the browser extension, I use FF, they also have for Chrome. https://www.zotero.org

It is pretty easy to set up. Then you add 1-3 titles to Zotero to use as dummy citations.

That is easy, just go to Amazon, select any book. Then click on the Zotero icon on your browser. That will save the citations into Zotero.

Then under "Tools>add-ons" make sure the integragation with your word processor is turned on. I use Word but you can also integrate with LibreOffice. Then restart Word and right click on Tool Bar to custumize it and add the Zotero Tool bar.

Then just open a new Word document click the cursor on the page. And on the Zotero tool bar choose to add a new citation. Zotero will then ask you which citation style you want, make your choice, and follow the steps to get to the citations you created and click on any of them and voilà Zotero has added a footnote to the page and at the bottom of the page is the citation.

At this point I would copy and paste the citation into the Scrivener.

It would be nice if I could skip Word and do this all just using Zotero and Scrivener.

I hope that is clear. You actually have to do it yourself a few times to understand the workflow involved.

User avatar
nontroppo
Posts: 755
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Mon Aug 07, 2017 10:00 am Post

@mijalce: what you are suggesting is sort of already possible if you use some automation (Folder actions etc). I compile from Scrivener to an .MD file and Pandoc handles my bibliography and spits out a fully outlined, beautifully styled, and bibliographically complete LaTeX/ODT/DOCX file. There have been mentions of an improved Pandoc workflow in the compile system for Scrivener 3 and I wouldn't be surprised if all this became even easier.

Regarding the "auto-paster" discussion you linked to, I don't get it really — Scrivener already allows you to link to a bibliography app. You hit ⌘Y, select a ref in Bookends, hit ⌘Y and the citation is sat in Scrivener. L&L have worked to implement this functionality, surely it is up to ZotPicker to support this mechanism as other reference managers can? Perhaps a technical description from Keith about how this works would help them to implement this?

@Orpheus: I'm sure several people have mentioned you can get temporary citations directly into Scrivener from Zotero without needing to first do this in Word (see LewF's posts), so there seems to be something not working on your machine rather than this not being possible?

All of this makes me glad I use Bookends! :P

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Mon Aug 07, 2017 2:35 pm Post

This is discussion about Zotero not Bookends.

I'm sure several people have mentioned you can get temporary citations directly into Scrivener from Zotero without needing to first do this in Word (see LewF's posts), so there seems to be something not working on your machine rather than this not being possible?

All of this makes me glad I use Bookends! :P


If you check my previous posts I tried that and it didn't work. I made a test page put in temp citataions then exported it to RTF and then used Zotero to scan it and Zotero just hung with the candy cane bar twirling for ever and nothing happening. So something didn't go right from someone's side Zotero or mine or both.

That is why it would be better if Zotero and Scrivener were BFF (-: Zotero can do a lot of things aside from citations that is why I want to use it. Things like this nifty trick https://youtu.be/4aDvAPLZwCY?t=7m15s

User avatar
lunk
Posts: 1741
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Mon Aug 07, 2017 2:58 pm Post

Continuing on Nontroppo's line of thought, isn't this an issue for Zotero, not for Scrivener?

The Papers app has a similar short-cut (ctrl -> ctrl) which opens a dialogue where you can choose a reference, which is then inserted as a "citekey" in the text which can later (after compilation to e.g. Word) be translated to a correctly formatted reference and also create a correctly formatted bibliography. But this is all controlled by the Papers app, not by Scrivener or Word.

So maybe it is Zotero that should provide the functionality?

User avatar
KB
Site Admin
Posts: 19190
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Mon Aug 07, 2017 6:22 pm Post

Orpheus wrote:This is discussion about Zotero not Bookends.


But nontroppo's comment is entirely relevant. For instance, with Bookends:

- In Scrivener, you set up Bookends as your bibliography manager in the Preferences.
- In Bookends, you set Scrivener as your word processor in its Preferences.
- In Scrivener, while typing, you hit Cmd-Y to open the bibliography manager set in the Preferences (Bookends in this example). Bookends is launched.
- In Bookends, you select your citation and hit Cmd-Y. This automatically inserts the temporary citation in Scrivener.
- In Scrivener, you compile to RTF and then scan the document with Bookends.

You can do something very similar with Endnote, I believe.

How can you do this in Zotero? You can set up Zotero as your bibliography manager in Scrivener, but there seems to be no way of adding an arbitrary word processor in Zotero. So there seems to be no way of hitting a keyboard shortcut in Zotero that will automatically open Scrivener and insert a temporary citation (which would need to be done on Zotero's end).

A quick look at the Zotero API seems to suggest that it comes with no UI and just a way of looking up citations in the user's online Zotero database. This would mean I would have to build my own UI for inserting Zotero citations in Scrivener if I were to user their API. But then, their Objective-C API has the warning: "This is still in early stages of development, so proceed with caution when using this in a production application."

All the best,
Keith
"You can't waltz in here, use my toaster, and start spouting universal truths without qualification."

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Mon Aug 07, 2017 6:36 pm Post

KB wrote:
Orpheus wrote:This is discussion about Zotero not Bookends.


But nontroppo's comment is entirely relevant. For instance, with Bookends:

- In Scrivener, you set up Bookends as your bibliography manager in the Preferences.
- In Bookends, you set Scrivener as your word processor in its Preferences.
- In Scrivener, while typing, you hit Cmd-Y to open the bibliography manager set in the Preferences (Bookends in this example). Bookends is launched.
- In Bookends, you select your citation and hit Cmd-Y. This automatically inserts the temporary citation in Scrivener.
- In Scrivener, you compile to RTF and then scan the document with Bookends.

You can do something very similar with Endnote, I believe.



So the problem is with Zotero and I should move my quacking duck over there 8)

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Mon Aug 07, 2017 7:17 pm Post

KB wrote:
Orpheus wrote:This is discussion about Zotero not Bookends.


But nontroppo's comment is entirely relevant. For instance, with Bookends:

- In Scrivener, you set up Bookends as your bibliography manager in the Preferences.
- In Bookends, you set Scrivener as your word processor in its Preferences.
- In Scrivener, while typing, you hit Cmd-Y to open the bibliography manager set in the Preferences (Bookends in this example). Bookends is launched.
- In Bookends, you select your citation and hit Cmd-Y. This automatically inserts the temporary citation in Scrivener.
- In Scrivener, you compile to RTF and then scan the document with Bookends.

You can do something very similar with Endnote, I believe.



Which citation manager works best with Scrivener?

User avatar
lunk
Posts: 1741
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Mon Aug 07, 2017 7:27 pm Post

I think both Bookends, Endnote and Papers work well with Scrivener, so it's more a question of which one you prefer.

mi
mijalce
Posts: 14
Joined: Fri Aug 14, 2015 9:55 pm
Platform: Mac

Mon Aug 07, 2017 8:59 pm Post

I would like to inform that RTF/ODF Scan for Zotero is actually doing what was identified by the others as an integration with other reference managers.
My idea for integration is probably biased because I write in Markdown and I want when I compile in Scrivener to get .docx with the expanded bibliography and the appropriate citation style. I think that an opportunity for custom processor, as it is in some form available in MultiMarkdown->Web page (.html) by using XSLT post-processing, can provide the flexibility for integration with Zotero. Will this be a problem to be developed? I do not know will Scrivener 3 use custom processors. If it has it it will be good option to be user friendly in order we to add our arguments.
Thanks!

Le
LewF
Posts: 34
Joined: Sun Dec 10, 2006 11:54 pm
Location: Madison,WI

Mon Aug 07, 2017 10:05 pm Post

Probably "Zotero" should. But Zotero is an open source project and so has a few key people working on it with a lot of demands. So I am guessing if the Zotero-using Scrivener community (all of us?) really wants a plug-in we may have to build it "ourselves." That is in quotes because I can't code (easy to volunteer for things you can't do). I wish I could.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Tue Aug 08, 2017 12:31 am Post

mijalce wrote:I would like to inform that RTF/ODF Scan for Zotero is actually doing what was identified by the others as an integration with other reference managers.


Except I can't can't get that to work. For a one page document it just spins its wheels and nothing happens. Until I can figure out what I I am doing wrong I am just going to to put parenthetical marker where I want to put a footnote with name of author and page number and then after I export it to word I will put in the footnoots with Zotero so that I can edit the footnotes and generate a bibliography. That way I will be spending more time writing and less time trying to get the software to work.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Tue Aug 08, 2017 12:52 am Post

KB wrote:
Orpheus wrote:This is discussion about Zotero not Bookends.


But nontroppo's comment is entirely relevant. For instance, with Bookends:

- In Scrivener, you set up Bookends as your bibliography manager in the Preferences.
- In Bookends, you set Scrivener as your word processor in its Preferences.
- In Scrivener, while typing, you hit Cmd-Y to open the bibliography manager set in the Preferences (Bookends in this example). Bookends is launched.
- In Bookends, you select your citation and hit Cmd-Y. This automatically inserts the temporary citation in Scrivener.
- In Scrivener, you compile to RTF and then scan the document with Bookends.

You can do something very similar with Endnote, I believe.

How can you do this in Zotero? You can set up Zotero as your bibliography manager in Scrivener, but there seems to be no way of adding an arbitrary word processor in Zotero. So there seems to be no way of hitting a keyboard shortcut in Zotero that will automatically open Scrivener and insert a temporary citation (which would need to be done on Zotero's end).

A quick look at the Zotero API seems to suggest that it comes with no UI and just a way of looking up citations in the user's online Zotero database. This would mean I would have to build my own UI for inserting Zotero citations in Scrivener if I were to user their API. But then, their Objective-C API has the warning: "This is still in early stages of development, so proceed with caution when using this in a production application."

All the best,
Keith



What exactly should I tell the people over at Zotero?
Last edited by Orpheus on Tue Aug 08, 2017 5:14 am, edited 1 time in total.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Tue Aug 08, 2017 1:29 am Post

nontroppo wrote:@Orpheus: I'm sure several people have mentioned you can get temporary citations directly into Scrivener from Zotero without needing to first do this in Word (see LewF's posts), so there seems to be something not working on your machine rather than this not being possible?

All of this makes me glad I use Bookends! :P


Yes it must be something wrong with my machine if everyone else is getting it to work. Will have to figure what is wrong. I suppose that would be on the Zotero forum. :?

I would prefer to be writing than tracking down software bugs. :evil:

User avatar
nontroppo
Posts: 755
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue Aug 08, 2017 2:28 am Post

Orpheus wrote:I would prefer to be writing than tracking down software bugs. :evil:


Honestly: don't use Zotero. You get what you pay for, and Zotero is free/open source[1]. I do use Zotero to maintain a shared reference list with one collaborator, but it is painful to use compared to Bookends. Bookends was released in 1984(!!!), yet still gets monthly updates by a super supportive developer, Jon. Bookends fully supports extracting annotations from the PDFs it manages — AND you can use Applescript to copy these notes (with citekeys linked to page numbers) directly into Scrivener. It does a thousand other things better than Zotero (managing keyword lists, detailed duplicate management, SQL querying, a very flexible UI, amazing format and import editor, a version for iOS with full sync, including annotations etc. etc. etc.). I see Scrivener and Bookends as critical tools for a fluid academic workflow, and each is carefully crafted by a dedicated paid developer. Most pertinently to this thread, Bookends works flawlessly with Scrivener. Or if you really want to save money, probably take some time off writing, learn to code, and contribute to build a better Zotero...

----
[1] Zotero is currently going through a major transition. V5 broke most tools that were built for V4, but V5 is itself transitional. Firefox killed its extension system, and therefore Zotero will have to completely rewrite everything for V6. Most bits of the Zotero ecosystem were built by volunteers, who have other commitments and this will impact how fast we get the tools to work. And as Keith mentions, "This is still in early stages of development, so proceed with caution when using this in a production application." is not a tempting prospect for a developer like Keith.