Zotero integration please!!!

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Tue Aug 08, 2017 5:11 am Post

nontroppo wrote:
Orpheus wrote:I would prefer to be writing than tracking down software bugs. :evil:


Honestly: don't use Zotero. You get what you pay for, and Zotero is free/open source[1]. I do use Zotero to maintain a shared reference list with one collaborator, but it is painful to use compared to Bookends. Bookends was released in 1984(!!!), yet still gets monthly updates by a super supportive developer, Jon. Bookends fully supports extracting annotations from the PDFs it manages — AND you can use Applescript to copy these notes (with citekeys linked to page numbers) directly into Scrivener. It does a thousand other things better than Zotero (managing keyword lists, detailed duplicate management, SQL querying, a very flexible UI, amazing format and import editor, a version for iOS with full sync, including annotations etc. etc. etc.). I see Scrivener and Bookends as critical tools for a fluid academic workflow, and each is carefully crafted by a dedicated paid developer. Most pertinently to this thread, Bookends works flawlessly with Scrivener. Or if you really want to save money, probably take some time off writing, learn to code, and contribute to build a better Zotero...

----
[1] Zotero is currently going through a major transition. V5 broke most tools that were built for V4, but V5 is itself transitional. Firefox killed its extension system, and therefore Zotero will have to completely rewrite everything for V6. Most bits of the Zotero ecosystem were built by volunteers, who have other commitments and this will impact how fast we get the tools to work. And as Keith mentions, "This is still in early stages of development, so proceed with caution when using this in a production application." is not a tempting prospect for a developer like Keith.


Thank you for your thoughts. Much appreciated. I had already installed Bookends before reading your post but am now more encouraged to learn how to use it and get a workflow going between Scrivener and Bookends so I can just focus on writing and not fiddling with niggling software quirks. I just want it to work.

If it is half as good as you say it is then it would be cheap at twice the price. I like Scrivener so much that even though I bought it I would like to give them a donation for having such good software and support.

I am curious regarding Bookends and PDFs. With Zotero it makes a duplicate PDF and puts it in a separate dBase rather than just making a link to the original. I don't like that. I have a large number of PDFs (20,000 +) that I organize into my own library according to topics and I would prefer that the software just uses an alias to find and access the PDF rather than taking up disk space by making duplicates.

Also can you recommend any good tuturials for Bookends?

Is there a separate group on Scrivener for Bookend users?

I have even considered Endnotes. How is that in comparison to Bookends? I also just downloaded a copy of Endnotes, Papers, and ReadCube.

I just want to something that works out of the box.

Now I am going to uninstall Zotero. :twisted:

User avatar
lunk
Posts: 1737
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Tue Aug 08, 2017 6:45 am Post

One thing to look at when you test the different softwares is their abilities to search for relevant litterature. I tested Bookends and it can't search e.g. Scopus or Web of Science, which are fundamental to my research. Papers can, and it is usually sufficient for my needs. As for exporting annotations to Scrivener, it is as simple as 'Copy as' in Papers and then choose what it is I want. The reference, all annotations, or something else. It also supports Apple script and their support has helped me and others to get script to do specific things for us. Everything is not perfect in Papers. It does end up with duplicate pdfs sometimes, but merging them is simple. But then I only have about a thousand references, not 20 000+.

A side note: where did you find the time to read and annotate 20 thousand references? I still have over a hundred references on my list that I haven't had time to read...

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Tue Aug 08, 2017 9:41 am Post

lunk wrote:One thing to look at when you test the different softwares is their abilities to search for relevant litterature. I tested Bookends and it can't search e.g. Scopus or Web of Science, which are fundamental to my research. Papers can, and it is usually sufficient for my needs. As for exporting annotations to Scrivener, it is as simple as 'Copy as' in Papers and then choose what it is I want. The reference, all annotations, or something else. It also supports Apple script and their support has helped me and others to get script to do specific things for us. Everything is not perfect in Papers. It does end up with duplicate pdfs sometimes, but merging them is simple. But then I only have about a thousand references, not 20 000+.

A side note: where did you find the time to read and annotate 20 thousand references? I still have over a hundred references on my list that I haven't had time to read...


Good point about search abilities, I never used that function before. Usually I just go into places like JSTOR to look for what I need. But search abilities would definitely be useful.

Just like I found out that Zotero is not the one for me, though it may be useful for grabing online citations if others don't do that, I will have to explore each software over time and see what they are good at for what I need and assemble a tool box.

To answer your last question. I am not a student. I'm an independent researcher. I paid someone to scan in my whole library (it took almost 7 years scanning 8hr/day, 6 days/week) that I collected over a 40+ year period. I read a lot. I learned to read fast. Most of them do not have digital annotations but hand written marginalia (from when I read them in paper form) that I use as landmarks for later digital annotation. My topics of interest are different than yours; things such as cross cultural interchange of knowledge in the ancient world. I have not read every item - yet. :wink: At one point I download anything that looked useful for my research for future reference as opposed to trying to find and download that interesting looking title I remember seeing last year. If it is useless then delete.

User avatar
nontroppo
Posts: 735
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue Aug 08, 2017 10:31 am Post

For Bookends, you can use an existing directory to link to your existing PDFs, but it probably will not handle sub-folders. But it does usually work with PDF files directly and not aliases (I think anyway, I haven't tested it). 20,000+ PDFs is a colossal amount :shock:

Lunk's point about search options is fair; Bookends does support Web of Science (and JSTOR, though I don't use it), it doesn't support Scopus (I asked why and Jon told me the Elsevier's access fees for a single developer app is exorbitantly high). But you can simply use a web browser, and export to your reference manager (bookends has a bookmarklet, and can find refs in a web page as long as it supports the CoINs standard). Papers is a very capable choice, and a fair distinction from Bookends is that Papers is more elegantly designed and is more PDF centric (it started off as a glorified PDF manager/reader), Bookends is better at detailed reference editing and customising bibliographic styles (it started off as a glorified database before PDFs existed :lol: )

I tend to prefer to want deeper reference management and read my PDFs in a dedicated program. I use PDF expert now, and annotations sync back to Bookends. However if I want an annotation with a page number then I do that in Bookends. What I miss in other reference managers is things like carefully standardising Journal names and abbreviations, combining multiple author name variants, detailed SQL searches and the like. But I would happily use both Bookends and Papers way over Zotero.

r.e. the behemoth Endnote (by Thompson Reuters): IMO inferior to Bookends or Papers. I occasionally use the latest version (X8) because of a collaborator (although I've now made a script to auto-convert Endnote citations to Pandoc ones during a Scrivener compile). Of course their federated search is second to none (especially Web of Science!), but everything else is second rate.

r.e Readcube — I remember downloading and playing with it some time ago and giving up, not designed for deep reference management but perhaps it has changed since then? Oh, Papers has been bought by Readcube so it seems? Well, if they combine the best of Papers and Readcube, they may have a juggernaut app on their hands! Something needs to take Endnote down! :twisted:

User avatar
lunk
Posts: 1737
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Tue Aug 08, 2017 12:09 pm Post

Springer bought Papers in 2016...

li
liz
Posts: 44
Joined: Sat Mar 25, 2017 6:40 pm
Platform: Win + iOS

Tue Aug 08, 2017 7:37 pm Post

I'm going to sidestep the whole "you get what you pay for" claim about open-source software and just point out that there are plenty of reasons besides price why someone might go with open source and be very happy with that choice (Pandoc, anyone?). On the other hand, it's also true that with open-source you might need to do a little more legwork on your own instead of dumping your problem on the developer. The issue is simply whether your individual needs are being met.

FWIW, it sounds like a lot of the magical things Bookends does--I'm a Windows user, so I don't have first-hand experience--can be handled with Zotero plugins such as ZotFile. And there are lots of Zotero plugins.

User avatar
lunk
Posts: 1737
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Tue Aug 08, 2017 9:17 pm Post

liz wrote:with open-source you might need to do a little more legwork on your own instead of dumping your problem on the developer.


And the "legwork" is coding, in some way?

I think it's a philosophical question to some degree, or personality trait. I did a fair bit of coding back in the 1980s and 1990s because the softwares that were available weren't capable of doing things we now take for granted. Today I don't code and I don't spend time asking developers to change their apps. Instead I find the apps capable of doing what I want and use them as they are, which gives me more time to do my job. :)

Online
User avatar
devinganger
Posts: 901
Joined: Sat Nov 06, 2010 1:55 pm
Platform: Mac + Windows
Location: Monroe, WA USA
Contact:

Tue Aug 08, 2017 10:19 pm Post

Not always coding. A lot of open source software requires the user to do more work integrating it into workflows. Look at some of the hijinks our Markup and LaTeX communities sometimes have to go through in order to ensure they're getting the right scripts/stylesheets to run, even when they're using scripts developed by other people.

Some people just don't want to spend the time rolling their own. That doesn't make open source bad or good, just a different experience, and it's not universal to all open source software. Some of the biggest pains-in-the-apps I've used were commercial, and some of the most user-friendly were open source.
--
Devin L. Ganger
Devin on Earth: http://www.devinonearth.com/
Plotter on the streets, pantser in the sheets

User avatar
nontroppo
Posts: 735
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue Aug 08, 2017 11:03 pm Post

Dear liz, you are absolutely right and I regret my wording somewhat. It is the case that with open-source projects, if the tool fits your needs, then it can be every bit as good (or far better) than a commercial one. Pandoc is a good example. It is an utterly brilliant tool, powerful and very well documented. But it isn't perfect, and there are people who've wanted Pandoc to do something they critically need and have had no luck for years, no one has been able to help and learning Haskell is a leap too far. In my case I do try to contribute back to Pandoc where I can (bug reports, templates and one simple haskell patch), and the benefits I gain from Pandoc far outweigh this time invested.

Indeed commercial tools like Scrivener comes with ample warnings about what it can and cannot do. So all software comes with limitations. But it is the case that when something doesn't work right in Bookends or Scrivener, I bug Jon or Keith, and they will fix it or tell me of an alternative route to do what I want. There really is something to be said about a well supported program from a dedicated and passionate developer & support team.

Zotero is likely a brilliant tool for some, and perhaps I am over harsh, but my personal take is that it is mostly just "good enough"; the original novelty that it was deeply intertwined into your browser doesn't make up for the other weaknesses. That is a personal opinion of someone who uses it for one project alone. But especially at the moment, Zotero's is going to go through a painful transition. And that results in friction for its users.

A workflow with a solid reference manager the minimises friction can hugely enhance the writing flow for writers who cite. So you have to know that Zotero may work for you but obviously doesn't work for numerous others (hence the regular Zotero threads here) and solutions that worked in V4 are broken in V5 and will likely be broken again in V6.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Wed Aug 09, 2017 10:48 pm Post

lunk wrote:Springer bought Papers in 2016...



ReadCube bought Papers https://www.readcube.com/papers/

User avatar
lunk
Posts: 1737
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Wed Aug 09, 2017 11:05 pm Post

True. I missed the date. Springer acquired Papers in 2013 and Readcube got it from Springer in 2016.

And they are working on a new version.

http://support.mekentosj.com/discussion ... or-papers3

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Thu Aug 10, 2017 2:33 am Post

lunk wrote:True. I missed the date. Springer acquired Papers in 2013 and Readcube got it from Springer in 2016.

And they are working on a new version.

http://support.mekentosj.com/discussion ... or-papers3



I haven't tried Papers yet. Will test drive BookEnds first. But I did try ReadCube.

On the positive side it indexes your PDF library without dupicating any files, and considering that I have about 167 GB of PDFs in my DropBox that would be a lot of duplcation. However I found out that it will choke if you give it too much at once. I tried to just have it index part of my JSTOR collection. That was not a good idea as it basically stopped working and hogged the CPU while trying to digest everything. Best to do something like that before going to bed. Or to just feed it smaller bites.

But the deal breaker for now with ReadCube is that it can not read any annotations in PDFs that you have made with Acrobat Pro. Nor will Acrobat Pro read any anntations you make with ReadCube. Their tech support told me that "It is just now coming" as they say in India. Apparently they are getting a lot of complaints about this incompatibility. Like they didn't think their users would like to read and use all their previous annotation? Duhh.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Thu Aug 10, 2017 2:51 am Post

I previously wrote:

To answer your last question. I am not a student. I'm an independent researcher. I paid someone to scan in my whole library (it took almost 7 years scanning 8hr/day, 6 days/week) that I collected over a 40+ year period.


If anyone has a few/many books they would like to scan and turn into PDFs you can take a look over here http://www.diybookscanner.org.

I made the mistake of getting some engineering friends to help me and they took a perfectly working plan and "improved" it to the point that it didn't work. Money and time wasted. Better to follow a plan or buy a kit. There current best model costs about as much as the yearly maintanence fee for the Bookeye scanner http://bookeye.us/index.html

With a two camera setup you can scan at a rate of 600-800 pages/hour. Half that with one camera.

I now have only a few items to scan since the project is over and I will just use the cardboard box method. Cheap and best. http://www.instructables.com/id/Bargain ... board-Box/

My library that previously took up several rooms now fits into a flash drive. I can take my library with me where ever I go. :D And it is also on DropBox.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Thu Aug 10, 2017 3:43 am Post

liz wrote:I'm going to sidestep the whole "you get what you pay for" claim about open-source software and just point out that there are plenty of reasons besides price why someone might go with open source and be very happy with that choice (Pandoc, anyone?). On the other hand, it's also true that with open-source you might need to do a little more legwork on your own instead of dumping your problem on the developer. The issue is simply whether your individual needs are being met.

FWIW, it sounds like a lot of the magical things Bookends does--I'm a Windows user, so I don't have first-hand experience--can be handled with Zotero plugins such as ZotFile. And there are lots of Zotero plugins.


My one and only concern is that it works well with Scrivener. Currently that is not Zotero. We'll shall soon see if BookEnds lives up to its press.

User avatar
Orpheus
Posts: 73
Joined: Thu Jan 05, 2017 3:27 pm
Platform: Mac
Location: Lost in the material world.

Thu Aug 10, 2017 3:47 am Post

lunk wrote:Today I don't code and I don't spend time asking developers to change their apps. Instead I find the apps capable of doing what I want and use them as they are, which gives me more time to do my job. :)


I feel exactly the same way. I want to write, just not write code.