Scriv4, Pandoc, Latex and citations

zs
zsteir
Posts: 17
Joined: Thu Oct 25, 2012 2:25 am
Platform: Mac

Tue Feb 13, 2018 12:44 pm Post

Thanks, nontroppo. Indeed, that did the trick! [I figured out the errant "$" in the gsub, but couldn't get any further].

My issue was that I need to run Biber, and outputting directly to pdf (so pandoc calls latexmk) didn't seem to do that. But no matter, this will work just fine. Thank you.

User avatar
nontroppo
Posts: 1034
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue Feb 13, 2018 2:18 pm Post

zsteir wrote:My issue was that I need to run Biber, and outputting directly to pdf (so pandoc calls latexmk) didn't seem to do that. But no matter, this will work just fine. Thank you.


I've updated scrivomatic to V1.0.14 which checks if pandocomatic specifies the pdf-engine setting, then uses that instead of the default xelatex. Regarding biber, not quite sure why --biblatex option didn't work for you, I've never used LaTeX for bibliographies so don't really know what the process is for using direct Pandoc⇨PDF with Biblatex. Anyway, if Scrivomatic⇨Latexmk does work then great!

fp
fpramme
Posts: 3
Joined: Thu May 03, 2018 12:07 pm
Platform: Mac

Thu May 03, 2018 12:27 pm Post

I am new concerning the handling of Scrivener. I would like to write my thesis with Scrivener. The advantages are obvious, but again and again I come across problems. I've worked a lot with latex and feel a bit more familiar with it, but I've got trouble getting started with Scrivener - scrivomatic and everything that follows.

The best way to describe the problem a bit. I downloaded "scrivomatic" and installed it according to the instructions. I have worked through the "getting started file" shown here in the forum. It lives. I can tell from exactly this file a .md file and it will also create a latex and a Word file. The latex file compiled. So far so good.

However, I would like to start my own Scrivener project and adapt it so that I can transfer it with the template (Eisvogel) in a latex document.

Now my problems start. How do I get the styles from the tutorial in my project? Does Scrivener subsequently create sections, subsections, .. by the folder structure?

Is it enough to have .bib-file, scrivomatic, authorSimplifyMetadata, preventall, ... in the target folder? Or must further precautions be taken there?

If this problem is solved and I can convert the first part of my thesis with scrivener I would like to use papers3 to incorporate the quotes. Are there traps there?

Thanks in advance

User avatar
nontroppo
Posts: 1034
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Thu May 03, 2018 11:00 pm Post

Hi fpramme, the getting started file is just a way to test this workflow without needing to install anything. Normally, the various filters and templates will live in subfolders of the Pandoc data directory (which is ~/.pandoc/ by default).

Now my problems start. How do I get the styles from the tutorial in my project?


Editor styles can be imported into a new project from another project using Format > Style > Import Styles... — Compiler styles are part of the compile format, which can be either specific to a project (Project Formats)or shared across projects (My Formats). You can convert between these by editing the format and changing Save to....

Does Scrivener subsequently create sections, subsections, .. by the folder structure?


Yes. The exact mapping between the Scrivener folder level and the LaTeX level can be controlled by changing the base-header-level metadata.

Is it enough to have .bib-file, scrivomatic, authorSimplifyMetadata, preventall, ... in the target folder? Or must further precautions be taken there?


As I said above, normally these files are not stored in the target folder, this is just to make the getting started project work simply. I normally store scrivomatic in ~/bin/, all my templates in ~/.pandoc/templates etc. but the point is you can store them anywhere, as long as you edit the paths so Pandoc can find them. I would recommend to try to use the Pandoc data directory as I do though as it simplifies the paths significantly (Pandoc assumes files will be in its data directory by default). You can see my pandoc data directory structure as an example here:

https://github.com/iandol/dotpandoc

I store links (symlinks) to my .bib file in ~/.pandoc also, but these are actually stored in my Dropbox so they sync across all my machines. You can place your bib file wherever you want , again as long as you tell Pandoc the proper path.

If this problem is solved and I can convert the first part of my thesis with scrivener I would like to use papers3 to incorporate the quotes. Are there traps there?


Papers 3 can generate Pandoc style temporary citations so it should work fine (you need to configure this in preferences somewhere). I don't use papers so don't know the specific details, but as long as you can add citekeys and build a .bib you should be fine.

By the way, my dotpandoc already includes Eisvogel and you can use it by specifying it in the Scrivener metadata like so:

Code: Select all

pandocomatic_:
  use-template: [docx-refs, eisvogel]


It can be fully configured from scrivener metadata or editing the pandocomatic.yaml directly:

https://github.com/iandol/dotpandoc/blo ... .yaml#L179

fp
fpramme
Posts: 3
Joined: Thu May 03, 2018 12:07 pm
Platform: Mac

Tue May 08, 2018 8:02 am Post

Thank you very much for your help.

I started with a new and clean scrivener file. Than I insert a File at the top level called "Pandoc metadata". After this I just copy the text out of your Tutorial-File.

Pandoc is installed via homebrew and the scrivomatic file is stored in /User/xxx/bin/scrivomatic and I selected this Path inside Scrivener.

I just insert some text in scrivener and now I am trying to compile this into an .m file. So I pressed the compile button, select scrivomatic-Format, use "Pandoc metadata" als front-metter and hit the compile button. Than I select a new folder called "output" and compile the project to an .m file. This works, and there is a scirvomatic.log and an html file now inside this folder. But no latex sources or an docx file.

I actually the pandomatic templates called docx, eisvogel and latex-with-toc. But there is just the html file inside my output folder.

Code: Select all

=== ------------------------------------------------------ ===
=== Scrivomatic V1.0.18 Report @ 2018-05-08 10:06:15 +0200 ===
=== ------------------------------------------------------ ===
 Running under Ruby 2.3.3
 Working directory: /Users/florianpramme/Documents/Dissertation/thesis/Output
===------ Input Options: ------===
#<struct Scrivomatic::OPT input="doctoral_thesis.m", output=nil, to=nil, yaml=nil, command="pandocomatic", envpath="/Users/florianpramme/bin:/usr/local/bin:/Library/TeX/texbin", build=false, verbose=true, dry_run=false, open_log=false>
===------ Final ENV PATH: ------===
/Users/florianpramme/bin:/usr/local/bin:/Library/TeX/texbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
===------ TOOL PATHS: ------===
---pandoc: /usr/local/bin/pandoc | V: 2.2
---pandocomatic: /usr/local/bin/pandocomatic | V: Pandocomatic version 0.2.2.1
---ruby: /usr/bin/ruby | V: ruby 2.3.3p222 (2016-11-21 revision 56859) [universal.x86_64-darwin17]
---python: /usr/bin/python
---xelatex: /Library/TeX/texbin/xelatex
---latexmk: /Library/TeX/texbin/latexmk

 … running pandocomatic, please wait …
===------ COMMAND OUTPUT: ------===
:: Running: /usr/local/bin/pandocomatic --debug doctoral_thesis.m
::: An unexpected error has occurred. You can report this bug via https://github.com/htdebeer/pandocomatic/issues/new.
::: /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:379:in `parse': (<unknown>): could not find expected ':' while scanning a simple key at line 6 column 1 (Psych::SyntaxError)
:::    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:379:in `parse_stream'
:::    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:327:in `parse'
:::    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:254:in `load'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/pandoc_metadata.rb:69:in `load_file'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/command/convert_file_multiple_command.rb:50:in `initialize'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/pandocomatic.rb:100:in `new'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/pandocomatic.rb:100:in `run'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/bin/pandocomatic:3:in `<top (required)>'
:::    from /usr/local/bin/pandocomatic:22:in `load'
:::    from /usr/local/bin/pandocomatic:22:in `<main>'
:: exit status: pid 23908 exit 1

!!!---scrivomatic::runCommand() RETURN non-zero value: /usr/local/bin/pandocomatic --debug doctoral_thesis.m !!!

=== ------------------------------------------------------ ===
=== Scrivomatic V1.0.18 Report @ 2018-05-08 10:22:26 +0200 ===
=== ------------------------------------------------------ ===
 Running under Ruby 2.3.3
 Working directory: /Users/florianpramme/Documents/Dissertation/thesis/Output
===------ Input Options: ------===
#<struct Scrivomatic::OPT input="doctoral_thesis.m", output=nil, to=nil, yaml=nil, command="pandocomatic", envpath="/Users/florianpramme/bin:/usr/local/bin:/Library/TeX/texbin", build=false, verbose=true, dry_run=false, open_log=false>
===------ Final ENV PATH: ------===
/Users/florianpramme/bin:/usr/local/bin:/Library/TeX/texbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
===------ TOOL PATHS: ------===
---pandoc: /usr/local/bin/pandoc | V: 2.2
---pandocomatic: /usr/local/bin/pandocomatic | V: Pandocomatic version 0.2.2.1
---ruby: /usr/bin/ruby | V: ruby 2.3.3p222 (2016-11-21 revision 56859) [universal.x86_64-darwin17]
---python: /usr/bin/python
---xelatex: /Library/TeX/texbin/xelatex
---latexmk: /Library/TeX/texbin/latexmk

 … running pandocomatic, please wait …
===------ COMMAND OUTPUT: ------===
:: Running: /usr/local/bin/pandocomatic --debug doctoral_thesis.m
::: An unexpected error has occurred. You can report this bug via https://github.com/htdebeer/pandocomatic/issues/new.
::: /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:379:in `parse': (<unknown>): could not find expected ':' while scanning a simple key at line 6 column 1 (Psych::SyntaxError)
:::    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:379:in `parse_stream'
:::    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:327:in `parse'
:::    from /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/psych.rb:254:in `load'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/pandoc_metadata.rb:69:in `load_file'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/command/convert_file_multiple_command.rb:50:in `initialize'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/pandocomatic.rb:100:in `new'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/lib/pandocomatic/pandocomatic.rb:100:in `run'
:::    from /Library/Ruby/Gems/2.3.0/gems/pandocomatic-0.2.2.1/bin/pandocomatic:3:in `<top (required)>'
:::    from /usr/local/bin/pandocomatic:22:in `load'
:::    from /usr/local/bin/pandocomatic:22:in `<main>'
:: exit status: pid 24818 exit 1

!!!---scrivomatic::runCommand() RETURN non-zero value: /usr/local/bin/pandocomatic --debug doctoral_thesis.m !!!

User avatar
nontroppo
Posts: 1034
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Tue May 08, 2018 9:13 am Post

fpramme wrote: I just insert some text in scrivener and now I am trying to compile this into an .m file. So I pressed the compile button, select scrivomatic-Format, use "Pandoc metadata" als front-metter and hit the compile button. Than I select a new folder called "output" and compile the project to an .m file. This works, and there is a scirvomatic.log and an html file now inside this folder. But no latex sources or an docx file.


The scrivomatic log is saying that pandocomatic cannot properly parse your metadata. You can send me your compiled .md file (either zipped up or via private message) and I can have a look to see what may be the problem (metadata can be fussy, no tabs etc).

fp
fpramme
Posts: 3
Joined: Thu May 03, 2018 12:07 pm
Platform: Mac

Tue May 08, 2018 5:32 pm Post

It works!!! Thank you so much for your great support.

Now I can arrange everything directly in scrivener. Are there any meta-informations for post processing? I just want a clean and easy workflow like yours, so I think it would be great to make out of the latex document a pdf and afterwards clean the folder by deleting stuff which is not longer needed like .aux data and so on. Is there a possibility inside scrivener to manage this or do I have to make an applescript or something else?

Thanks in advanced
Florian

User avatar
nontroppo
Posts: 1034
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Wed May 09, 2018 12:33 am Post

fpramme wrote:It works!!! Thank you so much for your great support.


Great! For others who may come along it was indeed the metadata was being escaped, and it is important to make sure that you use a Section Layout that compiles it "As-Is", and that you don't use "Convert Rich Text to MultiMarkdown".

There are many options with which to generate a clean PDF via latex. Pandocomatic allows you to specify PDF as the output, and choose the PDF engine you want to use, then you can specify a cleanup command ("latexmk -c") to run after the PDF was built like so:

https://github.com/iandol/dotpandoc/blo ... .yaml#L164

This example extends my default "latex-refs" template to build the PDF.

Scrivomatic also has an option to run "latexmk" and "latexmk -c" by itself, you just pass the -b option at the command line and as long as there was a .tex file output it will build and clean for you:

Code: Select all

> scrivomatic -v -b -i markdown.md


Screen Shot 2018-05-09 at 08.31.05.png
Screen Shot 2018-05-09 at 08.31.05.png (11.2 KiB) Viewed 1690 times

yo
your_liang
Posts: 12
Joined: Sat Sep 30, 2017 6:52 am
Platform: Mac + iOS

Mon May 28, 2018 9:00 am Post

OzPhil wrote:Hi again,
Thanks, I tried using defining a new format after duplicating the MMD one. My compile format pane looks like the attached. However, when I compile it says "cannot produce pdf output from pdf". (same result if I add <$inputfile> to the Arguments). I had thought the input file for the MMD compile format would be an md file?
- Phil

Why I cannot find out the Pandoc Options and Processing in the compiling settings. I installed the latest MacTex and Pandoc from HomeBrew along with Scrivener 3. How can I turn these two settings on? Thank you very much.
Screen Shot 2018-05-28 at 16.55.52.png
Screen Shot 2018-05-28 at 16.55.52.png (218.52 KiB) Viewed 1049 times

User avatar
AmberV
Posts: 22065
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Mon May 28, 2018 11:41 am Post

It looks like you are using the sandboxed version sold through the Mac App Store. Apple doesn’t allow software to make use of the file system in a normal fashion, so it cannot link up to Pandoc or the LaTeX binaries. You will need to use the direct-sale version available from the main website. Although that downloads as a demo, it will unlock with a valid MAS receipt on the system. There are migration tips on our knowledge base.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles