Scapple Manual sample project fail to compile: generate_user_manual.rb missing

User avatar
MarcoAbreu
Posts: 4
Joined: Sat Mar 03, 2018 9:42 pm
Platform: Mac

Sun Mar 04, 2018 8:10 am Post

I downloaded the WinMac-ScappleManual.scriv project, but its compilation fails. MacTex is installed along with the required URW-Garamond font and I folowed the rest of the instructions to the letter.

But then I noticed that the Processing / Post-process on command line, in the project's compilation format Scapple Bridge Format, calls a script: `/usr/local/bin/generate_user_manual.rb`. Well, that script does not exist and does not came included in the project files.

I didn't find any mention of the problem in the forum, so I'm most certanly missing something here, right?

User avatar
AmberV
Posts: 22418
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Sun Mar 04, 2018 12:52 pm Post

Sorry for the confusion. That was put together very quickly (in the middle of the Scrivener 3.0 launch), and it looks like I missed a few things. I’ll be uploading an updated version in a bit. Short story is that it is in the middle of being transitioned to MMD version 6, and there are still some outstanding issues with that process.

Long answer: one good thing is that the generate_user_manual.rb file shouldn’t be necessary. It does some tidying up and further replacements that the compiler cannot handle, but I don’t think it should be required to convert the .md file to .tex.

As for Garamond, those instructions are out of date. In fact you’ll most likely need to go into the header tex file and change all of those font declarations to fonts you own. We switched to using proprietary fonts this time around. In particular, Calluna, Effra and Bauer Bodini. Inconsolata XL and Linux Libertine are also custom, but freely available. The rest are all fonts provided in a default macOS install. If you’re on Windows—you might need to find some fonts that can handle the various Unicode characters used for technical symbols, cogs and such.

I’ve uploaded a preliminary zip which should in theory compile fine if what appears to be an MMD6 bug is fixed. It will not compile correctly at all right now—in either MMD5 or 6, so if you’re looking for a working example I would recommend either waiting for MMD6 to fix the problem, or if it turns out that is an expected limitation, the workarounds I’ll need to implement in the compile settings.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
MarcoAbreu
Posts: 4
Joined: Sat Mar 03, 2018 9:42 pm
Platform: Mac

Sun Mar 04, 2018 5:13 pm Post

Yes, you right, the compilation to Markdown occurs without the if the `generate_user_manual.rb` script if Post-process on command line is disabled. But no TeX file is generated.

Now, using the `generate_user_manual.rb` script in post-processing, a TeX is generated, but has only one line saying `output.puts " %!TEX encoding = UTF-8 Unicode"`. Running the script directly from the Terminal has the same result, but the `Unknown token type: 121` error can be seen. All other compilations with Markdown fail with that same error being shown.

I saw that the issue you mention was closed by Fletcher a few minutes ago in the develop branch. I checked it out, compiled it and replaced the Homebrew `multimarkdown` installed binary. Scrivener compilation fails in the same way. But it was a long shot anyway... 1st compiling from source is not my specialty. 2nd, only one binary, `multimarkdown`, appeared after `make` while Homebrew installed version has 10 binaries. I guess I'll wait a little longer until you guys sort that out.

User avatar
AmberV
Posts: 22418
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Sun Mar 04, 2018 9:20 pm Post

Yes, you right, the compilation to Markdown occurs without the if the generate_user_manual.rb script if Post-process on command line is disabled. But no TeX file is generated.


Correct, the script itself does some processing to the output in its MultiMarkdown form, then runs MMD to produce LaTeX data, does a little more processing to format tables and a few other details, and writes out a .tex file in the end.

Without the script, you would create the .tex file on the command-line, or you could possible just compile to .tex directly from Scrivener.

It would be a bit academic at the moment though as, whether the .tex is generated by my script or by the compiler, it still won’t work right with MMD6 (hence the errors you saw and the largely empty output file). I was basically assembling chunks of LaTeX code piecemeal with the compile settings, and the way in which I was doing so resulted in escaped characters—a flaw I missed that Fletcher pointed out. Knowing that it should be easy to convert the compile settings to produce a workable file. Basically, in MMD5 you could do this:

Code: Select all

<!--\--><!--command{-->


The result would be:

Code: Select all

\command{


Messy, but it worked and it doesn’t matter how messy the interim format is if nobody ever sees it. With the new code span based LaTeX passthrough syntax:

Code: Select all

`\`{=latex}`command{`{=latex}


The result is:

Code: Select all

\texttt{\textbackslash{}`{=latex\}}command\{`


It ends up that way because technically speaking the second backtick on the line is escaped, neutering its function. So I will have to fix the interplay between Replacements and Annotation delimiters to avoid running into that scenario.

The fix that Fletcher committed only resolves the error condition that occurs when a line has out of balance backticks like the above. It might possibly cause the file to typeset, but even if it did, you would get an messy PDF with visible code in it.

I need to get a Scrivener PDF out the door before I can tackle this properly, but it’s after that the next highest thing on my list as I don’t like having an example on the site that simply cannot be compiled.

But it was a long shot anyway… 1st compiling from source is not my specialty. 2nd, only one binary, multimarkdown, appeared after make while Homebrew installed version has 10 binaries. I guess I’ll wait a little longer until you guys sort that out.


Sounds like you got it compiled correctly. There is in fact only one binary with MultiMarkdown. If you open up the other “mm2tex” etc. files in a text editor you’ll see they are just shell wrappers. So to install a build from source, simply replace the old /usr/local/bin/multimarkdown with the compiled binary. The helper scripts will use the new binary.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
MarcoAbreu
Posts: 4
Joined: Sat Mar 03, 2018 9:42 pm
Platform: Mac

Mon Mar 05, 2018 7:55 am Post

Sounds like you got it compiled correctly. There is in fact only one binary with MultiMarkdown. If you open up the other “mm2tex” etc. files in a text editor you’ll see they are just shell wrappers. So to install a build from source, simply replace the old /usr/local/bin/multimarkdown with the compiled binary. The helper scripts will use the new binary.

Shell wrappers, of course! I ran a couple of them yesterday with the help flag, saw exactly the same message generated by the multimarkdown binary and concluded incorrectly they were binary wrappers :lol: . See, not my specialty.

Back to your generate_user_manual.rb script, I see that one of its functions is:
This will catch any of symbols that get missed by Replacements; edit both to revise


Some of the symbols listed are in the compilation Replacements and others, like the arrow symbols, are not. Is it possible that Replacements misses?

I have another question about MutiMarkdown and Pandoc, but is way more generic and so I decided to make a new post here.

User avatar
AmberV
Posts: 22418
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Mon Mar 05, 2018 11:56 am Post

Some of the symbols listed are in the compilation Replacements and others, like the arrow symbols, are not. Is it possible that Replacements misses?


Right, as mentioned before, the production of this was a bit rushed. The process being applied to compile this project was not designed for it, but rather the Scrivener manual. What you see is a crude hack n slash job of grafting a 2018 Scrivener 3 compile process onto a 2014 Scrivener 2 project with just enough effort to get the thing working without blowing up. :) And now with a major MultiMarkdown upgrade it is blowing up—little surprise.

But yes, Replacements are not universal; in most cases they are a pretty comprehensive tool—where I run into issues is when I am using replacements to build a placeholder that injects a string from another item, where that string needs replacements done on it as well. Probably not your typical use-case!
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
AmberV
Posts: 22418
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Mon Mar 05, 2018 7:53 pm Post

Okay! Give the updated .zip package a try. I think I’ve worked out all of the little rough edges in the compile settings at this point, and I’ve streamlined the compile folder setup a bit. There is now a separate font file and I’ve plugged in a few generic common fonts that should work on most systems out of the box.

That leaves some broken image links in the content, but I think as far as demonstrations go, it services it purpose at this point. There is still quite a bit of work to do—in the end I’d like it set up like the Scrivener project is, but I need to get the Scrivener project uploaded before tackling that one. Never ending to-do list. :)

Let me know if you run into any problems. I tested this on a blank macOS 10.12 account, with MacTeX 2017 installed, and it all seems to work.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles