Help sought to turn Pandoc fenced_divs into prefixes/suffixes in the Compiler Format Designer

en
enricoscarpella
Posts: 19
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Mon Mar 11, 2019 1:16 am Post

Dear members of the forum,

In a few documents of a scriv project, I used Pandoc fenced_divs to signify custom paragraph styles for docx output. At the moment, those fenced_divs are directly typed into the documents and formatted to preserve their format, so that they are passed as is through the compiler. So, when I compile for Multimarkdown and process with Pandoc using a reference docx that contains such custom styles, the text contained within those fenced_divs is styled according to the respective custom styles (as defined in the reference docx). For example, in

Code: Select all


::: {custom-style="Affiliations"}
Affiliations
:::



the text Affiliation will be styled according to the custom paragraph style called Affiliation that I had defined in the reference docx used by Pandoc.

This works just perfectly, and the resulting docx file is formatted exactly as I need it. However, I wondered whether I could achieve the same result with custom styles in Scrivener. So I created new styles in Scrivener that have the same names of the custom styles I had created in Word, and in the Styles pane of the Compiler Format Designer, I specified the prefix and suffix for each of those styles. For the example above the prefix is

Code: Select all


::: {custom-style="Affiliations"}


and the postfix is

Code: Select all

:::



However, in this case, when I compile for Multimarkdown and process with Pandoc using that reference docx, the output is not styled according to those custom styles: instead, it is styled according to the Normal style, which is what Pandoc falls back to if the syntax is not recognized as correct. I suspect the problem is that the correct syntax for those fenced_divs requires that the part that precedes the text to be custom-syled is on a line by itself and is preceded by an empty line, and that the part that follows the text to be custom-styled is on a separate line followed by an empty line, and that perhaps all those requirements cannot be inputed properly in the Prefix/suffix box of the Styles pane of the Compiler Format Designer. But perhaps I am doing something else wrong. Has anyone succeeded in turning Pandoc fenced-divs into prefixes/suffixes in the compiler? If so, could you please be so kind to point me in the right direction? If I have not been sufficiently clear, I would be happy to upload a test project (though I would not be able to upload a reference docx file). Thank you for your consideration.

Best regards,
Enrico

User avatar
nontroppo
Posts: 1116
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Mon Mar 11, 2019 4:01 am Post

To input newline characters in the prefix/suffix, you can use ⌥ENTER — this allows you to put newlines before and after your Markdown so that blocks have newlines before and after.

This is working for me at the moment, testing with Scrivener V3.1.2 and Pandoc V2.7. To test you can use Pandoc directly on the command line easily:

Code: Select all

pandoc -t docx -o test.docx; open test.docx


This puts terminal into STDIN mode where you can past in some text like:

Code: Select all

Dickinson starts the poem simply:

::: {custom-style="Poetry"}
| A Bird came down the Walk---
| He did not know I saw---
:::

And so it was…


Now press ⌃D and Pandoc generates a test.docx file and it gets opened and you should see the Poetry style has been correctly assigned to the paragraph (even though there is no reference.docx, recent Pandoc creates new styles if there is none present in the reference file).

So the question is what does your intermediate markdown look like (the .md file output from Scrivener's compile)? You should be able to see if this is a newline problem and fix it with ⌥ENTER then retry...

en
enricoscarpella
Posts: 19
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Mon Mar 11, 2019 4:46 am Post

Thank you very much, nontroppo, for your helpful reply.

The Option+Enter suggestion did indeed allow me to enter empty lines, so that’s solved: thank you for that! Nevertheless, I still did not get the correct docx output: once again, the text to be custom-styled was instead styled according to the default Normal style, so I inspected the md file created by Scrivener and noticed something else I had missed the first time. Instead of reading

Code: Select all

::: {custom-style="Affiliations"}
Affiliations
:::

The markdown file created by Scrivener read

Code: Select all

::: \{custom-style="Affiliations"\}
Affiliations
:::

even though I had never typed those backslashes in the Prefix boxes of the Styles pane of the Compile Format Designer and they are definitely not there in those boxes. Would you have any idea how those backslashes got added to the text I had typed in the Prefix boxes? They are definitely the reason Pandoc is not recognizing the syntax because if I delete them manually from the md file created by Scrivener and run the resulting md file through Pandoc through the Terminal, now the text is styled with my custom styles as it should be.

Thank you for your consideration.

Best regards,
Enrico

User avatar
nontroppo
Posts: 1116
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Mon Mar 11, 2019 9:07 am Post

Yes, this is a simple fix, you must make sure you disable "Convert rich text to Multimarkdown" in the main Compiler options:

Screenshot 2019-03-11 at 17.03.30_SMALL.png
Screenshot 2019-03-11 at 17.03.30_SMALL.png (56.23 KiB) Viewed 2723 times


....or, if you need to enable it, make sure you DISABLE "Escape special characters". I use Scrivener Styles for all "rich text" (i.e. I never use italics and bold, but Emphasis and Strong character styles, that are converted to markdown) and so simply disable "Convert rich text to Multimarkdown" for all my compiles...

en
enricoscarpella
Posts: 19
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Mon Mar 11, 2019 2:40 pm Post

Thank you very much, nontroppo, for your prompt and helpful reply: that indeed solved it! Thank you very much again for your time and help.

Best regards,
Enrico

en
enricoscarpella
Posts: 19
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Wed Mar 13, 2019 12:50 am Post

Dear nontroppo,

I use Scrivener Styles for all "rich text" (i.e. I never use italics and bold, but Emphasis and Strong character styles, that are converted to markdown) and so simply disable "Convert rich text to Multimarkdown" for all my compiles...


I normally let Scrivener convert my rich text into Multimarkdown and use prefix/suffix on "as is" text only when Multimarkdown lacks that syntax, so I was wondering whether there is a reason for your preference: is there any undesired effect that would result from letting Scrivener convert rich text into Multimarkdown?

Thank you again for your consideration.

Best regards,
Enrico

User avatar
nontroppo
Posts: 1116
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Wed Mar 13, 2019 3:00 am Post

My initial reason was due to what you experienced, that you cannot "mix" rich text and Markdown, because "Convert Rich Text" forcibly escaped all the special characters necessary for many types of Markdown formatting. Therefore I (and everyone else who needs to use Markdown in some way) developed a workflow to use Scrivener Styles for every formatting requirement[1].

But to be honest, this totally suited my philosophy to form vs. content. I think the semantic meaning of something should not be locked into its resultant form. This is why I love Scrivener's compiler as a general concept, and why I love the elegance of HTML (structure+semantics) + CSS (flexible form). So I much prefer "Emphasis" as a semantic container, and the compiler normally assigns italic, but it could also make it small caps, change its colour etc.). Keith made sure we could assign ⌘b and ⌘i to toggle Strong and Emphasis character styles (that make text look bold and italic in the editor), and so there is no real downside[2] to moving to use Style-driven-rich-text only to make text look a certain way in the editor, but utterly discard it during compile.



----
[1] I and several others nevertheless asked if we could have the flexibility of choosing to convert rich text without escaping, and Keith allowed this workflow in the V3.1.1 update... I personally won't use it, but this flexibly solves workflow problems some other users had (i.e. say you want to use TeX maths yet still use vanilla bold/italic to drive emphasis and strong).
[2] Unless you have a workflow that combines Markdown for some output formats but the Compiler to preserve in-editor styled text for other (non-markdown) outputs. But remember many of Scrivener's compile formats still override all rich text during compile anyway...

en
enricoscarpella
Posts: 19
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Wed Mar 13, 2019 7:17 pm Post

Thank you very much for the clarification, nontroppo.

For the most part, separating form from content works from me too, but there are a few exceptions in my field in which specific content is conveyed through specific form; for example, HOBBIT is a wild-type protein, hobbit is a mutant form of the protein, HOBBIT is a wild-type gene and hobbit is a mutant form of the gene.

Those exception notwithstanding, from your words it seems to me that nothing should prevent me from going with either options: letting Scrivener convert my rich text into Multimarkdown or letting Scrivener convert my (custom) styles to whatever I specify in the Styles pane of the Compile Format Designer, which seems to me an additional reason to move all my long-form writing to Scrivener as quickly as possible!

Thank you very much again for all your time and support!

Best regards,
Enrico

User avatar
nontroppo
Posts: 1116
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Thu Mar 14, 2019 12:27 am Post

Actually, that example is a nice use of semantic styles. You can create "WT Protein", "M Protein", "WT Gene" and "M Gene" character styles. Each style uses normal case/small caps/italic to visualise each type in the editor (use a font that has real smallcaps, assign using the typography panel, looks better in the editor). In the Compiler, you convert using:

WT Protein prefix: "[" | suffix: "]{.smallcaps}"
M Protein prefix: "" | suffix" ""
WT Gene prefix: "*[" | suffix: "]{.smallcaps}*"
M Gene prefix: "*" | suffix: "*"

This means you can use meaningful semantic styles that visually follow the conventions and work in both the Editor and the Compiler, with a flexibility to change this mapping in the future (though I doubt such meaning>form conventions will change in this case!)

en
enricoscarpella
Posts: 19
Joined: Fri Sep 14, 2018 9:46 pm
Platform: Mac + iOS

Thu Mar 14, 2019 4:44 pm Post

Thank you, nontropppo, for the suggestion. I was more referring to the fact that though I share the guiding principle of separating content from form, sometimes content conveys ambiguous meaning and that ambiguity is resolved through the form by which that content is expressed. But I do appreciate you relating this tangential observation of mine, to Scrivener styles — after all, that's what members of this Forum come here for: not to learn about my questionable opinions, so thank you for keeping me in check.

On a related note, I had some questions on the Workflow.scriv project you kindly shared with me through a different post. I was wondering whether it would be better to ask them here, there or in a new post. Of course, please feel free to ignore any of my questions: you have already been exceedingly generous with your time and have already helped me a lot! Thank you for your consideration.

Best regards,
Enrico

User avatar
nontroppo
Posts: 1116
Joined: Mon Mar 05, 2007 5:22 pm
Platform: Mac
Location: Airstrip One

Wed Apr 03, 2019 4:08 am Post

You can create a new post if you think it is a separate topic, or use the other thread otherwise 8)