Heading to graph databases

[Arkhaiel]

Hello there !
To respect the "be polite" rule, I just want you to know I use Scrivener for at least ten years, and your amazing software has been my daily companion since we first met. I love Scrivener very much, and using it is a great pleasure, every day ! So, a big thank to you sir ! (and to all the L&L team ofc).
The version 3 has been a huge update, and made its use more and more simple AND powerful, which is an amazing thing in the software development.

My wish of the day could be for the v4, if it implies huge core modifications, but I think that would improve Scrivener A LOT : thinking the core of Scrivener, the behind-the-scenes, the documents and the intra-Scriv links, as a graph database. Each document is linked to other documents through objectified relationships.

• each document seen as an object, adding a new metadata to identify it clearly
• each document can be linked to any other document thanks to as an objectified relationship, and the relationships can be created in the project settings
• you can show a "dynamic" view of the whole graph, and as the corkboard, the scrivenings and the outliner, the "graph" view can be shown to selected documents only (may need to choose between "include" or not the documents linked to the selected documents in the graph)
• templates and objects could be merged (next part of my wish)

And, to avoid making a second post, because I think these two ideas can be related : reforging the templates with two parts :
1st part : a "real" template, with placeholders where to display some variables, mainly some basic tweaks needed because documents are most of the time plain text, sometimes with styles
2nd part : some kind of form, where we note the values of the variables used in the template, so the variables used can be involved in filters and collections

Of course, the template should need a "free text zone" in which the user writes everything (s)he wants, and each instance of the template can also be filled freely, and the placeholders can be modified.
This second whish would allow documents to be objectified, and relationships exposed in the first wish could also be some "special" documents to link two documents.

I'm not sure about the "objectified" word, but in my french brain it's the best work I found to say "make like an object", and of course I talk about object in a development semantic field.

Well this 2 part wish could allow even more flexibility in Scrivener, allowing users to really work on some freeform organization. Thanks for reading me, thanks again for Scrivener, and long live this software !

Cheers from France !

[kewms]

Thank you for your suggestion.

I think the application you are describing would no longer be Scrivener, but something else.

Scrivener is intended to help writers complete long works. It is not intended to be a general purpose database.

Katherine

[Arkhaiel]

All the things I describe in my post would of course be implemented in the backend, actually there would be quite few changes in the frontend, except an optimized version of the freeform corkboard and better links between documents.
Thanks for you answer anyway !

[kewms]

If the change is mostly invisible, then what benefit to the user justifies essentially rewriting the back end?

Katherine

[Arkhaiel]

I'm not arguing to prove my wish of the day is worth the effort, if the dev thinks it's a good idea maybe he will implement it, if he thinks it's not he won't
I don't know how Scrivener is coded, maybe the changes I talked about in my wish are not necessary to implement better relationships between documents and better templating.

edit : tu summarize,
- that would be cool to be able to make different relationships between documents
- that would be cool to be able to display these relationships in a freeform corkboard-like view
- the template system is imo underexploited, and it could be interesting to create forms to fill templates, like real templating system

[devinganger]

I'm not arguing to prove my wish of the day is worth the effort, if the dev thinks it's a good idea maybe he will implement it, if he thinks it's not he won't

Be aware that up to this point, the person discussing this with you is long-term L&L staff and probably has a much better idea than you and I put together what KB's thoughts and preferences are when it comes to the future of Scrivener.

I, however, am just a random user!

One of the things I really like about how Scrivener's file format and back end are implemented right now is that it relies on commonly available data formats, like RTF, XML, and plain text. Although a Scrivener project is made up of a large number of interrelated files, I have access to free RTF and text editors on both Mac OS and Windows so that if something happened to Scrivener (or God forbid L&L) tomorrow, I can still get my data out of the Scrivener project and transfer it to some other program of choice. Depending on how much I broken down my project into separate documents, it might take some effort -- but all of my data is there and still accessible without Scrivener or separate expensive tools. KB (the developer) has stated that this accessibility is one of the design objectives for Scrivener.

If Scrivener moved to a graph database format (or even any database format) for its projects on the backend, what would that recovery scenario then look like? Would this make things more locked in to a proprietary format that required special tools and higher-than-average levels of computer skills, or would the average writing user still be able to recover their data if they had to?

[Arkhaiel]

I'm not arguing to prove my wish of the day is worth the effort, if the dev thinks it's a good idea maybe he will implement it, if he thinks it's not he won't

Be aware that up to this point, the person discussing this with you is long-term L&L staff and probably has a much better idea than you and I put together what KB's thoughts and preferences are when it comes to the future of Scrivener.

WoW, didn’t say "I’m not arguing WITH YOU", I just said I won’t enter a debate to prove anything, it’s just a wish on a wish list, and I summarised to get rid of the unnecessary graph and tech thing.
I’m not the dev nor the team, if they don’t want to keep my wish my life will be the same, not a big deal actually oO

[devinganger]

WoW, didn’t say "I’m not arguing WITH YOU", I just said I won’t enter a debate to prove anything, it’s just a wish on a wish list

Which is on a *community forum* with an active community. Don't take it as an attack or an attempt to make you prove something -- we discuss each others' ideas *all the time.* This isn't an argument This is a recognition that good ideas can come from anyone and anywhere, and there isn't a good idea so good that it can't be improved through feedback and discussion.

[gr]

@ Arkhaiel,

A)

If I understand correctly you are imagining adding to the free-form corkboard the ability to make connections between the cards. What you are describing is literally what L&L’s Scapple software does! This means the developer also thought this ability would be useful. But they also decided it should be separate from Scrivener. If you search on these Forums you will find others who wished that Scapple could be integrated with Scrivener, and you will also find the developers response and reasons against doing that.

B)
I don’t think I understand clearly what you second wish is. Is the idea something like: project template with placeholder text in various places and then a designated place to go to specify the replacement values for those placeholders for the particular project? (Something like the Replacement area in the third pane of the Compile dialog box, but built in to the project instead.)

[Arkhaiel]

WoW, didn’t say "I’m not arguing WITH YOU", I just said I won’t enter a debate to prove anything, it’s just a wish on a wish list

Which is on a *community forum* with an active community. Don't take it as an attack or an attempt to make you prove something -- we discuss each others' ideas *all the time.* This isn't an argument This is a recognition that good ideas can come from anyone and anywhere, and there isn't a good idea so good that it can't be improved through feedback and discussion.

Of course, I completely get it.

@ Arkhaiel,

A)

If I understand correctly you are imagining adding to the free-form corkboard the ability to make connections between the cards. What you are describing is literally what L&L’s Scapple software does! This means the developer also thought this ability would be useful. But they also decided it should be separate from Scrivener. If you search on these Forums you will find others who wished that Scapple could be integrated with Scrivener, and you will also find the developers response and reasons against doing that.

I understand your point, and I think my wish implies more things than just a visual organization. That could also allow you to make searches, or collections, based on these relationships. But I can't explain here what does the graph databases allow to do, not in english between two answers. You could see in this video ( https://www.youtube.com/watch?v=bL7wBbk0PIs ) a short presentation. Scrivener actually should work this way imo, since it encourages to split documents, and make connections between them. What's not in the video is the fact that different types of relationships can be used, so you can easily make very powerful queries.

B)
I don’t think I understand clearly what you second wish is. Is the idea something like: project template with placeholder text in various places and then a designated place to go to specify the replacement values for those placeholders for the particular project? (Something like the Replacement area in the third pane of the Compile dialog box, but built in to the project instead.)

Damn, the language barrier is sometimes hard to handle ^^
I think the templating feature can be way better. In web dev, for example, the templates systems separate the content from the form. This way, you can modify THE template and it will automatically modify the way ALL your pages look. Considering that, if you could make a solid templating system where some key informations (name, physical description, introduction scene...) are stored in metadata (or in another place), and displayed in a document, you could then be able to modify the "template" to update the way the document is displayed.
No template ? All the "template metadata" are displayed basically in a blank document, one per line.
Switch to template A ? If you defined "$NAME", "$DESCRIPTION", "$INTRO SCENE", into a well designed template A, then the document now is "template A + template metadata".
Switch to template B ? Juste change the "template" source and the whole document is now changed, keeping the templates metadata into the well designed template B.
Back to template A ? Just change the "template" source.
Want to modify template A ? Juste modify the "Template A" file, and all the document based on this template are now changed.

You can see that as a dynamic templating system, which would refresh each time is needed, and display in realtime the datas "template model + template metadata". You can also modify the template metadata whenever you want, of course.
And, in the template, you can define "free zones" where you can write whatever you want. Free zones are not related to "template metadata", it's just zones where you write some longer text, like a short bio, to keep the characters template example.