Maximum file size of a project?

Th
The.Creech
Posts: 3
Joined: Mon Dec 26, 2016 10:27 pm
Platform: Mac + iOS

Sat Jan 13, 2018 11:51 am Post

Hi everybody,

What is your experience with a maximal file size of a Scrivener Project?

I work with scanned hand written index cards - thus the file size of my Scrivener project increases over time. I wonder where Scrivener will stop me...

So, let's see what file sizes are around - I put a 3 GB in the round...

Hu
Hugh
Posts: 2417
Joined: Thu Mar 08, 2007 12:05 pm
Platform: Mac
Location: UK

Sat Jan 13, 2018 3:03 pm Post

In theory I suspect that there may be a maximum technical size for a Scrivener project, but I don't know what it is. (There have been discussions of this question in these forums in the past in which - if I remember correctly - L&L people participated, so a search should reveal them.)

In practice (as opposed to theory) with some other writing tools, saving to disk poses a bottleneck which can slow down use and provide a practical limit to size. Scrivener is different from most. You'll notice that in my first sentence I used the word "project", not "file". You may not be aware that a Scrivener project can be composed of hundreds or even thousands of files. This helps to protect the project from corruption, and it also means that only a few of the files in the project are loaded into RAM memory at any one time. So in practical (as opposed to theoretical) terms, as far as I know you should be unlikely to run into a limit when the project is being saved to disk in normal use.

There are two exceptions - that I know of - to this guidance. One is that if you have fewer, larger documents (and so files), that will (logically) slow up saving to disk. For the same project size, more, smaller documents are better, from this point of view.

The second is backing up. Backups are the only times when a complete project is saved to disk in one go. If, say, you have set your project to "backup on project close" (in Scrivener > Preferences > Backup), you may find that your 3 GB takes a while to be backed up, and your project takes a while to close.

A final thought: you are taking a risk by putting all your eggs in such a large basket. Although Scrivener has been built to help safeguard projects against corruption, no digital tool is 100 per cent proof against it. In this respect, two or more smaller projects would be safer than a single very large one.
'Listen, some quiet night, when you've shirked your work that day. Do you hear
that distant, almost inaudible clicking sound? That's one of your
competitors, working away in the night in
Paris or London or Erie, PA.'

User avatar
kewms
Posts: 3749
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Sat Jan 13, 2018 4:45 pm Post

To Hugh's thoughts, I would only add that OCR tools have gotten pretty good. It might be possible to digitize many of those index cards, which would dramatically reduce the space needed.

Katherine
Scrivener Support Team

Th
The.Creech
Posts: 3
Joined: Mon Dec 26, 2016 10:27 pm
Platform: Mac + iOS

Sat Jan 13, 2018 4:55 pm Post

Thanks a lot for your detailed answer and view, Hugh.

Your thoughts about the RAM usage was what I was looking for. The thoughts about the saving time/bottle neck with large files was already clear. So, Scrivener files are organized like any other folder storage, one can say (I think) - that helps a lot.

I got all my single files separately as copies on an external drive additionally, so I'm save. But to have them one click away within Scrivener makes it much more comfortable...

Thanks again for your thoughts
Last edited by The.Creech on Sat Jan 13, 2018 5:01 pm, edited 1 time in total.

Th
The.Creech
Posts: 3
Joined: Mon Dec 26, 2016 10:27 pm
Platform: Mac + iOS

Sat Jan 13, 2018 5:00 pm Post

Also thanks Katherine, OCR wouldn't match with my fast running scrawl, but would be an good advice when I could discipline myself to write slower. But then my thoughts would be boiled away before I could write them down.

I will challenge Scrivener on the file size to figure out how it can match with my special workflow...

Hu
Hugh
Posts: 2417
Joined: Thu Mar 08, 2007 12:05 pm
Platform: Mac
Location: UK

Sat Jan 13, 2018 5:12 pm Post

The.Creech wrote:Thanks a lot for your detailed answer and view, Hugh.

Your thoughts about the RAM usage was what I was looking for. The thoughts about the saving time/bottle neck with large files was already clear. So, Scrivener files are organized like any other folder storage, one can say (I think) - that helps a lot.



Well, yes and no. They are files in a folder, but all of them are indexed, and many are intricately linked together, in order to enable Scrivener's various attributes as a writing tool.

The.Creech wrote:Thanks again for your thoughts



You're welcome.
'Listen, some quiet night, when you've shirked your work that day. Do you hear
that distant, almost inaudible clicking sound? That's one of your
competitors, working away in the night in
Paris or London or Erie, PA.'

Ra
Raederle
Posts: 1
Joined: Mon Mar 05, 2018 10:20 pm
Platform: Windows

Mon Mar 05, 2018 10:24 pm Post

My largest project has roughly 3500 files and 700 MB. I find that this project takes significantly longer to save and load than my other projects and sometimes will say "not responding" and need to sit for a minute to sort itself out. My computer is fairly up to date (2016) and plays video games without an issue, so I'm sure this is Scrivener's capacity that is causing the delay.
To Scrivener's credit – it successfully accepted an import of 3000 files at once! It took it around an hour to import.
~ Raederle
www.Raederle.com

User avatar
kewms
Posts: 3749
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Tue Mar 06, 2018 3:54 am Post

Having 3500 files is going to slow down any indexing operation, since you'll have 3500 open/read/close operations. A project with the same overall size but fewer component files would probably be zippier.

Katherine
Scrivener Support Team