Scrivener and OneDrive's SaveSpace option.

St
StarDog2
Posts: 11
Joined: Wed Oct 10, 2018 2:30 am
Platform: Windows

Sun Nov 04, 2018 2:57 pm Post

This really applies to just about any software that you want to interface with OneDrive, particularly with the "Save Space" option. In it, Your local windows hard drive maintains the file name and folder structure locally, BUT OneDrive actually stores the files itself. With this option, you access your local folder just like always, but Win10 and OneDrive are smart enough to know to download the file from OneDrive. Response time is a tad slower than a local hard drive storage, but not too bad if your internet connection is reasonably fast.

I have learned (the hard way) to never interrupt the download/upload cycle that OneDrive Uses even if you think there is no good reason for OneDrive to be accessing files at that moment. It is doing the same sort of maintenance that your HD does that you never see.

However, it does mean you are, as has always been true, vulnerable to unfortunately timed power outages. For this reason, I make sure the backups are made to a LOCAL external drive (i.e. NOT 'C:', and probably NOT 'D:'). I would not use the "Save Space" option on any folders that contain data from programs that don't have an auto backup option.

I'm quite happy with this choice, and I have the save space option set up on both my laptop and desktop computers, and I am able to seamlessly work on my writing projects on either computer at will.

User avatar
kewms
Posts: 4063
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Sun Nov 04, 2018 4:25 pm Post

As a rule, we do not recommend the "Save Space" (or similar) option for any cloud service. Scrivener expects that your entire project will be stored locally and instantly accessible. If it is not, it is quite common to attempt to load a file and find that the contents are "missing," because the cloud service hasn't downloaded it to the local system yet.

Needless to say, many users find this distressing, leading to support queries and the (false) fear that "Scrivener deleted my work!"

Katherine
Scrivener Support Team

St
StarDog2
Posts: 11
Joined: Wed Oct 10, 2018 2:30 am
Platform: Windows

Mon Nov 05, 2018 2:34 pm Post

kewms wrote:As a rule, we do not recommend the "Save Space" (or similar) option for any cloud service. Scrivener expects that your entire project will be stored locally and instantly accessible. If it is not, it is quite common to attempt to load a file and find that the contents are "missing," because the cloud service hasn't downloaded it to the local system yet.

Needless to say, many users find this distressing, leading to support queries and the (false) fear that "Scrivener deleted my work!"

Katherine


Your statements are ABSOLUTELY correct, particularly so for the technically challenged.

However, I personally will continue to use OneDrive Save Space option as I feel that the convenience of sharing the space with both my Desktop and Laptop are too convenient to ignore. And, as a retired DBA, I know enough about data management to understand what you are saying in detail. I am also comfortable in my backup procedures. Also, I know enough about product support to never submit a "Scrivener Ate My Data" claim without some pretty serious supporting documentation, and a willingness to work with tech support to resolve such an issue.

My goal wasn't to convince others to use "Save Space" so much as it was to establish guidelines on how to use it, and the basic precautions one needs to take, on the assumption that I am not the only one to discover this possibility. People WILL experiment in this way, and they need to learn from the mistakes of others.

It seems that pretty much all creative writing software vendors take your position, and it is one I would take as well if I were a vendor. (I would guess that the hierarchical nature of this type of software, with lots of small-ish files, poses a bit of a problem for this type of software in "the cloud".) But 'the cloud', and data portability will become an increasingly important function to both the technically minded and the technically challenged, and not just to writers and writing software.

I've re-read this post, repeatedly, and I'm not sure if I come off as the south end of a northbound horse, or not. I apologize in advance if I have presented myself that way. That was not my intention!

User avatar
kewms
Posts: 4063
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Mon Nov 05, 2018 4:36 pm Post

StarDog2 wrote: It seems that pretty much all creative writing software vendors take your position, and it is one I would take as well if I were a vendor. (I would guess that the hierarchical nature of this type of software, with lots of small-ish files, poses a bit of a problem for this type of software in "the cloud".) But 'the cloud', and data portability will become an increasingly important function to both the technically minded and the technically challenged, and not just to writers and writing software.


The fundamental problem is that the internet is not and never will be as fast or reliable as a local hard drive. So the potential for slow or failed connections between locally running software and its remotely stored data will always exist.

I understand that you know what you're doing, and also know how to figure out the cause of this kind of error. My comments are intended primarily for less knowledgeable users who might come across this thread.

Katherine
Scrivener Support Team

User avatar
AmberV
Posts: 22315
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Santiago de Compostela, Galiza
Contact:

Mon Nov 05, 2018 7:53 pm Post

Have you confirmed that your project has been “space saved” yet, and if so, how are your backups? With Scrivener basically just zipping the project folder, if 98% of the folder is has been deleted from the disk all along, then wouldn’t the only backups of it be 2% of the data? That’s one of my main concerns—the thought of sync software making a mess in an active project is one thing, but it’s a thing we can relatively confidently work around the risk of because of full-project backups.

Ideally OneDrive is smart enough to download dependencies in a folder that is being zipped, and in that case the routine backups may if anything ensure that the project remains intact on account of frequent “usage”.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
devinganger
Posts: 1199
Joined: Sat Nov 06, 2010 1:55 pm
Platform: Mac, Win + iOS
Location: Monroe, WA 98272 (CN97au)
Contact:

Tue Nov 06, 2018 10:57 am Post

Where modern versions of Windows 10 and OneDrive allow you to selectively pick which folders and files to keep a local copy of, I would (if I were using OneDrive as my primary project storage) make sure I had an Active Projects folder that was set to always keep the local copy, and an Archived Projects folder that was set to reclaim local space. Move projects between the two as needed and wait a few moments for OneDrive to fully sync. That way, I don't have to worry about whether I have an Internet connection, if the Internet connection is fast enough, etc. to work on the projects I am actively working on.

Then again, this is pretty much the way I order all of my data regardless of program or sync service these days.
--
Devin L. Ganger, WA7DLG
Not a L&L employee; opinions are those of my cat
Winner "Best in Class", 2018 My First Supervillain Photo Shoot

St
StarDog2
Posts: 11
Joined: Wed Oct 10, 2018 2:30 am
Platform: Windows

Tue Nov 06, 2018 4:03 pm Post

AmberV wrote:Have you confirmed that your project has been “space saved” yet, and if so, how are your backups? With Scrivener basically just zipping the project folder, if 98% of the folder is has been deleted from the disk all along, then wouldn’t the only backups of it be 2% of the data? That’s one of my main concerns—the thought of sync software making a mess in an active project is one thing, but it’s a thing we can relatively confidently work around the risk of because of full-project backups.

Ideally OneDrive is smart enough to download dependencies in a folder that is being zipped, and in that case the routine backups may if anything ensure that the project remains intact on account of frequent “usage”.


Oh yes, I've used this setup on both my Laptop and my Desktop both synced to the same Onedrive account. The only problems I had was when I actively interrupted some download/upload activity. You've got to trust OneDrive to do its thing.

Basically, what happens is your writing software (i.e. Scrivener, Atomic Scribbler, etc.) saves to the normal storage location you have set up for it, and if you have told OneDrive to monitor that location (folder), OneDrive moves the file(s) just saved to your OneDrive Account, leaving an empty file with the same name and a little cloud next to it. That tells you the file has been moved to OneDrive. You access the project the same way, you tell your writing software to retrieve your project, and it goes to its NORMAL project storage location and loads the project. It appears to Scrivener (or, whatever) that the project is loading from that normal storage location, but what is really happening is that the project is being loaded from OneDrive. So far, it has been working great.

Obviously, if your internet service is unreliable, then you would not want to do something like this. Or at the very least, make sure you understand how your back up and restore procedures work, and you have TESTED the backup and restore. (That last is good advice for anyone, actually).

As kewms correctly pointed out, there is an inherent risk in using your software in this manner at this time. I am willing to take the risk because I find the convenience too great to ignore, and I know how to recover from almost any sort of catastrophic loss. (Backup, Backup, Backup, then back up the Backup)

However, I believe that in time, cloud-based storage will become the default saving option for all software, and creative writing applications will eventually have to address this issue. It isn't particularly difficult though it does take valuable development & testing time away from other work. And small shops will be stressed more than the larger shops.

But Microsoft has taken on a lot of the work for the developers, and I suspect eventually the other online storage vendors will do something similar. It is too convenient for everyone to ignore.
Last edited by StarDog2 on Tue Nov 06, 2018 4:14 pm, edited 2 times in total.

St
StarDog2
Posts: 11
Joined: Wed Oct 10, 2018 2:30 am
Platform: Windows

Tue Nov 06, 2018 4:04 pm Post

devinganger wrote:Where modern versions of Windows 10 and OneDrive allow you to selectively pick which folders and files to keep a local copy of, I would (if I were using OneDrive as my primary project storage) make sure I had an Active Projects folder that was set to always keep the local copy, and an Archived Projects folder that was set to reclaim local space. Move projects between the two as needed and wait a few moments for OneDrive to fully sync. That way, I don't have to worry about whether I have an Internet connection, if the Internet connection is fast enough, etc. to work on the projects I am actively working on.

Then again, this is pretty much the way I order all of my data regardless of program or sync service these days.


I can NOT fault your logic!

User avatar
kewms
Posts: 4063
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Tue Nov 06, 2018 4:42 pm Post

StarDog2 wrote:However, I believe that in time, cloud-based storage will become the default saving option for all software, and creative writing applications will eventually have to address this issue. It isn't particularly difficult though it does take valuable development & testing time away from other work. And small shops will be stressed more than the larger shops.


A point of clarification, though. The "save space" option is a step beyond merely synchronizing your project with a cloud service. The basic synchronization option keeps a copy of your project locally *and* a copy in the cloud. That's reasonably safe, provided you are alert to the possibility for synchronization conflicts. *Removing* the local copy is the specific practice that is especially problematic in my opinion.

As I said upthread, local storage will always be faster, cheaper, and more reliable than cloud storage. I can pay $100 per terabyte per year for OneDrive, or $50 for a 1TB drive that will sit on my desk for four years or more, and I don't see anything in the technology pipeline that will change that ratio. So I'm not as convinced as you are that universal cloud-only storage is the future.

Katherine
Scrivener Support Team

User avatar
devinganger
Posts: 1199
Joined: Sat Nov 06, 2010 1:55 pm
Platform: Mac, Win + iOS
Location: Monroe, WA 98272 (CN97au)
Contact:

Tue Nov 06, 2018 6:43 pm Post

kewms wrote:As I said upthread, local storage will always be faster, cheaper, and more reliable than cloud storage. I can pay $100 per terabyte per year for OneDrive, or $50 for a 1TB drive that will sit on my desk for four years or more, and I don't see anything in the technology pipeline that will change that ratio. So I'm not as convinced as you are that universal cloud-only storage is the future.


Especially true given the lack of ubiquitous, truly inexpensive, *reliable* connectivity to interact with said cloud storage. While cloud storage has come a long way and makes a lot of cool use cases possible, it is not the answer for all of them.
--
Devin L. Ganger, WA7DLG
Not a L&L employee; opinions are those of my cat
Winner "Best in Class", 2018 My First Supervillain Photo Shoot

St
StarDog2
Posts: 11
Joined: Wed Oct 10, 2018 2:30 am
Platform: Windows

Tue Nov 06, 2018 8:04 pm Post

devinganger wrote:
kewms wrote:As I said upthread, local storage will always be faster, cheaper, and more reliable than cloud storage. I can pay $100 per terabyte per year for OneDrive, or $50 for a 1TB drive that will sit on my desk for four years or more, and I don't see anything in the technology pipeline that will change that ratio. So I'm not as convinced as you are that universal cloud-only storage is the future.


Especially true given the lack of ubiquitous, truly inexpensive, *reliable* connectivity to interact with said cloud storage. While cloud storage has come a long way and makes a lot of cool use cases possible, it is not the answer for all of them.


That is true, I have several external drives of the sort you mention, they are a part of my backup/disaster recovery plan.. However, what I can't do with an external HD is continue to load and expand on my project from either Chicago or Texas and then continue seamlessly when I get home, without taking one more piece of gear with me. But inexpensive HDs are great for those who tend to write in one spot.

I also use this feature, with my ACDSee Photo Studio Ultimate 2019 software I can take photos anywhere in the world, and do intake, management, and even editing anytime I am near an internet service. For me, this has been a Godsend.

Again, I'm not trying to convince anyone to use this cloud storage feature, I'm just trying to explain my experiences with it if people are tempted to use it. Everyone has to judge the risk and rewards for themselves. I'm quite certain some will find it useful, and some will not. It's all good, in my eyes.

User avatar
kewms
Posts: 4063
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Tue Nov 06, 2018 10:41 pm Post

StarDog2 wrote: That is true, I have several external drives of the sort you mention, they are a part of my backup/disaster recovery plan.. However, what I can't do with an external HD is continue to load and expand on my project from either Chicago or Texas and then continue seamlessly when I get home, without taking one more piece of gear with me. But inexpensive HDs are great for those who tend to write in one spot.


Again, it should be possible to get the benefits of cloud storage without the risks of the "save space" option. If the service you're using doesn't allow you to keep a local copy as well, I'd recommend asking them why not.

Katherine
Scrivener Support Team

St
StarDog2
Posts: 11
Joined: Wed Oct 10, 2018 2:30 am
Platform: Windows

Wed Nov 07, 2018 1:41 am Post

kewms wrote:
StarDog2 wrote: That is true, I have several external drives of the sort you mention, they are a part of my backup/disaster recovery plan.. However, what I can't do with an external HD is continue to load and expand on my project from either Chicago or Texas and then continue seamlessly when I get home, without taking one more piece of gear with me. But inexpensive HDs are great for those who tend to write in one spot.


Again, it should be possible to get the benefits of cloud storage without the risks of the "save space" option. If the service you're using doesn't allow you to keep a local copy as well, I'd recommend asking them why not.

Katherine


They do, of course! "Save Space" is an OPTION that you actively have to set up.