Yup, this advice extends to all formats that use “cohesive bundles of files” for their data storage. DEVONthink and VoodooPad are two others that have known issues with the technology, not to mention Ulysses, which has a similar mechanism internally as Scrivener.
The best thing to do is check your file formats. Right-click on those saved files and if they say “Show package contents”, chances are you shouldn’t be storing it as a live file on Dropbox. There are certainly exceptions (applications being the main one), but to err on the side of caution, that’s a good place to start.
How about storing the .scriv package inside a disk image (sparse or regular) in the dropbox folder? If you happen to open the same image on 2 computers, or forget to eject the before leaving the computer, it simply creates a copy of the image (with a new name), so you have a record and can resolve the conflicts manually. It seems that the image file is not updated until the disk is unmounted, so you wouldn’t want to work off an image all day without unmounting it periodically so that your edits are saved to disk.
It should work for bigger disk images too, because dropbox only uploads the part of the image that has changed (dropbox.com/help/8), so I assume you wouldn’t have to upload the entire image if you make a small change. I wouldn’t want to use a sparsebundle, since that appears to be a package. You’d want to keep the disk image small if Time Machine storage is an issue.
Of course, this will make the files inaccessible to iPad/iPhone, but it should be OK for syncing between multiple macs.
Disk images are pretty stable, though some people worry about image corruption. But the good thing is that you’ll KNOW when a disk image is corrupted, because it won’t open. If that happens, you just go back to a previous version. That’s better than losing bits here and there without ever noticing, in my opinion.
there are a couple of things that make a disk image attractive. I am too lazy to put an exhaustive answer together, but imagine having non-scriv assets all packaged together. Think about a person writing a book about development in cocoa. Source code and executables all on the one disk image with the .scriv.
It is better because there are less steps. I find the zip method to be somewhat tedious, especially if I’m working in a number of different scrivener files or just want to add a quick note in one of my projects.
Steps for using a zip file:
open dropbox, find most recent zip file
copy zip file to desktop
try to find file amid desktop clutter
double-click on scrivener file to open project in Scrivener
write
make new zip file
copy zip file to dropbox
delete scrivener project file from desktop
Steps for using a disk image:
click alias of scrivener project in dock (alias points to a Scrivener project file inside the disk image); the disk image automatically mounts and your project opens in Scrivener).
write
eject disk image
If you use a dock alias, it’s just one click and you’re writing. Then one click and a keystroke to eject the disk when you’re done. Just one more step than keeping a Scrivener project on your hard drive.
The only thing to watch for: don’t want to use bundle images on Dropbox for the same reason you don’t want to use Scrivener projects on Dropbox. This format of disk images have an allocated upper size limit, but only use what space they actually require, in storage. So you can have a 5gb sparse disk image, but if it only has 20kb in it, the actual disk size will be closer to the latter. Apple’s format will create new sub-file entries in this bundle as it needs to, and therein lies the problem. Since it is a cohesive bundled format, like a .scriv project, Dropbox will introduce mayhem if its internal parts ever become conflicted.
Using regular disk images (which use all of their allocated space), or sparse images (which do not, but come with an overhead that might make them larger than regular images if they are small), should be safe.
There would definitely be advantages to working this way, as zip files require you to extract a copy of the project, while a disk image would let you work on a single copy. That is also a dis-advantage from a backup standpoint. Transferring via datestamped zip files gives you an implied sequence of backups. Editing a single project in place does not. Pick your poison, but if you choose to edit in place, make sure you keep external backups. In other words, this doesn’t really alleviate the workload of creating zipped projects, unless you choose to work without backups and favour convenience over security.
Otherwise, I can think of no reason why this would be risky. All dropbox sees is the string of binary information that the disk image is. As you work in Scrivener, this string of binary code gets modified, and Dropbox alters its image of that file appropriately. It is wholly unaware of what you are doing inside the image, and so will not cause the sort of problems that having a “bare” .scriv file would.
In short: Good idea! Just make sure you couple it with good backups (as always)
Thanks, Ioa. That sounds great, but are you sure the sparse image is a bundled format? I was under the impression that a .sparse image is one giant file, while the .sparsebundle image is a package.
Either way, I don’t need to use a sparse image at the moment — I’m only using a tiny image and at this scale sparse seems to use more overhead space than a regular image. That’s not an issue for DropBox with its binary diff method of syncing, but it might be an issue for Time Machine. BTW, I do use Time Machine (and Crashplan), plus DropBox keeps recent versions up to 30 days, I think. And those are all automated backups, so backing up doesn’t impact the workflow at all… I love technology.
Yes, sparse bundle is what I was thinking of, I forgot about sparse images being a separate type. Thanks for that, I’ve updated the original post!
P.S. And in 2.0 you’ll be able to add Scrivener’s own automatic backups to your list. By default it stores them in your Application Support folder, but you can point it anywhere. I’ve pointed mine to Dropbox, and turned the zip option on, which means every time I force save or close a project, I get duplicated backups spawned to all of my computers, and their servers, too.
Seriously? In this day and age we’re still having issues with syncing to the cloud?
I was actually thinking about buying this app but if it’s going to be more of a headache working on two machines then I hardly find it worth the time.
When is this going to be fixed? Or is it not going to be fixed?
Seems the Dropbox part could be solved by simply implementing the Dropbox API in the app. Since it’s free it seems like a nobrainer option for those working on multiple machines.
It doesn’t quite work like that. This is an issue with any application that saves to folders rather than a single file because of Dropbox’s renaming policy, and the API, which is for interacting with Dropbox accounts for mobile apps, has no bearing on that. So there’s nothing to be “fixed” here. For Scrivener to have the features it has - the importing of any research file - it requires a package file format; and Dropbox doesn’t work well with package file formats.
This does not mean that you cannot use Scrivener successfully with Dropbox - plenty of users do so, and I keep backups of all my projects on Dropbox so that if I need to I can access them anywhere. It just means that you need to take a little extra care, that’s all.
I don’t think you’d actually like that as much as you think you would—even if it were feasible. The thing that makes Dropbox cool is that it uses your hard disk as a parity cache and backgrounds the network activity. Switching to a network transfer API would introduce all of the reasons why we never really used Internet disks for serious large-scale real-time work in the first place. Would you actually tolerate having to sit there for 15 minutes every time you opened your 100mb PDF reference file, or wait 3 hours for your project to save, or 5 minutes just to load a project while local cache files were compared with network datestamps?
What I’ve read in the past led me to believe that work wouldn’t quite exactly be lost through dropbox. As I understand it, what in fact happens is something like this: when dropbox sees a potential sychronization conflict between two versions of a file it renames one of them (e.g. to something like “3 monday-oct-25th-version.rtf”). When those files are located within a .scriv package, scrivener doesn’t know to look for a file that has a title different from the title that scrivener originally gave it.
But the file continues to exist (albeit invisible to Scrivener, which only looks for files with names like “3.rtf”) within the package. So if you’re looking for your lost work, you just have to right-click and select “show package contents…” —and if you’re wondering if you’ve ever had lost work, you just have to do that and look for files with funny names like that.
The mechanics aren’t that important, really. Is my conclusion (dropbox-lost work isn’t really lost; it’s just invisible to scrivener, hiding in files inside the .scriv project) correct? If not, that would affect my risk-reward calculation here.
Yes, the worst that usually happens is that Dropbox renames some files inside the .scriv package “behind Scrivener’s back”, so fixing it is usually as easy as drilling into the .scriv file and changing the names back. Scrivener 2.0 is better at looking for the most recent binder file anyway - which is the most frequent problem - but that won’t prevent problems if Dropbox renames other files too. Most Dropbox issues are an easy fix and we really need to put something on our wiki page about it when we get chance to breathe, but it would be remiss of us to say that you can’t lose data, because there is a minor risk involved still. I’m sure Ioa will be able to say more as he’s a bit of an expert on these issues.
Yes the Dropbox data loss risk is more minimal than with MobileMe—keep in mind this post combines all network utilities to avoid confusing the issue. If misused badly, I can see some scenarios where stuff gets lost with Dropbox—but these would be good things to avoid no matter what you are using, like not opening the project twice and actively working on it from two different machines.
With MobileMe, yes, I have seen entire projects wiped off the face of the earth—nothing left, not even a search index. I’m not even sure how it happens, but I’ve seen more than one case. So since this includes both, the wording is more strict that it necessarily needs to be with Dropbox all by itself.
2.0 is going to be more resilient to the sorts of stunts DB pulls, too. Probably not enough to remove it from the list, but definitely enough to decrease that risk side of the equation a bit more.
Thanks for the quick reply! Of course it would be foolish to minimize the dropbox risk—but for those of us willing to take our fates into our own hands, knowledge of the risk helps us to make informed decisions.
As for mobileme killing projects entirely, hurp.
(FWIW, I only view this as an acceptable risk because I (a) only use dropbox on one computer (well, and one iphone), and (b) I set auto-save to 120 seconds.)
For what it’s worth, I’ve been keeping my primary Scrivener projects on Dropbox for the better part of two years, and I’ve only had a sync conflict once. I used to have considerably more trouble keeping my Scrivener files on iDisk.
(And I should note that one Dropbox sync issue was entirely my own fault; I went and wrote on the MacBook Pro at a coffeeshop that had no WiFi, came home and didn’t open the laptop to let Dropbox sync before trying to open the file on the Mac Pro. Oops…)
That’s probably the number one mistake with Dropbox: forgetting to let your computer sync up before you go and open a project and edit it. Fortunately it can be relatively easy to fix this using DB’s version roll-back features.
I’ve been using Scrivener with dropbox since (my) day 1 and never had an issue, but after reading this thread (and other mentions of the issue) I have just spent a few hours moving everything out of my dropbox folder and only using dropbox as a backup.zip destination.
I would much rather keep my live files in dropbox and not just the backups (Which could be half a day or more out of date if I don’t cmd-s and wait the 40+ secs for the backup to run). The backups take time to run, and the single zip files take a little while to upload to dropbox.
I’m not editing the project on several machines, I was just using dropbox as a near-realtime “backup” location to supplement mozy + timemachine + manual copies to my NAS.
Assuming dropbox doesn’t break and start syncing bad files (Never happened to me). Can I store my scrivener projects in a dropbox folder, and “normally” expect them to be intact, in-date and readable?
My experience said I could, but I’d really appreciate a “definitive” answer to this, as there does appear to be a bit of a mixed message as to what you should be able to do around waiting for it to sync etc.
I guess the answer is how much risk are you willing to take. I was, I believe, one of the first to say that I had lost work through keeping the live project file on DropBox and working on two computers, an MBA that I was carrying around and an MBP which was my “portable desktop”. Currently, I’m working only with the 17" MBP and lugging that around — even using a back-pack my aged frame is suffering! People here just don’t believe how much by back-pack weighs …
You say you only use one computer for working on Scrivener. I think issues you must ask yourself are: (1) what is your machine … for instance, if it’s an iMac or whatever that sits in the same location, the risk is much lower than if you use an MB/MBA/MBP and carry it around and use it in different locations with and without WiFi; (2) if you’re using a non-moving computer, do you leave it on and connected to DropBox all the time … if you do, the risk is lower as you don’t have to remember to wait for Dropbox to sync before you switch it off and walk away.
All that said, I realised only today, that for, I believe, the whole of this year or nearly — ever since the administrators of the Great Firewall of China decided that Dropbox was a danger to social stability — I have been storing a couple at least of my live projects in folders that are set to sync automatically by a similar, though Linux originated and UI-style, cloud system called SpiderOak. In SpiderOak you don’t have a single dedicated sync folder like Dropbox, but you can set up any folders you wish on your machine to sync automatically … an advantage in some ways, but that is the root of how I ended up with live projects in sync’d folders. For 6 months I was using it between the MBP and the MBA, but I have not had any problems. Since realising this, I am having to decide whether to continue, or change my system.
I think I’ll look into the auto-back-up system in Scrivener 2, and maybe keep the live projects in their current locations but have an automatic back-up set to a different, non-sync’d location on my internal hard disk … plus other back-ups of course.
I guess, if you want to keep your live project on Dropbox, you should be sort of Irish about it: Back-up early and back-up often.
Good luck!
Mark
I can only support what Mark has written above – and add that the automatic back-up in Version 2 is pretty good for your purposes, Jenny. You can choose your Dropbox folder as the recipient of your zipped back-ups, select the number of back-ups you want to keep, and check the points at which you want the back-ups to take place. Then set and forget.
)
Pick your poison, but if you choose to edit in place, make sure you keep external backups. In other words, this doesn’t really alleviate the workload of creating zipped projects, unless you choose to work without backups and favour convenience over security.