Working off of network drives (MobileMe, thumb drives...)

User avatar
Wock
Posts: 2540
Joined: Fri Aug 24, 2007 8:09 pm
Location: Gallatin, Tennessee (USA)
Contact:

Wed Jan 28, 2009 6:17 pm Post

SImply use the backup feature (To create a zip) or Zip the file before copying over and no worries. BUT make sure all the files are shut down before zipping or copying (unless you use the backup feature to make a copy and only copy the backup and not the original.
The wheel is turning but the hamster is still dead.

js
jswadley
Posts: 4
Joined: Sun Feb 03, 2008 6:19 pm

Tue Feb 17, 2009 3:46 pm Post

For those of us who work a lot on the road (or work frequently between two different computers), it would be great if perhaps in the 2.0 release the saving structure was modified to work with these services. I've corrupted a few files before I found this thread.

Several other well-regarded Mac apps have adapted their structure for such syncing (1Password, etc), and I'd love to see this in place for my favorite writing app as well. :-)

User avatar
KB
Site Admin
Posts: 20920
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Tue Feb 17, 2009 7:26 pm Post

For those of us who work a lot on the road (or work frequently between two different computers), it would be great if perhaps in the 2.0 release the saving structure was modified to work with these services. I've corrupted a few files before I found this thread.


I'd be interested in how this could be achieved. The problem - at least as I understand it - doesn't really lie with Scrivener so much as with how many of these services work. Scrivener projects are "packages", and any Cocoa application could recognise them as such - as distinct from a folder - using NSFileManager's (the Cocoa file manager) -isFilePackageAtPath: method. So in theory, any of these programs could look at a Scrivener file, determine that it is a package, and thus update it in its entirety (except for those services that just provide a box to drop in such as, er, DropBox, of course). I'm not sure why some of the Mac solutions don't do this, to be honest.

But if these services aren't offering a solution, how would Scrivener do it? How would the saving structure be modified? Surely the only solution would be for a .scriv project to be a single file (a zip package itself, perhaps). But in that situation, every time you saved, the whole project would have to be saved, not just the current document. For those with large projects with lots of research material in them, this would not be a fun experience, having to save 100MB or so every time. With the current set up, if you import a 100MB movie or sound file, once it's imported, that file never needs saving again. With a single file, it would have to be rewritten every save. The current format is actually more secure on a local drive than a single file - a single file corrupted would lose everything; on a local drive, if the .scriv binder structure gets corrupted, you still have access to the underlying writing; you can extract it easily enough. It is only in these network and shared drive instances that it becomes less safe. But the price for making it safer is getting rid of the multiple file format and having longer save times, updating every file unnecessarily every time there is a save...

All the best,
Keith

User avatar
Jaysen
Posts: 6309
Joined: Mon Dec 17, 2007 4:00 am
Platform: Mac + Windows
Location: East-Be-Jesus-Nowhere SC, USA

Tue Feb 17, 2009 7:46 pm Post

KB,

Forgive my ignorance, but can you lock the file from the OS level. I use flock on solaris which can be flagged to prevent even read only access. I this would theoretically prevent any backups until the lock is removed.

On a different front is the idea of leveraging the fs journal. I would expect that to be implemented on the backup software side, but it "could" be done by scriv. In this model you snapshot the package at the fs level and all calling progies see the static snaphost. Scriv makes changes to the real data and when done (or at regular intervals) it releases the new data. We use this method to backup the SAN (24TB).†

All that said, it might be more trouble than it is worth.

Ok to really do this right you would need to use a dmg with the package on it. I think this would require quite a bit of effort. It might be solvable as a service but that would be outside scriv.

This had got to be one of the worst solution I have come up with recently.
Last edited by Jaysen on Tue Feb 17, 2009 7:53 pm, edited 1 time in total.
Jaysen

I have a wife and 2 kids that I can only attribute to a wiggle, a giggle, and the realization that she was out of my league so I might as well be happy with her as a friend. 26 years marriage later, I can't imagine life without her. -Me 10/7/09

ImageImage

User avatar
Jaysen
Posts: 6309
Joined: Mon Dec 17, 2007 4:00 am
Platform: Mac + Windows
Location: East-Be-Jesus-Nowhere SC, USA

Tue Feb 17, 2009 7:49 pm Post

OH! OH! OH!

New idea... what about an "auto archive changed files to" preference? Using rsync you could update a copy pretty efficiently. All you would need to do is freeze auto-save for the duration of the sync. The interval could be large, say 30 minutes.

Ok. I'm done.
Jaysen

I have a wife and 2 kids that I can only attribute to a wiggle, a giggle, and the realization that she was out of my league so I might as well be happy with her as a friend. 26 years marriage later, I can't imagine life without her. -Me 10/7/09

ImageImage

ne
nextread
Posts: 4
Joined: Sat Jan 10, 2009 1:03 am

Wed Feb 18, 2009 12:10 am Post

Gah, Well I just found out the hard way that using a MobileMe box as my main box storage for Scrivener files and lost a couple of days worth of work. Nothing major, more annoying. I was thinking about not using MobileMe as my main folder yesterday. I should have gone with the urge.

I think that sync services are going to come on but I don't that relying on them as the main folder for stuff is a good idea. It might take more seconds to copy over files to the sync folder but at least you know you've got a good copy on the machine you've last been using.

My files managed to corrupt somehow - probably due to the way they are packaged and not a single file. Not that I'm complaining more explaining.

Scrivener is a great piece of software - it's user error this time!

User avatar
Wock
Posts: 2540
Joined: Fri Aug 24, 2007 8:09 pm
Location: Gallatin, Tennessee (USA)
Contact:

Wed Feb 18, 2009 1:13 am Post

Hey Jaysen,

How about using BACKUP PROJECT TO function to a directory you want "updated", then on the other end you can download the most current zip, open it and get to work. Once done BACKUP PROJECT TO a directory that is updated/backed up and repeat the process down stream.

Works like a charm since its a "zipped" file (no corruption) and you are always using the most current file if you always use the zip files as your main files and use the .scr as working files.

After all the only thing this will do is take a little more time depending on file size and you can't be in two places at once so....

:-P
The wheel is turning but the hamster is still dead.

Ap
Apollo16
Posts: 198
Joined: Sun Dec 07, 2008 7:23 pm

Wed Feb 18, 2009 1:28 am Post

Wock beat me to it but this is how I do it. I have backup to via zip and ::KnockOnWood:: have had zero issues. In fact, I prefer it this way so that I can control the backups. I do not want an auto back up as has been previously suggested.

Apollo16

User avatar
xiamenese
Posts: 4714
Joined: Mon Jan 29, 2007 1:32 am
Platform: Mac
Location: London or Exeter, UK.

Wed Feb 18, 2009 1:37 am Post

Actually, I have found that "Backup Project to ..." will always revert to the folder that it wrote to the previous time it was used. So, if you keep your active project on your internal hard disk, then do a first backup to a suitable folder on DropBox or MobileMe or whatever — like a "Scrivener" folder for instance ;) — then each time you do a backup it will be put in that folder, and in the case of DropBox will automatically be uploaded and synced to every linked computer.

I am one who has suffered losses — not catastrophic, though — through trying to use DropBox as the location for active Scriv projects, and am resolved on taking that route from now on. Perhaps if there were an overall preference, or maybe a preference that could be set in each project as to where the backups go, that would remove one more issue ... Now someone is going to tell me that there already is, over and above the defaulting to the last backup path used! :)

Mark
The Scrivenato sometimes known as Mr X.
M1 MacBook Air (2021), 11.2.2, 16GB RAM, !TB SSID
iMac 27" (late 2015) 10.15.7, 24GB RAM, 512GB SSID
2017 iPad, iPadOS 14.3, 128GB, Apple Pencil
Scrivener, Scapple, Nisus Writer Pro, Bookends …

User avatar
Jaysen
Posts: 6309
Joined: Mon Dec 17, 2007 4:00 am
Platform: Mac + Windows
Location: East-Be-Jesus-Nowhere SC, USA

Wed Feb 18, 2009 1:49 am Post

the other "problem" with backup up to zip is that you need to sync the whole file. This is horribly inefficient. Especially if you have one of those giant projects we hear about.

Maybe KB can be talked into a collaborative project where someone writes the sync and he tells us how he will indicate locked files. Maybe. Or maybe not.
Jaysen

I have a wife and 2 kids that I can only attribute to a wiggle, a giggle, and the realization that she was out of my league so I might as well be happy with her as a friend. 26 years marriage later, I can't imagine life without her. -Me 10/7/09

ImageImage

User avatar
xiamenese
Posts: 4714
Joined: Mon Jan 29, 2007 1:32 am
Platform: Mac
Location: London or Exeter, UK.

Wed Feb 18, 2009 1:55 am Post

Jaysen wrote:the other "problem" with backup up to zip is that you need to sync the whole file. This is horribly inefficient. Especially if you have one of those giant projects we hear about.

Maybe KB can be talked into a collaborative project where someone writes the sync and he tells us how he will indicate locked files. Maybe. Or maybe not.

:)

So does that mean we ought to go the "Subversion" route?

OK ... I admit it, my .scriv projects are tiny — diminutive, yes, but beautifully formed ;) — so backup times are not an issue.

Mark
The Scrivenato sometimes known as Mr X.
M1 MacBook Air (2021), 11.2.2, 16GB RAM, !TB SSID
iMac 27" (late 2015) 10.15.7, 24GB RAM, 512GB SSID
2017 iPad, iPadOS 14.3, 128GB, Apple Pencil
Scrivener, Scapple, Nisus Writer Pro, Bookends …

User avatar
Wock
Posts: 2540
Joined: Fri Aug 24, 2007 8:09 pm
Location: Gallatin, Tennessee (USA)
Contact:

Wed Feb 18, 2009 4:16 am Post

xiamenese wrote:Perhaps if there were an overall preference, or maybe a preference that could be set in each project as to where the backups go, that would remove one more issue ... Now someone is going to tell me that there already is, over and above the defaulting to the last backup path used! :)

Mark


I think I heard Keith mention somewhere at some time (my moonshine addled brain is a tad foggy) that in the next major upgrade (2.0) that he overhauled the preferences and I think he mentioned there was a "backup" preference that you could set the backup directory default and a few other neato things that Keith does.

Don't quote me on that but I think 2.0 will have some beefed up backup preferences from what I think I thought I heard (or actually read since I heard nothing).
:lol:
The wheel is turning but the hamster is still dead.

eo
eodeclas
Posts: 1
Joined: Thu Feb 26, 2009 6:21 pm

Thu Feb 26, 2009 6:32 pm Post

I would really love to work directly off of files saved to my iDisk as I find saving a backup to the idisk and then copying that backup to the local hard drive on the other end and uncompressing can get a bit convoluted. How about using the iWork 2009 trick of saving everything as glorified zip files. Would that work and/or be feasible?

This link discusses the new file format a bit: <http://www.macosxhints.com/article.php?story=20090225034801527>

Thanks!

User avatar
AmberV
Posts: 24669
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Thu Feb 26, 2009 6:44 pm Post

The main problem with using zip (or any single-file format) with Scrivener is that currently it is very efficient about what it saves. Only the bits of the project that you touch are saved, so you can have a 100 mb project, change a single line, and only that one file with the line gets saved. If Scrivener were using a zip format it would have to regenerate the entire 100 mb archive every time it saved. Consider how long it takes to back up to zip right now, on a large project like that, and then imagine that happening every time you paused for two seconds.

This technique is useful for single documents and such, but for applications which work with potentially many hundreds of individual documents, and deal with multimedia, it is really impractical.

The only way to make it work would be to remove auto-save altogether and make project saving a manual affair you have to remember to do. As it stands, we already have that "feature" in the form of zipped back-ups. The result isn't all that different than what you would get otherwise, too. Potentially better since you can automatically get a sequence of files instead of writing over the same one every time.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

st
steastu
Posts: 32
Joined: Mon Jan 14, 2008 7:59 am
Platform: Mac
Location: Norway
Contact:

Tue Mar 03, 2009 3:28 pm Post

But what if I save my project to directly to a thumb drive, can I then just work from the td?