Scrivener & Backing up via an NAS

Ta
Tacitus
Posts: 213
Joined: Sun Jun 17, 2007 10:33 am
Platform: Mac
Location: UK

Thu Jan 04, 2018 6:53 pm Post

kewms wrote:
rdale wrote:I've helped too many people who've lost data with cloud services to entirely agree with this. The speed with which a corrupted copy on one cloud-connected system will overwrite all other copies is quite impressive, and reverting a complex object like a Scrivener project is far from easy, no matter what the marketing materials say.
Katherine

This actually happened to me just before the holiday. I was working at home with an approaching deadline, did a decent amount of work on the file and bang, a sudden power cut whilst I'd got an open project. Power outages aren't very common around here so I' ve never bothered with a UPS. However once the power came back I realised that the project was broken and corrupted and then the penny dropped that DropBox had been syncing in the background whilst I was trying to sort things out. Needless to say I now had two corrupted files.

Fortunately I had a backup elsewhere. One was on the Airport Disk, the other was on the hard drive which does have a Time Machine backup. A lot of faffing around and I had a workable project but I'd lost enough work to be a hindrance.

Could have been much worse and I've always taken the view you can't have too many backups, but spread them around as long as you know where you're putting them.

I learnt that lesson some years ago from a final year student approaching his project. He used Zip disks (yes it was some time ago) and religiously backed up to another Zip disk. All his fieldwork recordings and project notes were there all religiously backed up. What could possibly go wrong?

He kept both disks safe in his bag until one night the bag got nicked whilst he was in the Union bar....
History is a nightmare from which I am trying to escape

ka
katzenjammer
Posts: 147
Joined: Sun Oct 01, 2006 6:19 pm
Location: Boston

Thu Jan 04, 2018 6:57 pm Post

kewms wrote:
katzenjammer wrote:Can this issue be avoided by using Time Machine -> NAS (Western Digital for example)? Or would it present the same risk of overwriting with corrupted files?


I recommend that all Mac users have a Time Machine volume. In my experience, it's much easier to go back to earlier versions with Time Machine than with Dropbox, and it stores enough history to take you back before the error occurred.

Katherine


That makes sense, thank you Katherine!

This might sound like a weird question: but how would I know if my files were corrupted?

User avatar
AmberV
Posts: 24411
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Thu Jan 04, 2018 7:02 pm Post

lunk wrote:Can a zip-file be corrupted in such a way that it can’t be read and its content can’t be restored but it can still be read and uploaded to a cloud service? I think not.

Maybe I misunderstand, but strictly speaking, I don’t see why that would make a difference to the server. It copies your bytes even if they represent a corrupted zip string. I’m not aware of any service that would actively scan your files and reject them if they don’t open.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

User avatar
pigfender
Posts: 2912
Joined: Tue Oct 12, 2010 10:25 am
Platform: Mac, Win + iOS
Location: I share a head with a great many personalities
Contact:

Thu Jan 04, 2018 7:04 pm Post

katzenjammer wrote:This might sound like a weird question: but how would I know if my files were corrupted?

Your writing would be gibberish, the structure would be all over the place and your research documents would be meaningless.
No wait... that’s just my regular project. Sorry. :oops:
http://www.pigfender.com | http://www.novelinaday.com
"Some dice only have sixes." nom, 19 Oct 2013
Image Image

User avatar
kewms
Posts: 6720
Joined: Fri Feb 02, 2007 5:22 pm
Platform: Mac

Thu Jan 04, 2018 7:04 pm Post

katzenjammer wrote:
That makes sense, thank you Katherine!

This might sound like a weird question: but how would I know if my files were corrupted?


If you open a project up and massive amounts of data is missing, that's usually a pretty good sign.

Katherine
Scrivener Support Team

User avatar
Silverdragon
Posts: 1653
Joined: Mon Jul 29, 2013 2:52 pm
Platform: Mac
Location: Tarzana, California, USA
Contact:

Thu Jan 04, 2018 7:07 pm Post

pigfender wrote:
katzenjammer wrote:This might sound like a weird question: but how would I know if my files were corrupted?

Your writing would be gibberish, the structure would be all over the place and your research documents would be meaningless.
No wait... that’s just my regular project. Sorry. :oops:

ROFL! :lol:

sc
scshrugged
Posts: 533
Joined: Wed Feb 10, 2016 6:55 pm
Platform: Mac + iOS

Thu Jan 04, 2018 7:10 pm Post

Side note of importance regarding Time Machine and Scrivener. Read the manual Sec. 5.2.4, Tips for Using Time Machine , an excerpt:
Time Machine will automatically back up your computer once every hour, and store backups as far back in time as possible, reducing the frequency of these backups the further back in time you go.
This presents a unique problem with Scrivener in that the hourly backup rou- tine is likely to run while you are working in Scrivener. This means that Time Machine will be capturing your project while it is open and in progress. Fre- quent users of Scrivener may very well keep their projects open for weeks at a time, meaning good backups of their project will be few and far between.
There are a few tips you can use to help Time Machine work effectively with Scrivener:
1. Time Machine can be set to run manually at any time of your choosing, using the menu status icon in the upper-right hand portion of your display. You can thus control when Time Machine makes a backup of your projects, making sure they are closed first.
2. As Time Machine starts to erase hourly backups that are old, it saves only the last backups made in a single day. In conjunction with the first tip, you can make certain that your “safe” backups are retained once Time Machine starts erasing old backups, by always running Time Machine manually at the end of every day with all of your projects closed.
3. Always use Apple’s Time Machine interface to restore projects. That goes for all forms of using Time Machine’s disk storage.

If you find it at all confusing, someone here will be glad to help.
I'm a Scrivener user, not an L&L employee.

User avatar
Silverdragon
Posts: 1653
Joined: Mon Jul 29, 2013 2:52 pm
Platform: Mac
Location: Tarzana, California, USA
Contact:

Thu Jan 04, 2018 7:17 pm Post

No, but seriously, corrupted project symptoms include but are not limited to:

  • Being unable to open a project that was working fine yesterday.
  • Blank files that should have stuff in them.
  • Garbage in the files.
  • Binder structure not being what you remember.

Of course, I’ve caused most of these myself with user error :D , as well as have them show up in a project that was on a hard drive that was slowly dying, But having a solid backup is a lifesaver whether there’s a power outage, hardware failure, or just klutziness.

User avatar
rdale
Posts: 2078
Joined: Tue Jul 14, 2015 1:07 pm
Platform: Mac, Win + iOS
Location: St. Louis, MO
Contact:

Thu Jan 04, 2018 7:49 pm Post

kewms wrote:
rdale wrote:To other posters;please don't discourage the "better than nothing" solution of cloud syncing services. Not everyone is willing to go to the effort of the ideal setup. If we can get more people to occasionally plug in a usb3 hard drive for use with Time Machine + sending automatic backups to Google Drive, then those people are a 1000% better off than advocating for a backup setup that they'll just give up on as too complicated. I know that I have given up on safer, co-located solutions for myself and my wife, and I work in IT--we even just moved our datacenter!


I've helped too many people who've lost data with cloud services to entirely agree with this. The speed with which a corrupted copy on one cloud-connected system will overwrite all other copies is quite impressive, and reverting a complex object like a Scrivener project is far from easy, no matter what the marketing materials say.

Katherine

I'm not advocating for live projects in the cloud. I specifically referenced zipped automatic backups + nearly universally available free cloud storage as a "better than no backups at all" solution. I want people whose eyes glaze over at the more complex, robust backup solutions to at least do the easy thing instead of nothing.

I am suggesting that [Scriv auto backups + cloud] + [external USB drive for Time Machine] is easily attainable, and will protect the vast majority of users due to its relatively simple implementation. Unless your goal is to only protect people who are willing and capable of understanding and implementing more complex backup systems.
FKA: robertdguthrie
AKA: R Dale Guthrie, Robert, Mr. Obscure, and "Oh, it's you again".

User avatar
lunk
Posts: 4274
Joined: Wed Aug 21, 2013 4:24 pm
Platform: Mac + iOS
Location: Sweden 64° N

Thu Jan 04, 2018 7:56 pm Post

AmberV wrote:
lunk wrote:Can a zip-file be corrupted in such a way that it can’t be read and its content can’t be restored but it can still be read and uploaded to a cloud service? I think not.

Maybe I misunderstand, but strictly speaking, I don’t see why that would make a difference to the server. It copies your bytes even if they represent a corrupted zip string. I’m not aware of any service that would actively scan your files and reject them if they don’t open.


I am talking avout Scrivener’s zip:ed backups.
The cloud server doesn’t actively scan my files, does it. It’s my own Dropbox, Mega, OneDrive, iCloudDrive, etc app that uploads it, right?
I am a user, writing non-fiction and science, using:
* Mac Scrivener 3 on a Macbook 12”, MacBook Pro 13”, and iMac 27”, running different OS.
* iOS Scrivener 1 on an iPhone 11 Pro, iPad Air 9.7”, and iPad Pro 12.9”, all running the latest iOS

User avatar
rdale
Posts: 2078
Joined: Tue Jul 14, 2015 1:07 pm
Platform: Mac, Win + iOS
Location: St. Louis, MO
Contact:

Thu Jan 04, 2018 8:11 pm Post

lunk wrote:
AmberV wrote:
lunk wrote:Can a zip-file be corrupted in such a way that it can’t be read and its content can’t be restored but it can still be read and uploaded to a cloud service? I think not.

Maybe I misunderstand, but strictly speaking, I don’t see why that would make a difference to the server. It copies your bytes even if they represent a corrupted zip string. I’m not aware of any service that would actively scan your files and reject them if they don’t open.


I am talking avout Scrivener’s zip:ed backups.
The cloud server doesn’t actively scan my files, does it. It’s my own Dropbox, Mega, OneDrive, iCloudDrive, etc app that uploads it, right?

I've never encountered a file that couldn't be copied, except where the underlying disk or file system was itself failing/corrupted. I don't think you can create a .zip file that cannot be copied, since copying a file doesn't require any program to interpret its contents. Syncing to the cloud is just copying one byte at a time, so as long as the meta-data is intact (operating system info telling it about the file), internal corruption of that .zip archive or any of it constituent files won't have any effect on the sync.
Last edited by rdale on Thu Jan 04, 2018 8:17 pm, edited 1 time in total.
FKA: robertdguthrie
AKA: R Dale Guthrie, Robert, Mr. Obscure, and "Oh, it's you again".

User avatar
Jaysen
Posts: 6294
Joined: Mon Dec 17, 2007 4:00 am
Platform: Mac + Windows
Location: East-Be-Jesus-Nowhere SC, USA

Thu Jan 04, 2018 8:12 pm Post

Silverdragon wrote:
pigfender wrote:
katzenjammer wrote:This might sound like a weird question: but how would I know if my files were corrupted?

Your writing would be gibberish, the structure would be all over the place and your research documents would be meaningless.
No wait... that’s just my regular project. Sorry. :oops:

ROFL! :lol:

It's a side effect of him being forced to make sense of my submissions for NiaD. Corruption by association.
Jaysen

I have a wife and 2 kids that I can only attribute to a wiggle, a giggle, and the realization that she was out of my league so I might as well be happy with her as a friend. 26 years marriage later, I can't imagine life without her. -Me 10/7/09

ImageImage

User avatar
AmberV
Posts: 24411
Joined: Sun Jun 18, 2006 4:30 am
Platform: Mac + Linux
Location: Ourense, Galiza
Contact:

Thu Jan 04, 2018 8:15 pm Post

A Scrivener zip or any other zip—it’s generally speaking going to be quite safe the way modern technology works. The main reason a zip would corrupt in the past would be an interrupted transfer. Most cloud utilities do not present files until they are fully downloaded, and are very good at resuming where they left off when transfer is interrupted.

The cloud server doesn’t actively scan my files, does it. It’s my own Dropbox, Mega, OneDrive, iCloudDrive, etc app that uploads it, right?

In general no, that’s what I was getting at when I said it wouldn’t care if you uploaded a corrupted zip. I wouldn’t categorically state that none do though. I know at least one of those did rifle through bytes in the past, back when it was known as SkyDrive. It’s not good business to do that though, for obvious reasons. Dropbox even caught some flack for passively scanning hashes looking for copyright violations in public share folders—not scanning per se, but matching data signatures.
.:.
Ioa Petra'ka
“Whole sight, or all the rest is desolation.” —John Fowles

ka
katzenjammer
Posts: 147
Joined: Sun Oct 01, 2006 6:19 pm
Location: Boston

Thu Jan 04, 2018 8:46 pm Post

Thanks everyone for the help. I'm just going to use Time Machine to Back Up my Macbook; and a few things that are truly irreplaceable I'm going to back up on various cloud services (icloud/drop box, etc.).

The thing is, while I'm doing a reorg of my computer/digital stuff, I also wanted to rip all my DVDs (some of which are pretty rare and not available to stream on netflix/amazon etc.), as well as 100s of hours of lectures - and I'd like to be able to stream those films to my laptop, phone, etc.

So, the question is: back up to what? If I want to be able to also use the device as a server for films, the best rated NAS seems to be Synology DS218+

Or, should I just get a friggen USB drive, and connect it and and back up once a week via Time Machine? And keep it simple?

Prolly the latter. This stuff gives me a headache.

Ji
JimRac
Posts: 1722
Joined: Wed Aug 27, 2014 2:06 pm
Platform: Win + iOS

Fri Jan 05, 2018 12:20 am Post

katzenjammer wrote:Or, should I just get a friggen USB drive, and connect it and and back up once a week via Time Machine? And keep it simple?


I do the equivalent of this on a PC with Acronis backup software and a USB drive. (Actually, 2 USB drives—I swap them every month and store one off-site, as I live in a city near an area that has experienced wildfires. The house burning down with the PC and the USB drive inside is a scenario I need to mitigate.)

If you backup once a week, then you know at most you’re at risk for losing one week’s worth of stuff, and if you can make yourself backup changes more frequently, you can improve that risk, It’s definitely worth it for the minimal effort involved, so might as well go for it. When you’re feeling more ambitious, you can add a NAS or whatever.

Thank you for your OP and question, by the way. This thread has got me to considering that I may need to add a true cloud backup (not sync) to my configuration.
I’m just a customer.