Subversion issue (a bit long & technical)

Qu
Quorcork
Posts: 23
Joined: Tue Oct 02, 2007 2:47 pm

Wed Jun 11, 2008 8:46 am Post

Hi Folks!
Some days ago i took a second start at using a large scrivener project with svn, single user scenario (only I am editing). In this scenario using the svn mainly works when editing, commiting, checking out from repository, etc.

However from time to time svn bailed out reporting obstructed directories. Now, obstructed elements are rather nasty with svn as they require some manual file operations in order to get everything nice and smooth again with your working copy.

My error-search came up with the fact that a xxx_notes.rtfd dir caused the conflict and after some time i think i have a clue what is causing this issue:

a) svn uses hidden .svn directories to store meta information about all files in a dir, every directory of a svn file tree has it's own .svn dir. If you check out a scriv project from a svn repository a .svn dir in every dir of the project is created to monitor changes in the dir's contents.
b) Some operations in Scrivener create or DELETE whole directories. In my case i removed the content from a notes field and later on added new notes to that specific field. It seems to me that scrivener removed the specific notes dir when i deleted the content. As i added new content later on the dir was recreated by scrivener.
c) What happens behind the scenes? Scriv deletes the unneeded dir and later on creates a new one with the same name and with new content. The next time the user asks svn to commit his changes to the repository svn checks the folders for changes and finds that the folder has a name known in the repository but without an .svn dir inside. There is no .svn dir because scriv deleted and recreated the note dir in the meantime, effectively removing the .svn dir inside.
d) At this point svn snaps out, reports major mojo and the user has to get manual in order to repair the structure and get svn into accepting the modified dir.

This is only a guess and i'm no expert with scriv or svn implementation so maybe the wizards can check back on this one. If they find my guess to be true it would be nice to find a fix (what else?). ;-)

Currently i'd recommend using svn with scriv only if you've a bit of svn experience because correcting the obstructed dirs, although no rocket science, requires you to better know what you're doing.

Cheers & I hope this helps
Silvan

PS: After re-reading my post several hours later i'd like to add that i absolutely do NOT want to blame scrivener for these conflicts! It is just that the two programs (scriv & svn) have interaction issues. This is by no means a fault of scrivener but only a technological result of the different implementations.
Last edited by Quorcork on Wed Jun 11, 2008 7:31 pm, edited 1 time in total.

User avatar
KB
Site Admin
Posts: 20902
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Wed Jun 11, 2008 5:43 pm Post

I'm hoping a Subversion expert can help you more, as I don't use it. I can, however, tell you what is happening behind the scenes.

Basically, you are correct. RTFD files are actually OS X packages, which are directories with extensions that look like regular files (a .scriv file is also a package). If you delete all the content from a file, when it saves, Scrivener sees that the file is empty and thus no longer requires a representation on disk, and so deletes it. That seems to be what you are describing.

It may be possible for Scrivener to look for the .svn file inside the file and only delete it if no such .svn file exists, but I need more information. For instance, if you deleted the file entirely and it was destroyed from the trash, the file would have to be deleted from file too, otherwise internal conflicts could occur. Would this also cause problems for Subversion?

All the best,
Keith

Qu
Quorcork
Posts: 23
Joined: Tue Oct 02, 2007 2:47 pm

Fri Jun 13, 2008 12:15 pm Post

Hi!
Sorry, i do not completely understand your example, maybe you can clarify.

Which file is deleted and destroyed from the trash? Which other file has to be deleted, too?

Maybe you can make it a quick example with file & folder names like File A, Folder B, etc.
I would be happy to help

Generally creating new files & folders is no issue with svn as you can add those additions pretty easily to a repository.
Deleting files in a folder is a bit of a hassle as svn wants to know exactly which files you want deleted but can be handled ok.
Deleting complete folders is something svn absolutely does not like you to do without telling svn first. Normal process, e.g. when developing code, is to delete such folders via svn commands which makes sure the deletion happens synchronously in the repository and on your working copy.
If you delete a folder manually svn get's really upset. But even more so if you first delete and later on readd the folder without the .svn dir inside. In the latter case svn does not know how to handle the situation as it does not know which changes have been made. Without the .svn information the meta data on this folder is broken.

Cheers
Silvan

User avatar
KB
Site Admin
Posts: 20902
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Fri Jun 13, 2008 5:26 pm Post

As I say, this is a problem because .rtfd files are technically folders - svn treats them as such. But Scrivener - and OS X in general - treats them as files. So Scrivener will delete a blank RTFD file because it has nothing in it. But an RTFD file is really a package so could have an .svn file inside it. It seems that this is what is causing problems here - a conflict between the OS X package file format and .svn. Perhaps other Subversion users can offer some insights?
Thanks,
Keith

Qu
Quorcork
Posts: 23
Joined: Tue Oct 02, 2007 2:47 pm

Tue Jun 17, 2008 9:32 am Post

I'd think it would be enough if scrivener would simply not delete empty .rtfd directories if svn/cvs compatibility is enabled.
Cheers
Silvan

User avatar
Wock
Posts: 2540
Joined: Fri Aug 24, 2007 8:09 pm
Location: Gallatin, Tennessee (USA)
Contact:

Tue Jun 17, 2008 6:32 pm Post

I don't think SCR was designed as being used with SVN in mind really since SVN is more for code development and collaboration. Since it (SCR) uses the OSX package system it can be limited to those OSX guidelines unless large amounts of custom alteration are done. Deleting the blank files is done to keep the package tidy and in order and from becoming quite bloated with "blank" directories or corruption occurring due to large amounts of "useless files" (deleted) contained within..

I am afraid that the option of not deleting the empty directories would probably cause more harm than good for most users especially since many SCR users tend to create and delete many files, many times, through out a given project.

In the end you may have to do some manual work arounds or use other 3rd party software to achieve the results you are looking for since this is a behavior of the OS itself and would take extensive coding to customize a preference that would probably be rarely used.

You might try experimenting with permissions on files inside a package (DO ONLY ON A COPY AND BE FORWARNED!)

Read Only would allow something to NOT be deleted.
The wheel is turning but the hamster is still dead.

User avatar
Jaysen
Posts: 6294
Joined: Mon Dec 17, 2007 4:00 am
Platform: Mac + Windows
Location: East-Be-Jesus-Nowhere SC, USA

Tue Jun 17, 2008 6:50 pm Post

I wrote some perl to do this once. I will have to see if I can find it. What it did was
1) check out your branch new dir.
2) compare check out to your local copy (the edited scriv file)
3) submit individual deletes
4) go back to 1 until no "to be deleted" files exist in the repo

Then all you do is check in the modified files. It was a hack. I inherited a system and was asked to managed configs in SVN. problem was the config was created via a web form and stored in various directories (this was pre-XML implementation).

I probably won't find it, but the above should give you an idea on how you might be able to create your own filters. Might actually be easy to do in applescript or automator...
Jaysen

I have a wife and 2 kids that I can only attribute to a wiggle, a giggle, and the realization that she was out of my league so I might as well be happy with her as a friend. 26 years marriage later, I can't imagine life without her. -Me 10/7/09

ImageImage

User avatar
KB
Site Admin
Posts: 20902
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Tue Jun 17, 2008 6:58 pm Post

The problem doesn't only lie in blank files, though - what about files that the user deletes from the trash? In that case, Scrivener has to delete those RTFD files internally to avoid clashes when new files are added that may take the same internal identification number as a deleted file.
Best,
Keith

Qu
Quorcork
Posts: 23
Joined: Tue Oct 02, 2007 2:47 pm

Wed Jun 25, 2008 7:41 am Post

Yep, this is a problem, currently i've no idea how to solve this. :(

User avatar
KB
Site Admin
Posts: 20902
Joined: Tue Jun 13, 2006 11:23 pm
Platform: Mac
Location: Truro, Cornwall
Contact:

Wed Jun 25, 2008 4:54 pm Post

I'm hoping some other Subversion users will come along with some suggestions...

User avatar
MarcustheBlacksmith
Posts: 220
Joined: Mon Sep 10, 2007 4:48 am
Location: Brisbane, Australia
Contact:

Sun Jul 06, 2008 1:09 pm Post

The only way I could possibly figure this scenario working would be if you could queue an SVN delete every time you deleted a file or folder in Scrivener.

That way SVN knows that if the folder (or file) is recreated that it's a new file (and will just queue and SVN add) even if it has the same name.

But we're talking about integrating SVN functions into Scrivener itself - and therein lies madness. If you have native SVN deletes, then you want native SVN moves, SVN renames...nuh uh. It's crazy overkill. Not to mention outside the 'zen' of Scrivener we all love so much.

I adore Subversion. I use it daily for my development work and study. But I wouldn't use it with Scrivener - firstly, because it's not necessary. I tend to see a lot of writers/nerds getting into subversion as some sort of incremental backup system. It's weird...I know very few professional writers who would actually work that way - like, revert just a paragraph or a scene back to an earlier revision, rather than doing an additional redraft that perhaps resembled the older revision but is more refined again...

If you're working in a team and desperately need the peace of mind (and in NO way should version control be a substitute for proper communication...and if you communicate properly you probably don't need version control as writers...) then I suggest hosting a Document Management System such as Alfresco (also free and open source), which is designed for versioning and collaborating on documents. These systems will version the whole .scriv package, rather than each individual file, leaving Scrivener to manage its internal document format by itself (as it should be).
sincères amitiés,
Monsieur Clangy Bangy

But t'weren't the Almighty that lifted her nighty...'twas Roger the lodger, the dodger!