Scrivener as QDA software

[Fanlike]

I'm an anthropologist trying to use scrivener as both a word processor and a qualitative data analysis software. I'm quite new to Scrivener but I can see it has an incredible potential to substitute QDA such as Nvivo and Atlat.ti. I''ve only found two existing topics on this matter and they are quite old so I'd like to open a new discussion.

viewtopic.php?f=4&t=14655&p=110844&hilit=QDA#p110844
viewtopic.php?f=18&t=12713&p=110839&hilit=QDA#p110839

I'm still trying to figure out my own coding style and the overall workflow to analyze fieldnotes and transcriptions of interviews. However, It seems that the complex system of comments, keywords, documents links etc. that Scrivener provides could work very well for this porpuse. I’d like to suggest how to code material, within scrivener, by using a system similar to Nvivo's "nodes" . I’d also like to suggest a new feature that allows the direct coding/marking of files audio/video (my wish list!). And lastly, I'd like to ask some advice for working with co-occurrencies in the coding system I’ve designed for my research. Any suggestion is more than welcome!.

Analytical nodes in scrivener. In Scrivener you can easily create "nodes" like in Nvivo. You only need to create a folder that works as "container" of quotations, exerpts or sections that you want to link from other documents. Let’s say you are trying to analyze the opinion about housing prices that emerges in your transcriptions. You first create a document/node called “opinion about housing prices”. After that, any time you find a section in which informants give their opinion, you select it and link it to the document/node you have created. In this way you can easily retrive all the sections in which informants have discussed the topic by browsing the document/node’s inspector panel. In particular the "document references" panel will show all those links you have created, thus allowing you to open the sections in windows, re-read them, make comparisons etc.
I find this method a bit better than working with keywords, because it gives the user more flexibility. Keywords can be added to folders or documents that have previuosly been created in the binder. So there no way to select excerpts or passages arbitrarily, which is exaclty what you need to do when you are re-reading your material for coding. Also the use of comments can be confusing, since from the perspective of the researcher, comment and coding are two different mental processes at two different analytical levels. By creating nodes like this, and by coding through links, you don’t need to change the structure of your main document, nor you need to mix comments with codes.
It is also very flexible in terms of reorganizing the nodes with the progression of the analysis. When you find new connections and possibly create new nodes, you can easily move existing nodes in new folders and sub-folders. Moreover, nodes are also texts files in which you can start writing memos that refers to that connection.

Coding and marking files audio/video. However, I see at least two limitations in this system. As far as I know, Scrivener has limited features for working with audio/video. It doesn't allow the selection of portions of files audio/video for adding comment or links, and it also doesn’t have an audio/video marking system (Am I right?). As a consequence, since you can’t link portions of files audio/video to other documents, my coding workaround, works only with texts. If you want to select the section of the audio/video interview in which informants (again) talk about “housing price”, you can’t do it. YOu have to transcribe it first and then select the text to be linked.
I would suggest to add two simple functions, which will benefit all sorts of academics/journalists/writers working with audio and video material. 1) As I just said, some feature that allows the selection of portion of audio/video files. It would be nice to add comments or annotations that refer to that specific section instead of the whole file (like you do with a text file). It would be also very useful to link those sections to other documents, like the “nodes”.
2) I would suggest to add some kind of audio/video marking system thought which you can link your transcriptions to the audio/video file themselves. In this way, when you are re-reading the transcription, you could open the audio file directly in the section you want, without having to find it manually (this feature is also present in QDA software).

Co-occurencies. Another limitations regards co-occurencies. The method I have designed allows the user to link the same excerpt to different nodes. If for example you find a section of the transcription in which the interviewee link “housing prices” to “gentrification”, you can code the exerpt with both codes (just by linking it to the nodes you have created). If you were using comments, for example, this wouldn’t be possible, since you cannot add two different comments to the same exerpt. However there is no way to rethrive and group toghether co-occurrrencies like this one. By using this method you will create two different document/nodes with their own document references (“housing prices” and “gentrification”), but you cannot detect the documents references (or link) they share. Let’s say you want to recall all the sections in which informants link “housing prices” to “gentrification”, you have to do it manually by browsing the document references of the two nodes. It would be great to have some functions that help to do this. Something that could compare the references of two or more documents (in my case document/codes) and detect the common links. Any suggestions to solve this? maybe there is also some other way you can achieve the same result.

I hope I made myself clear, it is not an easy task for a beginner like myself.
I love Scrivener and I think it is a great software at an affordable price. As an academic working with qualitative data, I’d love it even more if it could integrate these functions. Users like me would be able to carry out all the research process (data gathering, analysis and writing) within the same software. Just amazing!

[nom]

I think I followed most of that. I think you're missing out by not using keywords for nodes. By using keywords as node labels, you can then create collections based on each individual node (keyword) or any combination of nodes (Keywords) - this then addresses the co-occurrencies problem you describe.

I tried using your process (or at least my understanding of it) in a dummy Scrivener file and, unless I'm missing something critical, it resulted in lots of duplication and required lots and lots of cutting and pasting and manually creating links. I like to minimise the amount of work I have to do - qualitative research has enough mental overhead without adding more.

To minimise the issue of knowing which part of a document a keyword was pointing to, I'd probably do three things. First, I'd create a folder for each source "document" (defined in whatever way makes sense for your material). Then I would split it down to smaller, more meaningful, levels using sub-documents to help keep the structure apparent. e.g. If it was transcript of a focus group interview (source material I'm most familiar with), I would create subdocuments for each speaker and then break up their speech at paragraph, perhaps even sentence, level into separate documents under that subdocument (or folder if you prefer - remember that in Scrivener they are equivalent). Each of these mini-documents could then be coded as needed.

The second thing I would probably do is use comments to mark the specific parts of the text that I have identified as a specific node. I thought about using inline annotations (easier, and more visible) but I think that comments are more flexible because they highlight the relevant text (whether it be a single letter or several lines) and they can overlap, or even be nested within, text containing other comments. The latter then allows the same text to be associated with different nodes.

Third, I would place detailed annotations and comments as memos in separate documents with links back to the source document within Scrivener (much like you described). Although it would be easier to use inline annotations (to mark the node) and comments to do this (meeting my first requirement), by using separate documents this you can then treat your own comments as research material (a common process, so my colleagues tell me, within anthropology and ethnographic research) - I've spent enough time in qualitative research supervision groups and methodology forums to see the benefits and this is something I would do better if I was re-doing my own doctoral research now.

I can't comment on using media files in Scrivener as I've never tried. I'll leave that for others more qualified than I.

I thought about qualitative coding in Scrivener quite a lot as I finished my thesis and again when conducting qualitative research since then. So far, I have continued to use other software (Nvivo mainly, but also a basic spreadsheet when conducting simple research). However, as I am about to complete another round of qualitative analysis I am deciding whether to trial the new Mac version of Nvivo (it can't be worse than the Windows version, so it has that going for it, but it is also licensed by my university and so is well supported) or test my ideas from above in Scrivener.

[Fanlike]

Hi Nom, thanks for your reply, this is really helpful. Thanks also for spending some of your time trying out my idea. It seems you actually understood my proposal, even if my writing is not always clear.

Your method sounds like another very nice alternative, however I'm not sure it could work well for carrying out the kind of analysis I have in mind, in particular for what concerns your first suggestion. Correct me if I'm wrong.

First, I'd create a folder for each source "document" (defined in whatever way makes sense for your material).

Anthropologists working in different cultural contexts (sometimes using other languages like in my case) have to deal with different layers of interpretation when analyze their data. Those different layers emerge with the unfolding of the research. It is almost impossible to anticipate your understanding of a text. As a consequence, it is also very difficult to split it in advance in meaningful sub-folders. You could do it in progress, but wouldn't it turn out in an endless creation of micro subfolders? I will give it a try anyway, I'm not sure mine is the I the best way.
Moreover, my method doesn't require any cut and paste. You just have to select a sentence or paragraph that relates to the document/node you have previously created, and then link them (right click/link to document). After that, just by browsing the document/node’s inspector panel (in particular the "document references") you can easily retrieve all the sentence or paragraph that you linked to the node. In this way, when you are analyzing a specific node, you can visually organize and make comparisons in a very intuitive way. Those documents listed in the inspector panel ( "document references" section) can be opened in separated quick reference windows that you can move around as you like (Right click/open as quick reference). I'm not sure, but this seems an efficient workflow.

that comments are more flexible because they highlight the relevant text (whether it be a single letter or several lines) and they can overlap, or even be nested within, text containing other comments.

You are absolutely right. Comments cannot overlap only in case you select the exact same letter or line. As soon as you add few more letters you can also add a separate comment. Thanks for clarifying this.

Third, I would place detailed annotations and comments as memos in separate documents with links back to the source document within Scrivener (much like you described).

This also sounds more than reasonable. For now I prefer to write memos in the documents' notes, just because I'm in the field and need them to be "closed" to first hand data. I guess when I'll have to work with comparisons and generalization it could work very well also in your way.


Btw, I raise this discussion basically because I'd like to take a decision about whether to purchase a QDA or carry out my analysis within scrivener. It seems you have a long experience in this, what is your suggestion? Is a proper QDA software absolutely necessary?

Thanks for sharing your ideas

[nom]

Hi Nom, thanks for your reply, this is really helpful. Thanks also for spending some of your time trying out my idea. It seems you actually understood my proposal, even if my writing is not always clear.

You're welcome. It was kind of fun, and a nice change from my normal evening routine. And there was an ulterior motive, I have some analysis to do soon and I am facing the same decision as you: use Scrivener or something else.

Your method sounds like another very nice alternative, however I'm not sure it could work well for carrying out the kind of analysis I have in mind, in particular for what concerns your first suggestion. Correct me if I'm wrong.

First, I'd create a folder for each source "document" (defined in whatever way makes sense for your material).

Anthropologists working in different cultural contexts (sometimes using other languages like in my case) have to deal with different layers of interpretation when analyze their data. Those different layers emerge with the unfolding of the research. It is almost impossible to anticipate your understanding of a text. As a consequence, it is also very difficult to split it in advance in meaningful sub-folders. You could do it in progress, but wouldn't it turn out in an endless creation of micro subfolders? I will give it a try anyway, I'm not sure mine is the I the best way.

Just to be clear, you most certainly don't have to do this in advance. If you simply have "a text" then I would import this into its own folder. Then I would split it into as many subdocuments as possible.
Although I haven't imported large amounts into Scrivener for a while, when I first started using it I imported lots - one of the first things I did was then break up the imported text into ever smaller documents within Scrivener.

Let me see if I can give a relevant, research related, example. In my doctoral research my qualitative data stemmed from focus groups, ethnographic observation (of a kind), and survey responses. If I were to set that up in Scrivener now, I might create a "Source data" folder and then 3 folders within that ("Focus groups", "Observations" and "Surveys"). Focus groups might have subfolders named "Past participants" and "Current participants" and Surveys might have subfolders titled something like "Post" and "Follow-up". That would be the limit of my pre-research structure because that would be all that I knew in advance. Anything after that would emerge from the process of doing the research (which is a strength of qualitative research).

To pick one of the above at random, Surveys I would probably have a document with the survey questions (so they are listed separately) and then I imagine I would then create a subdocument for each participant. As I received their responses, I would enter their data into a sub-document, then break that into sub-documents (one for each question) then break those up at paragraph level (or even beyond). Repeat as needed.

You can see that I will end up with lots of documents and subdocuments. Despite how unwieldy this seems, because of the binder it is actually very easy to still see how everything relates. And because of Scrivener's "Scrivenings" feature, I can still view them as if they were one document (in fact, you can view any combination of documents as if they were one simply by selecting them concurrently in the binder). Using this approach, I would get visual feedback from the Binder on the relationship of the text to both the respondent and question (e.g. person A said this in response to question Z) while at the same time I could code ever smaller snippets of text as needed.

Remember also that, within Scrivener, folders and documents behave identically (although you can trigger different behaviours at compile). So documents can have sub-documents and subfolders as needed, and vice-versa. Also they don't all have to be child documents, you can create "sibling"documents as well (which, in fact, is probably more common).

An added benefit is that you can select in the binder only the sections you are currently working on, removing everything else from the editor. Further, you can collapse sections of the binder that aren't relevant to whatever you are working on, reducing visual clutter and helping improve focus (well, it works for me anyway).

Moreover, my method doesn't require any cut and paste. You just have to select a sentence or paragraph that relates to the document/node you have previously created, and then link them (right click/link to document). After that, just by browsing the document/node’s inspector panel (in particular the "document references") you can easily retrieve all the sentence or paragraph that you linked to the node. In this way, when you are analyzing a specific node, you can visually organize and make comparisons in a very intuitive way. Those documents listed in the inspector panel ( "document references" section) can be opened in separated quick reference windows that you can move around as you like (Right click/open as quick reference). I'm not sure, but this seems an efficient workflow.

I'll take your word for it. I haven't used links much, so when I tried again last night... it got messy.

Third, I would place detailed annotations and comments as memos in separate documents with links back to the source document within Scrivener (much like you described).

This also sounds more than reasonable. For now I prefer to write memos in the documents' notes, just because I'm in the field and need them to be "closed" to first hand data. I guess when I'll have to work with comparisons and generalization it could work very well also in your way.

True, but if your memos are all in document notes then you can also only see them with the data. If you have them in their own documents then you can still see them simultaneously if you wish (via split-screen view) and view them on their own and create memos independent of a specific data point. Plus, you gain the added benefit of being able to code your own memos. My colleagues who know far more about qualitative research than I do tell me this is crucial - I usually feel inadequate and slightly embarrassed at that point, but I share their wisdom in the hope it helps (and, maybe, I can take a step towards overcoming my own inadequacies in the process).

Btw, I raise this discussion basically because I'd like to take a decision about whether to purchase a QDA or carry out my analysis within scrivener. It seems you have a long experience in this, what is your suggestion? Is a proper QDA software absolutely necessary?

Thanks for sharing your ideas

"Is a proper QDA software absolutely necessary?" No. You could do it the old fashioned way with print-outs, scissors, glue and a range of coloured highlighters. The last piece of qualitative research I did, I simply used a spreadsheet (I don't recommend it unless you have limited quantity of source data and a narrow focus). But then again, qualitative analysis software was created specifically to overcome the shortcomings of these manual techniques, especially as data grows big and unwieldy.

If budget is set aside, then I think the issues have to do with the complexity of the project(s) and comfort in using the tools available. In favour of using a dedicated qualitative research tool like Nvivo is that it is dedicated to qualitative research. There are tens of thousands people using it, many of whom provide direct feedback into the software regarding the features they would find helpful (Note: I mention Nvivo because it is the QDA software I know best, and because I was a member of a study group with one of its founders). The downside of Nvivo is it has become ugly, hard to use, slow, frustrating and, until very recently, Windows only. However, if I was starting a complex project, then I would choose it over Scrivener because I could be reasonably confident it would scale up as I needed it (I know people with many thousands of nodes, hundreds of hours of recorded audio, and hundreds of thousands of words of source text). I also know that, should I need help, there will be many people able to offer it - both in terms of the software and the analysis. There are colleagues at my university who have been using QDA software for as long as it has been around and can make it do things that leave my head spinning. If I get stuck, they can free me.

On the other hand, if my project were less grand in scope, then Scrivener becomes much more attractive. I already know how to use it, it is very flexible, I already own a licence, and it can achieve most of the basic tasks a dedicated QDA app could do. The downside is I would need to establish my own workflow, adapting Scrivener's features to create a new research workflow. While I would also need to do that to some extent in a QDA app, the very terminology with an app like Nvivo is going to guide my decisions and, as I said, there are many people who could help. With Nvivo, those who know the software know the research field. With Scrivener however, while there are people who can help with the technical know-how, in terms of applying it to qualitative research I'd be on my own.

With that stated, I am almost convinced to complete my impending analysis in Scrivener. My project is of limited scope, uses exclusively textual sources (written observation notes and typed participant journals) and the timeframe is short (limiting my time to come to grips with, or re-establish familiarity with, other apps).

How do these variables play out for you?

[Fanlike]

Thanks again, that is what I call a detailed and exhaustive answer!

1) It is becoming clear that both methods could work quite well. I guess the only difference is the tool you want to employ for rethriving your codes. I suggest to use a previously created document (which contains links listed in the document reference panel), you suggest to use keywords. By using your method you complicate the process when you edit the main document and create sub-folders (the advantage is that you can then employ direclty the keyword system). By using mine you complicate the process when you have to create documents to be used as nodes container (the advantage is that you don't need to split your main document in too many pieces; you can select your codes very flexibly since all links can be created choosing words/sentences/paragraphs from other documents without changing the structure). it seems that we achieve the same result with a slighly different procedure.

2) You persuaded me that creating memos in a seperate document might be worth doing! It seems very handy for in-depth analysis at later stages.

3) My research is a quite complicated project. At the end of the year I'll have collected tens of interviews, hundreds of fieldnotes, and also some material audio/video (in chinese!). The most appealing aspect of using Scrivener for QDA is that you can process your data within your own word processor. It seems an incredible advantage when you have to start write reports (and later your dissertation). However, I'll take a definiote decision later on. Now I can't figure out how easy or difficult will be to manage the coding process within scrivener. Both our methods have limited capacity in terms of checking double coding, cross references, detecting semantic relationship etc. For now I'll keep storing my data in Scrivener, maybe I'll have to start using some other software when It gets more complicated.

4) How do you transcribe your interviews? Don't you think that a feature that allows coding and marking files audio/video would be very handy? This is absolutely my wish for next versions of scrivener.

Please keep me posted about any new path you might discover...

[reepicheep]

I'd never consider Scrivener as an equal of Nud*ist let alone of Nvivo. Too much manual intervention involved. If it was the only tool at my disposal I grab other tools to do specialised functions; NLTK for the grammar/textual analysus, Lucene for fast retrival of texts, one of the concordancing applications for co-locations (I use AntConc), and the R Project's R to handle that statistical analysis with one of the re-implementations of VarBrl. But it would be a lot of work to manage all that. Might prove simpler and easier to write an open-source Nvivo work-a-like.

[nom]

...By using your method you complicate the process when you edit the main document and create sub-folders

Well, it doesn't complicate it for me: That's how I naturally use Scrivener. I break text down to the smallest semantically useful size. Typically that's two or three paragraphs, but is often just one. I find this incredibly helpful when writing. I find not splitting up my documents a huge disadvantage and can't face using Word for creating long documents for exactly this reason (among others). Since coding involves a similar approach, semantically coding pieces within a larger text, the idea of breaking a larger text into smaller chunks seems natural. This is even more helpful since both the binder and Scrivenings mode mean the containing structure, or context, is maintained and always readily accessible. A "document" in Scrivener is not like a document in Word. While documents in Word are isolated things, there is no such separation in Scrivener. Document are just another of structuring data within a project and I find that structure semantically useful. Coming to terms with this feature of Scrivener was liberating and remains the greatest benefit to me!

4) How do you transcribe your interviews?

I budgeted for a professional transcription service.
What would have taken me months, took less than two weeks and was very reasonably priced. All I needed to do was listen to the interviews as I read the transcription to correct any transcription errors. I'm pleased to say that, even given the rather unique terminology of my interview subjects, there were very few errors. If you want the name of a good transcription service in Adelaide, South Australia, let me know...

Don't you think that a feature that allows coding and marking files audio/video would be very handy? This is absolutely my wish for next versions of scrivener.

Probably, but if this was essential to my needs, I'd use Nvivo or something that already had this built in. During my doctoral research, I mostly relied on the (corrected) transcripts and only listening to the source audio when the transcript wasn't clear or I wanted to hear the inflection. I should note that my task was made easier because my interviewer did a better job than I dared to hope. She would say things like, "For the record, everyone's nodding, so everybody agrees" or "I notice you smiling <name>, was that because...". It was her first time as a focus group interviewer and she was brilliant! Further, the transcription service time stamped every line. Hence I could find, within a few seconds, every word in the transcript by skipping to the specified time in the audio. Highly recommended.

Given that my university has a licence for Nvivo, if I were starting a complex project now, I'd use it. Given that you are using Mac, you might be interested in this (public beta available here, but only until the end of the month).

I have been trying to decide whether my project is sufficiently complex to warrant re-learning Nvivo - I have registered for the public beta, but am yet to install it. Because my current project is relatively small, I'm leaning towards bending Scrivener to the task at hand. Also, I do not have fond memories of Nvivo on Windows, but am very familiar with Scrivener on a Mac. However, I'm curious about Nvivo on the Mac, so remain undecided...

[nom]

I'd never consider Scrivener as an equal of Nud*ist let alone of Nvivo. Too much manual intervention involved. If it was the only tool at my disposal I grab other tools to do specialised functions; NLTK for the grammar/textual analysus, Lucene for fast retrival of texts, one of the concordancing applications for co-locations (I use AntConc), and the R Project's R to handle that statistical analysis with one of the re-implementations of VarBrl. But it would be a lot of work to manage all that. Might prove simpler and easier to write an open-source Nvivo work-a-like.

I agree: Nvivo is no way equal to Scrivener.
They are each very unique tools designed for very different tasks.

While I generally prefer to use tools designed for the job at hand (e.g. I use a Day One for journaling, Evernote for notes, Wordpress for my website and Word or Pages for printable documents), sometimes it's simpler to use the tools you know, and have, rather than learn or buy a new one. I can't use a hammer to tighten a screw on my kitchen coffer grinder, but I can use a knife from the draw beside me. If it's not up to the task, I can fetch a Phillips-head screwdriver from my toolbox. If the task was bigger, I might get the cordless drill and the screw-head set. I'm unlikely to do any home maintenance where I need a hammer drill with a phillips-head...

Similar process applies with data analysis. For basic statistics I can, and do, use Excel. For more complex analyses, however, I use SPSS. I can't use Scrivener to conduct statistical analyses, but I can use it for qualitative analysis. I have Scrivener at hand and available (in my software "kitchen draw" so to speak), so it is worth considering whether it is up to the task. But it's not just Scrivener, I could adapt any of a large range of software to do qualitative research. I have used both Word and Excel in the past and have considered using Numbers, Pages, NovaMind, Devonthink Pro (serious contender!) and even Scapple. Or I could simply do it the old fashioned way with paper, scissors, a bunch of highlighters, sticky tape and a spare wall. But if the project is too complex, then I know that all of these tools will struggle. The alternative then is to use a specialist, more powerful, app like Nvivo. However, there is a cost. Apart from the potential financial impact, there is also the overhead of learning the software—the assumptions, shortcuts, language, hidden tricks, etc—rapidly enough to complete the current project. At some point, this cost will outweigh the benefits.

I have colleagues that can make Nvivo sing. I can't. Using it will cost me time and significant effort. So I need to decide if that time and effort is worth it. For very small projects, it clearly is not worth the effort. For very large projects, it clearly is. But for in between projects? Where is the transition point? While I won't speak for Fanlike, this is the issue that I am grappling with. I won't be doing any other qualitative analyses this year, so the current project has to justify the effort expended.

[urbanmkr]

Thanks so much to everybody who contributed to this thread. I've been using Scrivener to store my (many) fieldnotes for a fairly long-term ethnographic research project, and I'm convinced I can also use it to code all my transcripts - I also have about 35 interviews, some of which run to 80 pages in Word...
I hadn't thought of using the "links" function, as I haven't used it much at all, so it was good to know about that. Othwerwise I was just thinking of using keywords attached to fairly small units of text (a "paragraph" or so - a chunk of interview discussing one topic).
I love using Scrivener, way more than NVivo, which I find clunky, ugly, slow, prone to crashing, annoying and expensive. My only concern is what happens if my Scrivener project gets too big. One of the contributors here said, if I understood right, that what slows down Scrivener is when a single document is very long, so if that's right, then splitting into subdocuments seems the right way to go.
Does anyone have any updates on their QDA projects in Scrivener?

[derick]

I’ve posted on this in another thread earlier - basically you’re not going to get paragraph-level coding in Scrivener like you can in MaxQDA or NVivo etc.

http://www.literatureandlatte.com/forum ... 20#p258520