Is there a way to RegEx Search for Keywords (or exclude results that contain specific keyword)?

ma
mattgor123
Posts: 5
Joined: Tue Jun 09, 2020 8:47 pm
Platform: Windows

Tue Jun 09, 2020 9:00 pm Post

Hi guys, I'm currently using Scrivener : 1.9.8.0 and absolutely loving it for keeping notes and working on a little non-Fiction manuscript. Perhaps I'm using the functionality completely wrong, but all of my Notes are organized with Keywords, so for example I have keywords for things like "Table Needed" and "Organized" and "Free" and "Subscription".

Is there a way I can search through all of my notes for items that have the below keyword combinations::

[*] "Table Needed" and "Free"? (meaning: all items that have BOTH the Table Needed and the Free Keyword)
[*] "Table Needed" and NOT "Subscription"? (meaning: all items that have the keyword Table Needed that DO NOT also have the Subscription keyword)?
[*] "Free" AND "Subscription"? (ideally, this would be an empty set, but I'd like to double-check)
[*] "Free" OR "Subscription"? (ideally, this would be all of my notes, but I'd like to double-check)

Basically I'm hoping to use Scrivener as a combination of a note-taking device and a personal database and I'm struggling to figure out how to search more effectively. I've seen RegEx references on the forums, but struggling my way through it.

I'd really, really appreciate any advice! Thanks!

User avatar
rdale
Posts: 2065
Joined: Tue Jul 14, 2015 1:07 pm
Platform: Mac, Win + iOS
Location: St. Louis, MO
Contact:

Wed Jun 10, 2020 1:27 pm Post

One thing that will simplify searching is to make sure your keywords are all one string of (visible) characters. This is because the search fields in Scrivener are typically space-delimited, so if you search for "Table Needed", and don't specify "all words" (when using non-regex searches), you end up matching to any key word that contains "table" or "needed". And if you do specify "all words", that makes it impossible for you to search for anything with an "OR" relationship. So either use TableNeeded or Table_Needed, or some other variation that keeps the words connected by non-space characters.

Now, on to your problem.

One thing you can do for OR relationships in regex, is to use the pipe | symbol. That counts as an or.

You can also try parentheses:
(This|That) It
... to get This OR That AND It.I'm not sure if a space is the best delimiter after the parenthesis, however. Not sure how the keywords are presented to the RegEx engine. You'll have to fiddle with it a bit, see if you can discover a workable solution.

If you want a NOT operator, I think this would work for excluding a document that is tagged with either or both of the keywords This and That.
^(This|That)

If you're trying to find keywords with an "AND" relationship... I'm not sure how to accomplish that. "This That", would match a line that had both words, in that order, with a space between them. But if the keywords are presented to the RegEx engine as "That This", for instance, that wouldn't match. Better to turn off the regex option, and just choose "all words".

I hope some of this turns out to be helpful. Sorry that I didn't have a chance to check out if those solutions actually work, but I don't currently have access to the Windows computer where I had the beta installed. Good luck!
FKA: robertdguthrie
AKA: R Dale Guthrie, Robert, Mr. Obscure, and "Oh, it's you again".

User avatar
krastev
Posts: 548
Joined: Fri Feb 05, 2016 5:21 am
Platform: Windows
Contact:

Wed Jun 10, 2020 2:17 pm Post

[quote="rdale"][/quote]
I don't think the RegEx is working in 1.9, but I may be wrong.
Krastev
Windows 10 Home 64-bit
Scrivener for Windows Version 1.9.16.0
Testing Scrivener Beta for Windows

ma
mattgor123
Posts: 5
Joined: Tue Jun 09, 2020 8:47 pm
Platform: Windows

Wed Jun 10, 2020 4:13 pm Post

Oh gosh, I feel like such a dummy! I didn't realize there was a Beta version out with full RegEx support. I will try installing that and working forward (worst case, I'll just revert to my existing build).

Thanks a lot for the really helpful tip to make all Keywords "single strings" without spaces. I'll start using CamelCase for them.

I'm still struggling to get the NOT operator to work with RegEx. I can't seem to find all documents without a certain keyword.

I noticed when I installed the Release Candidate it expires at the end of June -- do we typically get new releases in time? Or am I gonna be stuck?

EDIT: Now it seems all my Keyword searches are broken. I'm a bit confused :/

User avatar
krastev
Posts: 548
Joined: Fri Feb 05, 2016 5:21 am
Platform: Windows
Contact:

Wed Jun 10, 2020 4:33 pm Post

mattgor123 wrote:I'm still struggling to get the NOT operator to work with RegEx.

There is no NOT operator in RegEx.
Yes, there will be a new version
Krastev
Windows 10 Home 64-bit
Scrivener for Windows Version 1.9.16.0
Testing Scrivener Beta for Windows

ma
mattgor123
Posts: 5
Joined: Tue Jun 09, 2020 8:47 pm
Platform: Windows

Wed Jun 10, 2020 5:46 pm Post

krastev wrote:
mattgor123 wrote:I'm still struggling to get the NOT operator to work with RegEx.

There is no NOT operator in RegEx.
Yes, there will be a new version


Is there a possible workaround you can think of to get a search to give me all notes I have with "Table_Needed" Keyword and NOT the "Free" Keyword as one search result.

User avatar
krastev
Posts: 548
Joined: Fri Feb 05, 2016 5:21 am
Platform: Windows
Contact:

Wed Jun 10, 2020 8:24 pm Post

I don't know how you can achieve that. And there may be a possible bug with the negative lookahead regex. Hope someone more knowledgeable will chime in.
Krastev
Windows 10 Home 64-bit
Scrivener for Windows Version 1.9.16.0
Testing Scrivener Beta for Windows

User avatar
theswede
Posts: 101
Joined: Thu May 16, 2013 10:55 am
Platform: Mac, Win + iOS

Thu Jun 11, 2020 7:53 am Post

rdale wrote:If you want a NOT operator, I think this would work for excluding a document that is tagged with either or both of the keywords This and That.
^(This|That)


Scrivener appears to be using PCRE for RegExp, which would make ^ match start of string. PCRE has no NOT operator, to my knowledge.
Jesper Anderson SA5NEO
Computer scientist. Software engineer. Scifi writer. Scuba diver.

ma
mattgor123
Posts: 5
Joined: Tue Jun 09, 2020 8:47 pm
Platform: Windows

Thu Jun 11, 2020 5:41 pm Post

Hm, it seems like Scrivener may not be the perfect tool for the job then. I was hoping it could be nice as a personal database & manuscript all-in-one but if I can't exclude certain keywords, it won't work for my needs.

I could try to make a feature suggestion, but it seems like people have been using this perfectly for ten years without it, so I think I should probably just find the perfect tool for the job before I go too far.

I guess one solution would be to add "negative" keywords as well, like "TableNotNeeded" ... but doing that (and remembering to apply it) seems a bit farfetched for all of my keywords.

User avatar
DavidR
Posts: 878
Joined: Fri Mar 08, 2013 12:21 am
Platform: Windows

Thu Jun 11, 2020 5:57 pm Post

mattgor123 wrote:Hm, it seems like Scrivener may not be the perfect tool for the job then. I was hoping it could be nice as a personal database & manuscript all-in-one but if I can't exclude certain keywords, it won't work for my needs.

I could try to make a feature suggestion, but it seems like people have been using this perfectly for ten years without it, so I think I should probably just find the perfect tool for the job before I go too far.

I guess one solution would be to add "negative" keywords as well, like "TableNotNeeded" ... but doing that (and remembering to apply it) seems a bit farfetched for all of my keywords.

For the personal database aspect, you might take a look at Evernote. It doesn't use RegEx searching, but does have a search grammar that allows you to either include or exclude tags applied to a note.
David
Scrivener for Windows Version 1.9.9
Windows 7 Pro 64-bit

What's the difference between a free lance and a loose cannon?

User avatar
devinganger
Posts: 2197
Joined: Sat Nov 06, 2010 1:55 pm
Platform: Mac, Win + iOS
Location: Monroe, WA 98272 (CN97au)
Contact:

Thu Jun 11, 2020 8:58 pm Post

theswede wrote:
rdale wrote:If you want a NOT operator, I think this would work for excluding a document that is tagged with either or both of the keywords This and That.
^(This|That)


Scrivener appears to be using PCRE for RegExp, which would make ^ match start of string. PCRE has no NOT operator, to my knowledge.


https://stackoverflow.com/questions/139 ... -operation


EDIT: But, this may run you into the bugs that someone (was it Lunk, I think?) mentioned with these features.
Last edited by devinganger on Thu Jun 11, 2020 9:08 pm, edited 1 time in total.
--
Devin L. Ganger, WA7DLG
Not a L&L employee; opinions are those of my cat
Life has a way of moving you past wants and hopes

ma
mattgor123
Posts: 5
Joined: Tue Jun 09, 2020 8:47 pm
Platform: Windows

Thu Jun 11, 2020 9:06 pm Post

DavidR wrote:For the personal database aspect, you might take a look at Evernote. It doesn't use RegEx searching, but does have a search grammar that allows you to either include or exclude tags applied to a note.

Thanks a lot for the suggestion. I am loving Scrivener and the organizational / hierarchical structure it has. The synopsis/corkboard feature is just amazing. Plus I've already typed up so many notes! I'd hate to have to copy and paste them all over into another piece of software, lol.

But, I guess if there's no simple way to do the powerful search like they have in Evernote, it may be my only choice. I hope there's some workaround I can come up with, though I'm certainly not a Regex wizard.

User avatar
rdale
Posts: 2065
Joined: Tue Jul 14, 2015 1:07 pm
Platform: Mac, Win + iOS
Location: St. Louis, MO
Contact:

Fri Jun 12, 2020 12:44 pm Post

You could try adding another custom metadata field that has 2 values: Free/Paid. Then a document can only have one or the other; and if you select your other search results, and view them in outline mode so that you can see which documents have which value. You can even sort such an outline view by that column to group the documents by that custom metadata field.
FKA: robertdguthrie
AKA: R Dale Guthrie, Robert, Mr. Obscure, and "Oh, it's you again".

User avatar
krastev
Posts: 548
Joined: Fri Feb 05, 2016 5:21 am
Platform: Windows
Contact:

Fri Jun 12, 2020 1:57 pm Post

Or the devs can look into why the negative lookahead doesn't work. :mrgreen:
Krastev
Windows 10 Home 64-bit
Scrivener for Windows Version 1.9.16.0
Testing Scrivener Beta for Windows

User avatar
theswede
Posts: 101
Joined: Thu May 16, 2013 10:55 am
Platform: Mac, Win + iOS

Sun Jun 14, 2020 1:52 pm Post

devinganger wrote:stackoverflow.com


Good to see that was not the case, even though not all those answers are correct for most PCRE engines. And notably, the Scrivener engine seems to have some bugs.

So the [^exlude_this] pattern works in the PCRE, at least. I just did a quick test in the Windows RC, but it didn't do what I expected. I don't have time to look at that right now, but I probably did something strange; PCRE is not my usual regex engine.
Jesper Anderson SA5NEO
Computer scientist. Software engineer. Scifi writer. Scuba diver.