Creating a local searchable index

[_lt_svs_gt]

I started this within another thread and realised that it should be in its own thread so here it is. The original post can be found here:
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118145.

A couple of folks commented here:
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118185
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118253
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118255

I teach theology amongst other things. I have extensive teaching material that I have prepared and have been looking for a way to classify it. I came across an excellent classification system that is extremely detailed and each theme and sub-theme has a unique number identifier, although sub-themes carry the major themes number. So for example God is 1000, God being all-knowing is 1020, the anger of God is 1025. Jesus is 2000 and all the sub-themes start with a 2… and so on.

I want to use these numbers in my info manager for naming documents of a particular subject. It means I can reduce the folders I use as the numbers of a theme tend to sit together. However, the problem I currently have is that this thematic system runs into 10,000 words. I have them all in a single text document and each theme and sub-theme is on a separate line. My problem is in finding themes and sub-themes. The major themes are fairly simple, but many sub-themes appear in different major themes. For example, ‘Anger’ appears under the theme of God and Jesus. If I want to search for ‘Anger’ to find its numerical classification I have to cycle through using the ‘find next’ rather than having all the lines only with ‘Anger’ in them showing up.

How can I create and index that would show all the themes with this word in them? Here is a small sample of what the text document currently looks like:

God 1000 1015 activity of 1305 all-knowing 1020 anger of 1025,256 as judge 1310 as redeemer 1315 as Saviour 1320 as shepherd 1220 as Spirit 1225 compassion of 1030 descriptions of See God, titles and descriptions of 1200 face of See face of God 1255 faithfulness of 1035 fatherhood of 1040 ... Jesus Christ 2000 anger of 2009 as creator 2303 as high priest 2306,1293 as judge 2309 as king 2312 as king of the Jews 2 225 as Lamb 2315 ...

I do have DevonThink Pro Office and was wondering if it was possible to use this.

Some suggestions have already been made (please see the above links), but I’ll précis here:

nom suggested:

At first blush, it seems DevonThink would be ideal (I’m thinking SmartGroups for a start), but I’m hesitant to respond in more detail as I am not sure I fully understand the system you are describing and how it might work in practice. Perhaps you could start a new thread as I am sure there are many ways your needs could be addressed.

nom

Posts: 425
Joined: Sun Aug 31, 2008 12:02 am
Location: Melbourne, Australia
Platform: Mac

Druid suggested:

You might try this experiment. Create a DTP database and import all of your coded items. Then match the codes with tags. You search for all the 1000 items and tag them (God), then all the 1310 items become (God) (as judge), and so on. I’m using parens ( ) to simulate the oval “pill” shape of a tag.

The tags appear in the Inspector panel. There may be ways to select items and apply a tag to all of them, which would speed up this kind of editing a lot. I haven’t used tags myself very much; Katherine (kewms) is a very sharp DTP user and may be able to correct/enhance my advice. Good luck!

AndreasE suggested:

Yes, that’s easy. As soon as you’ve defined a tag the first time, it appears in the tag collection (yellow symbol in DTPs “binder”/tree structure). If you select several items in the list, you can simply drag them on the tag you want to assign to them, and it will be assigned to all of them at once.

I really like the tags idea, but have hit a bit of a snag. How do I get a single text document of some 10,000 lines into DTPO as separate items so that they can be tagged?

Also, is DTPO the best option? Would a Wiki be better or some other software I have not heard of yet?

Your help is much appreciated!

[exegete77]

Howdy. For sheer power and addressing some of these issues (exploding a document into parts), you might consider Tinderbox. I also teach theology and have begun putting course materials, lecture notes, etc. into TB (from scratch). TB will offer you tremendous growth, in fact, you will never outgrow its potential. Yes, a steep learning curve, but well worth it.

TB Learning Curve

[_lt_svs_gt]

it looks really good, but at $249 it’s well out of my price range at this time.

I’ll continue looking for an alternative.

many thanks

[exegete77]

I know. But check, because for a while they offered a discount to Scrivener users. And they occasionally have sales (that’s how I could afford it).

[_lt_svs_gt]

It does have a discount for Scrivener users, but still too high. Upgrades are $70-$100 once you’ve bought it which is a bit steep for me.

I’m quite happy with how I make notes and store them. I also own DTPO and have a corresponding iPhone app which is great. Just need to figure out how to sort my index without spending lots of money!

Many thanks though, Tinderbox looks good. I just have too many apps already and cannot justify the cost of Tinderbox.

[brookter]

How many main themes do you have and how many sub-themes do they have each, roughly on average? I think I can see a couple of ways to do what you want (but I may be way off the mark…)

How you go about it depends really on how you want to see the results. For example, if you have each main theme (“God”) as a DTP document, then searching for a sub-theme (“anger”) will bring up all the main themes in a list. You can then choose the most likely one(s) to select to see the contents, where the word “anger” will be highlighted next to the code.

If you need more than that - e.g. you want to see an explicit list of the relevant sub-themes with their code without having to move through each document - then you’re going to have to have a single DTP document per sub-theme, I think.

Both are possible, if a bit convoluted, but it’s a one-off process that I think can be done with basic Office tools. (It could also be done a lot simpler by an experienced programmer, but I’m not one of those )

Briefly, for the first scenario, assuming that there’s a standard structure such as a blank line before each main theme, I’d manipulate the file in Word using Find and Replace to add a marker (say ) between each main theme. Then I’d import it into Scrivener using File > Import > Import and Split, with "" as split identifier. Export back to DTP into their own folder and you have your separate main theme documents. Search for “anger” (only within your new folder) and you’ll get the list of main theme documents in which anger appears – the word will be highlighted in each, but you’ll have to select individual themes to see the code.

The second scenario is a bit more labour-intensive:

• in Word, highlight each main theme section, then find and replace the indents within that selection with the title of the main theme so you get “God - Anger 1033” etc. [E.g. your replacement string would be “***^pGod -”]
• your document should now consist of

---

Main-theme - code

---

Main-theme - sub-theme code

---

Main-theme - sub-theme code
etc.

• Import into Scrivener, export to DTP, and you have a single document per sub-theme with the code explicit in the title. Searching for anger will bring up your list of codes…

It’s not a particularly difficult process – easier to do than to describe. I’m sure other people will have other (probably far better) ideas, but this may give you some things to explore.