Creating a local searchable index

I started this within another thread and realised that it should be in its own thread so here it is. The original post can be found here:
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118145.

A couple of folks commented here:
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118185
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118253
http://literatureandlatte.com/forum/viewtopic.php?f=15&t=2092&p=118145#p118255

I teach theology amongst other things. I have extensive teaching material that I have prepared and have been looking for a way to classify it. I came across an excellent classification system that is extremely detailed and each theme and sub-theme has a unique number identifier, although sub-themes carry the major themes number. So for example God is 1000, God being all-knowing is 1020, the anger of God is 1025. Jesus is 2000 and all the sub-themes start with a 2… and so on.

I want to use these numbers in my info manager for naming documents of a particular subject. It means I can reduce the folders I use as the numbers of a theme tend to sit together. However, the problem I currently have is that this thematic system runs into 10,000 words. I have them all in a single text document and each theme and sub-theme is on a separate line. My problem is in finding themes and sub-themes. The major themes are fairly simple, but many sub-themes appear in different major themes. For example, ‘Anger’ appears under the theme of God and Jesus. If I want to search for ‘Anger’ to find its numerical classification I have to cycle through using the ‘find next’ rather than having all the lines only with ‘Anger’ in them showing up.

How can I create and index that would show all the themes with this word in them? Here is a small sample of what the text document currently looks like:

God 1000 1015 activity of 1305 all-knowing 1020 anger of 1025,256 as judge 1310 as redeemer 1315 as Saviour 1320 as shepherd 1220 as Spirit 1225 compassion of 1030 descriptions of See God, titles and descriptions of 1200 face of See face of God 1255 faithfulness of 1035 fatherhood of 1040 ... Jesus Christ 2000 anger of 2009 as creator 2303 as high priest 2306,1293 as judge 2309 as king 2312 as king of the Jews 2 225 as Lamb 2315 ...

I do have DevonThink Pro Office and was wondering if it was possible to use this.

Some suggestions have already been made (please see the above links), but I’ll précis here:

nom suggested:

Druid suggested:

AndreasE suggested:

I really like the tags idea, but have hit a bit of a snag. How do I get a single text document of some 10,000 lines into DTPO as separate items so that they can be tagged?

Also, is DTPO the best option? Would a Wiki be better or some other software I have not heard of yet?

Your help is much appreciated!

Howdy. For sheer power and addressing some of these issues (exploding a document into parts), you might consider Tinderbox. I also teach theology and have begun putting course materials, lecture notes, etc. into TB (from scratch). TB will offer you tremendous growth, in fact, you will never outgrow its potential. Yes, a steep learning curve, but well worth it.

TB Learning Curve

it looks really good, but at $249 it’s well out of my price range at this time.

I’ll continue looking for an alternative.

many thanks

I know. But check, because for a while they offered a discount to Scrivener users. And they occasionally have sales (that’s how I could afford it).

It does have a discount for Scrivener users, but still too high. Upgrades are $70-$100 once you’ve bought it which is a bit steep for me.

I’m quite happy with how I make notes and store them. I also own DTPO and have a corresponding iPhone app which is great. Just need to figure out how to sort my index without spending lots of money!

Many thanks though, Tinderbox looks good. I just have too many apps already and cannot justify the cost of Tinderbox.

How many main themes do you have and how many sub-themes do they have each, roughly on average? I think I can see a couple of ways to do what you want (but I may be way off the mark…)

How you go about it depends really on how you want to see the results. For example, if you have each main theme (“God”) as a DTP document, then searching for a sub-theme (“anger”) will bring up all the main themes in a list. You can then choose the most likely one(s) to select to see the contents, where the word “anger” will be highlighted next to the code.

If you need more than that - e.g. you want to see an explicit list of the relevant sub-themes with their code without having to move through each document - then you’re going to have to have a single DTP document per sub-theme, I think.

Both are possible, if a bit convoluted, but it’s a one-off process that I think can be done with basic Office tools. (It could also be done a lot simpler by an experienced programmer, but I’m not one of those :wink:)

Briefly, for the first scenario, assuming that there’s a standard structure such as a blank line before each main theme, I’d manipulate the file in Word using Find and Replace to add a marker (say ) between each main theme. Then I’d import it into Scrivener using File > Import > Import and Split, with "" as split identifier. Export back to DTP into their own folder and you have your separate main theme documents. Search for “anger” (only within your new folder) and you’ll get the list of main theme documents in which anger appears – the word will be highlighted in each, but you’ll have to select individual themes to see the code.

The second scenario is a bit more labour-intensive:

  • in Word, highlight each main theme section, then find and replace the indents within that selection with the title of the main theme so you get “God - Anger 1033” etc. [E.g. your replacement string would be “***^pGod -”]
  • your document should now consist of

Main-theme - code


Main-theme - sub-theme code


Main-theme - sub-theme code
etc.

  • Import into Scrivener, export to DTP, and you have a single document per sub-theme with the code explicit in the title. Searching for anger will bring up your list of codes…

It’s not a particularly difficult process – easier to do than to describe. I’m sure other people will have other (probably far better) ideas, but this may give you some things to explore.