Improved ways to get items into Collections

[mcoad]

Collections are great, but getting items into them once they’re up and running is proving really clunky, especially when they get long. At the moment the only way of updating a collection with a new file is to drag it from the Binder into the collection tab. This is laborious, especially if you are adding several items, and most of all if their position in the list is important.

Would it not be possible to include a drop-down list of collections in the meta-data pane of the Inspector? All you would have to do then would be to choose one for the file (as you do now for compilation, for example), and bob’s your uncle.

Even better would be if there was also an option for the collection to keep the same order as the binder, so the new item appears in the correct position. This would avoid the very clunky procedure necessary at the moment to achieve this. Currently you have to either (i) open the collection and drag and drop up from the bottom of the list; or (ii) spring-open the collection by hovering over the tab and then drag and drop down. If your lists are long - and mine are several hundred files long - this can drive you crazy. I wish I had a cent for every time I’ve run out of mousepad space trying to do it…

For illustration, I have a collection for syncing to Index Card for the iPad (as you have to to do this). Keeping this up to date so that syncs work is really quite demanding. Every time you create a new file, you have to remember to drag it to the appropriate place in the collection list. In the heat of work, this is very easy to overlook, with the result that syncs constantly get screwed up. If all you had to do was to check a box in the Inspector, it would all be much more fluid.

Cheers,

Malcolm

[KB]

I don’t really agree that this is laborious at all. There is already a Documents > Add to Collection feature if you prefer to use that, but I can’t see how dragging to a collection tab is that difficult or awkward, especially seeing as if you leave the mouse on the tab for a second the collection opens, so you can choose where to place the selected items. The whole point of collections is that you don’t have to keep them in binder order, so having things inserted in-between existing items would be very annoying.

I can see how managing a collection for syncing with Index Card may become cumbersome, but that is the exception. The idea there though is that you just set the collection up when you want to take your stuff to Index Card, not micro-manage it while working in Scrivener.

So, that would be a “no”. Not that I’m ruling out refinements to collections in the future, of course - I’ll be refining many things. But the fundamental way collections work and are operated on will not change for the foreseeable future.

All the best,
Keith

[AmberV]

So wait, just to get this straight: you don’t like how it takes a moment of pause to hover over the tab so you can precisely place an item when adding it to a collection—and the solution to that is to add a generic drop-down menu in the inspector which would give one no placement options at all? Sorry if I am misunderstanding what you meant by that.

If all you want is a menu to place items into collections with, see Documents/Add to Collection/. There is Bob for you. I don’t see how that solves your original issue with placement, but there is a menu already.

Even better would be if there was also an option for the collection to keep the same order as the binder, so the new item appears in the correct position.

But, the whole idea for collections is to have a place where binder order isn’t important—which isn’t to say order isn’t important, but that such an order can be determined based on the context of the collection. It sounds like what you really want here is Documents/Hoist Binder. There now you have a small sub-section of the binder in binder-order and no hassle with keeping things organise.

You’ve got these 100+ file long collections that must be in binder order at all times—it really sounds like you’d do better working in the binder for that kind of thing. I have a few collections that are that large, but frankly I don’t care what their order is and I never even look at them, they are just big compile filters or sync filters. I can’t disagree with you that keeping several hundred files in order by hand is clunky. That’s not what collections were meant to be used for. Feel free to try, but just be aware they were never meant to be “binders”.

Keeping this up to date so that syncs work is really quite demanding. Every time you create a new file, you have to remember to drag it to the appropriate place in the collection list. In the heat of work, this is very easy to overlook, with the result that syncs constantly get screwed up. If all you had to do was to check a box in the Inspector, it would all be much more fluid.

Granted, you have to configure what gets synced with Index Card. The computer just can’t guess that information for you. So that aside, I’m not sure what good a checkbox in the inspector would do (collections don’t have an inspector) that forces the collection to conform to binder order. That would rather nullify the entire point of using Index Card! You re-arrange stuff in IC and make a few edits, bring those back to Scrivener—and it obliterates everything you changed because of a checkbox you forgot about?

I will say one thing I do agree with in all of this: a menu command that does a One Time binder order sort to a collection would be quite nice. This option to keep collections always in binder order though—that makes no sense to me at all.

[mcoad]

Hi Keith -

Thanks as ever for the super-fast response. And apols, I hadn’t noticed the Documents/Add to Collection menu item. However, in my set-up it appears to be dead. When I select it nothing happens - no list of collections appears. With Move and Sort immediately above and below, the alternatives appear, but with Add to Collection, zilch. I’ve tried creating new collections to test, but nowt.

Believe me, keeping the Index Card list up to date isn’t a matter of micro-management. It’s what you have to do if you want regular syncs to work. Unless, of course, the expectation is that you recreate the list every time you sync - which is really cumbersome. I’m talking several hundred files, here.

I’m not suggesting that the collection order be fixed, but that there be an option. I wonder, if there was a poll of users, how many usually need to reproduce their binder order in the list and how many not. I suspect you’d get a thumping majority for the first.

As for the awkwardness of dragging-and-dropping down a list of several hundred files every time you need to update with a new file, well…

No polemic intended, but it does seem to me that these points are a bit more substantial than you suggest. But, certainly, if the Add to Collection menu item worked, it would be great.

Best,

Malcolm

[AmberV]

Thought of one other thing: if you want a bunch of rigid collections that don’t let you move stuff around and violate binder order, and add new items automatically in the right spot: you’ve already got just such a feature.

Use Keywords. Search for keywords. Create saved search collection.

Now you can even add items to the collection in the Inspector by just popping the right keyword into its list.

[KB]

I don’t write the software based on votes, sorry - I don’t believe that leads to good software. I also think you’re wrong about the “thumping majority”. Add to Collection works fine for me - obviously there needs to be collections in the project (non-search ones) for it to show up though.

I still don’t see the big deal here - I’m not sure why you would take hundreds of files to Index Card, either. But really, I used a collection to provide the ability to sync with Index Card, which is a pretty nifty feature, but that is not the raison d’etre of collections.

[mcoad]

Hi Amber -

My main use of Collections so far has been for the Index Card sync. This a great feature, and makes using the iPad in conjunction with Scrivener a possibility. But unless you’re only doing this rarely, making this work is currently demanding, especially if you add several files a session. Each has to be added manually to the collection and put in the right order. You tend to forget this if you’re caught up in what you’re writing. Obviously, if it the collection isn’t in the same order as the binder, the sync to Index Card will be meaningless.

Point taken over the Add to Collections menu item as against the Inspector checkbox. Except that it’s dead in my copy of the app, as mentioned already.

People work in different ways. Order of files may not be important to you, but I’m writing a long and complicated novel in which it most definitely is! And, yes, sometimes I will need long collections including lengthy chunks of the binder list, and if they’re not in the right order and stay that way as new files are added, I will be royally screwed. So anything that helps this is very welcome.

Sure, keywords are a possibility. But just having an option to keep the original order would be so much more convenient. It wouldn’t “force” anyone to stick to it - though I guarantee a lot of users would for much of the time - and don’t understand why it produces such a strong reaction against. And it would be a lot more convenient, when necessary. Which obviously it wouldn’t be all the time.

[mcoad]

Hi Keith -

Okay, I really didn’t intend any polemics here. Just making a point and a suggestion as a devoted user.

I am worried about this menu item not working, though. I’m sure I’m using it correctly. I create non-search collections by selecting items in the binder and clicking on the Collections plus icon. The collection appears in the Collections list, it’s tab above the Binder, but no drop-down list appears beside the Add to Collection item. If I select an item in the binder and choose the menu item, there is nothing there. To be clear, these are non-search collections.

I’m glad this is working fine for you, but for me it’s like this. Any suggestions would be much appreciated.

Cheers,

Malcolm

[AmberV]

Each has to be added manually to the collection and put in the right order. You tend to forget this if you’re caught up in what you’re writing. Obviously, if it the collection isn’t in the same order as the binder, the sync to Index Card will be meaningless.

I really don’t get why you are syncing that much to IC. Yeah, everyone works differently, but that feels like overkill to me, given the design constraints of IC. At any rate: what’s wrong with keywords? This seems to solve the whole problem. Everything you want to sync gets a “Sync” keyword. If it’s a bother adding that all of the time, create a document template with it already added. At the end of the day, open the Keywords HUD, search for “Sync”, +Saved Search Collection; convert to standard collection. Sync to IC. Done. It’s perfect because it is instantiated in binder order and automatically retains IC order upon round-tripping. That’s what you want. You don’t want to have to remember to disable that “Keep Binder Order” option before syncing to something that is primarily designed to conflict with binder order.

No struggle of remembering to do this or that or constantly trying to “be the machine” and operate multiple sequential binders by hand.

Can you duplicate this problem with the menu in a new project? What are the symptoms precisely? Does the menu not show anything at all when you attempt to activate it, or does it have some stuff that doesn’t match the current stuff? Does a project restart fix it?

Keith: A placeholder menu here like in New From Templates might be good when this item is empty. Something with a hint toward what the feature does and why it isn’t doing anything.

[mcoad]

PS - Nor does anything appear by the Documents/Move/To menu item.

[AmberV]

Okay, at this point I would recommend switching to Console and seeing if you have any error messages. Sounds like you have some problems with the automatic project-generated menus. Launch Console.app from Applications/Utilities, and paste anything you see from Scrivener, here.

[KB]

Which build are you using? Does the main window have the focus? This is very strange…

[mcoad]

Hi guys -

Yes, it is weird - it’s giving me a bit of deja vu, after the double document launching saga a few days ago! Here’s hoping that it’s just me being dumb…

To recap: The main window does have the focus, and there is nothing in the menu item at all. I attach a screenshot (if you’d like a vid of the whole procedure, no problem). The same thing happens with the Move/To menu.

There’s nothing in Console at all.

I’ve created test projects and the same thing happens with all of them, so presumably it’s app-wide.

The build is 6227.

Thanks for the follow-up re my binder/colection order beef. I’ll drop a comment later, for what it’s worth, but this more important.

Best,

Malcolm

[mcoad]

It looks as though the attachment was too heavy, so here it is again:

Malcolm

[KB]

Can you please post the console log (~/Applications/Utilities/Console.app)?

Thanks!

[mcoad]

Yes, but you’ll laugh - or weep. This is all there is. To be clear, I relaunched Scrivener and went to the offending menu item, as in the screenshot. I then selected All Messages in Console and did a search for Scrivener. The result:

11/10/10 7:03:47 PM TextExpander[169] keyLogChars:
, Scrivener

Nuffink else.

• Malcolm

[mcoad]

On the original point I was making - and casting no aspersions at an app I dearly love and a dev who for me rules the pantheon - the background is that I’ve been doing some serious testing of ways to work on my Scrivener project on an iPad during a long upcoming trip: basically to see how much is possible or whether I’m going to have to invest in a new laptop. This means putting all of a large project on the iPad, as I can’t take bits. It’s complicated because as well as Scrivener I’m working in Tinderbox (a complex file also) for planning, for which there is no iPad equivalent at all, and DEVONThink Pro Office for databases (which does have an iPad app, hallelujah).

I know this is a very tall order, and probably asking far too much of the little guy, but I wanted to try, and push the envelope as far as possible. The kind of thing great apps like Scrivener encourage you to do, btw.

Hence the need to transfer hundreds of files to Index Card (and Plain Text, but that’s no problem at all), preserving the order, etc. So, I’ve been trying it out by working on the iPad for a few days, which has meant regular syncs with Scrivener itself as I test things out.

Plain Text works like a dream (apart from the strange numbers it adds at the end of document titles) and is surprisingly pleasant to work in. Index Card is also good, but syncing is far clunkier, in part because of the issues I mention in the thread and partly because it’s own integration with Dropbox is not so elegant as Plain Text’s.

Obviously, this is a very special set of circumstances and makes extreme demands on the iPad and syncing. Very likely it’s just not possible, though I reckon you can get pretty close. But in the process these two more general points emerged:

1. I know you’re both sceptical, but if you are dealing with a binder with hundreds of files, dragging one all the way up to the collection tab and then back down again in order to place it precisely when the tab springs open is indeed laborious and plays havoc with the mousepad, especially if you have to do this a lot - really, truly, honestly, kidding you not. Hence the suggestion of a simple drop-down list in the Inspector, which would at least enable you to do the first part on the fly. I didn’t imagine this would arouse such strong feelings. But I’d also overlooked the menu item which allows you to do it anyway (except on my machine, apparently!). Mea culpa.

2. When you’re moving stuff about in this kind of project, a novel where narrative is complex and key, not having an option to keep the basic order in collections makes things more difficult. This has been my experience, anyway. I’m not being anal retentive or talking about being “rigid” or forcing anyone to do anything, and obviously there will be other times when you want everything free-floating; it’s just what has arisen in practice, so I was suggesting an option for users with similar needs. Index Card syncing is one special case, but if, for example, I want to create collections of scenes involving particular characters, I would also need these to be in order, or my narrative will soon get in a twist. However, if as Amber says, this will happen automatically when keywords are added, the problem is solved, and I bid you goodnight…

That’s all. 'Nuff said.

Cheers,

Malcolm

[AmberV]

I know you’re both sceptical, but if you are dealing with a binder with hundreds of files, dragging one all the way up to the collection tag and then back down again in order to place it precisely when the tab springs open is indeed laborious and plays havoc with the mousepad, especially if you have to do this a lot - really, truly, honestly, kidding you not. Hence the suggestion of a simply drop-down list in the Inspector, which would enable you to do it just so.

Sorry if it came across that way. It wasn’t really so much a strong reaction as genuine confusion. I don’t see how a menu in the Inspector, or the Add To menu in Documents replaces or fixes what you are defining as a problem. How is a simple “Add to collection” function that dumps the thing at the bottom of the list address the placement problem?

I agree, fifty times in four hours of doing that would get old. Under my own usage patterns for collections, I care about the actual order of the thing maybe once a week. I add things maybe once or twice a day and usually just drop it on the tab and let it fall to the bottom. None of this stuff is a bother.

Like you say, you are doing some hardcore testing right now that is, I would submit, way outside of what is ordinary usage. Most people are not going to be managing +100 item collections several times a day. That’s not normal usage, even for you I would bet, it’s you playing around and trying to find an optimum workflow for iPad stuff while on the trip. That’s great, but you can’t judge the elegance of the UI based on that. It’s like saying it takes too many steps to compile a document when you are firing off 30 tests and trying to tweak some little quibble in Formatting when 99 times out of a 100 compile is something you do four or five times at the end of five months of writing. There might be one button too many, but is it important to spend a lot of time optimising that when its something that occupies so little actual-user-time? That’s my question with this workflow. Yeah, it’s a bit clumsy how you are describing it, but it’s like a clumsy routine you do once ever five years. Diminishing returns and all that.

When you’re moving stuff about in this kind of project, a novel where narrative is complex and key, not having an option to keep the basic order in collections makes things more difficult.

I think Saved Search Collections already fulfil the role for needing static order collections. You just have to expand the role of them a bit in your mind to see that they can be just like standard collections in their arbitrary inclusion of items. They already do everything you want though, and are super simple to curate.

Making standard collections outline-order rigid makes no sense because the whole point of those is to eschew the binder order. Yes, there are other tangential uses of them that arise because of their convenience in terms of defining groups of items for special purposes. There are compile filters and compile groups; some sync selection methods (and for most of these, actual order within the collection is meaningless)—but by and large the reason for this feature is so you can maintain curated collections of documents and arrange them creatively with no regard for the binder order.

For everything else that would benefit from outliner-order coherence: saved search is perfect! That’s what it does. It’s designed to be an elegant and streamlined way to do precisely the sort of stuff you are describing. There is a need for that, and this feature satisfies it. Essentially, Standard Collections and Saved Search Collections are your user-choice. It just might not be immediately obvious because of the activation mechanism. SSCs aren’t quite as immediate as dragging stuff onto a tab. I’ll grant you that. It’s a matter for education.

The one area where there is a little bit of over-fiddling is IC sync. This might be improved a little with some refinement, but IC sync is a very small, minor part of the collection system. It was literally tacked on in the 11th hour to something that had been developed and refined for years.

So yeah. Not a strong reaction; I can come across that way at times and for that I apologise. More just an attempt to say, “I think we already have this covered: and here is why”.

[mcoad]

@AmberV

I don’t see how a menu in the Inspector, or the Add To menu in Documents replaces or fixes what you are defining as a problem. How is a simple “Add to collection” function that dumps the thing at the bottom of the list address the placement problem?

It doesn’t. I was talking about two problems: (i) the ease of getting items into collections; (ii) the placement issue. Sorry of that wasn’t clear. The famous menu item would solve the first, having an option to preserve order the second.

But you’re right. My demands were extreme and I certainly wasn’t “judg(ing) the elegance of the UI based on that,” btw. I LOOOOOOOVE the interface. I just felt there was room for a bit of improvement in this respect - unnecessarily as it’s in the menu item (that I can’t get to work, but doubtless that will get sorted). As for the other, I’ll now get to grips with Saved Searches. Thanks very much for this advice, as I hadn’t grasped this properly.

The one area where there is a little bit of over-fiddling is IC sync. This might be improved a little with some refinement, but IC sync is a very small, minor part of the collection system. It was literally tacked on in the 11th hour to something that had been developed and refined for years.

A lot of the trouble is with IC, too, not Scrivener. Syncing to Plain Text through the External Folder is a doddle. You set it up and it just works seamlessly, both ways, as you work. With IC, though, apart from getting the collection list right, you have to go to Dropbox on the iPad and open the altered IC document into IC, making sure first that the one you’re replacing is closed in the app, otherwise it doesn’t take. And even then it frequently doesn’t work properly - or it’s easy to screw up. Syncing back to Scrivener often doesn’t seem to work properly, either, but that may just be me.

But it’s great you did tack it on. It may not be perfect, but it’s the way to go - barring a bona fide Scrivener iPad app we’d all give an arm and a leg for - and will doubtless get better.

So, all good, and thanks for the time. Here’s hoping the menu item problem can be solved…

Very best,

Malcolm

[AmberV]

Did you try making a new blank project and duplicating the menu problem there?

As for the other, I’ll now get to grips with Saved Searches. Thanks very much for this advice, as I hadn’t grasped this properly.

Yeah, that’s not quite as intuitive as it could be. It’s almost an abstraction of multiple features, and what I mean by it being an education problem since it isn’t immediately obvious. I already have a section of the manual on the differences between these two types of collections, and it might be a good idea to put a paragraph in there describing this workflow and maybe a Wiki how-to on the topic as well once I have time.

A lot of the trouble is with IC, too, not Scrivener. Syncing to Plain Text through the External Folder is a doddle. You set it up and it just works seamlessly, both ways, as you work.

Oh and by the way those funky numbers on the end in PlainText (and Simplenote) are quite important! Those are the internal ID numbers for those documents, and how Scrivener knows which .txt files correlate to which binder items when checking for changes and integrating them. It’s not the best solution in the world, but everything else we could think of wouldn’t have been any more elegant on all systems. “Hidden” dot files could have accomplished quite a bit, but those are visible on some systems, and some workflows might even end up with the dot files getting separated from the visible files, which would be really bad.