A Future With A Plug-In Architecture?

This has probably been asked before, BUT:

I keep on posting all these various wish-lists for future features on here, and the one thing I keep having to remind myself of is that Scrivener is the work of one (very talented!) developer, and though talented the dev might be, he cannot make Scrivener all things to all people, or he would never actually get any real work done doing all the usual under-the-hood stuff that developers gotta do: making core improvements, squishing bugs dead, communicating with users, and all that jazz. So it then occurred to me, after my latest post full of wishful-thinking . . . what if I could take it upon myself to learn to code in Objective-C (or whatever language Scrivener is written in), and I could implement these features all on my lonesome? If Scrivener accepted “plug-ins”, I bet I could knock a Citation Manager out of the park! And, the developer could do what Apple does with Apps – he could review each plugin, or at least some of them, and then feature them on the website! How awesome would THAT be?

Of course, thinking it through a bit more, I’m sure that from the developer’s perspective, this would probably be a living nightmare. First off, the whole app would probably have to be rethought and recoded (don’t know for sure, just guessing). Then it would have to work right and behave itself all over again, as though it were only version 1 again. The dev would have to add “reviewing another batch of ^%#! plugins!” to his daily list of things to do, as well as would have to keep track of which plugin broke what and why, and whenever someone needed support, the dev would have to ensure that all their plugins were copacetic and behaving properly, as well as “authorized” by the dev team. Ugh. I can almost see why it would be a sucky endeavor.

However, I do think that plugins (and maybe exposing some functions to AppleScript and Automator) could open up whole new vistas of possibility for us Scrivener users, whole new avenues of creativity with our binders and our documents and our formatting choices. A new universe – a Scrivener-verse, if you will. What sayeth you?

I agree, a plugin architecture would make Scrivener a fantastic tool to work with. Several others and myself have raised this issue over at the Windows Feedback section with a request to implement Python scripting to allow developers and Scrivener users to develop their own plugins.

Reasons for choosing Python are (as I’ve detailed in my post in the thread linked above):

  • Open source,
  • Cross platform,
  • Easy and straightforward syntax,
  • Widespread and very popular (many books written, substantial community behind it),
  • Already used in a number of other applications as a scripting language,
  • Large and powerful library of code which would make writing plugins easy and allow Scrivener to be extended very far.

Implementing several scripting languages across the Scrivener suite (AppleScript for Mac, Python for Windows and Linux as an example) wouldn’t allow scripts to be dropped straight into other environments for others to use.

Indeed many of the feature requests on this forum could be easily knocked out into Python scripts without the need to rip apart Scriveners code, implement the changes and test it to make sure it works before deploying it. A community could be built around script development and sharing further strengthening the community and supporting features across each of the three ecosystems (Windows, Mac OS and Linux).

Regarding your point about plugin review, that could be left up to each individual plugin developer and the community. If a plugin doesn’t work, it can simply be disabled still leaving Scrivener working.

I very heavily use Thompson Reuters EndNote (citation software) for my academic work and is really the only reason why I have not adopted Scrivener fully as I need EndNote (unfortunately). If I was able to write a plugin to allow EndNote to talk to Scrivener, I would be a very happy person and would instantly purchase Scrivener.

Some are likely to disagree with me but I believe that implementing AppleScript (or even Objective-C) alone would only really benefit Apple users and not Windows and Linux users as well.

I’ve read that Scrivener strives to be bloat free, which is one of it’s great qualities. A lot of the feature requests made could be implemented as plugins in Python whilst keeping Scrivener on a diet so-to-speak. Users can choose to install which plugins they want for their specific needs (an EndNote manager myself) whilst others may not need any additional functionality and choose not to install any.

KB, if you’re reading this, what do you think?

+1 for plugin architecture.

Like the previous posters I want Scrivener interaction with my Reference Management software (in my case, Mendeley). I have tried but not bought Scrivener solely because I can’t make effective use of it without interaction with my reference management software.

Until plugins are available for reference managers, Scrivener will only scratch the surface of the potentially huge academic market. Only a small percentage of academics will be willing to use Scrivener in a workflow that requires rtf scan of compiled documents (ie the current way of working with Endnote).

@subgeniuszero: I have no knowledge of the process of creating applications, but why would the Scrivener developers need to approve 3rd party plugins or monitor how well they are working? I assume Microsoft, for example, does not vet the many 3rd party plugins that are available for MS Office products.

Forgive me, but can you explain how you know this? Have you interviewed them, or conducted a survey? What is the size of the sample? And is it a properly balanced sample? What value have you assigned to p? Are you sure we are not (once again) in the presence of the false consensus effect? I don’t want to sound dismissive, but I really do think your statement is stretching things a bit.

Best, Martin
(An academic who scans rtfs compiled from Scrivener with Bookends or Sente.)

@osm — I was only bringing that up because I like the way that Apple does things with Apps; you register as an Apple developer, then you build your apps to spec, then submit them for review. The finished apps are available in the app store for download, and their availability there gives the impression that their overall quality/compatibility/etc has been verified, and that they have met with a certain amount of scrutiny before being made available. This centralizes the process of app development and deployment and has so far met with fantastic levels of acceptance from the Apple ecosystem. I figured a similar ecosystem should exist for Scrivener plugins; that is, they are crafted to a spec outlined by the developer, then submitted for his review and approval, then made available. This would cut down on bugs, incompatibilities, and problems, as well as would allow the developer to maintain tight control of the Scrivener software ecosystem, and to deliver a seamless, consistent user experience. (The way Microsoft does things is, in and of itself, an argument in favor of this way of doing them.)

Of course, this is all academic until Keith actually chimes in and says whether he will or will not support such an architecture . . . but it’s one of those things that’s nice to think about ahead of time, regardless.

I’m a heavy user of EndNote on the Mac and it works with Scrivener quite well. If I had to choose between them though, I’d dump EndNote in a heartbeat in order to stick with Scrivener. EndNote has saved me many hours, probably even more than it’s cost me, but Scrivener has saved me hundreds of hours.

I’m also an academic, I use EndNote and Scrivener. Not sure what workflow you are referring to but mine does not require rtf scan of compiled documents and this is not the current way I work with EndNote. Not sure if that makes me part of your hypothetical minority or majority, but I’m pretty confident that I do form part of the “potentially huge academic market” along with the 6 (at last count) colleagues that now use Scrivener.

Out of curiosity, like Martin, I’d really like to know how those hypothetical stats were derived. Since I’m not as fussed by the p-value (useful, but over-rated as it provides minimal practical information), I’d want to know the confidence intervals and, especially, your sampling techniques, inclusion criteria and metrics. There’s probably a grant in this for someone…

Gentlemen:

The original purpose of my post here was to discuss the possibility of a plug-in architecture for Scrivener, as a way of “pitching” the idea to the dev. However, I guess that original topic has been lost in the shuffle of discussing of Scrivener’s precise usage statistics amongst academics. This is infuriating, because the idea is, I think, one that is worthy of discussion. The previous poster’s original post was merely meant to drive home the point that a plug-in architecture would be a useful way for users to solve problems that fell outside the developer’s main areas of focus, to solve niche problems that would not be solvable any other way, and to allow end-users to determine the nature of those solutions. The point was not to make specific, statistical claims about Scrivener’s usage in the academic world, and the inaccuracy of the statistical figure quoted has now been dealt with twice already, so let’s just move on, shall we?

If you look around the forums, you will find there is a thread entitled: “AppleScript: what do you want to see in the Scrivener Suite?” (viewtopic.php?f=5&t=1921&hilit=AppleScript) It was begun by Keith on 2nd June 2007, and is still active. In other words users have been waiting (patiently) for an extension to Scrivener’s capabilities in this area for about four years, occasionally expressing a hope that it will eventually arrive when Keith has time to get round to it.
On 20 November 2011, in response to a question about the development of AppleScript support, Keith replied:

“It’s not being actively developed yet, no - it keeps getting delayed by other things. It will be one big effort for an update at some point (and then just adding to it based on suggestions after that). It’s non-trivial and involves a lot of planning, thought, coding and work. I’m hoping to get to work on it finally in the new year.”

Perhaps this might go some way to indicating how likely we are to see a plug-in architecture being developed in the near future. It is true that Keith may see plug-ins as a great idea, but having been reading things on this forum for some time now, I usually get the impression that he spends a long time thinking about things before doing them (which is what he says in the post quoted above, but I think it is a general tendency). Scrivener, it seems to me, is a piece of software that has been carefully thought out, and meticulously planned. And that takes time – four years in the case of AppleScript, and it still hasn’t arrived yet! And if you take into consideration that another priority is to make localisations of the program, so that speakers of French, German, Italian, Japanese, Chinese, etc can use it (that must be a fairly large user base) you can see (I hope) why a new idea such as yours might have to wait a while before coming to fruition, no matter how good it is. What I am trying to say is “be a little patient”. Ideas are good, but there are plenty out there which were very good and are still waiting for their moment.

As for the other question – what academics might want – I’m not sure I like other people claiming to speak for me when they haven’t asked me, and I feel it is permissible to say “hang on a minute, that’s not what I think, can you support what you say?” That’s what academia is about, basically. But I’m happy to move on. Particularly because I’m in danger of appearing to speak for Keith if I don’t! (Sorry Keith.)

Cheers, Martin.

What Martin said (he writes better than me, so I won’t duplicate). But, speaking for myself, I’ll add that I contributed for two reasons. First, the original post mentioned that the possibility of creating a citation manager as a key reason for plug-ins and this was followed by a comment on the limitations of EndNote and Scrivener that, in my experience on a Mac, do not apply. Hence I responded to correct a potential misunderstanding that other users may have reading the thread (i.e. EndNote and Scriv do in fact play well together) and to point out that at least some citation managers do work with Scrivener (Papers 2 apparently works very well with Scrivener, although I haven’t upgraded yet). This obviates the only concrete example given of the benefit of plug-ins (although see Martin’s comment regarding the discussion on AppleScript).

Second I actually do think it would be interesting to understand how staff and students use Scrivener in academic settings. There might even be a grant in it. However, my comment was meant to be light hearted, so maybe I should have used more smileys… :blush: :neutral_face: :confused: :wink: :smiley: :question:

I’m theoretically in favor of a plug-in architecture in python, but I’d bet one megabyte of prose that it won’t happen anytime soon. If implementing applescript is a daunting task, I don’t want to imagine how difficult it’d be to implement a safe plug-in architecture from the scratch. It’d probably entail hiring a new programmer.

The advantage of a plug-in architecture is that it can create a secondary scrivener market of plug-ins, or allow third party applications to interact with Scrivener in sophisticated ways. No developer will invest in Scrivener until it reaches the status of world domination, which I can glance in the horizon, but it’s still very far away :frowning:

The advantage of plug-in for user customization is overrated imho. It benefits very few people. I don’t see users, academics or not, to train themselves in a language named after a serpent to implement a feature. Look, it happened with Mellel. They spent over two years developing the custom XML format with the expectation that it’d be a bounty for users to import/export to any format they wanted via XSLT sheets. Then we learned how “easy” it was to work with XSLT sheets. A couple of users put lots of effort and contributed xslt filters to the community but it never went beyond (good) beta versions. Had those two years been spent in improving support for rtf and gui, I wouldn’t probably be in this forum (p=0.75, CI=+/-0.25)

Moreover, the type of use suggested here (for bibliography) would require changes in the file format and perhaps in scheduling tasks. I can see Scrivener ideally suited for this, and would love to see it tailored for academic use, but this is unlikely to drive users in large numbers, which would be required for Endnote/Bookends/Sente/Zotero and third party developers to jump into the scrivener platform. I think Scrivener would have much more to win by keeping its focus on the user interface, improving work with popular file formats, syncing, etc. Build userbase, then hire people to work on plugins, that seems to me the logic sequence.

Well, I’ll tell you what I’d be most likely to write, were there a Plug-in API added: a more extensive version tracking system. Coming out of the software development world, where I’m used to robust source control solutions that keep good track of every change made to a project and allow comparisons of different versions, and also being someone whose writing projects are sometimes the sort that follow a non-linear evolution (for instance, presentations that might be given in different forms at different times, leaving me with no single progression from first draft to final version, but rather a whole collection of related versions, each one a branch of the project’s evolutionary tree culminating in its own final, polished version appropriate for that particular audience), I would love to be able to handle multiple revisions in a way that allows me to better compare versions and see a graph of how these versions are related to one another.

An alternate request would be to build something like this into the software itself, a request to which I’d also add my vote, but I know that has been requested before and isn’t necessarily an easy problem…but it might be an easier problem if a plug-in architecture allowed us to combine Scrivener with pre-existing tools designed for source control, instead of having to re-invent inside of Scrivener all the functionality of, say, Mercurial or Subversion along with KDiff or WinMerge or other freely available tools.

I found this thread via a search in this forum for subversion having just watched the tutorial on snapshots. The functionality I saw described provides only a subset of that provided by a mature version control system. Note that most version control systems will already work with scrivener and subversion, one of the most popular, is available for OS X, Windows and Linux but integrating version control into scrivener would provide so much more.

Which returns my contribution to plug-ins. Many applications already integrate features such as version control via plug-ins. A robust plug-in architecture is what makes these applications such world leaders. Many of these applications use Java to be OS independent and others are based on Qt. Both of these environments have the ability to use plug-ins built in but they aren’t the only ones.

I am new to Scrivener having just bought my copy yesterday and, like trajanmcgill, I come from a software background (only 26 years) so I have seen both the joys and headaches of plug-ins. Implementing a robust plug-in architecture requires careful thought to avoid instability problems. I don’t think anyone loosing hours, if not days of work, would see an unstable plug-in as a feature. However, the advantages of such an architecture would be incalculable, allowing integration with a whole host of applications, not just those mentioned in this forum.

trajanmcgill and I can’t be the only users from a software background. deltafalcon sounds like another. Does anyone know if there are any community developer projects associated with Scrivener?

While I can’t read Keith’s mind, I wouldn’t hold my breath waiting for plugin support in Scriv.

Reason: Mac OS X is the main platform for Scriv. (I expect this to be followed by iOS, when the iOS port is ready.) Scriv is sold direct by lit’n’latte, and via the Mac App store; the iOS port can only be sold via the Mac App store.

Apple’s App Store TOS mandates sandboxing of all apps sold via the App store(s), and I believe this is interpreted in such a way as to ban plugins or make them difficult to implement (more here). In any event, Apple’s iOS license explicitly bans apps from executing downloaded code from the internet (unless it’s in JavaScript). A plugin architecture based on JavaScript might therefore be legal … but would adding a JavaScript interpreter to Scriv be an easy or sensible job for Keith? (I suspect it’d be more of a ground-up re-write task.)

And while there might be scope for a non-App Store “Scrivener Pro” with plugin support for those who care, I’m not sure how profitable it’d be.

(EDIT: This makes me sad, because I, too, would welcome plugin support … but I think there are reasons it’s unlikely to happen, unless Apple execute a U-turn on their TOS.)

FWIW, as a developer I’d be delighted to have a scriptable version of Scrivener—it’s a no-brainer. But as a writer of books, three stunning qualities make Scrivener a very appealing tool: (1) it’s efficient, (2) it’s simple, and (3) it’s inexpensive.

I’d love to know how the Scrivener customer base breaks down, especially in terms of how many writers might be likely to get into the more specialized areas that scripting enables. Over the years, I’ve customized several editors of various kinds, but it often became difficult to maintain all the scripts as the product evolved. In some cases, I ended up getting along without the plugins just to keep the original functionality clean and convenient.

At the same time, as an app evolves, an internal plugin architecture is almost inevitable, at some point. Managing complexity for the developer is comparably important to minimizing complexity for the user, and modular design usually gives way to some kind of script or plugin architecture. So I’m all for scripting, but I’d rather see it as an outgrowth of an evolving internal structure than as a “feature” added on to an already elegant design.

Allen