“Views” are not an indication of much of anything. I’ve personally viewed this thread on multiple occasions.
As previously noted, a major sticking point on this request is that no one has defined what “Scrivener support for reference managers” actually means. What specific thing does Scrivener not do that you think it should? It’s very difficult to write code without a design specification.
@mixhaba03I repeat: Exactly how do you want it to work? Click by click?
As Katherine writes: If you can’t describe exactly what you want, how is anybody gonna be able to code it?
I’m surprised that out of all those disappointed academics, not a single one can describe how they want it to work. Not a single scientist that can formulate their idea in words? Unbelievable!
This isn’t entirely fair. Not all users can understand how software works, for them it is a black box: press button, get result. I suspect most users in this category have used Cite-While-You-Write1, which at least for Endnote is installed for you and integrates into Word as a ribbon element. I suspect those who complain about Scrivener mostly all fall into this category, they want the same thing: in-text citations + bibliography are formatted and updated “live” and some sort of extension UI to manage references…
For the CWYW crowd there appears a general reluctance to try a slightly different way of working. It seems that scanning a document at the end is something scary or just too inconvenient to bare. I understand that argument from inconvenience I suppose, it was a factor for me to move to Pandoc with its automatic bibliography generation. Zotero users in particular seem to have buggy processing plugins and users then blame Scrivener when the scanning workflow fails…
Those of us that use Scrivener successfully with reference managers and bibliographies, given Scrivener’s clear design and instructions can get grumpy when someone comes with some ill-conceived generalism that Scrivener is “incompatible” with, or doesn’t “support” reference managers. They get upset in return, or perhaps do not understand what we say as they may not really understand the technical details.
3 possible ways to build a more integrated bibliography tool are (mutually exclusive by design):
Embedded database and native UI management tools + formatting (i.e. Word’s native reference tools [which are awful IMO]). Would IMO be relatively straight-forward to develop a basic tool like this, but like Word’s mess of a tool, it would be limited, and raise endless feature requests (can you support X citation style etc.)
Formatting-alone and a floating extension-made UI using an editor extension (aka Endnote’s CWYW). This I think is by far the hardest to implement technically for Scrivener. It requires designing and implementing into the existing RTF editor a full plugin system with an API that can manipulate some kind of document structural representation.
Automating a post-processing tool, like citeproc, to handle citation formatting as part of the compile process. Requires a tool than can be integrated. Easy to implement, but requires users to still use temporary citations.
Users who provide general complaints are unlikely to really understand these differences, wanting something that just “works” like they were used to without understanding any underlying technical stuff, and that is IMO a valid position. Some go further by then wanting to go off on a crusade, and do not have the humility to give context that Scrivener can and is used successfully by others, if you adjust your workflow slightly.
[1] … and I’ve sat and watched several collaborators / students who really don’t understand why their Word is slow, and hangs everytime they copy and paste, or add a reference. CWYW is a moving target of buggy and slow, and trivially breaks reference connections when its “travelling library” (a kludgy and fragile workaround for collaboration) and the source become detached. This is the price of dealing with an “extension”. Turning it off and suddenly their Word “works”. They simply didn’t understand there was an alternative: temporary citations and one-off scan once finished. I ask all my collaborators who insist on Word+Endnote to at least turn CWYW off, at which they often then decide to change the default for themselves…
Katherine, did you take a look at any of the threads in the Zotero forums where this has been asked about many times? If you did click on any of those links, you’ll see numerous very clear suggestions. Click-by-click & Cite-While-You-Write would certainly be the ideal, but I’m not too fussed what exactly it looks like for me to come aboard; the essential thing is to provide some sort of API for third-party apps & programs to be able to integrate with the software itself. The main experience that all these people saying these things have is with programs like Pages or Word that have very tight integration with Mendeley, Zotero, EndNote, etc. They work as add-ons that appear on the ribbon of the word-processor. Note that Word didn’t develop those add-ons - they just provided the API.
That’s a very fair & insightful comment, thank you. I would just add that the scanning feature is much more likely to go wrong for long academic pieces & is not as simple as it seems: if you write your doctoral dissertation with Scrivener & then scan for Zotero references, you will need to go through & check the entire document to make sure that all the references scanned correctly (giving your possible typos while writing the placeholder in Scrivener or possible mistakes on the Zotero side), which would take hours & hours of work. With an integration, you would either i) see the citation immediately as you write without having to separate laborious checks back through the whole document, AND ii) you would just have to correct those wrong (dynamic rather than static) citations without having to go back, correct, then outputting & scanning again etc.
I feel like you’re being quite pedantic. Is this or is this not number 1 on the wishlist in Scrivener’s own forum?
Keep in mind, this forum is also filled almost entirely with people who have bought your software, and so is in addition to the thousands & thousands (millions?) of students & academics that use Zotero and other reference managers but haven’t dived into Scrivener.
I also gave 3 other lines of evidence beyond this on which you haven’t commented.
Katherine is hardly being pedantic. I visit this thread (and others) every time there is a post, mainly to see if anyone has finally responded to Keith’s request for a clear statement. No, telling L&L staff to go to another company’s forum doesn’t count. (Insulting L&L staff sure as hell won’t increase the odds of achieving your desired outcome)
I have Zero interest, so take my hundred or so visits to such threads off the 15000. I suspect there are many others in the same position. Add to that Keith’s statement that development is not based on a popularity contest and the continued demands seem pointless. Several posters have shared their process to achieve the desired output.
For this to be true, the guaranteed unique ID that e.g. ODF Scan uses (something like zu:2433:WQVBH98K), would have to be somehow different to the plugin. This is not the case, both the temporary citation and the LibreOffice CWYW plugin use the same identifier[1]. If this unique-ID goes wrong, this would apply to either mechanism. You do not write the temporary citation yourself, Zotero generates it (uses its export setting) so I don’t think a typo is an issue.
Being able to see the reference id (e.g. WQVBH98K), you can also link back to the source the temporary citation is from (search for it in Zotero), so you can quickly check the reference, look at the abstract etc. I do this for Bookends, I can double-click my temporary citation in Scrivener, and using Applescript automatically highlight the reference in the Bookends list. An extension could do the same for Zotero I’m sure.
[1] I did just test this using Scrivener 3 beta + Zotero on windows — using temporary citations and ODF export and scan, then comparing to the same refs added using the plugin. I do understand why users complain, setting up BetterBibTeX / ODF Scan / ZotPicker requires some searching for solutions online. First the Java I had wasn’t found by Zotero (it only supports the original Oracle JDK, not more recent open source variants). Then the ‘scannable cite.js’ needed to be manually placed, which was not in the instructions page, only buried in the forum post and github issue. Windows drag-n-drop messes up focus and dropped references often move a paragraph or so. Copy and paste works. Zotpicker also causes focus issues and is quite hit-and-miss. Getting it all set up took about 12 restarts of Zotero!
So for KB, to implement this you would need to do the following at a minimum (trying to be helpful and constructive):
Add a field “object” as an extension to the RTF engine. Fields contain visible text as well as metadata which would not be seen by the user, but can be used by an extension to store information (in Zotero’s case this is CSL JSON representation IINM). This could be similar in a way to a comment (displayed text different from the comment text, without any UI to show comments).
A [READ] API that allows a tool to “walk” through the document and retrieve the information from each field.
A [WRITE] API to enable an extension to change the displayed text. This would be the formatted in-text citation. The API should also be able to update the data, if a user edits the item in Zotero they can refresh the metadata.
An API to add a button or drop-down so a user can interact somehow with the extension.
I suspect adding a new field item to Scrivener’s customised RTF is quite straightforward (though KB probably doesn’t like adding too many patches on top of RTF). BUT the API is an absolutely huge undertaking I believe. Scrivener would ideally need some sort of semantic document hierarchy to allow walking along (i.e. what Pandoc does to allow granular document manipulations), and to add this probably would require a whole new text engine or very ugly kludges. The UI API could be easy to work around (using http or applescript etc. and third-party tools).
And what is the benefit of this? Feature-parity with LibreOffice and Word for a feature that many of us think is slow, buggy, kludgy and unstable. My testing today for Zotero didn’t convince me its solution for “dynamic” refs is any better than Endnote’s. Honestly, temporary citations through simplicity have distinct advantages while writing…
It’s also worth bearing in mind that Pages and Words - given as examples above - are coded by two of the biggest software giants on the planet, whereas Scrivener is coded by… me. (And Lee and Tiho on Windows.) Adding some kind of extendable API to Scrivener is just not feasible unless we could hire whole teams of coders, some of whom could be solely responsible for developing and maintaining an integrated API. It boggles my mind even to think about what that might involve, knowing the complexity of Scrivener’s code base as it stands.
And another thing to bear in mind is that Scrivener is very different from a word processor such as Word or Pages. In Word or Pages, all of your text is right there in a single document. So whenever a dynamic field needs updating, it just has to be updated throughout the text open in front of you. This becomes much more difficult in Scrivener, in which any given project can contain hundreds of documents, many of which may be very long, and all of which would need scanning and updating “dynamically”.
It’s not that I am not interested in making working with Reference Managers easier for academic writers, but I do have to be realistic about what is achievable with our resources, and also about what I want to spend time implementing. I’d certainly be interested in finding a way of making it easier to have temporary citations scanned and converted during Compile, but even this gets complicated, because there then has to be options for where the bibliography goes in the compiled text and so on.
Also, this definitely is not the number one user request for Scrivener (that accolade goes to iCloud support on macOS, and to feature parity with Mac on Windows). It’s something that comes up sporadically, though.
Scrivener is not a democracy. In addition to popularity, feature requests are evaluated based on factors like whether the desired feature is within Scrivener’s scope and how technically challenging implementation might be. Those factors, in turn, require a clear understanding of what the feature request actually is. What specific thing are you asking Scrivener to do? A question that remains unanswered.
This really does seem to be the most “Scrivener-like” solution. The whole raison d’être of Scrivener is writing without distractions and then compilin’ fer stylin’ at the end — and temporary citations fit this model perfectly…
As an aside, Pandoc is going to build citeproc directly into its core, thus it will become easier to set up. Also, current Pandoc installer is ~25MB, much smaller than it used to be[1]. So one route worth considering is replacing MMD with Pandoc (as it supports MMD-syntax and much more), then utilising its new citeproc system as the built-in bibliography tool. There are even extensions for e.g. Zotero (see zotxt) that pull references directly from the database without any intermediate files required. These can be passed as options at runtime, just like we do currently for compile post-processing… And it is cross-platform, so can be used identically on Windows and macOS Scrivener…
[1] I’ve also tried to UPX compress pandoc and it is ~24MB and takes 0.3s to start…
Hi friends! Fellow academic here. Know that my reply is simply to share my personal experience— not speaking for all academics by any means.
I personally would deeply appreciate better integration with reference managers such as Zotero. I did buy Scrivener and am trying to make the placeholder citations work, but it has taken a lot of time to figure out (multiple days) and I still don’t think I quite have it.
What would help my writing process personally would be a couple of things:
the ability to sync my references in Scrivener when I notice a change that needs to be made in Zotero (or insert preferred reference manager here).
the ability to easily change formatting styles. Sometimes I write up a document in one format (e.g. APA) but then it gets rejected from a publication, so I need to switch the whole document to Chicago Endnote, which is a huge pain. In word, this process is made less time consuming with the Zotero plugin, and while I recognize Scrivener is superior to word for many of my purposes (that’s why I bought it), this is an improvement that could help.
I’m not a computer scientist—I’m in the humanities— so this is the clearest way I can articulate what I would like to see. I personally look for software to help make my workflow easier and software that doesn’t have a huge learning curve. While Scrivener checks most of these boxes for me, this is one area I personally would benefit from.
It seems that many other academics here have been finding ways to make this work for a long time now, which is encouraging, but I don’t know if I’ll be able to convince others in my cohort to invest in Scrivener without some of these fixes. I know right now this is not a solution Scrivener plans on working on at all due to the number of resources it would require of their coding team, but hopefully, in the future, you’ll have the resources to do so.