I just think that it seems more logical that the needed routines are provided by the company who produce the reference manager, like Papers already do.
lunk: donāt worry, Scrivener V7 will handle semantic analysis for imported reference material and a full coreAI model of the writing style and thought process finesse of the great and the good in each of the major fields of endeavour ā our job will be to merely add our name to the author list and try to pretend we know what the paper is about at conferences and meetings 
Formatting a finished bibliography is not a ānaturalā fit for a reference manager, which is why they have to farm this work out to various plugins and applescript contraptions that hijack word processors (and often force you to use an intermediate format like RTF that can be fragile with other useful features like maths etc). And this is why numerous reference managers have standardised on CSL universally for style definition and bibtex or json for data transfer. We still have a general problem in academic bibliography production ā there is no universal citekey standard (Papers proposed one but no-one else has adopted it)ā¦
ā¦ but I still wonder how much of the total time spent on your type of academic project that has to do with fo
I understand that it of course can be automated in various ways, but my experience is that the scientific journal do more of this nowadays. As long as the references in the reference list are formatted in the same way, itās quite easy for the journal to re-format it to their own style.
I just donāt see why L&L (i.e. Keith) should spend a lot of time on this and not on other stuff, and if you canāt explain it to me, whoās been working as an academic for quite some time, I think you have a difficult task trying to persuade Keith.
I never recommend to my students to sloppily style their bibliographies: they must make them (and the rest of the manuscript) exactly fit the requirements (you can get rejected from high impact journals for not respecting their requirements). And when I review I get really annoyed by papers when references are missing, improperly cited (missing volumes or doi etc.) so I canāt find them ā that impacts my peer review. I personally do not want to burden the reviewer and editors with sloppy formatting.
We do get repeated users who ask about bibliography workflows on the forums and elsewhere. Keith has emphatically said no to reference management, a decision I wholeheartedly agree with (KB would be wasting time reproducing what others have done so well). But finalising a bibliography (text replacement) is exactly the thing that Scriveners compiler is designed for. Scrivener uses styles, customised RTF, and replacements to slice and dice our text to make post-processing less work.
Scrivener provides several tools and users combine them in a way that optimises their workflow. You are happy to shuffle through RTF intermediates, manual scanning and other finalising steps. Just as some are happy with RTF and do not want to bother with all the advantages that more advanced workflows afford. But that does not mean we should all use your workflow.
I would argue that, for example, being able to go straight to PDF with a bibliography without manual fudging would be gratefully employed by many academic users. You do not know how many users would benefit, and neither do I. I do know lots of people who just continue to stick to Word+Endnote because they donāt have to bother with manual fudging and post-process scanning, and they are users that would benefit who are not currently using Scrivener. Also we get repeated calls to make Scrivener work better with Zotero, and using citeproc would be a contrete example that would benefit many of them. Advanced users (complicated compilers!!!) like me prefer to use Pandoc or other tools, and citeproc support is not in our interest. It is for L&L to estimate how many academics have asked for better bibliographic support over the years, how many zotero users moan, and weigh up the development cost with the projected user benefit.
As I understood it, integrating citeproc is mostly a process of text replacement, well within the compiler scope. MMD and Java tools are already bundled within scrivener, and if it is relatively straight forward to integrate the citeproc-js engine into Scrivener, then I still maintain it would be a useful addition. But this seems to now depend on how easy it will be to add citeproc into the compiler, so input from the developer would be welcomeā¦
There would be technical difficulties adding citeproc-js purely because there is very little out there on running Javascript apps inside Cocoa apps (it seems youād need to do it inside a WebView, but as I say, thereās not much out there for something complex like this).
But also: I am still utterly unclear on what any of this really does. I tried installing citeproc-js but to test it out seemed very complex from the instructions. What Iād like to see is an example of, say, an RTF document, a sample bibliography file, and simple instructions on how I can get use citeproc-js to insert the bibliography into the RTF file. If I can see how that works, I can consider how or whether it would fit into Compile, But Iām not seeing any way of doing this right now.
All the best,
Keith
Keith you might contact Jesse Grosjean, the developer of TaskPaper etc. who has been rewriting his text editors in Javascript within a Cocoa exterior over the last few years.
See
support.hogbaysoftware.com/t/ar ā¦ cript/2228
Not for examples of bibliography etc. but for ideas on integrating JS into existing Cocoa frameworks.
I think the first step is to give Keith the example files heās requested. Thereās no reason to explore the technical details of how to do it until heās decided if itās worth doing.
Katherine
citeproc-js does not have a command line tool, so it is not possible to really demonstrate without writing some code. citeproc-java (which wraps citeproc-js) does contain a command line tool (8MB including all dependencies), and demonstrates the workflow:
michel-kraemer.github.io/citepr ā¦ line-tool/
I attach a test.bib file below (citeproc-java bundles the CSL style files so no need to pass it seperately, the example assumes RTF output). So for example, Scrivenerās compiler would scan and find a citekey like [@dehaene2017]. It would then pass that citekey to retrieve the in-text citation (replaces the [@dehaene2017] text fragment):
Command:
citeproc-java citation --style apa -format rtf -i test.bib dehaene2017Returns:
(Dehaene {\i et al.}, 2017)And the bibliography entry command (appended to a new section):
citeproc-java bibliography --style apa -format rtf -i test.bib dehaene2017Returns:
{\pard \ql \f0 \sa180 \li0 \fi0 Dehaene, S, Lau, H, & Kouider, S. (2017). What is consciousness, and could machines have it. {\i Science}, {\i 358}(6362), 486\u8211-492. {\field{\*\fldinst{HYPERLINK "https://doi.org/10.1126/science.aan8871"}}{\fldrslt{\ul
https://doi.org/10.1126/science.aan8871
}}}
\par}
Though you need to call citation for each in-text citation, you only need to call the bibliography command once by collecting the citekeys for the end.
citeproc-js is used in the same way, except it needs a javascript shim run using webcore instead of a command line program. I assume using citeproc-js directly will be faster, but they do exactly the same thing (as do citeproc-python, citeproc-ruby, pandoc-citeproc and a plethora of other interfaces).
test.bib.zip (3.61 KB)
Iāve posted an issue to citeproc.js and have PMed Frank Bennett directly to see what his input is. He will be much more knowledgeable and although he isnāt a Scrivener user, he supports students who are so is amenable to help.
Thanks, nontroppo. I donāt understand this, though:
Surely I only need to call citation for each different citation, not for those that are the same? And surely I need to call the bibliography command once for each different citation?
Iām also unclear on how I would do this in citeproc-js, since Iām not au fait with Javascript and the instructions look very complex.
EDIT:
Testing out citeproc-java with the options and bib file you provided I just get this error:
error: Exception in thread "main" java.lang.NullPointerException
at de.undercouch.citeproc.script.JREScriptRunner.eval(JREScriptRunner.java:60)
at de.undercouch.citeproc.script.AbstractScriptRunner.loadScript(AbstractScriptRunner.java:29)
at de.undercouch.citeproc.CSL.getRunner(CSL.java:386)
at de.undercouch.citeproc.CSL.<init>(CSL.java:264)
at de.undercouch.citeproc.CSL.<init>(CSL.java:242)
at de.undercouch.citeproc.CSL.<init>(CSL.java:221)
at de.undercouch.citeproc.CSL.<init>(CSL.java:185)
at de.undercouch.citeproc.tool.BibliographyCommand.generateCSL(BibliographyCommand.java:162)
at de.undercouch.citeproc.tool.BibliographyCommand.doRun(BibliographyCommand.java:137)
at de.undercouch.citeproc.tool.InputFileCommand.doRun(InputFileCommand.java:121)
at de.undercouch.citeproc.tool.AbstractCSLToolCommand.run(AbstractCSLToolCommand.java:113)
at de.undercouch.citeproc.CSLTool.doRun(CSLTool.java:170)
at de.undercouch.citeproc.tool.AbstractCSLToolCommand.run(AbstractCSLToolCommand.java:113)
at de.undercouch.citeproc.CSLTool.main(CSLTool.java:132)
Also, reading the site, it says about the command-line tool:
That doesnāt sound as though itās intended for something like Scrivener, which would presumably need to use the rather complicated API instead.
Yeah Iām not sure why the OP recommended the JS implementation of citeproc - I also found its docs rather cryptic. I have always used pandora-citeproc which works very well:
Yes, you could check if a citekey is unique, if it is then call citation and cache the result; if youāve called citation before then use the cached replacement. You only need to call bibliography once because you can pass multiple citekeys and it will sort and organise the bibliography for you in one go.
Yes, Iām waiting for the citeproc-js developer to chime it to help, I could probably work it out (though javascript makes my head hurt) but Iām snowed under with work and do not have any free nerd time available to, well, nerdle. There are also python and ruby versions (which are also both bundled with macOS), but from what Iāve read, the javascript version is higher quality and less buggyā¦
Yes, I think the command-line tool is not the main intended interface. I think the API is not too bad (Iāve had to deal with worse), but does require a familiarity with bibliographies etc. And as citeproc-java actually wraps citeproc-js, I think it is better to see if citeproc-js can be made to work.
derick: Scrivener would need to package both Pandoc & pandoc-citeproc which would require ~110MB of files to be bundled within the Scrivener app bundle. KB has rightly said before that he would not do that.So that forces a user to do install manually, which partially negates the purpose of the wish in the first place 
ā citeproc-js is a tiny library, using the javascript engine builtin to macOS to run. It would add no extra āweightā to Scriv, and is considered the best citeproc implementation apart from the Haskell derived pandoc version.
What? Doesnāt everyone already have pandoc installed?!?
Shockingly, some people seem to imagine they can survive without it!?!?!?! 
A small update: a new citeproc is being developed in Rust with the goal of allowing much easier linking of citation processing into any other app, including Cocoa apps like Scrivener. This should make the integration part straightforward at least. Iāll update this thread when it is readyā¦
Here is an up-to-date list of citeproc engines:
citationstyles.org/developers/#csl-processors
The rust one I mentioned nearly two years ago is mostly feature complete (for standard CSL bibliographies AFAICT) but still under development: github.com/cormacrelf/citeproc-rs
Note, there are ways to embed a Node JS server into Cocoa apps: github.com/nodekit-io/nodekit-darwin
The developer of Pandoc has created a new citeproc library. I asked here about integration into other programs, and while it may be possible to integrate a Haskell library into an objective-c runtime, the easiest interface will be to call an external binary (as you do for MMD or Java etc.). The citeproc binary will receive a JSON list of citations and emit formatted JSON structure.
This already supports almost all of the CSL 1.02 specification and is much faster than the previous system. Once the interface stabilises, and I get time I will test how it could be utilised by Scrivenerā¦
citeproc-rs has Swift bindings which should facilitate the integration into objective-c apps like Scrivener (see here for more details)
https://citeprocrskit.cormacrelf.net/documentation/CiteprocRsKit
So the current best options for integration seem to be to use either JSON serialisation and the citeproc binary (from the pandoc developer) or this library and an objective-c > swift interface to citeproc-rs. Either of these should allow Scrivener to generate fully formatted bibliographies and in text citation from temporary citations at compile timeā¦
1 Like