Editing imported documents very slow - how to improve?

[Fant]

I have imported an rtf (4,48 MB big) document (after saving it as rtf from docx in Word) and a (the original) docx (about 671 KB big), each about 280 pages:

It is not possible to (properly) use them in Scrivener, because they behave extremely sluggish, slow. For example, adding a tab needs about 2, 3 seconds from pressing the key to the execution, marking (and un-marking) the entire text about 5, 10 seconds, writing with unacceptable hesitation only.

How could I make the imported docs / text work properly?

[brookter]

First of all, do you really both the doc and the rtf? They’re big files and Scrivener will convert the .doc to rtf internally anyway… so I’d delete them both, then just reimport the .doc version to see if that makes a difference.

BTW, Scrivener’s not really designed to work on such big monolithic documents (and 280 pages is very big) – its forte is manipulating individual sections and then compiling them into a single large document at the end. So, even though you can import the .doc into the binder, you’re going to want to split it into smaller sub-documents. You can do that manually, going through the document and using Document > Split (with or without selection as title) at appropriate point.

But in this case there’s an easier way, particularly if the original already has defined section: Parts, Chapters, sections and so on.

You’ll want to use the ‘Import and Split’ option on the Import menu – this automatically breaks the important document down to smaller sub-documents (which is how you’ll want such a big document in Scrivener anyway).

The way it works is that you give it a character sequence and it will create a new document every time it encounters that sequence. The default is # but you can change that.

So, say your document is in 10 Chapters, then all you need to do in Word is to insert a # at the beginning of each new chapter title (e.g. #Chapter 1, #Chapter 2, etc). (Of course, if you’ve used Styles in Word, then you can do a simple Search by Style to do this insertion instantly.)

All that remains is to use Scrivener’s File > Import > Import and Split and identify your chosen ‘split’ character. The binder will then have 10 new documents, named Chapter 1, Chapter 2, etc.

Even if you don’t have sections in the document already, then it would be worth adding them every 20 pages or so (just insert a #…) because this is the quickest way to break the document into manageable slices if it’s too big to use comfortably in Scrivener.

HTH

[Fant]

Many thanks, brookter.

First of all, do you really both the doc and the rtf? They’re big files and Scrivener will convert the .doc to rtf internally anyway… so I’d delete them both,…

Sorry, I am not quite sure to understand correctly. The .docx file is the original file, the file I worked with, after importing it I noticed one cannot work with it in Scrivener, because it is extremely sluggish, so I saved the original docx file as rtf (with Word) and imported the rtf file (from Scrivener), but it has the same behaviour.

then just reimport the .doc version to see if that makes a difference.

So import the docx file again? OK…hmm…what is the difference with a second import? I had imported that file a few times before.

BTW, Scrivener’s not really designed to work on such big monolithic documents (and 280 pages is very big)

Whoops, I would not have thought that…and I would have thought, 280 pages are rather a small amount of pages. With how many pages / slices can Scrivener cope? (until it gets sluggish)? Suppose, the output would be 1000 pages, could Scrivener manage them, were the editable in Scrivener without sluggishness?

its forte is manipulating individual sections and then compiling them into a single large document at the end.

So how can one get text being already written to Scrivener then? May be with that import split option, but where / how should the text / the doc be splitted?

I first tried it with a line break (just for testing), but Scrivener crashed after some time, too many splittings, obviously:

After opening it again, it shows a folder containing the slices, the folder is not accessible, the slices are, after clicking that folder Scrivener obviously freezed, so I closed it with the Task Manager.

You can do that manually, going through the document and using Document > Split (with or without selection as title) at appropriate point.

Going through the original docx file? And with or without selections as title? Where can I find those options?

I have imported the doc with that option:

Leaving it like it is:

And the doc then was imported splitted in about 20 slices. When I click the folder (containing the slices) and scroll through the document (or the gathered slices) there is partially shown white space (I have to press the image down / up key 10, 20, 40 times to get to the next text) like:

When I click, seeing the blank area, “Reveal in Binder” a slice / text in Binder is marked and when I click it text (of this slice) is shown. Why is that? So, why does Scrivener show a blank area in the “folder view”, (so after clicking the folder containing the slices), but show text after clicking “Reveal in Binder”?

After importing such a big document to Scrivener into slices, how could I (automatically, not manually) verify that no text is missing, that each word is really imported?

Next try: I have added “###” to different positions in the original docx and imported it:

So it is splitted now in about 10 slices / documents. And it appears to be not being sluggish anymore. Again there is displayed a huge blank area, revealing it / different parts of the area shows slices containing text. What could that mean?

In the folder view (or whatever it is called, I guess “document” or “text”?) there is a lot of text missing / not displayed which is available - at least partially have not checked it yet - in the view you have clicking a document / item contained in the folder.

After compiling this message is shown when trying to open the compiled docx file in Word:

As an rtf file it can be opened.

Very many thanks!

[brookter]

OK. Lots of questions there and I think that if we try to go through them one by one it could get confusing. I think it will be better if we start from scratch.

First a couple of definitions. A Scrivener ‘document’ is an individual entry in the Binder (which basically contains folders and documents). The entire file (documents, folders and all) is a called a ‘project’.

This matters, because in this case it’s wrong to think of your very long Word document as being the same as a Scrivener document: it isn’t. A document of this length is really the equivalent of a Scrivener Project.

Scrivener is designed to work with relatively short documents (typically a chapter, a scene, a section) which are later stitched together in the Compilation process into the final output, which may be for publication in the Kindle, or for sending back to Word for final polishing, or for a host of other purposes. That’s Scrivener’s main strength: instead of having to copy and paste scenes to rearrange a Word novel, all you have to do is move the scene in the Binder and work on a small unit of text at one time.

So by trying to treat your 280 page Word document as a single Scrivener document, you’re making it work against the grain. It’s simply too long for the way the program was designed to work AND it’s missing all the opportunities that you buy Scrivener for in the first place!

Hope that makes sense as an explanation of WHY you’re having problems: now for the HOW to work round them .

It sounds to me as though you’re nearly there. You can’t use a blank line as the ‘split character’ as there will be too many results, as you’ve found. #### is fine.

If you imported the document into a folder (let’s call it ‘Imported’) and you now have 10 subdocuments (Chapter 1, Chapter 2… etc), then the process has worked.

As for the empty blank space, then I think I know what’s happened – you’re seeing the editor view of the folder itself (which is empty), not the text of the subdocuments.

It’s important to understand that ‘Folders’ are themselves like documents – they can contain text. So if you click on a folder and you haven’t entered any text, then it will be empty in the editor – that’s because it is an empty document.

If you want to see all the text from all the documents which are children of Import, then you need Scrivenings mode. This is essential a ‘virtual view’ which combines any number of documents from anywhere in the binder into one temporary document for you to view or edit. If you select multiple documents then you will always see them all in the Editor. If you highlight a folder on its own, you have a choice of seeing just the folder text (Document mode), or the text of all the subdocuments (‘Scrivenings mode’).

You can get to Scrivenings mode in three ways, but the easiest is simply to hit Ctl+1. It’s a toggle, so if you highlight Import and press Ctl+1 repeatedly, it will toggle between ‘Document mode’ (you’ll see no text in the editor) and ‘Scrivenings mode’ (you’ll see the text of the individual subdocuments, separated by dotted lines).

Compare these two screenshots (Documents mode first).

The second picture shows the empty text of the folder Part 1 Basics, followed by the text of Step 1 Beginnings. Note there is a group of three icons towards the right of the screenshot: the left hand one shows stacked documents. It’s white when in Document mode, yellow in Scrivenings mode and clicking on it toggles between the modes.

Scrivenings in fully explained in the tutorial – it’s one of the reasons why I suggested you’re better off working through it before attempting any work. The concepts and features are brilliant, but they can be confusing if you’ve not come across them before.

Hope this helps…

[Fant]

Thank you very much, sorry for the amount of questions, I am confused already.

First a couple of definitions. A Scrivener ‘document’ is an individual entry in the Binder (which basically contains folders and documents). The entire file (documents, folders and all) is a called a ‘project’.

This matters, because in this case it’s wrong to think of your very long Word document as being the same as a Scrivener document: it isn’t. A document of this length is really the equivalent of a Scrivener Project.

Scrivener is designed to work with relatively short documents (typically a chapter, a scene, a section) which are later stitched together in the Compilation process into the final output, which may be for publication in the Kindle, or for sending back to Word for final polishing, or for a host of other purposes. That’s Scrivener’s main strength: instead of having to copy and paste scenes to rearrange a Word novel, all you have to do is move the scene in the Binder and work on a small unit of text at one time.

So by trying to treat your 280 page Word document as a single Scrivener document, you’re making it work against the grain. It’s simply too long for the way the program was designed to work AND it’s missing all the opportunities that you buy Scrivener for in the first place!

Hope that makes sense as an explanation of WHY you’re having problems: now for the HOW to work round them .

Yes, yes, it does, I have understand it.

If you want to see all the text from all the documents which are children of Import, then you need Scrivenings mode. This is essential a ‘virtual view’ which combines any number of documents from anywhere in the binder into one temporary document for you to view or edit. If you select multiple documents then you will always see them all in the Editor. If you highlight a folder on its own, you have a choice of seeing just the folder text (Document mode), or the text of all the subdocuments (‘Scrivenings mode’).

I see, I should have provided more information, respectively the correct ones:
These items are the result of importing the doc, the Scrivening mode should be activated, so I guess the imported doc should entirely be shown:

I do not understand: when I click the folder in the binder “Ideen 2” the writer area is / gets blank. When I click that button

the doc is shown. When I click the folder “Ideen 2” the writer area turns blank:

At the beginning of the write area there is text and at the end also:

When I scroll to the middle of the area there is shown a huge blank area:

When I then click “Reveal in Binder”

this is shown:

When I then click the marked item this, text, is shown:

Is this like it meant to be? So I do not understand why there is such a huge area of blank space in the middle?

When I press STRG+1 this happens (it is one press):

Or clicking that icon:

I do not understand why the text fades in and then (automatically, without pressing the icon) disappears.

The second picture shows the empty text of the folder Part 1 Basics, followed by the text of Step 1 Beginnings. Note there is a group of three icons towards the right of the screenshot: the left hand one shows stacked documents. It’s white when in Document mode, yellow in Scrivenings mode and clicking on it toggles between the modes.

Yes, that is how I had understood it, but obviously I am missing anything.

Scrivenings in fully explained in the tutorial – it’s one of the reasons why I suggested you’re better off working through it before attempting any work. The concepts and features are brilliant, but they can be confusing if you’ve not come across them before.

Yes, yes, that is plausible.

[brookter]

Thanks for the pictures. What I suggested is the ‘obvious’ solution – always worth trying first! I think it’s getting difficult to diagnose remotely without having the files to hand (Scrivener and Word). It could be something simple (but not immediately obvious) or it could indeed be a bug.

If you want, send me a Private Message and I’ll send you my email so you can forward the files for me to have a look.

But I’m only another user, and I may not find anything… so you may be better off going directly to the official support email in the first place. Send them the files and explain the trouble you’re having.

Either way, good luck!

[Fant]

Yes, trying first is a good way.

Thank you very much for your offer, I appreciate it very much.

In the so called Scrivenings mode view between the end of the text where the blank area begins and the end of the blank area where the next text begins is text missing, but this missing text is shown when I click one of the items in the binder, so when I press the item containing the missing text it is shown.

OK, sending such a document containing information like this to the support might be a bit displeasing. May be I should try to split the original file at some other positions or split the original docx file manually or so.

Thank you very much, indeed!

[kewms]

If you split a document into chunks, and then create a large Scrivenings session from the chunks, you’ve sort of defeated the purpose.

What are you actually trying to do with the project? What need inspired you to switch over from Word?

Also, if you haven’t already, it’s worth your time to go through the Interactive Tutorial project, available from the Help menu.

Katherine

[Fant]

Thank you, Katherine.

…and then create a large Scrivenings session from the chunks, you’ve sort of defeated the purpose.

OK, why?

What are you actually trying to do with the project? What need inspired you to switch over from Word?

Besides of that Word, all Word versions I know of Win since ever run instable (on my systems), crashes very often (on my systems) one of the main reasons are or the main reason is, I assume, the display of the text (with “Collections”, tags, labels, savable searches, etc.), one can show items / text in Scrivener the way a data base work, show the text / text pieces / ideas (and above all organize / manage them) according to special specifications, criterions (e.g. you have projects with the same locations, same / different ideas which could work with different projects, and many more, you can show all of one’s ideas only, ideas belonging to special projects etc. ), so the feature to show just the text you want to see. I hope, it really works like that, I will find out in practice, I assume.

Also, if you haven’t already, it’s worth your time to go through the Interactive Tutorial project, available from the Help menu.

Yes, I partially have, I will finish it very soon.

Thanks again

[kewms]

To create a Scrivenings session, Scrivener has to read in all of the component subdocuments. So if you split into chunks, then read them all back in, you’ve created just as large a memory load as the original file, plus some additional overhead.

A more “Scrivener-like” approach would be to work with a single chapter or scene at a time, which both reduces the load on Scrivener and makes it easier to keep track of where you are.

Moreover, all the features you mentioned – Collections, labels, keywords, etc. – work at the level of individual documents. Using smaller chunks therefore allows you finer granularity for these other tools: if you want, you can break your project down into paragraph-sized chunks, each with a single idea.

Katherine

[Fant]

Many thanks.

Moreover, all the features you mentioned – Collections, labels, keywords, etc. – work at the level of individual documents. Using smaller chunks therefore allows you finer granularity for these other tools: if you want, you can break your project down into paragraph-sized chunks, each with a single idea.

So I could even create one or more or many sub chunks for each single paragraph-sized chunk? That would be great. It sounds like there might be thousands or ten thousands or even more of chunks / pieces / “Texts” / “Folders” at the end. So Scrivener can properly cope with such an amount of slices without slowing down? When I press CTRL+N or CTRL+SHIFT+N the cursor is inserted to the name field in the Binder. How could I make Scrivener add the cursor to the writer / the writing area instead when pressing those shortcuts? And is there a shortcut to create a child item (“Folder” and “Text”) of the item the cursor is inserted to (in the editor) at the moment?

When there are different ideas / expressions for a single sentence / paragraph / scene, how could I manage this with Scrivener, how could I organize those thoughts / ideas best?

[kewms]

When you create a new document, Scrivener needs a name for it. That’s why the cursor stays in the Binder.

The Documents -> Split and -> Split with Selection as Title commands will automatically assign titles to the split documents.

A “chunk” can be as small as you like, down to a single sentence or even a single word. Personally, I use keywords to associate ideas or sources with each chunk. You can have as many keywords per item as you like. In my own work, I tend to start with fairly small chunks and then merge them together as I get closer to a final version.

By default, if you select a text item in the Binder and create a new text, the new item will be a sibling. If you select a folder, the new text item will be a child. If you create a new folder, it will always be a sibling of the selected item.

Katherine

[Fant]

Thank you very much, Katherine.

When you create a new document, Scrivener needs a name for it. That’s why the cursor stays in the Binder.

After entering that name, can you insert the cursor with a single shortcut to the editor (instead of pressing “Enter” and twice the TAB key)?

A “chunk” can be as small as you like, down to a single sentence or even a single word. Personally, I use keywords to associate ideas or sources with each chunk. You can have as many keywords per item as you like. In my own work, I tend to start with fairly small chunks and then merge them together as I get closer to a final version.

Alright, so I could create ten / hundred thousands or more of (sub / sub, etc.) chunks / pieces / “Texts” / “Folders” without causing sluggishness.

[tiho_d]

Ctrl + Tab within the Binder will jump directly within the Editor.
Having many small chunks of text without causing sluggishness is within the design of Scrivener. Having one big chunk of text is more likely to cause sluggishness. HTH.

[Fant]

Alright, thank you very much