Web Pages

Two things (related)

I want to important web pages into my research space - I am running OSX Snow Leopard (10.6) (and firefox is my default browser, but I do have Safari on my mac-book pro.

When select “import web page” and put in a URL and title, I get a message “Error importing web page” import of the web page 'http://www.websiteiamlooking for" failed with the following error: Invalid URL The RL as invalid and so could not be converted to a web archive. Try saving the problematic web page from Safari as webarchives and then importing the .webarchive files into Scrivener.

I have tried all sorts of web sites - they all appear to come up with the same message (which makes one wonder why the “import webpage” is there at all. As mentioned, I tend to run Firefox, so opening a safari browser, cutting and pasting the URL, doing a save as, and then importing to scrivener is tedious and convoluted (Fixing the import would be great, but .webarchive" support in FF would be a second best (yup, understand that that is probably a FF problem not a scrivener problem

Second point - when I have imported a web page, It would be handy to be able to highlight relevant important bits. BUT… when I import a web page, all of the font/bold/highlight options at the top are greyed out - is there a different way of doing this such that I would be able to annotate those imported bits…

  1. Update to the latest version of OSX.
    I run 10.9.4 on an '08 iMac with no problems.

  2. Try another browser, like Chrome for Macintosh.
    I never have a problem importing URLs.
    And I just drag/drop, rather than use a menu command.

  3. Good luck!

This is my standard MO for getting research on the web into my Scrivener project.

  1. On the web page, I select the parts I want to preserve. Ads and navigation columns I usually skip as I am interested in the content of the article and any pertinent graphics. Command-C to copy my selection.

  2. I make a new document in my Scrivener project (this can be in my Research folder or in the Draft directly). Then I Command-V paste in the selection. I go to the heading and select it and use selection as title.

  3. Any annotations and notes I make to the document, I will either do as document notes outside the text proper, or as inline annotations to make sure that I can see what was the original and what was my own note.

  • asotir

First off, your default web browser shouldn’t matter; it’s my understanding that Scrivener does the import on its own, using programming code that is similar (maybe identical) to what is used in Safari for saving a web page as a .webarchive. Don’t hold me to that; I’m no expert.

Does this happen for all web pages? Could you give an example of the two fields, address and title? Try using CMD-A with your cursor in the URL field and then CMD-C to copy it. Is there a visible difference between that and what you usually end up pasting into the address field?

Also, what version of Scrivener are you running?

Re Druid - update to latest OSX - I wish - it’s a 2.16 Ghz Intel Core Duo - the A1150 model - 2006 hardware - yup, 8 years old, but a new one isn’t in the budget at the moment.

Version of Scrivener 2.5

Using Chrome - not an option - it doesn’t like the age of the hardware.

I am off to try several more websites, to see if there is something specific that is blocking this from work…

I don’t use Firefox, as I don’t feel at home in it and found that it didn’t like some sites that I needed.

My default is OmniWeb — now 64-bit in version 6 betas, though not all functions have been restored yet and development is slow — and Chrome for use when I need access to 1Password and bank security software. Safari I find irritating and problematic.

But Opera, which used to be my second-string, has re-appeared — I think in 64-bit form — and I’ve always liked that and found it capable. So you might like to try that.

Mr X

PS I should think the 32-bit versions are still available … I used it on 2005 hardware.

Scrivener is acting like a web browser. So as in web browsers you cannot bold text as in a word processor. You could PDF the web page and highlight using adobe reader … and then bring it into Scrivener.

For page content that I want to trim down, edit or annotate, I convert it to ordinary text. You can either do this by using the Documents/Convert/Convert Web Page to Text, or by setting the “Convert imported WebArchives and web pages to text” option, in the Import/Export preference pane.

But for myself, I find it very rare that I value the original web page layout over the ability to remove all of the navigation, ads and other junk. Your mileage may vary, as they say. :slight_smile:

Contrary to what I said a while ago, I am now having trouble with drag-drop URLs.
Happened recently with some that contain https:// code; the colon doesn’t copy
It happens both when I drag the URL code to the Binder, or when I create a link
With Cmd-L in text. Very puzzling, and I’m trying to figure out if there’s a pattern.

To make sure we are on the same page, do you have Cmd-L rebound to the standard hyperlink tool, Edit/Add Link…? Because by default that adds a Scrivener Link. I had no issues adding a hyperlink to “https://www.duckduckgo.com”, but do make sure you set the tool to not add a prefix (such as “http://”) as that will of course make a mess of the URL.

With dragging the URL in, I tried dragging from the URL bar icon in the browser:

  • …into the text editor: OK
  • …into the References pane: OK
  • …into the Binder as:
    [list][*] WebArchive: OK
  • Text conversion: OK
    [/*:m][/list:u]
    All tests were done with the latest 2.5.5 public beta.

Thanks for that answer; I see now the source of my problem.
Long ago, I re-mapped Cmd-L to bring up the dialog for inserting web links.
The same command works in Apple Mail, and I wanted them consistent.
(Scrivener Links I insert via the Control-Click menu.)
On the standard Insert Link dialog, the prefix is http:// with no s
And that produces a 404 Not Found error.
But the dialog, I now see, also has a No Prefix option.
When I choose that, the URL link works fine.
Whew. :open_mouth:

Yup, what was happening is that it was trying to create a link with the URL, http://https://whateverwhatever.com. Now, this dialogue will detect and strip out a redundant ‘http://’, but it’s missing the potential for there to be a secure URL, and that’s probably an easy thing to fix.

Genius ! :bulb:

I am still not seeing the ability to Import web pages. I even tried this with literatureandlatte.com

Still get the "Error importing web page.

Hi OP … no idea if you have fixed this issue or not … but I thought I would make a comment in passing after some tidying up work that I did to my current project.

I wanted to know why my current book/project had increased to ~80Mb from my first book of ~35Mb. So I explored the file contents of the .scriv package, and viewed the files in size order.
Right at the top were three web archives. Three of them were ~7Mb each. When I previewed them it turned out they were the web pages I had added to my Research section.

So what I did was, I revisited the web pages, copied their URL, went back to Scrivener and added a txt Doc in the research section, named it the name of the web page, and pasted the URL into the text body. When I hit return the URL turned into a link. Next I deleted the original imported web page and emptied the Trash. Job done.

I did this for several of my imported web pages and the Book.Project is now down to 31Mb.

I am not saying everyone should do this. I am simply saying that the web pages certainly add to the file size, and considering I am connected to the web 24/7 … when push comes to shove I don’t need to keep a copy of the site.

Also … for another web site that I do want a copy of, I saved as a PDF and kept that instead … also a much much smaller file.