Advanced Search and Replace: emojis to text in a specific font

Scrivener Scrivener for macOS
, ,
1

There’s no way, is there, to Find and Replace every occurrence of an Apple-included Emoji with text; but not just any old text - text in a certain font?

Using Scrivener 3.3.1 on macOS 13.4.1 I have a Scrivener Project where I have used Apple emojis to represent musical symbols (sharps, flats, time signatures etc). I can now do this using a special font.

For instance, ‘#’ displays a true sharp sign - instead of the emoji, which is the wrong size, has the wrong metrics etc.

This seems like something that’s unlikely to be possible - especially forcing the replace string to be in a specific font. But everyone here is so resourceful, imaginative and helpful, that I thought I’d give it a try.

Thanks in advance!

2

If the emoji you no longer want is of a specific font, perhaps find one then search similar formatting. (?)

image
image363×592 12.9 KB

(It doesn’t need to have a style applied to use this function.)

1 Like
3

I’ve been checking. Within Scrivener, you can’t do it in one go for an emoji, but you could Find-and-Replace each emoji with the character that will become the music symbol… e.g :slight_smile: → #, then Find All # and choose your notation font from the font viewer (Cmd-T), which should change them all to the chosen font, I think. You’d have to do that separately for each emoji, of course.

Personally, I’d wait until I’d compiled and do it in Nisus Writer Pro, where you can do it (I don’t know about Word).

:slight_smile:
Mark

P.S., If you can do it in your word processor of choice, the other option would be to use “Sync with External Folder”, and do it in the WP. Going forward just set up a character style for those symbols, if you haven’t done it already. :slight_smile:

1 Like
4

This doesn’t help you out right now, but going forward I would consider using a dedicated character style for such symbols. Had the emojis been in a style you could have then done a global search and replace to #, and then redefined the style to use a particular font.

1 Like
5

Thanks, @Vincent_Vincent.

There are only a few emojis, fortunately; but they’re scattered all over a fairly big Project.

I’m ‘converting’ them to their infinitely-better ‘equivalents’ in Musglyphs.

6

Thanks, @xiamenese.

That two-step procedure ought to work, oughtn’t it.

But, because they’re musical symbols in this font, Musglyphs, I’d be changing a flat (♭), say to just ‘b’ in Musglyphs.

Unfortunately, I can’t guarantee a space either side of ‘b’ (or even ‘#’) to make replacing all such occurrences reliable.

I do see, though, that emojis seem to carry Unicode info - viz:

♭MUSIC FLAT SIGN Unicode: U+266D, UTF-8: E2 99 AD

I wonder whether that could be searched on somehow?

1 Like
7

Thanks, @AmberV. That’s exactly what I’ll do once I’ve converted all the emojis, Yes!

8

See your point. When you said “emoji” I took you literally as using an emoji for future replacement. I don’t look into the farther reaches of the symbol pages:

What never?
No never!
What never?
No never!
What never?
Well… hardly ever!

:laughing:

Anyway, so I’d forgotten that unicode includes those musical symbols.

So do it the other way round:

  • Find All ♭ etc. (just copy and paste them into the find field)
  • Set font to Musglyphs
  • Replace with b or whatever.

The only problem is at the second stage you might end up with an empty glyph symbol if Musglyphs doesn’t include those codepoints, but it should be resolved at stage 3 if they are all still selected.

I’m trying to work out how to enter the code points rather than the symbols into RegEx.

Mark

1 Like
9

Thanks, Mark!

There is hope then… :slight_smile:

Am I Finding in the wrong place:

search
search1920×1337 134 KB

?

10

Check your search criterias (?)…
Click the magnifier glass on the left of the search input. (Your big red arrow.)
As it is, perhaps you are not searching inside the text at all.

1 Like
11

I definitely was not. But in Title (the default?)

Now I see.

Thanks for your patience, @Vincent_Vincent.

I’m still learning Scrivener.

With help like this, it’s all getting easier and easier :slight_smile: !

1 Like
12

My patience ? :slight_smile: @xiamenese did all the work.

1 Like
13

@Vincent_Vincent,

Yes, your patience. I should have known about defining my criteria using the dropdown and thereby selecting which area of the page etc to look in. I do now.

Very grateful to Mark as well.

What a great community.

1 Like
14

Mark - hello again!

I finally had time to experiment with this and have a couple of questions, please because what I thought was pretty straightforward I’m not having much success with.

Not sure which find field to use:

  • Edit > Find > Search in Project, or
  • Edit > Find > Project Replace, or
  • Edit > Find > Find

or does it not matter?

Yes, I get the Font dialogue box with cmnd-T. But…

… I can’t so far get it to apply to the character (e.g. ‘b’ for a flat) which I want to change to emoji to… I get ‘not found’.

So, at what stage do I set the replacement font, please?

Appreciate your help here; I know this must be simple :slight_smile: .

15

Hi Mark,

I haven’t forgotten your problem, but I have been busy with other things, I’m afraid. Also, I’m going to be out this morning, but will get back to it this afternoon.

:slight_smile:

Mark

16

Anything but!

I’ve been doing a lot of experimenting. The first thing I discovered—since I hardly ever use Find and Replace in Scrivener; I tend to shove any such operations, especially if they’re complex, over into Nisus Writer Pro—is how basic the Find and Replace in the Scrivener text editor is—Cmd-F—with no “Find All” command, only a “Replace All” command.

And although you can find every instance using Project Find, you cannot modify all the finds in one go. You would think therefore that you can go through them one-by-one, highlighting them, then changing the font and then replacing them with the character you need—i.e. “\u266D” to “b”. But the thing is that automatic font substitution means that whatever font you choose, and that probably includes Musglyphs, if it doesn’t have a glyph for that codepoint, it will not accept the change. So you can type the “b”, and though it is no longer highlighted so you can’t change the font, you can Shift-Left-Arrow to re-highlight it and then change the font.

However, although all the other instances of \u266D are marked, because you have exited the Find by typing “b” and so you can’t use Cmd-G to move on to the next one.

So the best way I have found is:

  1. Go into Scrivenings mode.
  2. In the editor, Cmd-F
  3. Either copy and paste the “flat” glyph or switch on RegEx and enter \u266D
  4. Find Next
  5. Select the ‘found’ glyph
  6. Type “b”
  7. Cmd-Left-Arrow to reselect
  8. Change font
  9. Cmd-G
  10. Repeat from 5–9
  11. Rinse and repeat.

As I said, it is a one-step process in NWP, as you can “Find All” and “Replace All” changing codepoint and the font. I presume you could do that in Word or LibreOffice, though I don’t know as I don’t use either.

Just for information for anyone who reads this, not only have Nisus done a lot of work on their Find and Replace dialog, but it turns out that they use the open-source Oniguruma RegEx engine, not the one provided by Apple, hence the slight differences.

:slight_smile:

Mark

1 Like
17

Mark,

Absolutely no problem at all: I’m extremely grateful!

No kidding! No wonder I’m a little stuck atm…

So have I. It’s a lot more complicated than I thought - and also perhaps than you did by the look of it?

I hope it can do more than a few hours experimenting can have so far revealed to me. I read and reread section 11 of the manual. But I couldn’t immediately find a way to force a font upon a ‘Replace’ string.

At least not in terms of font, that I could see.

Sidetrack: what I thought (and what definitely started out in TextEdit as an Emoji chosen from Apple’s Character Viewer) was an Emoji shows in Scrivener’s Font dropdown at the top of the Editor (though I’m not sure it’s correct) as a character in the Hiragino Mincho ProN font. But I’m doubtful that this is relevant anyway because the same conundrum of applying fonts as Replace properties still applies… at least I think it does.

Anyway, I can reliably replace the Emoji/character from the Hiragino Mincho ProN font (?) with another string - such as the very word ‘flat’ - throughout the document, using Edit > Find > Project Replace.

But I can’t change its font, any (such) font at the Replace stage.

Maybe @AmberV could help here, please?

Next I tried this:

  1. created a new Note
  2. (moved it into ‘Research’ for the sake of tidiness)
  3. successfully started a lookup table/reminder for me in that Note (my step 1 here) with lines like this: ‘Flat ♭: b’ etc where I have both the Musglyph wanted character and the non-Musglyph key used to achieve it; this I hoped to use both as a list to refer to and a ‘base version’ of each glyph to use for pasting purposes
  4. tried pasting that resulting symbol to take the place of the Emoji into the Replace field of Edit > Find > Project Replace, from where it certainly copied as a Musglyph character. The Copy seemed to be loaded with the (correct) font; perhaps it was not.

Certainly it only ever showed up/was pasted into the Replace field of Edit > Find > Project Replace as the original ‘b’ in Palatino; IOW the font format is not carried across the paste.

I would hope so, Yes - or something similar.

Unfortunately, I could not get that to work. Thanks: it seems logical, and likely to do the trick, Mark. But I have - in addition several hundred of at least half a dozen such glyphs (music notation characters). Your step 10 - unless it could be automated/scripted - even if it worked with Musglyphs) seems a bit of a slog.

But I’m still extremely grateful for your experimentation - really :slight_smile: !!

Nor I Word. I’m nervous about trying LibreOffice (which has saved my skin more than once - chiefly in reviving old ClarisWorks documents (!)… Yes, ( go back that far :slight_smile: ) for fear of losing other formatting and links etc since I’d have to take the document out of Scrivener to work in LibreOffice on Find and Replace.

The next thing I did - thanks to this post from @AmberV - was to try and define a new character style using the selection of one of the Emojis/characters apparently in the Hiragino Mincho ProN font and then Edit > Find > Find by formatting. But that never seemed to move on from the single selected flat I’d used to base the new style on. Could I be getting closer?

18

For a bit of light relief:

I too go back that far, though I never actually used Claris Works myself—I used Word 5.1a, which was absolutely brilliant… and then Microsoft re-imagined it following the ghastly Word 6 for Windows, which is the point where I gave up on Word!

Anyway, back then, when I was happily using Word 5.1a, I was working at the University of Westminster School of Languages, and the IT-support chappie sent a tearful student to me. She was one week away from handing in her dissertation (UK: Thesis for PhD, Dissertation for lower levels!) but could no longer open the file on the floppy! She had been using Claris Works—the School’s Mac lab was running Nisus Writer which couldn’t open it—could I help?

Since nothing else worked, I thought, “I wonder what would happen if I tried to open it in Word.” Guess what? It started to open, then came up with an alert “Bad sector on the disc. Do you want to continue opening?” So I said “Yes!”. It opened minus the 512 Bytes of that sector, and saved out onto a new floppy. All she had to retype was the 512 Bytes of lost words!

As for your current problem, my approach worked for me on a short trial project, though I don’t have Musglyph to try it against. I only tried with ‘flat’… I don’t know what character Musglyph needs for ‘natural’ and it seemed to me that ‘sharp’ as # shouldn’t be a problem as I didn’t think you’d be using the octothorp for any other purpose in your text.

As I said, I have no knowledge of whether or how you can do it in Word or LibreOffice. I can only think of one other person on the Scrivener forum who uses NWP, which is a pity, as it also uses RTF as it’s native format, being built on the same text engine as Scrivener, though more heavily developed as a traditional word processor. I have not had reason to try doing it, but I should think using Sync > With External Folder… ought not lose other formatting and links as the document format doesn’t get converted. But anyway…

And yes, I agree with Ioa (@AmberV) the way forward is using a character style.

By the way, in my experimenting, I found the default Hiragino Mincho ProN glyph didn’t affect the line hight, but as a font designed for two-byte characters, it had too much space before and after. Arial Unicode MS is one that includes those glyphs but without the extra spacing, so using that within TNR 12pt looked OK.

19

Mark,

Thanks again!

:slight_smile:
I think I must have begun on System 6! What was that - mid 1990s?

Before then I was (also) on Acorn… Archimedes (having begun with a BBC Micro etc), which I expect you’ll have known (and possibly used), also being British?

which was also how we distributed operating systems then; seems hard to believe, doesn’t it. I even had some limited experience of CP/M on 8" Winchester disks in the 1980s.

Word must have been good to cope with that. I’m not a fan of what we used to denigrate as M$ :frowning: .

I bet you felt good having rescued that thesis, Mark. Do you remember what its subject was?

In those days we also had floppy disk byte and sector editors (for Acorn, at least) to try and ‘inspect’ some software’s copy protection.

Thanks. So I tried again - and it did work for me. Thanks.

Being still a learner with Scrivener I was going wrong with your step 6 and entering ‘b’ inside (each of) the Find & Replace boxes with which I was experimenting.

I can now get it to work perfectly, glyph by glyph. But with hundreds of them - even in Screenings mode - it’d take ages. If it were possible to script or automate it, though…

But I think you got what you do have spot on.

Not used that yet.

I did succeed in LibreOffice, however; it allows you to set the format of the Replace string.

Was I right (after making and saving, of course, the same identical backup of my Scrivener project as I have been doing all along) to:

  1. select a single Note (could I even select them all in Scrivenings and expect to be able to put them back afterwards)?
  2. copy that Note’s text in its entirety
  3. create a new Text doc in LibreOffice
  4. carry out the Find and Replace
  5. copy and past that text back into the Note where it came from in Scrivener?

Would I run the risk of losing anything integral to Scrivener by doing this, please?

I do want to keep everything in one Scrivener document and not be referencing files outside it - if that’s possible.

(If not, I see that NWP has a 15-day trial version. Not sure how the file exchange would work between it and Scrivener. A little sensitive, too, to the ethics of ‘exploiting’ a trial version.)

It certainly does: looks really messy; yet another reason to replace it with the standard now - MusGlyphs.

20

I was not an IT professional and never a programmer. I was a lecturer in Linguistics, but found that I have a geeky underpinning so felt at home with computers. I was introduced to them using a BBC micro as a terminal to a mainframe and using NROFF piped to a dot-matrix printer for producing text. At one point, when the colleague responsible for staff computing left (my colleagues could never understand what he was saying so only 7 of us out of 50 or so had computers in our offices!), I was asked to take over the rôle. Not being an IT professional actually helped because I talked to them like another ordinary user in their own language, so to speak.

In two years, only 1 member of staff was not computerised, but basically because his office was so piled up with paper everywhere that there was no surface to install it on. After two years, I repurposed it because it was still sitting packed in its box… and had to put up with his howls of complaint, “That’s my computer!” “Clear your desk and you can have it!” It never happened.

Shortly after that I was relieved of the rôle because it was taking too much of my time… but colleagues still came to me rather than IT support for help!

Anyway, back to your problem.

If you want to try NWP, here’s a zipped folder, which contains: (1) “flat.rtf”, a short text with a couple of flat-emojis; (2) text-1.png, a screenshot of it before running Find and Replace; (3) find-and-replace.png, a marked-up screenshot of the Find and Replace dialog (see below); (4) text-2.png, a screenshot of the text after running Find and Replace; and (5) a screenshot of the Formatting Examiner palette showing that the replacements are in the Comic Sans font (a random choice to show the difference); (6) creating-regex.png, a screenshot showing how it can help you construct complex RegEx expressions.

Nisus replacements.zip (1.3 MB)

I suggest you start with that file to try things out. To set up your F&R dialog:

  1. Click on this area to set up “Powerfind Pro (regex)” though you could use ordinary find and copy-paste the glyphs you need to replace.
  2. Check that “where” is the entire file, which it should be as that is the default.
  3. Enter the code or paste in the glyph in “Find what:”
  4. Turn “Replace formatting” on.
  5. Enter b (for flat), then select it and choose MusGlyphs from the Font dialog (Cmd-T)
  6. Click “Replace All”.

All your flat-emojis should now be converted into MusGlyph flats, just as mine were converted into “b” in Comic Sans as you can see in the Formatting Examiner—if you look at text-2.png, you’ll see they are all still highlighted.

I say, should because I don’t have MusGlyphs to test and don’t know if it works like ordinary fonts.

Note, in the F&R dialog, if you turn on “Formatting sensitive” below “Find what:”, you can search for all text in a given font, variant, style… screenshot (6). It’s very powerful.

So play around with that file, trying out all the others you need to convert. When you know if it’s working for you, I’d make a duplicate project or new project with a representative document dragged into it, then try Sync with External folder, using RTF, and try opening that in Nisus, running the F&R and sync’ing back to see what happens.

I might give that a try myself later to see if there are any gotchas.

:slight_smile:
Mark

1 Like