port x-symbol to GNU emacs 24.

[David Kastrup]

What features are those you don't see elsewhere in Emacs? (I'm
specifically asking about GNU Emacs, not XEmacs.)

I think a picture says more than 1000 words, here is a screenshot. (I
hope it will not be blocked).

Actually, it doesn't. This screenshot is not dissimilar what you'll get
from using AUCTeX, preview-latex and TeX-fold-mode.

The main "not elsewhere" category I see are XEmacs' extensive input
modes (grid which is sort of a buffer-based menu, keyboard input methods
quite better sorted and convenient than what Quail offers and with
better feedback). Its buffer reencoding is a rather mixed blessing, but
it works with more than just TeX modes.

[Uwe Brauer]

What features are those you don't see elsewhere in Emacs? (I'm
specifically asking about GNU Emacs, not XEmacs.)

I think a picture says more than 1000 words, here is a screenshot. (I
hope it will not be blocked).

Well, I guess you could use Emacs' built-in `prettify-symbols-mode' in
order to display TeX macros using some unicode characters (or
preview-latex of course).

I just switched `prettify-symbols-mode' on in a latex buffer but nothing
happens to \int or \alpha

GNU Emacs doesn't have extents and specifiers.

But what would provide a similar feature? I hoped somebody on the
list could tell me.

Emacs' overlays serve the same purpose as XEmacs' extents, and I think
there is or has been some 3rd-party package providing a unified API for
overlays and extents.
But I have no clue what specifiers are...

My suggestion is to use the equivalent features provided by GNU
Emacs. If you tell which features you miss, people here could
advise you about the replacements, either in core or in add-on
packages.

Ok, so I have to dig more into the code or hope that somebody in
xemacs-beta could point me out what is the GNU emacs equivalent to
those functions. I still wonder how x-symbol could have worked under
GNU emacs 21, given that extents and specifiers did not exist neither
for emacs 21.

I just downloaded x-symbol 4.5.1, and in its x-symbol-emacs.el there are
several aliases defined for things that error for you. For example,
`map-extents' is made an alias for `cl-map-overlays'. So you have to
load that first.

I thought that as well, I had a (require 'x-symbol-emacs) in my init
file, but it seemed not to have worked. I will give it again a try.

Well, eventually when loading x-symbol.el I also got an
wrong-number-of-arguments error but that's probably an incompatibility
which is easy to fix.

Do you mean the

- x-symbol-set-face-font: Wrong type argument: listp,
"-adobe-helvetica-medium-r-normal-*-14-*-*-*-*-*-iso8859-1"

error? I hoped this is a wrong-number-of-arguments error but I don't
know how to fix it.

I will try again to load x-symbol-emacs manually and then will see.
Thanks

Uwe

[Stephen J. Turnbull]

[XEmacs's] buffer reencoding is a rather mixed blessing,

What do you mean by "buffer reencoding"? We decode external streams
to Mule encoding on input, and then encode them as specified on
output. "Reencoding" (duplicate encoding or transcoding) of buffer
text occurs only if explicitly requested by a Lisp program or as a
user command.

If you mean the fact that XEmacs can't handle the buggy output of TeX
(which produces invalid UTF-8 in error messages), that's a non-standard
encoding we'd like to implement, but the bug is in TeX, not in XEmacs.

[David Kastrup]

Post by Stephen J. Turnbull

[XEmacs's] buffer reencoding is a rather mixed blessing,

What do you mean by "buffer reencoding"?

This earns you a major "Huh?!?" but indeed, looking up the Email you
replied to, it is

The main "not elsewhere" category I see are XEmacs' extensive input
modes (grid which is sort of a buffer-based menu, keyboard input
methods quite better sorted and convenient than what Quail offers
and with better feedback). Its buffer reencoding is a rather mixed
blessing, but it works with more than just TeX modes.

If you replace the brain fart/typo "XEmacs'" with "X-Symbol's", the
paragraph makes wagonloads more sense as the topic of discussion never
was XEmacs. So I've been neither singing XEmacs' praises here nor
complaining about it. This is about X-Symbol.

Post by Stephen J. Turnbull
We decode external streams to Mule encoding on input, and then encode
them as specified on output. "Reencoding" (duplicate encoding or
transcoding) of buffer text occurs only if explicitly requested by a
Lisp program or as a user command.

X-Symbol is such a "Lisp program". And its core operation works via
buffer reencoding. Whether running on Emacs or XEmacs.

[Stephen J. Turnbull]

Post by David Kastrup
If you replace the brain fart/typo "XEmacs'" with "X-Symbol's", the
paragraph makes wagonloads more sense as the topic of discussion
never was XEmacs.

Indeed it does, although I was also confused by "re-encoding". I
wouldn't call what X-Symbol does "re-encoding", I'd call it
"translating". (What you call it is up to you, now that you've
explained I can live with "re-encoding". I'm explaining why the
typo wasn't obvious to me.)

Thanks for explaining.

Steve

[Stephen J. Turnbull]

Ok, so I have to dig more into the code or hope that somebody in
xemacs-beta could point me out what is the GNU emacs equivalent to those
functions. I still wonder how x-symbol could have worked under GNU
emacs 21, given that extents and specifiers did not exist neither for
emacs 21.

AFAIK you're the only user left, and you've spent quite a bit of time,
both yours and others', on trying to keep this code working (which was
idiosyncratic when written, to say the least). I'm giving up. This
is my last post, ever, on x-symbol (except if necessary to veto
changes to XEmacs :-/ ).

Extents are a unification[1] of Emacs's overlays and text properties,
with a similar API. Most likely you want to use the text property API
rather than the overlay API.

Specifiers are a unification and generalization of buffer-local and
frame-local variables[2], but with a rather different API that has to
be invoked explicitly to reference and mutate them, rather than being
a special type of variable that is referenced and mutated in the usual
way. I suppose that you can probably ignore the specifier stuff,
however, and just use Customize to define the appropriate faces
(defface) which are selected for the x-symbol charset.

Another problem you'll run into in more recent GNU Emacs is that the
internal encoding (a UTF-8 extension) no longer has charsets in the
sense that Mule encoding did, as part of each character. So you will
need to define a charset to hold the x-symbol characters (a Unicode
block, possibly in Unicode private space but perhaps outside of the
official range of Unicode scalars) using an API that is not relevant
to XEmacs, I suspect.

Note that it's probably possible to achieve what x-symbol does in 1/5
the code and without defining a new charset or glyphs by simply taking
advantage of the fact that most (all?) of those symbols are already in
Unicode. I believe there are TeX add-ons that allow you to use them
instead of the traditional macros, as well, so you might not even need
to fiddle the display of the TeX file at all, and x-symbol would be
completely irrelevant.

Footnotes:
[1] IIRC, there's a corner case involving text properties that can't
be properly simulated using only extent properties, but it's otherwise
equivalent.

[2] ISTR frame-locals have been deprecated or removed in recent Emacs?

[Uwe Brauer]

Post by Stephen J. Turnbull

Ok, so I have to dig more into the code or hope that somebody in
xemacs-beta could point me out what is the GNU emacs equivalent to those
functions. I still wonder how x-symbol could have worked under GNU
emacs 21, given that extents and specifiers did not exist neither for
emacs 21.

Note that it's probably possible to achieve what x-symbol does in 1/5
the code and without defining a new charset or glyphs by simply taking
advantage of the fact that most (all?) of those symbols are already in
Unicode. I believe there are TeX add-ons that allow you to use them
instead of the traditional macros, as well, so you might not even need
to fiddle the display of the TeX file at all, and x-symbol would be
completely irrelevant.

Do you have any reference, or is this just a guess?

[Stephen J. Turnbull]

Post by Uwe Brauer
Do you have any reference, or is this just a guess?

Post by Stephen J. Turnbull
Note that it's probably possible to achieve what x-symbol does
in 1/5 the code and without defining a new charset or glyphs
by simply taking advantage of the fact that most (all?) of
those symbols are already in Unicode.

That's almost certainly true; I don't know what kind of math you do so
I can't be sure the symbols you need are in Unicode, but certainly all
the math symbols I can remember are in it (I admit I haven't memorized
the symbol tables in the TeXBook.) So I think the odds are quite high
that you can rip out of x-symbol all of the charset code and glyph
definitions, and just use the macro/character swapping code and input
methods.

Post by Uwe Brauer

Post by Stephen J. Turnbull
I believe there are TeX add-ons that allow you to use them
instead of the traditional macros, as well, so you might not
even need to fiddle the display of the TeX file at all, and
x-symbol would be completely irrelevant.

That's a guess, based on the fact that recent versions of xelatex
uglified documents derived from my older templates until I set the
parameter that enables the traditional double-APOSTROPHE,
double-GRAVE, and multiple-hyphen ligatures again.

It's certainly possible to do this, but since these are all going to
be multibyte characters you'd need to do some really obnoxious
gymnastics with active characters.

[David Kastrup]

Post by Stephen J. Turnbull

Post by Uwe Brauer
Do you have any reference, or is this just a guess?

Post by Stephen J. Turnbull
Note that it's probably possible to achieve what x-symbol does
in 1/5 the code and without defining a new charset or glyphs
by simply taking advantage of the fact that most (all?) of
those symbols are already in Unicode.

That's almost certainly true; I don't know what kind of math you do so
I can't be sure the symbols you need are in Unicode, but certainly all
the math symbols I can remember are in it (I admit I haven't memorized
the symbol tables in the TeXBook.) So I think the odds are quite high
that you can rip out of x-symbol all of the charset code and glyph
definitions, and just use the macro/character swapping code and input
methods.

Well, X-Symbol was focused on providing math mainly for LaTeX, and the
STIX project's mission <URL:http://www.stixfonts.org/project.html>
states "All characters/glyphs have been incorporated into Unicode
representation or comparable representation and browsers include program
logic to fully utilize the STIX font set in the electronic
representation of scholarly scientific documents" can more succinctly be
expressed as "let Barbara Beeton dump all of AMSTeX into Unicode".

So indeed I should be surprised if the current state of Unicode did not
actually cover most of X-Symbol satisfactorily by now.

[Julian Bradfield]

Post by Uwe Brauer

Post by Stephen J. Turnbull
advantage of the fact that most (all?) of those symbols are already in
Unicode. I believe there are TeX add-ons that allow you to use them
instead of the traditional macros, as well, so you might not even need
to fiddle the display of the TeX file at all, and x-symbol would be
completely irrelevant.

Do you have any reference, or is this just a guess?

Has it occurred to you to do a web search?

In summary, if you want to do Unicode maths properly (or Unicode
anything else properly), you should really switch to Xe(La)TeX.
If you don't want to do that (and it's not entirely trivial), you can
hack things with [uft8x]inputenc and mathletters.

[Uwe Brauer]

Post by Julian Bradfield
Has it occurred to you to do a web search?

Yes.

Post by Julian Bradfield
In summary, if you want to do Unicode maths properly (or Unicode
anything else properly), you should really switch to Xe(La)TeX.

I am using Xelatex on a regular base, basically for hebrew. I presume
you mean by unicode math, the following package

\usepackage{fontspec,unicode-math}

That however requires special fonts, standard as it seems for TeXlive
2011, but not necessarily standard in older distributions. So if you
wish to send a file with utf8 math symbols to a colleague (as I do
frequently) you might run into a problem.

Post by Julian Bradfield
If you don't want to do that (and it's not entirely trivial), you can
hack things with [uft8x]inputenc and mathletters.

The same comment applies to mathletters.


In short the advantage of x-symbol over utf8 math symbol is:

- it produces standard Latex which is compatible back even versions
prior to Latex2e.

- the x-symbols are much nicer than the utf-8 ones.

[Stefan Monnier]

Emacs' overlays serve the same purpose as XEmacs' extents, and I think
there is or has been some 3rd-party package providing a unified API for
overlays and extents.

Indeed there is: it's overlay.el (which adds Emacs's API to XEmacs and
hence doesn't help here).

But I have no clue what specifiers are...

Presumably this code was not run in Emacs-21, so you'll have to figure
out why (maybe some `fboundp' which returned t in Emacs-21 but doesn't
any more?).

I just downloaded x-symbol 4.5.1, and in its x-symbol-emacs.el there are
several aliases defined for things that error for you. For example,
`map-extents' is made an alias for `cl-map-overlays'.

Was renamed to cl--map-overlays back in the cl-lib overhaul.

The main "not elsewhere" category I see are XEmacs' extensive input
modes (grid which is sort of a buffer-based menu, keyboard input methods
quite better sorted and convenient than what Quail offers and with
better feedback). Its buffer reencoding is a rather mixed blessing, but
it works with more than just TeX modes.

I never found the buffer-reencoding very convincing/convenient and even
less so nowadays, but indeed I sometimes miss the various input methods
of X-Symbol. Ideally, those should be extracted into their own package.


Stefan