Generated Headers - Quite the Can of Worms

[Vin Shelton]

Henry -

First the good news - with your patches applied, I succeeded in building a
64-bit cygwin XEmacs executable.

Now the bad news - that patched XEmacs does not build under Windows native
(at least on the 32-bit XP VM I use for Native Windows kits). The problem,
as I know you suspect is the encapsulated unicode headers. Thanks for
taking on the challenge of fixing up our interface to the w32api, BTW, but,
as you know, it's a real bear.

After running the make-mswin-unicode.pl script, I get a bunch of
definitions that refer to WINBOOL, which is not a native Windows type.

Here is the first such message:

console-msw.c
C:\Cyg_opt\src\xemacs-21.5-2015-03-26\src\intl-auto-encap-win32.h(105) :
error C2146: syntax error : missing ';' before identifier
'qxeSHGetNewLinkInfo'
C:\Cyg_opt\src\xemacs-21.5-2015-03-26\src\intl-auto-encap-win32.h(105) :
error C2501: 'WINBOOL' : missing storage-class or type specifiers
C:\Cyg_opt\src\xemacs-21.5-2015-03-26\src\intl-auto-encap-win32.h(105) :
fatal error C1004: unexpected end of file found
NMAKE: fatal error U1077: 'cl' : return code '0x2'

I have tried 2 different approaches: defining WINBOOL and LONG_PTR and a
few others in syswindows.h. In hindsight, this was the more fruitful
approach, but not all the necessary source modules seem to include
syswindows.h (the details escape me at this exact moment).

My second and more braindead approach was to change each of the functions
like SHGetNewLinkInfo from "yes" to "review". I did this for probably 10
functions, but I didn't achieve joy (or a clean compile) and this is,
fundamentally not the right way to go.

Any thoughts on how to get the WINBOOL, etc types defined for the native
Windows build?

Thanks,
Vin

[Henry S. Thompson]

Post by Vin Shelton
First the good news - with your patches applied, I succeeded in building a
64-bit cygwin XEmacs executable.

Good news, yes, thanks for pressing ahead!

Post by Vin Shelton
Now the bad news - that patched XEmacs does not build under Windows native
(at least on the 32-bit XP VM I use for Native Windows kits).

So, what compiler are you using at this point -- some MS compiler, or
are we talking mingw here?

Post by Vin Shelton
After running the make-mswin-unicode.pl script, I get a bunch of
definitions that refer to WINBOOL, which is not a native Windows type.

Hmm. I suspect a combination of my bad memory and unfamiliarity with
the overall build architecture is to blame, but I didn't think your
should have to run that at all, should you? If you did, it should be
part of configure, right? I will have to try harder to understand the
division of labour.

Post by Vin Shelton
console-msw.c
error C2146: syntax error : missing ';' before identifier
'qxeSHGetNewLinkInfo'

Can you send a copy of your intl-auto-encap-win32.h, so I can look at
what is different?

Post by Vin Shelton
Any thoughts on how to get the WINBOOL, etc types defined for the native
Windows build?

Not yet :-(.

ht

[Henry S. Thompson]

Post by Henry S. Thompson
So, what compiler are you using at this point -- some MS compiler, or
are we talking mingw here?

This is Visual Studio 6. Building with a newer version of VS will
require a whole 'nother set of work.
Mingw (either 32 or 64) remains a dream for the future - ideally we
could cross-compile using one of those compilers. Still dreaming.
:-)

Ah, OK. I will see if I can remember how to install/run the free VS6
setup (I think there is one???).

Post by Henry S. Thompson

Post by Vin Shelton
After running the make-mswin-unicode.pl script, I get a bunch of
definitions that refer to WINBOOL, which is not a native Windows type.

Hmm. I suspect a combination of my bad memory and unfamiliarity with
the overall build architecture is to blame, but I didn't think your
should have to run that at all, should you?

No, but I had compile difficulties building a Windows native (XP,
32-bit) executable with your new intl-auto-encap files, so I decided
to see if regenerating the intl-auto-encap files made a difference.

Understood.

Post by Henry S. Thompson
If you did, it should be part of configure, right? I will have to try harder to understand the
division of labour.

The Windows native build does not use configure. See nt/config.inc
and nt/xemacs.mak if you're curious (and you have a strong stomach -
MS nmake syntax is not for the weak).

Indeed -- been years since I had to use it, but the memory lingers. . .

In general, we could treat the intl-auto-encap files like configure -
we check them in, but people who are sufficiently interested should be
able to regenerate them. Prior to the most recent w32api changes, the
intl-auto-encap files had not been updated for years.

Understood.

Post by Henry S. Thompson

Post by Vin Shelton
console-msw.c
error C2146: syntax error : missing ';' before identifier
'qxeSHGetNewLinkInfo'

Can you send a copy of your intl-auto-encap-win32.h, so I can look at
what is different?

Sorry if I misled you, but I don't think it makes any difference.
...
Stop.

[good advice :-]

Thanks for thinking about this.

You're welcome, of course, and whatever it was you thought you needed
to apologise for, you didn't.

ht

[Henry S. Thompson]

Just a minor note that WINBOOL is defined in various header files under
w32api/
including ntdef.h and minwindef.h

It's the latter that gets included when you compile under Cygwin.

No joy yet in finding a copy of Visual Studio 6.0

ht

[Vin Shelton]

Hi, Henry!

Thanks for your efforts on this. I have copied xemacs-beta on my reply.

Adding the following to syswindows.h at around line 605, in the
_non_-CYGWIN_HEADERS branch
typedef int WINBOOL;
typedef intptr_t LONG_PTR;
typedef uintptr_t ULONG_PTR;
typedef uintptr_t DWORD_PTR;
/* #define PCIDLIST_ABSOLUTE_ARRAY LPCITEMIDLIST *
typedef const PCIDLIST_ABSOLUTE *PCIDLIST_ABSOLUTE_ARRAY;* */
typedef LPCITEMIDLIST PCIDLIST_ABSOLUTE;
allows several of the -msw.c files to compile.
More similar work is required, I think.
Oops: the last typedef above is wrong - I read the commented-lines above
that backwards!

Thanks for making sure the native Windows build continues to work.
I appreciate your taking the lead on this, because I haven't made any
substantive progress on this over the weekend.

Here are a few comments:

1. I have checked in the configure changes, so please make your next
patchsets based off of current version control. Please split your
patchset into two patches: one containing the w32api refactoring /
unicode regeneration and a second one containing the 64-bit
(DEVICE_TYPE_) changes.

2. Back when I was looking at updating the unicode definitions, I had
a slightly different regexp in lib-sr/make-mswin-unicode.pl:

my $rettype_re = "(SHSTDAPI_\\(${tok_ch}+${ws_re}\\*?\\)|${tok_ch}" .
"[A-Za-z_0-9 \t\n\r\f]*?${tok_ch})";

I think this regexp is a little more restrictive than the one you
used, but I'm not sure that it makes a difference.

3. Here are my current definitions for the syswindows.h block:

#else /* !CYGWIN_HEADERS */

#define W32API_VER(major,minor) 0
#define W32API_INSTALLED_VER 0

/* Some types that show up in Cygwin headers but not in Visual Studio headers,
and cause problems if we used Cygwin headers to generate
intl-auto-encap-win32.[ch]. */
typedef LPCVOID PCVOID;

typedef int WINBOOL;
typedef intptr_t LONG_PTR;
typedef uintptr_t ULONG_PTR;
typedef uintptr_t DWORD_PTR;

typedef ITEMIDLIST ITEMIDLIST_ABSOLUTE;

typedef ITEMIDLIST_ABSOLUTE *PIDLIST_ABSOLUTE;
typedef const ITEMIDLIST_ABSOLUTE *PCIDLIST_ABSOLUTE;

/* Defined in w32api/winuser.h */
#define GCLP_MENUNAME (-8)
#define GCLP_HBRBACKGROUND (-10)
#define GCLP_HCURSOR (-12)
#define GCLP_HICON (-14)
#define GCLP_HMODULE (-16)
#define GCLP_WNDPROC (-24)
#define GCLP_HICONSM (-34)
#define GWLP_USERDATA (-21)
#define GWLP_WNDPROC (-4)
#define GWLP_HINSTANCE (-6)
#define GWLP_HWNDPARENT (-8)
#define GWLP_USERDATA (-21)
#define GWLP_ID (-12)

#define DWL_MSGRESULT 0
#define DWL_DLGPROC 4
#define DWL_USER 8

#ifdef _WIN64

#undef DWL_MSGRESULT
#undef DWL_DLGPROC
#undef DWL_USER
#endif

#define DWLP_MSGRESULT 0
#define DWLP_DLGPROC DWLP_MSGRESULT + sizeof(LRESULT)
#define DWLP_USER DWLP_DLGPROC + sizeof(DLGPROC)

#endif /* CYGWIN_HEADERS */


Not all of these definitions are necessary, but I think it's probably
best to include the entire GCLP/GWLP definition block.

Thanks for digging in on this.

-Vin

[Henry S. Thompson]

I can confirm your version gets me exactly as far as the version I
just posted, and that your $rettype_re produces the same result as
mine...

ht

[Vin Shelton]

Post by Henry S. Thompson
I can confirm your version gets me exactly as far as the version I
just posted, and that your $rettype_re produces the same result as
mine...

OK. There are several differences in const-ness (LPCSTR vs. LPSTR) in
the newly-generated intl-auto-encap.c, but fixing those by hand only
gets to the next step.

- Vin

[Henry S. Thompson]

Post by Vin Shelton

Post by Henry S. Thompson
I can confirm your version gets me exactly as far as the version I
just posted, and that your $rettype_re produces the same result as
mine...

OK. There are several differences in const-ness (LPCSTR vs. LPSTR) in
the newly-generated intl-auto-encap.c, but fixing those by hand only
gets to the next step.

Not sure what "the next step" is. . .

Note that the source of the LPCWSTR vs LPWSTR (which is the first
problem I hit:

C:\C64\usr\local\src\xemacs-21.5-64bit\src\intl-auto-encap-win32.c(634)
: error C2664: 'OpenDesktopW' : cannot convert parameter 1 from
'const unsigned short *' to 'unsigned short *' Conversion loses
qualifiers

is caused by this discontinuity:

VC98/include/winuser.h:

OpenDesktopW(
LPWSTR lpszDesktop,
DWORD dwFlags,
BOOL fInherit,
ACCESS_MASK dwDesiredAccess);

/usr/include/w32api/winuser.h:

WINUSERAPI HDESK WINAPI OpenDesktopW(LPCWSTR lpszDesktop,
DWORD dwFlags,
WINBOOL fInherit,
ACCESS_MASK dwDesiredAccess);

I'm guessing this is simply because w32api has been updated to a later
version of Visual Studio include files. . .
Hmm, maybe not -- online MS documentation has LPWSTR, as above. . .

Some apparently mingw versions of winuser.h I can find on the web
have the non-const version. . ., but I can't immediately find any
discussion of the issue.

Since the only function with the LP[C]WSTR issue which is actually
_called_ by XEmacs is GetNamedSecurityInfo, for the time being I tried
editting intl-encap-win32.c to skip all the others, editting
w32api/aclapi.h to change LPCWSTR to LPWSTR and then rebuilding
intl-auto...

Ah -- I guess I see what you were worrying about. GetWindowLongPtr
and friends are not supported by VS6. Full stop. What level is the
VS6 requirement/where is it coming from? VS2008 an option?

ht

[Vin Shelton]

Hi, Henry et al,

Post by Henry S. Thompson
Some apparently mingw versions of winuser.h I can find on the web
have the non-const version. . ., but I can't immediately find any
discussion of the issue.
Since the only function with the LP[C]WSTR issue which is actually
_called_ by XEmacs is GetNamedSecurityInfo, for the time being I tried
editting intl-encap-win32.c to skip all the others, editting
w32api/aclapi.h to change LPCWSTR to LPWSTR and then rebuilding
intl-auto...
Ah -- I guess I see what you were worrying about. GetWindowLongPtr
and friends are not supported by VS6. Full stop. What level is the
VS6 requirement/where is it coming from? VS2008 an option?

I have been thinking about this, too. The only thing tying us to VS6
is that it works and converting to VS2010 would require some effort.
With 21.4 reaching the EOL (I hope not to make any more setup kits for
21.4), this is a good time to start thinking about building the 21.5
branch with a newer toolset.

VS6 was, actually, a good compiler for its day, but that day is long
past. I would encourage us to move to the newest free (as in beer)
version of Visual Studio available.
https://www.visualstudio.com/en-us/products/visual-studio-express-vs.aspx
seems to indicate that Express 2013 is the latest free version
available.

I think Mats has patches to support VS2010 - Mats, can you post your
patches?

- Vin

[Mats Lidell]

Here's what I found from November of 2010.

That should be it! Thanks for doing the digging Vin!

Yours

[Henry S. Thompson]

Right, looks essentially the same as what I had found, plus the
Solution and Project files.

I'm now hitting a segfault at the batch-update-elc-2 stage, the first
time it tries to use xemacs.exe instead of temacs.exe.

More later,

ht

[Henry S. Thompson]

The failures I'm now seeing are all somewhere in dumper.c, at various
places under pdump_load_finish depending on the values of USE_KKCC and
NEW_GC. I haven't found a setting that works. . .

First, I would try to build a 32-bit executable without any of your changes.

Doing that, as near as possible.

I assume you're using the portable dumper.

Yes.

Suggestion: turn off both KKCC and NEW_GC.

Done.

Still seeing the same failures: temacs.exe runs fine, but the dumped
xemacs.exe can't get started.

Here's how temacs.exe is being linked:

link -nologo -debug -opt:noref -incremental:no -base:0x1000000
-stack:0x800000 -entry:mainCRTStartup -subsystem:windows
-heap:0x00100000 -nodefaultlib setargv.obj
-pdb:C:\C32\usr\local\src\xemacs-21.5-native\src\temacs.pdb
-map:C:\C32\usr\local\src\xemacs-21.5-native\src\temacs.map
-out:C:\C32\usr\local\src\xemacs-21.5-native\src\temacs.exe ...

Any chance those base/stack/heap values are out-of-date?

Any possibility the problem lies somewhere in the vicinity of the
problem reported in the parallel thread for the OS X Yosemite build [1]?
I.e. is there any equivalent for VC to the -Wl,-no_pie flag Aidan
mentions in that thread?

ht

[1] http://permalink.gmane.org/gmane.emacs.xemacs.beta/39151