 |
 |  |  | ||
| Using Jactivex to Create a User-Defined Type for Java/COM Marshaling | Previous | Tools | Next |
Introduction to Java/COM Custom Marshaling for User-Defined Data Types
This article explains how to implement custom marshaling of user-defined types between Java and COM. Basically, this involves writing a hook class to do the marshaling, and using the jactivex tool with the /javatlb command-line switch to create a Java type library information file (.jnf) that describes the custom interface.
A hook class is a special class that you write for each user-defined type you want to marshal from COM to Java. It contains the code that actually does the marshaling and translates the COM type to a Java type and the Java type to a COM type. Much of this article addresses what a hook class is and how to author one.
Jactivex is a tool that can create Java libraries from existing COM interfaces. COM interfaces are based on type libraries, so a Java type library is a file that maps user-defined types in existing COM type libraries to hook classes.
What Is Custom Marshaling and Who Uses It?
Marshaling of data types is essential whenever a program spans languages, since data types are never exactly the same in any two languages. Since the Microsoft VM provides the ability to wrap COM libraries as Java classes, marshaling must occur between Java classes and COM type libraries. The Microsoft VM supports a certain level of intrinsic marshaling by default.
As an example, suppose you have a COM library that uses various data types as parameters and return values for its methods. For these methods to be callable from Java, the data types need to be converted between the COM types and Java types. For some types, like integers, floats, booleans, and strings, the Microsoft VM for Java provides intrinsic marshaling support that automatically does the type conversion.
In addition to the conversion work, the COM types must be represented as Java types to be callable from Java. This requires a Java .class file that contains type declarations in Java format. This class file is usually created by running a type-conversion tool such as jactivex (Jcom or Javatlb in older versions of the SDK) on the type library of the COM server. These conversion tools generate the correct type mappings for all types that are supported by the intrinsic marshaling of the Microsoft VM.
The Microsoft VM does not, however, have intrinsic support for complex COM library data types, such as C structures. To marshal such data types requires that you create a user-defined Java type and write a hook class that does the marshaling.
Since jactivex cannot automatically create type mappings that work for user-defined types, you will need to provide jactivex with information on how to generate the Java type declarations. This is done by creating a Java type library information file (.jnf) file, which describes your custom data types, and passing it to jactivex it creates the COM library class header file.
The .jnf file has the following format:
[Custom] <TypeLib Type>= <Java Type>, <Hook Class>
Where:
| TypeLib Type | A user-defined type declared in the COM type library. |
| Java Type | The Java type to map to the COM type. |
| Hook Class | The class that does the translation. |
An Example of Custom Marshaling
The following is an example of how to marshal a simple structure type between a COM type library and Java. Suppose you have a simple COM library, represented by the following pseudo-ODL (Object Description Language) example file.
typedef struct POINT
{ long x;
long y;
} POINT;
interface IPlotter : IUnknown
{
HRESULT DrawLine([in] POINT *start, [in] POINT *end);
}
coclass CPlotter
{
[default] interface IPlotter;
}
To implement or access this library with Java, you would use the following procedure.
- Write a plotter.idl or plotter.odl file for the COM library and pass it to the mktyplib tool (which can be found in Microsoft Visual C++) to generate a type library file called plotter.tlb.
- Write a .jnf file that describes all the user-defined types.
Here is what the .jnf file would look like for the
POINT type:
PLOTTER.JNF
[Custom] POINT = java.awt.Point, PointMarshaler
This declaration instructs jactivex to substitute the Java type java.awt.Point when it encounters a reference to the type named POINT in the type library. It also tells the Microsoft VM to use the PointMarshaler class as the hook class for the POINT type.
Important: The .idl and .odl files differ on how they expose the type name. For example, consider the following type definition:
typedef struct _Foo { ... } FOO;If you are writing an .idl file, you need to put the struct tag name ("_Foo") in the .jnf; if you are writing an .odl file, you must put the type definition name ("FOO").
- Pass the names of the type library file (.tlb) and the Java type library information file (.jnf) file to jactivex using the following command:
Jactivex /javatlb /n plotter.jnf plotter.tlb
(Remember that for jactivex, like its predecessor Javatlb, the default output directory is in your TrustLib directory, so you may want to use the /d switch to specify a base directory.)
Jactivex will create two .java files that include special @com javadoc syntax required for the jvc compiler to generate the extra bits needed by the Microsoft VM to treat these files as Java/COM classes. For example, the name of the PointMarshaler class is part of these bits.
For this example, jactivex generates two files: CPlotter.java (representing the coclass), and IPlotter.java (representing the interface). Excluding the @com javadoc syntax, emitted by jactivex and used by the jvc compiler, the generated source for IPlotter.java looks like this:
// In the actual file, you will see extra syntax that // represents the extra bits needed for the VM Java/COM // marshaling layer. interface IPlotter { public DrawLine(java.awt.Point start, java.awt.Point end); } - Compile CPlotter.java and IPlotter.java using jvc. You must use jvc version 1.02.3920 or later, which interprets the extra @com syntax that jactivex generates. This version of jvc is included in the /BIN directory of the Microsoft SDK for Java version 3.0 pre-release 2.
- A Java application can now load and use an installed Plotter
library by invoking:
IPlotter plotter = (IPlotter)(new CPlotter()); plotter.DrawLine(new Point(x1,y1), new Point(x2,y2));
A Java class can also implement the Plotter library by using IPlotter as shown in the following example:
class MyPlotterImp implements IPlotter { void DrawLine(Point start, Point end) { ... } }
The sample files for this article can be found in the SDK for Java version 3.0 pre-release 2. The sources for a working copy of this example is under the SKD-Java.20\Samples\CustDoc\plotter subdirectory. The sources for PointMarshaler are under the \CustDoc\point subdirectory. To execute the sample, change the current directory to the \CustDoc\prundir subdirectory, and run the go.bat file.
How JAVA/COM Custom Marshaling Works
How does the Microsoft VM for Java actually marshal the call from Java to Plotter? In the interface file IPlotter.class, there are special bits associated with the start and end parameters that indicate that these parameters are a user-defined class and that the hook class is PointMarshaler.class. These bits are all generated by jactivex.exe.
When a Java application invokes the DrawLine method, the Microsoft VM loads the PointMarshaler class (if it hasn't already). It then looks for a public static field named cbByValSize, the presence of which indicates that the COM type is a fixed-size. In this example, PointMarshaler happens to expose this field. It is equal to 8, which is the size (in bytes) of a POINT structure.
The Microsoft VM, therefore, allocates 8 bytes of space on the stack, and invokes another public static method on PointMarshaler called copyToExternal. The Microsoft VM passes the copyToExternal method a pointer to this 8-byte space and a reference to the end parameter, which in this case is java.awt.Point. Be aware that the pointer to the 8-byte space is passed as an integer value.
With this information, copyToExternal translates the java.awt.Point to a POINT structure in that 8-byte space. The same routine is repeated for the end argument. The COM DrawLine method ultimately receives a pointer to the two stack buffers as its start and end parameters. Since both parameters were marked [in], there is no further action for this simple type.
Note: The previous explanation was necessarily simplified for brevity. Much of the hook class analysis implied here is done prior to any method calls and does not actually occur at method call time.
Hook Class Overview
Aside from the extra step of writing a hook class and listing your data types in the .jnf file, the previous procedure is no different from that of integrating any other COM library with Java.
As previously discussed, a hook class is a collection of static methods and fields packaged as a Java class. The hook class must be installed and visible to the classpath on any system where the user-defined types are used. Otherwise, hook classes remain fairly hidden. The Microsoft VM loads the hook class as part of loading a jactivex-created interface file that references them, and invokes methods on the hook class to marshal the user-defined types.
All hook methods and fields are static (that is, per-class, rather than per-instance). The Java/COM integration layer of the Microsoft VM will never create an instance of a hook class. There are approximately 10 fields and methods defined by the Hook Class Specification. All hook members, however, are either optional or have reasonable defaults. Most hook classes will need only to implement a subset of the members.
The following list shows the responsibilities of a hook class:
- Specifying the Java type (referred to generically as JTYPE). JTYPE can be a primitive type (int, double, and so on) or a reference type (objects and arrays).
- Specifying the size, in bytes, of the COM type (referred to generically as ETYPE). ETYPE can be variable-sized, in which case, the hook class will indicate this fact by not offering a size.
- Providing code (methods) to translate JTYPEs to ETYPEs and back.
- (Optional) Providing destructor code for ETYPEs (to release embedded resources).
- (Optional) Providing code to allocate and free ETYPEs.
In turn, the Microsoft VM is responsible for calling the appropriate hook methods at the right time whenever ETYPEs are passed to or from COM methods. Once the hook class is written, the VM automatically supports passing ETYPEs (if ETYPE is fixed-size), ETYPE*, and ETYPE**, as well as returning ETYPE*. The VM also implements the correct semantics for [in], [out], [in,out], and [out,retval] type library attributes, which jactivex passes along using the extra bits in the .class file. An [out] attribute or double indirection is mapped to a one-element array of JTYPE rather than to JTYPE itself.
Hook Class Implementation
Although hook classes are packaged as Java classes, they cannot be written in Java. Hook classes must receive and pass machine addresses (pointers). Java methods can receive pointers by masquerading them as an int type. The Java language offers no way, however, to dereference, allocate, destroy, or otherwise manipulate pointers. For this reason, use of the Raw Native Interface (RNI) is a necessity for authoring hook classes. RNI allows Java methods to be implemented as C functions inside a DLL. RNI also offers a small set of APIs that can be called from C to create, manipulate, and destroy Java objects.
Using Template Files to Write a Hook Class
The SDK for Java contains samples that show how to write hook classes to achieve custom marshaling. These samples are installed in several directories under the SDK-Java.20\Samples\CustDoc directory. All directory paths referenced here are in relationship to the CustDoc directory.
The easiest way to start is by copying the files under the template subdirectory to a new directory. The key files in template are:
- TemplateMarshaler.java — a skeleton hook class
- TemplateMarshaler.c — contains a blank C function for each method
- lateMarshaler.def — exports the C functions in template.c
To make the files compilable, you must follow the instructions in each of the comments marked "TODO" in TemplateMarshaler.java and TemplateMarshaler.c. This involves replacing ETYPE and JTYPE with your particular type, and uncommenting some optional methods.
The template contains C functions for eight hook methods. The comments inside each function describe what that hook function is supposed to do. We will go into more detail about this as we proceed through the examples. First, a few salient points about the hook functions as a whole:
- Each C function starts with an unused parameter "OBJECT*x". This is an artifact of RNI and will always be NULL for a static method. This parameter can be ignored.
- Several C functions have parameters "JTYPE javaval". "JTYPE" should be replaced throughout by the C type that represents your Java type. To find out what C type to use, consult the table in TemplateMarshaler.c or use msjavah.
- Several C functions have parameters "PPETYPE ppetype" (a pointer to
a pointer to an ETYPE). These parameters are represented as "int"
in the Java method, but they are true pointers.
Important All pointers passed to hook methods are double-indirect ETYPE (PPETYPE). For many methods (such as releaseExternal), this will seem like an unnecessary level of indirection. The reason ETYPE is always double-indirected has to do with an obscure feature of hook classes that allows you to redefine the size and nature of A POINTER to ETYPE. This is a very arcane use of hook classes and will not be mentioned again in this primer. Just remember that you will, in some cases, have to deal with an extra level of indirection because of this. - Except for the functions that return Java types, hook methods return VOID or return a native pointer through an output buffer. To indicate failures, hook methods should use SignalError or SignalErrorPrintf, which cause RNI to throw a Java exception upon the exit of the C function. Remember that SignalError and SignalErrorPrintf return to their callers, so you must arrange to exit from your C function immediately after calling these APIs.
EXAMPLE 1: FixedPtMarshaler (The Basic Hook Class)
The FixedPtMarshaler sample is about the simplest useful hook class. It exposes only two methods and one field. The JTYPE is "double", and the ETYPE is the Win32 FIXED structure for representing fixed point fractions, as shown here:
struct FIXED {
WORD fract;
short //covvalue;
} FIXED;
The source code for FixedPtMarshaler can be found under the \fixedpt directory. The executables (FixedPtMarshaler.class and FixedPtMarshaler.dll) are under the \rundir directory.
The members implemented by FixedPtMarshaler are as follows:
public FixedPtMarshaler {
public static int cbByValSize; // set of sizeof(FIXED) = 8
public static double toJava(int ppFIXED, int flags)
{
// convert **ppFIXED to a double
// return the double
}
public static void copyToExternal(double javaval,
int ppFIXED,
int Flags)
{
// convert double to a FIXED
// copy the FIXED to **ppFIXED
}
}
Even with only these methods, this hook class can now be used in the
following ways:
| COM Type | Marshaled to Java As |
| HRESULT func([in] FIXED) | func(double) |
| HRESULT func([out,retval] FIXED*) | double func() |
| HRESULT func([in] FIXED*) | func(double) |
| HRESULT func([out] FIXED*) | func(double[]) |
| HRESULT func([in,out] FIXED*) | func(double[]) |
Note that when using this hook class in a .jnf file, you should preceed the Java type double with the const modifier as in the following example:
[Custom]
FIXED=const double, FixedPtMarshaler
If you do not include the const modifier, jactivex will interpret that [out] parameters are intended to pass along single values passed in by the caller (as a side-effect), rather than passing a new value out using a single-element array. That is, jactivex would map the FIXED type as follows:
| COM | Marshaled to Java As |
| HRESULT func([out] FIXED*) | func(double) |
This is not what you want. Because double is immutable, this method prototype cannot possibly return a value to the caller.
EXAMPLE 2: VarStrMarshaler (Embedded Resources)
The basic hook class, demonstrated by FixedPtMarshaler, assumes that ETYPE contained no embedded pointers or handles to allocated resources that need to be freed when ETYPE is no longer needed. In C++ terms, if ETYPE is a class in a basic hook class, it is assumed ETYPE does not have a destructor. Some structures, however, do need to clean up embedded resources. A well known example is the VARIANT structure used in ActiveX Automation. A VARIANT is a fixed-size structure, yet it can have allocated objects such as BSTR, SAFEARRAYS and COM objects referenced by it. The "destructor" for a VARIANT is the VariantClear API, which checks the type of the variant and performs the appropriate cleanup (for example, freeing the BSTR, freeing the SAFEARRAY, calling Release on the COM object, and so on).
Arranging for proper cleanup of embedded resources requires only one new method: releaseByValExternal.
The VarStr marshaler example maps VARIANT types, confining itself to only one case: marshaling BSTR VARIANT types to Java String objects. That is, the JTYPE is String, and the ETYPE is as follows:
struct {
short vt; // Always VT_BSTR for this example
short unused;
short unused1;
short unused2;
BSTR bstrVal; // Points to characters in BSTR
long unused3; // never used in this example.
} VARIANT;
The source code for VarStrMarshaler can be found under the \varstr directory. The executables VarStrMarshaler.class and VarStrMarshaler.dll are under the \rundir directory.
The members implemented by VarStrMarshaler are as follows:
public VarStrMarshaler {
public static int cbByValSize; // set of sizeof(VARIANT) = 16
public static String toJava(int ppVARIANT, int flags)
{
// convert **ppVARIANT to a String
// return the String
}
public static void copyToExternal(String javaval,
int ppVARIANT,
int Flags)
{
// convert String to a VARIANT
// copy the VARIANT to **ppVARIANT
}
/* NEW! */
public static void releaseByValExternal(int ppVARIANT, int Flags)
{
SysStringFree( (*ppVARIANT)->bstrVal );
}
}
This hook class can be used in the following ways.
| COM Type | Marshaled to Java As |
| HRESULT func([in] VARIANT) | func(String) |
| HRESULT func([out,retval] VARIANT*) | String func() |
| HRESULT func([in] VARIANT*) | func(String) |
| HRESULT func([out] VARIANT*) | func(String[]) |
| HRESULT func([in,out] VARIANT*) | func(String[]) |
As before, String is immutable so the const modifier should be used for it in the .jnf file.
EXAMPLE 3: PointMarshaler (Mutable Java Objects)
In the examples up to this point, all JTYPEs were immutable. This is why [out] parameters could only be handled by single element arrays: that is, to deal with immutable types, a caller has to write code similar to the following:
{
double ad1[] = {0};
double ad2[] = {0};
func(ad1, ad2);
System.out.println("func gave back: " + ad1[0] + ", " + ad1[1]);
}
For immutable objects, it should be clear that the function:
HRESULT func([out] FIXED *, [out] FIXED *);
could not be usefully mapped to:
func(double d1, double d2);
because func() would have no way of returning information to the caller.
However, consider the java class java.awt.Point (which represents a point in 2-dimensional space.) The Point class is mutable as its x and y fields are public and can set by anyone. Therefore, it is advantageous to use the non-array form for passing [out] parameters. To do this only requires two new methods: copyToJava and toUninitJava.
The PointMarshaler class maps java.awt.Point to the Win32 POINT structure. The source code for PointMarshaler can be found under the \point directory. The executables PointMarshaler.class and PointMarshaler.dll are under the rundir directory.
The members implemented by PointMarshaler are:
public PointMarshaler {
public static int cbByValSize; // set of sizeof(POINT) = 8
public static Point toJava(int ppPOINT, int flags)
{
// convert **ppPOINT to a Point
// return the Point
}
public static void copyToExternal(Point javaval,
int ppPOINT,
int Flags)
{
// convert Point to a POINT
// copy the POINT to **ppPOINT
}
/* NEW! */
public static void copyToJava(Point javaval, int ppPOINT, int Flags)
{
// modify "javaval" in place so it is "equivalent" to **ppPOINT;
}
/* NEW! */
public static Point toUninitJava(int ppPOINT, int flags)
{
// create a new Point with arbitrary x and y values.
// the contents of **ppPOINT are completely undefined and should
// be ignored for fixed-size hook classes.
}
}
This hook class can be used in the following ways.
| COM Type | Marshaled to Java As |
| HRESULT func([in] POINT) | func(Point) |
| HRESULT func([out,retval] POINT*) | Point func() |
| HRESULT func([in] POINT*) | func(Point) |
| HRESULT func([out] POINT*) | func(Point) |
| HRESULT func([in,out] POINT*) | func(Point) |
This differs from the basic hook class in that [out] POINT* becomes a single element, Point, rather than a single-element array, Point[]. To force jactivex to generate the non-array mapping, you must omit the const modifier from the Point entry in the .jnf file.
EXAMPLE 4: RectMarshaler (Support for Custom Allocation)
The hook class examples up to this point have provided no method for allocating or freeing the memory for ETYPE itself. They have relied entirely on the VM to allocate the actual memory for ETYPE. As a result, the use of custom data types up until now has been restricted to cases where ETYPE can be allocated on the stack and its lifetime is bounded by the lifetime of the call. While this is sufficient for some data types, there are cases where stack allocation is insufficient. The first case is when there is a need to marshal methods where an ETYPE is allocated by the callee but freed by the caller (or the other way around, as can happen with [in,out] parameters.) The second case is if the data type is variable-sized (such as a string), in which case, the VM cannot do the stack-allocation because the size is unknown. The first case is considered next, followed by the second case.
It is the responsibility of the hook class to specify which API is used to allocate and release the memory for ETYPE. The new hook methods required to support this are toExternal and releaseExternal.
The RectMarshaler example demonstrates this by marshaling the Win32 RECT structure into java.awt.Rect objects. The source code for RectMarshaler can be found under the \rect directory. The executables RectMarshaler.class and RectMarshaler.dll are under the \rundir directory.
The members implemented by RectMarshaler are:
public RectMarshaler {
public static int cbByValSize; // set of sizeof(RECT) = 8
public static Rect toJava(int ppRECT, int flags)
{
// convert **ppRECT to a Rect
// return the Rect
}
public static void copyToExternal(Rect javaval,
int ppRECT,
int Flags)
{
// convert Rect to a RECT
// copy the RECT to **ppRECT
}
/*NEW*/
public static void toExternal(Rect javaval, int ppRECT, int Flags)
{
// allocate a new RECT, initialize it using javaval,
// store pointer in *ppRECT
}
/*NEW*/
public static void releaseExternal(int ppRECT, int Flags)
{
// release *ppRECT. If RECT required a releaseByValExternal
// (which it doesn't), this routine must do that work as well.
}
}
Note this example could also have implemented copyToJava and toUninitJava here since Rect is mutable (like Point). However, for the sake of clarity, this was not done.
Important Note: Even in the presence of these new Marshaling methods, the VM will optimize by allocating the data structure on the stack and thus override certain allocations. For example, for "[in] ETYPE*" calls, the COM method will receive a pointer to a VM-allocated ETYPE on the stack rather than one allocated by the toExternal method. If you want all ETYPEs to be allocated using toExternal(), you must omit the cbByValSize field. This will prevent the VM from optimizing to stack allocations as it cannot predict how many bytes to allocate.
This hook class can be used in the following ways:
| COM Type | Marshaled to Java As |
| HRESULT func([in] RECT) | func(Rect) |
| HRESULT func([out,retval] RECT*) | Rect func() |
| HRESULT func([in] RECT*) | func(Rect) |
| HRESULT func([out] RECT*) | func(Rect[]) |
| HRESULT func([in,out] RECT*) | func(Rect[]) |
| HRESULT func([out,retval] RECT**) | Rect func() |
| HRESULT func([in] RECT**) | func(Rect[]) |
| HRESULT func([out] RECT**) | func(Rect[]) |
| HRESULT func([in,out] RECT**) | func(Rect[]) |
EXAMPLE 5: AnsiMarshaler (Variable-sized data types)
So far in this article, all of the ETYPE examples have been fixed-size structures. For some data structures, this is a prohibitive limitation. The classic example is a simple null-terminated string.
To define a variable-sized structure, the hook class must omit the cbByValSize field. The absence of this field marks the hook class as variable-size. Unlike a fixed-sized hook class, a variable-size hook class must support toExternal and releaseExternal in order to be useful (the VM cannot allocate a variable-sized structure on the stack). In addition certain mappings available to fixed-size hooks (in particular, those that pass the data type by value) are not available to variable-size hooks.
More specifically, consider the simple case where ETYPE is char (not LPSTR!), and JTYPE is java.lang.String. (Both types can be considered immutable for this example.) This structure can be defined by a hook class with only three methods: toJava, toExternal and releaseExternal; the minimum useful variable-size hook class.
The AnsiMarshaler class maps Strings to the LPSTR (ANSI null-terminated strings). The source code for AnsiMarshaler can be found under the \ansistr directory. The executables (AnsiMarshaler.class and AnsiMarshaler.dll) are under the rundir directory.
The members implemented by AnsiMarshaler are:
public AnsiMarshaler {
public static String toJava(int ppCHAR, int flags)
{
// convert **ppCHAR to a String
// return the String
}
public static void toExternal(String javaval, int ppCHAR, int Flags)
{
// allocate a new LPSTR, initialize it using javaval,
// store pointer in *ppCHAR
}
public static void releaseExternal(int ppCHAR, int Flags)
{
// release *ppCHAR. If LPSTR required a releaseByValExternal
// (which it doesn't), this routine must do that work as well.
}
}
The allowed usages are as follows:
| COM Type | Marshaled to Java As |
| HRESULT func([in] CHAR*) | func(String) |
| HRESULT func([out] CHAR*) | func(String[]) |
| HRESULT func([in,out] CHAR*) | func(String[]) |
| HRESULT func([out,retval] CHAR**) | String func() |
| HRESULT func([in] CHAR**) | func(String[]) |
| HRESULT func([out] CHAR**) | func(String[]) |
| HRESULT func([in,out] CHAR**) | func(String[]) |
Running and Building the Sample Client
The \client directory contains the sources for a sample Java client and a C++ inproc COM server which exchange all the various types implemented by the example.
To run the example, simply change the current directory to "rundir", and type "GO".
To build the sample, you will need to provide a makefile compatible with your build environment. Here are the steps:
- Compile sysdata.odl to produce sysdata.tlb:
mktypelib sysdata.odl.
- Compile CustSample.odl to produce CustSample.tlb and CustSample.h
mktyplib /h CustSample.h CustSample.odl
- Compile CustSample.tlb to produce CCustSample.java and ICustSample.java
jactivex /javatlb CustSample.tlb /nCustSample.jnf
- Compile CCustSample.java to produce CCustSample.class
jvc CCustSample (You must use Jvc version 1.02.3920 or later)
- Compile ICustSample.java to produce ICustSample.class
jvc ICustSample (You must use Jvc version 1.02.3920 or later)
- Compile CustSample.java to produce CustSample.class and callbacks.class
jvc CustSample (You must use Jvc version 1.02.3920 or later)
- Compile CustSample.cpp & CustSample.def to produce CustSample.dll.
cl + link.
