HotSwap Users Manual

July 25, 2001

Copyright (C) 2001 Paul Cody Johnston


Table of Contents


Introduction

../../images/hotswap-1

HotSwap provides a robust library for updating the implementation of an object at runtime, otherwise known as hotswapping. This is achieved through recompilation, dynamic class reloading, and object state migration throughout the life of an application. Hotswapping makes Java feel more like an interpreted language than a compiled language; it is fundamentally useful for incremental development of an application, so-called application evolution or dynamic application development.

API Documentation: Javadocs
Homepage: http://www.inxar.org/hotswap
Download: http://www.inxar.org/download
Bugs & Feedback: mailto:hotswap@inxar.org

HotSwap Library Usage Summary
// Get a ProxyClassLoader instance
ProxyClassLoader loader = new ProxyClassLoader();

// Get a ProxyCompiler instance (or use default)
ProxyCompiler compiler = new JavacCompiler();
ProxyCompiler compiler = new SystemCompiler("jikes");
ProxyCompiler compiler = loader.getDefaultProxyCompiler();

// Configure it (if necessary)
compiler.setDestinationpath("/tmp/classes");
compiler.setSourcepath("/home/myname/myproject/src");
compiler.getClasspath().add("/home/myname/myproject/classes");
compiler.getClasspath().add("/usr/share/java/com/acme/lib/acme.jar");

// JDK1.2 or JDK1.3 usage
ProxyClass cls = loader.loadJKD12("com.myname.myproject.MyObject");
Proxy proxy = cls.newInstance();
MyObject obj = (MyObject)proxy.hotswap();

// JDK1.3 usage only; assumes MyObject implements MyInterface
ProxyClass cls = loader.load("com.myname.myproject.MyObject");
MyInterface obj = (MyInterface)cls.newInstance();

// Additional Stuff; assumes mylistener implements ProxyEventListener
cls.addCompileListener(mylistener);
cls.addClassSwapListener(mylistener);
cls.addHotSwapListener(mylistener);
proxy.addObjectSwapListener(mylistener);
compiler.addCompileListener(mylistener);

Hotswapping is the process of changing the implementation of an object at run-time. This requires monitoring the state of the code that implements the object and, if necessary, recompiling and reloading the Class using dynamic classloading techniques.

The Java Virtual Machine does not support hotswapping directly. Once an Object has been constructed from a Class, it will retain its identity to that Class for the entire lifetime of the Object. Therefore, true hotswapping is impossible, but it can be simulated at the application level and requires the notion of a proxy, meaning: the agency, function, or office of a deputy who acts as a substitute for another (MerriamWebster).

When an object o is hotswapped, the class c is recompiled and reloaded into memory. If the reload step is successful, the ProxyClass instance mediating the reload will free the old Class c and replace it with the new Class c'. Once c' has been reloaded, a new object o' is constructed by reflection from c' under the auspice of the delegate Proxy.

The hotswap operation is a two-phase commit protocol. If the object o' is an instanceof of ProxyObject, it will be allowed to participate in the transaction and synchronize its state from the old object o via the hotswap_onPrepare(ProxyObject oldInstance) method. Likewise, the old object o will be notified of its impending release via the hotswap_onRelease() method. The reference to the old object o is then dropped and o will be subsequently garbage collected. See section Transaction Semantics for more detail.

Garbage collection of the old object o is of course contingent on its reachability; if any other object references to o exists in the JVM, it will not be garbage collected. This means it is important for the developer not to keep direct references to o but rather the Proxy instance acting on o's behalf.

HotSwap also provides transparent support for JDK1.3 Dynamic Proxy Classes within the context of a 1.3-compatible JVM.

Note: The PNG images in this document look, well, "funny" in Netscape.

Assumptions

The first assumption is that you expect the implementation of a particular object to change throughout the life of your application.

The second assumption is that the changes you make to an object at run-time have little or ideally no impact on the memory layout of the object. Runtime alteration of instance variables creates a situation where two different sized boxes are supposed to hold the same information; the 'ol square peg in a round hole issue. Hotswapping is most appropriate when you are changing procedural code, not data layout.

The third assumption is that you have a working understanding of how classloading works in Java. This is necessary because otherwise you'll get frustrated. See the section on ClassLoaders below.

Components

These are the main classes that you need to know about to understand how to use HotSwap.

ProxyClassLoader

The ProxyClassLoader is the "root" object in the system. It acts as a factory for ProxyClass instances as well as a place where some default things can be specified. The ProxyClassLoader can configure these default things from a properties file.

ProxyCompiler

The ProxyCompiler handles two important things: (1) it mediates sourcefile compilation and (2) it is reponsible for determining when the classfile should be recompiled or reloaded. It needs to be configured to know where the sourcefiles are, where to put new classfiles, and other things like compilation options and the classpath.

ProxyClass

The ProxyClass is a factory for Proxy instances. It depends upon a the ProxyCompiler to decide when compilation and classloading should take place, and relies on it to mediate the sourcefile (re)compilation.

Each ProxyClass can have a different ProxyCompiler, but for convenience a "default" one can be placed on the ProxyClassLoader and the ProxyClass will use that if one has not been explicitly given.

Proxy

A Proxy is a wrapper for an Object that is expected to be hotswapped.

ProxyObject

ProxyObject is an optional interface that can be implemented by end-user classes such that if an instance of that class is being hotswapped (ie wrapped in a Proxy), it will receive lifecycle event notifications.

ProxyLog

ProxyLog is an interface that can be implemented if an application wants to have HotSwap send its log messages to their logging system (such as log4j) rather than the default implementation.

components

The diagram shows the chain of allocation in blue: the ProxyClassLoader is instantiated by the end-user application, which is used to instantiate ProxyClass instances, which is used to Proxy instances, which is responsible for instantiation of Object instances, which is used by the end-user application.

Red lines show dependence: The ProxyClass needs a ProxyCompiler and a parent ClassLoader to function. If these are not explicitly set on the ProxyClass it will use the defaults that are accessible from the ProxyClassLoader. These defaults are setup using the configution properties file or explicitly by the user.

Example

You are developing a website that processes input from a CGI form. You have a `controller' class (MVC paradigm) within the scope of a servlet that checks the form input and conditionally stores some information in a database table. Call this the com.example.bank.FormProcessor class.

Problem

You want to tune the development of FormProcessor while the application is running. Since it's about a 30 second round-trip for the application to reload and to get the application in the required state (you have to login to the website first), you're looking to speed up the development cycle. Sure, the servet container you're using provides autoreloading, but the granularity is too large -- any changes cause the entire application to be restarted.

Solution

Rather than your servlet holding a reference to the FormProcessor directly, you change the implementation temporarily (you'll change it back later during preparation of the final production code) such that the servlet holds a Proxy to the FormProcessor.

During each HttpRequest the servlet fetches the FormProcessor object from the proxy. When a request is made to fetch the FormProcessor object from the proxy, the internals of the Proxy implementation (provided by this library) checks to see of the sourcefile has changed. If it has, the sourcefile will be automatically recompiled, the FormProcessor class will be reloaded into memory, and a new FormProcessor instance will be reflected.

As you can see, before the FormProcessor object gets its hands on the incoming request, a check is executed to see if the code is outdated. This allows you to incrementally develop the FormProcessor implementation without having to restart the application.

Implementation

In order to get the necessary Proxy object, you'll need to setup some other stuff such that the library knows what the name of the FormProcessor class is, where in the filesystem it's located, and how to compile it.

Step 1: Obtain a ProxyClassLoader instance

The root object in the HotSwap library is the ProxyClassLoader. It acts as a factory for ProxyClass instances which in turn act as factories for Proxy instances. Additionally, the ProxyClassLoader holds references to "global" and "default" things that are used by child ProxyClass instances during Hotswapping. These include the global ProxyLog, a default ProxyCompiler, and a default parent ClassLoader. Note that ProxyClassLoader does not extend java.lang.ClassLoader -- the name simply means a thing that loads ProxyClass objects (a factory).

Instantiation of the ProxyClassLoader
ProxyClassLoader loader = new ProxyClassLoader();
ProxyClassLoader loader = new ProxyClassLoader(propertiesFilename);

By default, the ProxyClassLoader configures itself from a properties file in the hotswap.jar archive. This can be overridden by explicitly passing a properties filename to the constructor (or using a JVM -D option). See section Configuration for more details.

Step 2: Configure a ProxyCompiler instance

The ProxyCompiler is the thing that mediates sourcefile compilation. In order to do this correctly, it needs to know where the sourcecode is, where to put the compiled classes, what classpath contains, and what other compilation options to include. The SystemCompiler implementation runs a system command whereas JavacCompiler interfaces with sun.tools.javac.Main and requires tools.jar in the CLASSPATH. Choose the implementation that best fits your setup.

The ProxyCompiler can be configured indirectly from the properties file, directly using the API, or both. Here is how this is done directly using the API:

Direct Configuration of the ProxyCompiler
ProxyCompiler compiler = new JavacCompiler();

compiler.setDestinationpath("/tmp/classes");
compiler.setSourcepath("/home/myname/myproject/src");
compiler.getClasspath().add("/home/myname/myproject/classes");
compiler.getClasspath().add("/usr/share/java/jsdk2.0/lib/jsdk.jar");

loader.setDefaultProxyCompiler(compiler);

Alternatively, use the default one (can be configured using the properties file):

Configuration of the ProxyCompiler
ProxyCompiler compiler = loader.getDefaultProxyCompiler();

compiler.setDestinationpath("/tmp/classes");
compiler.setSourcepath("/home/myname/myproject/src");
compiler.getClasspath().add("/home/myname/myproject/classes");
compiler.getClasspath().add("/usr/share/java/jsdk2.0/lib/jsdk.jar");

Take a good look at the filepaths given in the example. The sourcepath is required to help the compiler find the source file `/home/myname/myproject/src/com/example/bank/FormProcessor.java'.

The compiled class `/tmp/classes/com/example/bank/FormProcessor.class' will be read and its bytes used to define the Class. It is important that this classfile NOT be visible to other classloaders in your application. For example, if `/tmp/classes' or some other `FormProcessor.class' happened to be in your classpath when starting the application, the system classloader would load that classfile instead, the ClassLoader internal to HotSwap would never get the opportunity; hence, it will not be reloadable.

Step 3: Obtain the ProxyClass

The ProxyClass holds a reference to the Class object that will be periodically reloaded. Use the ProxyClassLoader to create the ProxyClass:

Obtaining a ProxyClass instance
ProxyClass cls = loader.loadJDK12("com.example.bank.FormProcessor");

Step 4: Get the Proxy

You're finally ready to get the Proxy object so patiently waited for. One last question to ask is whether the FormProcessor object requires constructor arguments. It turns out that the only public FormProcessor constructor looks like:

The FormProcessor constructor
public FormProcessor(com.example.bank.ConfigObject conf) {
    this.dbConnection = conf.getConnection();
}

It this case you have to give constructor arguments to the Proxy object such that it can reflect the right constructor:

Obtaining a Proxy instance
Proxy proxy = cls.newInstance(new Object[]{ configObject });

//
// Equivalent but more verbose
//
Proxy proxy = cls.newInstance();
Class[] params = new Class[1];
params[0] = com.example.bank.ConfigObject.class;
Object[] args = new Object[1];
args[0] = configObject;
proxy.hotswap_setConstructorParameters(args);
proxy.hotswap_setConstructorArguments(args);

Step 5: Use the Proxy

Now inside the doGet() method of the servlet:

Using the Proxy
FormProcessor p = (FormProcessor)proxy.hotswap();
p.process(httpRequest);

Each request will trigger a potential hotswap of the FormProcessor class. How RAD! (ahem... rapid application development, that is).

Step 6 [optional]: Implement ProxyObject on FormProcessor

It turns out that the FormProcessor implementation is not completely vacuous. In particular, it holds a database connection that it got from the ConfigObject in its constructor. The problem is that you are now leaking database connections every time the class is reloaded since the object never gets a chance to release the connection.

To solve this problem, the FormProcessor object needs to get clues about it's lifecycle in the context of a hotswap transaction. This is the purpose of the ProxyObject interface: if the thing enclosed within the Proxy instance implements ProxyObject, it gets special treatment. When the object is created, the hotswap_onPrepare(ProxyObject old) method is called; likewise hotswap_onRelease() is invoked when the object is being thrown away. Here is the updated implementation of FormProcessor.java:

Sample implementation of ProxyObject
public class FormProcessor 
implements com.example.bank.Processor, org.inxar.hotswap.ProxyObject
{
    ...

    public boolean hotswap_onPrepare(ProxyObject oldInstance) {
	return true;
    }

    public void hotswap_onCommit() {
    }

    public void hotswap_onRollback() {
    }

    public void hotswap_onRelease() {
        if (dbConnection != null) {
            dbConnection.close();
        }
    }

    public Object hotswap_get(Object key) {
        return null;
    }  

    protected DBConnection dbConnection;
}

This solves the connection leak problem. But it turns out that all the problems have not been solved. In particular, the DBConnection is a stateful object that contains important information relevant to its use within the FormProcessor. Therefore, rather than destroying it and making a new one each time the FormProcessor object is hotswapped, you want to transfer the dbConnection to the new object. You do that as follows:

Sample implementation of ProxyObject
public class FormProcessor 
implements com.example.bank.Processor, org.inxar.hotswap.ProxyObject
{
    ...

    public boolean hotswap_onPrepare(ProxyObject oldInstance) {
        this.oldInstance = oldInstance;
	return true;
    }

    public void hotswap_onCommit() {
        if (dbConnection != null)
	    dbConnection.close();

        dbConnection = (DBConnection)oldInstance.get("conn");
	oldInstance = null;
    }

    public void hotswap_onRollback() {
        oldInstance = null;
    }

    public void hotswap_onRelease() {
        if (dbConnection != null) {
            dbConnection.close();
	    dbConnection = null;
        }
    }

    public Object hotswap_get(Object key) {
        if ("conn".equals(key))
	    return dbConnection;
    }  

    protected DBConnection dbConnection;
    protected ProxyObject oldInstance;
}

You are now able to move the same dbConnection object through multiple FormProcessor generations.

But what's with the hotswap_get(Object) method anyway? Why not just cast to FormProcessor and fetch the dbConnection object directly, similar to how you might do it if implementing Cloneable? (Stop and think before reading the next paragraph).

Ready? The problem is that you can't cast a FormProcessor object to a FormProcessor object, you'll get a ClassCastException if you try. Sounds crazy, but it's true. Though oldInstance().getClass().getName() and newInstance().getClass().getName() will return identical strings, they are indeed different classes since they were loaded by different classloaders.

Though the classloader is the foundation of the type system, the Java Language provides no mechanism to express classloader membership within the Java syntax. In other words, the full notion of type in Java exists only at runtime.

For this reason, there needs to be a type-safe window from the old object to the new object such that state can be passed between them. hotswap_get(Object) is that window. Since the ProxyObject interface classfile does not change, it is a stable interface both objects have in common.

Step 7 [optional]: Better Error Reporting

The only problem is that there is no output when a compilation error occurs; it's as though the standard output has disappeared. The is because, in fact, it has disappeared. Rather than printing out to System.out (or System.err), the ProxyCompiler slurps up all the output into a string and wraps it in a ProxyCompileEvent. Unfortunately, there are no ProxyEventListener instances that are monitoring for this event, so nothing it shown. A modification of the setup of the ProxyCompiler in step 2 with an anonymous class fixes the problem:

Printing Compilation Output
compiler.addCompileListener(new ProxyEventListener() {
    public void notify(ProxyEvent evt) {
        System.out.println(evt);
    }
});

JDK1.3 Dynamic Proxy Classes

In this example it was necessary to replace any persistent references to the FormProcessor object with a reference to the delegate Proxy instance. This requires somewhat significant changes to our servlet application, though certainly liveable.

An arguably better option would have the Proxy instance be type-assignable to FormProcessor; rather than having the FormProcessor object swaddled in a Proxy blanket, it would exist as a transparent implementation of FormProcessor itself that handles hotswapping behind the scenes.

If this were possible, it would not be necessary to change the instance variables in our servlet. All we would have to do is instantiate a Proxy implementation rather than the concrete FormProcessor implementation directly.

This is the notion of dynamic proxy classes introduced in JDK1.3. A proxy class is a concrete type constructed at runtime by the JVM that implements the interfaces you specify. The ProxyClass in this library has convience methods that support the construction of JDK1.3 proxy classes.

To support this design in this example it is necessary to abstract the FormProcessor as an interface. Luckily for us, FormProcessor is a concrete implementation of the Processor interface. Design by interface is required to use dynamic proxy classes.

In order to take advantage of dynamic proxy classes, we have to make sure we are running in a JDK1.3 environment and make a few changes to the example code:

Implementation changes to use dynamic proxy classes
//
// Change Step 3 to load a JDK1.3 proxy class rather than not.
//
/* ProxyClass cls = loader.loadJDK12("com.example.bank.FormProcessor"); */
ProxyClass cls = loader.load("com.example.bank.FormProcessor");

//
// Change Step 4 such that we cast to Processor.  We can forget that
// the object is not the real thing.
//
/* Proxy proxy = cls.newInstance(new Object[] { configObject }); */
Processor p = (Processor)cls.newInstance(new Object[] { configObject });

//
// Change Step 5 back to the original, natural usage pattern 
// (hotswap is now called automagically within the dynamic proxy).
//
/* FormProcessor p = (FormProcessor)proxy.hotswap(); */
p.process(httpRequest);

The Processor instance is still a Proxy and will continue to hotswap as necessary, but the interface has changed; we only have to concern ourselves with the original type system.

Note that even if you are running JDK1.3, dynamic proxy classes are not always appropriate. The granularity of hotswap checking becomes much finer: rather than hotswapping only when the Proxy.hotswap() is called, the proxy will attempt hotswapping at every method invocation on the dynamic proxy object (using the default ProxyInvocationHandler implementation). Depending on the usage if the target object, this could have performance issues. Also, dynamic proxy class instances require more processing per invocation than non-proxy class instances, so that's another thing to consider.

Note: You can subclass ProxyInvocationHandler and use the ProxyClass.newInstanceH factory methods to provide custom proxy behavior, including but not limited to tuning hotswap granularity to your preference.

Configuration

The ProxyClassLoader will try to configure the system from a properties file. The default properties file is `org/inxar/hotswap/hotswap.properties' in the `hotswap.jar' file, but another can be specified either by passing the name of the properties file to the ProxyClassLoader constructor or passing the name of the file to the JVM from the command line via the -Dorg.inxar.hotswap.properties system property:

Custom Configuration
// Use constructor
String propertiesFilename = "/tmp/hotswap.properties";
ProxyClassLoader loader = new ProxyClassLoader(propertiesFilename);

# At JVM startup, from command line
$ java -cp $CLASSPATH -Dorg.inxar.hotswap.properties=/tmp/hotswap.properties ...

The configuration file is commented, here are the current default "factory" settings:

org/inxar/hotswap/hotswap.properties
#####################################################################
#
# This is the default HotSwap configuration file.  It is exactly like
# a normal properties file with the exception that multiple "key =
# value" entries having the same key are converted to a List.
# Therefore, the properties.get("compiler.classpath") would return a
# List object rather than a String if there are more than one
# compiler.classpath entries in the file.
#
# You can supply extra entries directly using the bin/java -D option
# by prefixing the key with "org.inxar.hotswap.".  These are put at
# the FRONT of the List which has the effect of overriding the base
# values.  For example, to turn on debugging from the command line you
# would do: "java -D org.inxar.hotswap.log.debug=true".
#
# Also: key names (like log.classname) are automatically converted to
# lowercase: CASE IS NOT SIGNIFICANT.
#
# Another Note: The idea of the properties file is to provide
# convenience and to the minimize hardcoding of implementation and
# deployment dependent things.  But it's not required -- everything
# you can say here can be coded directly within an application.  That
# is, stuff that's not included in the properties file should not
# break an application or prevent it from starting up.
#
#####################################################################



#####################################################################
# ProxyLog 
#
# This section configures log the instance.  To hook HotSwap into your
# own logging system, use the name of your own class that implements
# ProxyLog for the log.classname property.
#####################################################################

#-----------------------------------------------------------------
# The name of a class that implements ProxyLog.  If more than one
# entry is listed, the first one that initializes without error is
# used.
#-----------------------------------------------------------------
log.classname = org.inxar.hotswap.ConsoleLog

#-----------------------------------------------------------------
# Flags that are used to by ConsoleLog, the default Log
# implementation.  Values must be true or false.
#-----------------------------------------------------------------
log.fatal = true
log.warn = true
log.info = true
log.debug = true



#####################################################################
# Parent ClassLoader 
#
# This section names the class from which the default
# java.lang.ClassLoader should be retrieved.  It can be overridden
# using the ProxyClassLoader.setDefaultParentClassLoader method or set
# on a ProxyClass by ProxyClass basis using the
# ProxyClass.setParentClassLoader method.
#####################################################################

#-----------------------------------------------------------------
# The name of a class to get the ClassLoader from, for example
# (Class.forName(classname).getClassLoader()).  If more than one entry
# is listed, the first one that initializes without error is used.
#-----------------------------------------------------------------
classloader.classname = org.inxar.hotswap.ProxyClassLoader



#####################################################################
# ProxyInvocationhandler 
#
# This section configures the default ProxyinvocationHandler used to
# service JDK1.3 proxy class objects.  It can be overridden on using
# the newInstanceH methods.
#####################################################################

#-----------------------------------------------------------------
# The name of a class that extends ProxyInvocationHandler.  If more
# than one entry is listed, the first one that initializes without
# error is used.
#-----------------------------------------------------------------
invocationhandler.classname = org.inxar.hotswap.ProxyInvocationHandler



#####################################################################
# ProxyCompiler 
#
# This section configures the default ProxyCompiler.  It can be
# overridden using the ProxyClassLoader.setDefaultProxyCompiler method
# or on a ProxyClass-by-ProxyClass basis using the
# ProxyClass.setProxyCompiler method.
#####################################################################

#-----------------------------------------------------------------
# The name of a class that extends ProxyCompiler.  If more than one
# entry is listed, the first one that initializes without error is
# used.
#-----------------------------------------------------------------
compiler.classname = org.inxar.hotswap.JavacCompiler
compiler.classname = org.inxar.hotswap.SystemCompiler

#-----------------------------------------------------------------
# The name of the compiler command; used by SystemCompiler. If more
# than one entry is listed, the first one that initializes without
# error is used.  For example, the default is to list jikes first
# since jikes is faster.  For platforms that don't have jikes
# installed, the system compiler will fall back to javac.
#-----------------------------------------------------------------
compiler.command = jikes
compiler.command = javac

#-----------------------------------------------------------------
# The name of the compiler sourcepath.
#-----------------------------------------------------------------
!compiler.sourcepath = src

#-----------------------------------------------------------------
# The name of the compiler destinationpath.
#-----------------------------------------------------------------
!compiler.destinationpath = classes

#-----------------------------------------------------------------
# A list of classpaths to be passed to the compiler.
#-----------------------------------------------------------------
!compiler.classpath = rt.jar
!compiler.classpath = tools.jar

#-----------------------------------------------------------------
# A list of options to be passed to the compiler.
#-----------------------------------------------------------------
!compiler.option = -nowarn
!compiler.option = -verbose

#-----------------------------------------------------------------
# A flag that, if true, will tell the HotSwap to place all the entries
# from the system classpath (System.getProperty("java.class.path"))
# into the compilers' classpath.
#-----------------------------------------------------------------
compiler.usejavaclasspath = false

#-----------------------------------------------------------------
# A flag that, if true, will force the compiler to run in asynchronous
# mode.  Asynchronous mode changes the getStatus function to naiively
# return STATUS_CURRENT except at periodic intervals.  This changes
# the HotSwap granularity.
#-----------------------------------------------------------------
compiler.asynchronous = false

#-----------------------------------------------------------------
# A number, in milliseconds, to wait between asynchronous
# notifications.  This is required if the compiler is in async mode.
#-----------------------------------------------------------------
compiler.asyncinterval = 60000

Asynchronous Mode

In order to determine if a compilation or classload needs to occur, HotSwap refers to the lastModified timestamps on the sourcefile and classfile; this occurs within the ProxyCompiler.getStatus method (go take a look). Under high application load, these system calls could presumably become a performance bottleneck.

Asynchronous Mode is a ProxyCompiler setting that changes the logic of the getStatus method to naiively return STATUS_CURRENT except at certain times. This has the effect of tricking the ProxyClass into thinking that the class is up to date when, in fact, it may be out of date. When asynchronous mode is on, a separate thread periodically trips a flag that allows the ProxyCompiler.getStatus to genuinely check the status of the classfile.

To enable asynchronous mode, change the compiler.asynchronous setting to true and set the corresponding interval:

Using the Proxy
compiler.asynchronous = true
compiler.asyncinterval = 20000

The above settings would guarantee a maximum of three HotSwap transactions per minute.

HotSwap Dependence

During execution, hotswap may recompile and/or reload classes depending on the state of the corresponding sourcefile and classfile for a particular class. It is useful to know in order to understand when and why HotSwap will recompile and reload classes in your application.

Variables

For each Class that is being monitored for HotSwapping, three timestamps are watched to decide what to do: the lastModified timestamp of the sourcefile (`.java'), the lastModified timestamp of the classfile (`.class'), and a timestamp that is recorded when a class is loaded into memory. These are timestamps are symbolized in this section as src, cls, and obj. To say that the timestamp of the classfile is newer than the sourcefile one would write `cls > src'.

Also taken into account is whether or not the sourcefile exists in the filesystem, whether the classfile exists in the filesystem, and whether the object exists in memory (whether the classfile has been loaded into memory).

State Description

There are at least 15 different possible combinations of these variables, or states. Each state is written in two parts; the first part describes what things exist; the second part, if appropriate, describes how the timestamps are ordered. Finally, a brief description of each state is given:

State 1a
!src, !cls, !obj: Both the source and classfile are missing and the class has not been loaded into memory.
State 2a
src, !cls, !obj: The sourcefile exists, but the classfile is missing and the class has not been loaded into memory.
State 2b
!src, cls, !obj: The classfile exists, but the sourcefile is missing and the class has not been loaded into memory.
State 2c
!src, !cls, obj: Somehow, both the sourcefile and classfile are missing, but the class has been loaded into memory.
State 3a
src, cls, !obj; cls >= src: The sourcefile and classfile both exist but the class has not been loaded. The sourcefile is not newer than the sourcefile (classfile is up to date).
State 3b
src, cls, !obj; src > cls: The sourcefile and classfile both exist but the class has not been loaded. The sourcefile is newer than the classfile (classfile is out of date).
State 4a
!src, cls, obj; obj >= cls: The classfile exists and has been loaded into memory, but the sourcefile is missing. The classfile is not newer than the object (object is up to date).
State 4b
!src, cls, obj; cls > obj: The classfile exists and has been loaded into memory, but the sourcefile is missing. The classfile is newer than the object (object is out of date).
State 5a
src, !cls, obj; obj >= src: The object has been loaded and the sourcefile exists, but for some reason the classfile is missing. The sourcefile is not newer than the object (object seems up to date).
State 5b
src, !cls, obj; src > obj: The object has been loaded and the sourcefile exists, but for some reason the classfile is missing. The sourcefile is newer than the object (object seems out of date).
State 6a
src, cls, obj; obj >= cls >= src: Both files exist and the class has been loaded. The classfile is not newer than the object (object is up to date) and the sourcefile is not newer than the classfile (classfile is up to date).
State 6b
src, cls, obj; cls > obj >= src: The classfile is newer than the object (object is out of date) but the sourcefile is not newer than the classfile (classfile is up to date). Only the classfile has been updated (the object is still newer than the sourcefile).
State 6c
src, cls, obj; cls > src > obj: The sourcefile is is not newer than the classfile (classfile is up to date relative) but both files are newer than the object (object is out of date relative to both files).
State 6d
src, cls, obj; src > obj >= cls: The sourcefile is newer than the classfile (classfile is out of date), but the classfile is not newer than the object (object is up to date relative to the classfile, but out of date relative to the sourcefile).
State 6e
src, cls, obj; src > cls > obj: The sourcefile is newer than the classfile (classfile is out of date), and the classfile is newer than the object (object is out of date). Somehow, both are newer than the object (the object is out of date relative to both).

State Interpretation

HotSwap interprets these states as follows and will try to take the described action.

State 1a
The object is not in memory and no files exist. In this case it is not possible for HotSwap to proceed under the assumption that it is mandatory to load the class. Action: Since no classfile exists and no sourcefile exists to compile, HotSwap will report a a fatal error.
State 2a
The object has not been loaded and no classfile exists. However, a sourcefile does exist and we can presumably compile it to make a classfile. Action: Try to compile the sourcefile and proceed to state 3a.
State 2b
The object is not in memory. The classfile exists but not the sourcefile. No compilation is possible. Action: Try to load the class and proceed to state 4a.
State 2c
The object is in memory, but no files exist. Earlier in the life of the application a classfile must have existed but was subsequently deleted. Action: No update is possible, but signal a warning through the logging interface that critial resources (namely the classfile) are now missing.
State 3a
The object has not been loaded, both files exist, and the classfile is current. This state occurs naturally after a compile or upon startup and no changes have been made to the source. Action: Try to load the class and proceed to state 6a.
State 3b
The object is not in memory, both files exist but the classfile is out of date. Since the sourcefile is newer, we should try to compile it before loading the class. Action: Compile and proceed to state 3a.
State 4a
The object has been loaded and does not need to be updated since it is newer than the classfile. The sourcefile is missing so no recompilation is possible. Action: Do nothing, the object is current and no other update is possible.
State 4b
The object is in memory but out of date relative to the classfile. The source is missing so no recompilation is possible. Action: Reload the class and proceed to state 4a.
State 5a
The object is in memory and the source exists, but the classfile is for some reason missing. Presumably it has been deleted by an external agent since the class was last loaded. The object is newer than the sourcefile, however, so one possibility is to do nothing. However, we assume that the external agent that deleted the classfile did so because of good reason, perhaps because although the sourcefile itself has not changed, some other class upon which this one depends has changed and thus the class needs to be recompiled to reflect those "external" changes. Action: Recompile the class and proceed to state 6b.
State 5b
The object is in memory, the class is missing, and the object is out of date relative to the source. This case is like 5a, but the external agent has not only deleted the classfile, but also updated the source for this class. Action: Recompile and proceed to state 6c.
State 6a
All files exist and all things are up to date. This is the desired end-state. Action: Do nothing, no update is possible.
State 6b
The object is in memory but out of date relative to the classfile. The classfile is up to date relative to the sourcefile, so no recompilation is necessary. Strangely, the object is up to date relative to the sourcefile, however, meaning that there must have been some funny business to find outselves in this state. Action: Reload the class and proceed to state 6a.
State 6c
The object is in memory but out of date. The classfile is up to date, however, no recompilation is necessary. The object is also out of date relative to the sourcefile, which is normal. Action: Reload the class and proceed to state 6a.
State 6d
The object is in memory and up to date relative to the classfile, but the classfile is out of date and this needs to be recompiled. Action: Recompile the class and proceed to state 6c.
State 6e
The object is in memory and out of date relative to the classfile, which is also out of date relative to the sourcefile. Presumably an outside agent updated the classfile and then later updated the sourcefile, as though it made changes to the source, externally recompiled it, and then updated the source again without again recompiling it. Action: Recompile the class and proceed to state 6c.

State Diagram

In the following diagram, each state is shown as a record with three different boxes, one box to represent the sourcefile (`s'), one box to represent the classfile (`c'), and one box to represent the Class object in memory `o'. If a letter is shown then it exists, otherwise it does not exist. The ordering of the boxes indicates the relative order of the timestamps with the topmost box being the "newest" time, as though newer things are being stacked on top. Thus `OCS' (`6a') means "the timestamp of the Class Object is greater than or equal to the timestamp of the classfile, and the timestamp of the classfile is greater than or equal to the timestamp of the sourcefile".

The application may find itself in one of four scenarios after initialization. The solid blue and orange lines indicate action by the HotSwap engine itself. In constrast, the dark gray dotted lines indicate action by some external agent that is updating the state of the files, either by deleting files or changing their timestamps. This external agent could be a human with a text editor making changes to the sourcecode directly, some other part of the application, or another application entirely.

states

Records `2c', `4a', and `6a' are shown darker than the other records to indicate that if a HotSwap is attempted, no action will be taken since the Class object is up to date.

In the case where no sourcefile and no classfile exists (`1a'), classloading cannot occur and the HotSwap signals an exception. Note this is the ONLY scenario in which hotswap throws an exception.

If the application finds itself in case `2c', a warning is signaled through the logging interface. This is because although HotSwap is able to continue execution, both the sourcefile and classfile where deleted "under its nose", so to speak.

Note that this diagram does not show all the possible states transitions caused by the external agent upon the system. For diagrammatic clarity, only actions from the "ideal state" `6a' are shown.

Transaction Semantics

The hotswap operation, triggered by invocation of either the Proxy.hotswap() or ProxyClass.hotswap(), is a two-phase commit protocol. It involves a single ProxyClass pc and a set of Proxy instances P that are the children of pc and willfully elect to be a participant of the transaction. The ProxyClass pc manages a Class c; each Proxy p in P manages a single proxy Object po that may or may not be an instance of ProxyObject. PO is the set of all po objects in P. The sourcefile fs and classfile fc of c are used to determine if hotswapping is necessary (when the sourcefile is newer than the classfile).

Conditions

A hotswap is an all-or-none-operation that requires the following conditions for transactional commit:

Protocol

Step 1: Recompile

The ProxyCompiler is used to run a compilation command on the filesystem. If the classfile fc changes to fc', compilation is assumed to have succeeded.

Step 2: Classload

ProxyClass pc attempts to load a new Class c' from fc'. The old class c is not discarded unless the entire hotswap succeeds.

Step 3: Preparation

ProxyClass holds a list of participating (aka enqueued) p child Proxy objects. Each po in PO is allowed to prepare via the hotswap_onPrepare(ProxyObject) method (if it implements ProxyObject). If any po returns false, the hotswap is aborted. All po objects that have been prepared will be rolled back via the hotswap_onRollback() method and the new class c' is discarded.

Step 4: Commit

All objects po that have participated in the transaction will be notified of the commit via the hotswap_onCommit() method.

Step 5: Release

All old objects po that implement ProxyObject will be notified of their senescence via the hotswap_onRelease() method. The old class c is discarded and is replaced with c' (the changing of the guard).

Miscellaneous

ClassLoaders

Some of the most interesting bugs I encountered while developing this library stemmed from an incomplete understanding of the ClassLoader. Needless to say it is a central concept in Java, one that I had underestimated. If you don't know much about classloading I suggest your edify:

Here are some points that I think are important:

Identity

The identity of a Class is NOT the name of the class, it is the name of the class AND the defining ClassLoader. You can have more than one com.example.MyClass in a single JVM without conflict if they have different ClassLoader instances.

Delegation

Every ClassLoader has a parent ClassLoader except for the one at the root, sometimes called the primordial classloader.

Classloading follows a delegation model such that the parent ClassLoader is given the opportunity to load a Class before the child. This means that if any classloaders up the heirarchy can see the classfile, the ClassLoader implementation in this library will not be used to load the class and therefore will not be reloadable. You have to pay careful attention to what's in your CLASSPATH.

The parent ClassLoader is significant. I struggled with a weird bug for about 4 straight hours one day: I was getting NoClassDefFoundExceptions for seemingly no reason; my classes were not being found despite the classfiles clearly being in the right place. The problem was that I was testing under JServ, which has it's own special AdaptiveClassLoader (or something like that), and it was the AdaptiveClassLoader that was loading the `missing' classes in question. Since delegation only goes UP the ClassLoader heirarchy tree, the classes were not being found since they were stuck off in a branch. The solution was to grab the JServ ClassLoader and use that as the parent to my ClassLoader.

Relationship to JDK1.3 Dynamic Proxy Classes

The HotSwap Library and dynamic proxy classes (JDK1.3 - java.lang.reflect.Proxy and related interfaces) are two completely different things, they are designed to solve different problems. HotSwap is designed to provide dynamic class reloading and object state migration, while JDK1.3 dynamic proxy class are a general mechanism to create type-safe interface implementations on the fly. Both are designed to operate within the scope of a running application.

As it turns out, the problem addressed by HotSwap is a perfect application of dynamic proxy classes. That is, we can use dynamic proxy classes as a tool to solve the hotswap problem in a particularly elegant way. Dynamic proxy classes complement HotSwap. But support for dynamic proxy classes in HotSwap is dependent on a JDK1.3 environment and their usage is optional.


This document was generated on 27 July 2001 using texi2html 1.56k.