Playing Movies in a Java 3D World, Part 1

by Andrew Davison, author of Killer Game Programming in Java
06/01/2005

The ability to play a movie clip inside of a Java 3D scene opens up opportunities for richer, more interesting 3D content. A movie can display more believable backgrounds, such as moving clouds, a busy city street, or the view out of a window. Movies can be employed in help screens, or as transitions between game levels.

This article, which is split into two parts, describes how I implemented a Java 3D movie screen. In this part, I'll explain how I utilized the Java Media Framework (JMF), more specifically the JMF Performance Pack for Windows v.2.1.1e. The other tools in my arsenal were J2SE 5.0 and Java 3D 1.3.2. In part two, I'll discuss another version of the movie screen, using Quicktime for Java.

Figure 1 shows two screenshots of the JMF Movie3D application, taken at different times: the one on the right is a view of the screen from the back.

Figure 1. Two views of the Movie3D application

The important elements of this application are:

An integration of JMF and Java 3D. There can be multiple screens in an application, of any size. Since a screen is a subclass of Java 3D's Shape3D class, it can be easily integrated into different Java 3D scenes.
The implementation uses the Model-View-Controller design pattern. The screen is the view element, represented by the JMFMovieScreen class. The movie is the model part, and is managed by the JMFSnapper class. A Java 3D Behavior class, TimeBehavior, is the controller, triggering periodic updates of the movie. All of the JMF code is localized in the JMFSnapper class, making it easier to test any changes. Part two of this article essentially replaces JMFSnapper with a QuickTime for Java version called QTSnapper.
The use of Java 3D performance tricks to speed up rendering; the result is a movie that runs at 25 frames per second without any difficulty.
A discussion of the problems I had with JMF; problems that meant that my preferred solution wouldn't work--JMF has the potential to be a great API, but beneath its gleaming surface there are some poorly implemented features lying in wait.

1. I'm Sitting on a Mountain

Actually, no, I'm sitting on a chair in a very cold office with a thermostat that's out of reach. What I really mean is that this article rests on top of a lot of background knowledge about Java 3D and JMF.

I'm not going to explain the Java 3D elements in much detail, since they're covered in my O'Reilly book, Killer Game Programming in Java (henceforward known as KGPJ). For example, the checkerboard scene shown in Figure 1 is a slightly modified version of the Checkers3D example in Chapter 15. I've reused the code for creating the checkerboard floor, the blue sky, and the lighting, and the code for allowing the user to move the viewpoint around the scene.

If you don't want to buy the book, then early drafts of all the chapters, and all of the code, can be found at the book's website.

In this article, I'll explain the JMF techniques I've used for extracting frames from the movie. I won't be talking about streaming media, capture, or transcoding.

2. Two Overviews of the Application

The movie is loaded and played by the JMFSnapper class, and plays in a continuous loop until told to stop.

The movie screen is created by JMFMovieScreen, which manages a Java 3D quadrilateral (a quad) resting on the checkerboard floor.

One way of visualizing these classes is to look at the application's scene graph in Figure 2. (A scene graph shows how the Java 3D nodes in a scene are linked together.)

Figure 2. Scene graph for Movie3D

A lot of the detail in Figure 2 can be ignored, but the graph bears a striking resemblance to the one for the Checkers3D example in Chapter 15 of KGPJ. Only the movie-specific nodes are new.

The JMFMovieScreen and TimeBehavior objects are shown as triangles since they're nodes in the scene graph. The JMFSnapper object isn't part of the graph, but is called by JMFMovieScreen.

Every 40 milliseconds, the TimeBehavior object calls the nextFrame() method in JMFMovieScreen. That in turn calls getFrame() in JMFSnapper to get the current frame in the playing movie, which is then laid over the quad managed by JMFMovieScreen.

TimeBehavior is a subclass of Java 3D's Behavior class, and is the Java 3D way of implementing a timer. It's very similar to the TimeBehavior class used in the 3D sprites example of Chapter 18 of KGPJ.

Another way of gaining some insight about the application is to look at its UML class diagrams, given in Figure 3. Only the public methods of the classes are shown.

Figure 3. Class diagrams for Movie3D

Movie3D subclasses JFrame, while WrapMovie3D is a subclass of JPanel. WrapMovie3D constructs the scene graph shown in Figure 2, and renders it into the application's JPanel. It uses the CheckerFloor and ColouredTiles classes to build the checkerboard floor.

JMFMovieScreen creates the movie screen, adds it to the scene, and starts the movie by creating a JMFSnapper object. TimeBehavior calls JMFMovieScreen's nextFrame() method every 40 milliseconds. nextFrame() calls getFrame() in JMFSnapper to retrieve the current frame.

All of the code for this example, as well as an early version of this article, can be found at the KGPJ website.

3. Going to the Movies

The movie, its screen, and the TimeBehavior object for updating the screen, are set up by the addMovieScreen() method in WrapMovie3D:


// globals
private BranchGroup sceneBG;
private JMFMovieScreen ms;  // the movie screen
private TimeBehavior timer; // to update screen


private void addMovieScreen(String fnm)
{
  // put the movie in fnm onto a movie screen
  ms = new JMFMovieScreen( 
          new Point3f(1.5f, 0, -1), 2.0f, fnm);
  sceneBG.addChild(ms);

  // set up the timer for animating the movie
  timer = new TimeBehavior(40, ms);  
    // update movie every 40ms (== 25 frames/sec)
  timer.setSchedulingBounds(bounds);
  sceneBG.addChild(timer);
}

The two Java 3D addChild() calls link the JMFMovieScreen and TimeBehavior nodes into the scene graph. The setSchedulingBounds() call activates the TimeBehavior node (that is, it starts it ticking).

Pages: 1, 2, 3