The aim of this software development was to create a simple application for playback of multimedia streams (in particular MPEG stream). Additional, important requirement was to allow playback not only from file/URL, but also from local network broadcast (simple broadcast over UDP). Application was required to work properly on Sun/Solaris platform, but should also be easily portable to other JMF enabled systems.

There are several multimedia libraries available under public licenses, that can provide stream decoding and processing. As the base multimedia library for this project, we have chosen Sun’s implementation of Java Media Framework. Just like other modules, JMF takes care about proper MPEG decoding, and provides basic display controls, demanding from the application developer only data supply, application logic control, and User Interface implementation. However, this library was chosen due to a few important advantages. The most important for this project, is the ability of JMF MPEG decoder, to playback MPEG stream starting from any point of the stream (i.e. to pick up playback in the middle of the stream transmission). Initial analysis have proved that most of other available libraries fall into trouble under such conditions (sometimes they need multiple restarting, or providing them with stored initial part of an MPEG stream). The second advantage of JMF, is the ability to playback a variety of multimedia audio and video streams, all under control of the same, unified programming interface (API). Thus application implemented for MPEG stream playback, could probably be used for many different kinds of multimedia streams without any modifications. JMF has also additional advantages over other considered libraries: synchronized handling of MPEG audio and video tracks (many other libraries process only video track, or output each track separately, causing synchronization problems) and multi-platform support, thanks to Java VM used (however, there are several restrictions about JMF functionality, depending on the platform chosen, the all-Java implementation of JMF will work on every Java VM, but has weak performance and significantly restricted range of streams supported). Due to Sun/Solaris platform requirements, the fact that JMF is provided by Sun was also considered meaningful (so we can expect JMF to be compatible and optimized for Sun hardware, and work seamlessly under Solaris). The final JMF advantage over other public libraries, is the continuous development and improvement of this library. Support for additional stream formats will be added, and new functionality (transcoding among different stream formats, audio/video capture, RTP receiving/transmitting, playback speed control, access to every video/audio frame, etc.) are to be provided in the next generation of JMF – 2.0 .

Obviously, decision to use JMF library has significantly influenced choice of Java as the implementation language for this application (while most of the available libraries provide C interfaces). Despite the initial uncertainty, Java has finally proven to be quite a good choice for such project, keeping the source code clear and readable. The all-over performance of application also seems to be good enough, if only proper JMF performance pack is used (with performance pack, most of time-critical stream processing is handled with native platform libraries).

Functionality

Current features of the application cover playback of all streams supported by JMF library from file (you can find list of supported formats in appendix to this document), and playback of several (limited choice) streams from HTTP source (URL) and from network broadcast. Among the application abilities are:

• pausing and restarting playback of the stream at any time,
• positioning of playback for streams read from files,
• control over the sound volume,
• ability to disable sound (mute),
• and for video playback ability to adjust the screen size.

Many streams can be played back concurrently and independently (for each stream new window will be opened with separate set of controls). For most streams there is also additional control provided, which can display stream processing statistics, encoding type, playback duration, sound frequency, frame rate, and other stream-specific data. The application can be supplied with stream localization parameters from the command line, so it will pick up the stream and start playing immediately after starting.

The initial user interface consists of menu bar, available to user right after starting the application. This menu contains basic commands and options for the application control. There are two groups of menu items: "File" and "Options". Choosing "Open File" from "File" menu will open a new dialog window with directory navigation controls, so the user can choose the file for playback. Next menu item "Open URL" will pop up a dialog window allowing user to type in the URL for the stream to be played back. Accepted protocols are "file:", "http:" and "netdoc:" (the last one is kind of a back-door solution for starting the playback of broadcasted stream, parameters must be provided with address, port and stream type, e.g. "netdoc://224.0.0.100:23461/mpg"). "New Player" menu item will open new player window, where separate stream could be played back. "Close player" will close playback in current window and free machine resources allocated for this playback. Finally, the "Exit" menu item will close the current window, and closing the last window will terminate the application. Menu "Options" contains settings for the application, actually only one setting "Auto replay" – when enabled the stream will be played back again and again.

After supplying valid stream source location, the player for the stream is created, and playback begins. The video track is displayed in a panel right under the menu bar, but this panel is absent when the stream doesn’t contain video track. After the player initializes, another panel is available on the bottom of the window, with all playback controls that are supported for the current stream. Usually user can find there PLAY/PAUSE button allowing him to pause and resume the playback, MUTE button, for immediately disabling the sound, volume control buttons for adjusting the sound volume, and the stream status report button (the right-most one). For most streams played back from file, additional slider control will be available, allowing repositioning the playback at any moment. If the video display panel is visible, you can adjust video size with dragging the window edges with mouse, or by invoking the pop-up menu and choosing the zoom factor (you can invoke the pop-up menu by right-clicking the video screen).

Due to Java programming rules, application contains several classes, providing various aspects of its functionality. The base application class is NetPlayer class, extending the java.awt.Frame class (so it can display own, separate window) and implementing ControllerListener interface (this interface is defined in JMF library and allows control of the stream decoding and processing). Every object instantiated from this class provides playback functionality for single multimedia stream, managing own, separate window, playback controls and using separate JMF decoder. This class also contains static main() method, instantiating first object and possibly starting the playback if stream location is provided on the command line. All the rest of application logic and playback control is stored within objects of this class. Upon creation each of them, within constructor code, will create separate window and own menu bar, so they work independently, only updating the common, static variable windowCount, so we know when to shutdown the application. Each of them can be used to create another player window, but they don’t keep any relationships. For the purpose of starting playback, one common method openURL() is provided, supporting whole set of possible data sources. Calling this method with valid URL string will lead to creating the player for given stream format and attempt to start playback of the stream. Further stream processing is controlled through controllerUpdate method (from implemented controllerListener interface), which is used by the decoder to report events that occur during processing the stream (detailed communication description is provided below). All additional methods of this class are auxiliary and utility methods, embedding tasks like creating new dialog window, properly resizing the UI after user command, etc. Every object of this class will also maintain a set of internal variables, storing references to GUI and decoder objects used. Within this class additional internal classes are declared, used for creating dialog windows. Separate, but also auxiliary class is a MessageBox class, used for displaying the error messages to the user in convenient way. Additional classes are defined for receiving stream data from network broadcast, as this functionality needs special handling for JMF. Class Receiver is defined for receiving and buffering data incoming from the network broadcast, in particular this class handles separate thread used exclusively for awaiting the data, receiving them and storing into memory buffer. This thread is given relatively high priority, so we can minimize data loss (however as UDP protocol is unreliable, we cannot avoid losing some packets at all). Classes NetDataSource and NetSourceStream are created to handle picking such buffered data up and passing it to JMF decoder, whenever it calls for new data. Those classes must also implement methods for identifying stream type, so the valid decoder can be created – currently this information must be passed from outside. More detailed comments on purpose of every class and method are provided in source code.

There are two general steps of cooperation with JMF framework. First, proper decoder (player) object must be created, able to playback given multimedia stream. When the stream is to be played back from file or HTTP, we can use library method createPlayer() defined in common Manager object, passing URL as argument. But when we want to playback from non-standard source of stream data (e.g. network UDP broadcast), we have to create proper, custom object for handling the data supply, extending DataSource class. Such object must provide information about stream format, and expose data stream (object implementing SourceStream interface) used by decoder. The possibility of creating custom data sources is very powerful feature of this library, but sometimes also leads to restrictions in range of formats supported for such data source (described in details below). When we finally have the data source object ready, we can create the proper decoder, and second cooperation step begins. Now decoder (also called controller or player) will inform us about its situation through common controllerListener interface, calling method controllerUpdate in our object (previously registered to listen for those events) and passing event objects, so we can analyze the event and take proper action. Such events contain information about controller state transitions (unrealized, realized, prefetching, started), or processing errors. Newly created controller is initially unrealized, which means that it doesn’t know anything about the stream and is absolutely not ready to start. We must first call realize(), so it will prepare itself, allocating system resources and initially analyzing the stream. If everything goes right, controller moves to realized state. Now application can fetch from the controller the set of UI controls available for current stream, and the panel for displaying the video (if available). But to actually start playback, the controller still needs quite a long time. We can order it to start() immediately, so it will finally start playing, or we can go through two steps, asking controller to prefetch() data first, so later start() call will be much faster. All those states are separated due to time consuming actions the controller must take to actually start playing. With those states we can also have better control over system resources allocation. However we can simply call start(), and controller will move through all states automatically.

Restrictions

Despite the common API, every stream format is handled by separate decoder, causing different requirements about the data source properties, and different set of formats supported on different platforms. In particular, for MPEG stream decoding on Win32 platform (Windows 9x/NT), JMF uses native Microsoft DirectShow decoder, which allows stream playback only from files. Similar restrictions come from the Quicktime stream decoder, expecting the data source to be seekable (positionable), thus disabling possibility to playback this stream from network broadcast. Due to those restrictions, actual abilities of the application must be tested separately for every platform and data source type. Below we have presented current test results.

‘+’ symbol stands for ability to playback, ‘-‘ for disability.

More detailed list of formats supported by JMF 1.1 is provided as appendix to this document.

1. Optimization

JMF is available under few distribution versions. A cross-platform, all Java version, can run on any Java VM, but has the widest restrictions of supported formats and weak performance. However there are also specialized distributions of JMF "performance pack", optimized for Sun/Solaris and Win32 platforms. In fact all tests and development of this application have been performed with "JMF Solaris performance pack".

Further optimization on Solaris platform could be performed by adjusting the cache size, threads type (moving from "green" threads to "native" available under newer Solaris) and adjusting the platform settings.

2. Test results and possible further improvements

Test performed for MPEG stream on Sun/Solaris platform have proved proper application functionality for all data source types (file, URL or network broadcast), no problems with picking the transmission up in the middle of the stream (network broadcast), good quality of video and audio tracks together with video frame rate (25-30 fps on Ultra 1 workstation for stream encoded with 30 fps).

Starting the player on a remote server with display on local machine leads to reduced performance, dependent on video size (probably due to sending all video data from the server for local rendering), and no sound is available (as the player in fact runs on remote machine).

We must notice that during playback from network broadcast, from time to time disturbances in playback appear. The reason seems to be loss of data packets by the receiver (server is sending lots of short data packets really quick, so it is quite possible to lose a packet). However, if only the workstation is not too burdened, those disturbances do not have greater impact over the video quality.

The application could be improved by adding playback of streams transmitted by RTP server, programmable buttons with stream locations (like TV channel buttons) for easy switching among different streams, and probably further optimizations could be achieved. Together with development of JMF, new functionality like capturing and transmitting or accessing separate video frames (possibly providing On Screen Display and such features) could probably be added.

To install and run the application, two components must be installed first:

• Java VM – JDK 1.1.4 or later, works with JDK 1.2 (tested under JDK 1.1.6)
• Java Media Framework - JMF 1.0 or later (tested under JMF 1.1)

There is a makefile provided with the application. Several environment variables must be set to start the player, so the makefile must be supplied with valid JDK and JMF path locations (defined at the top of the file, see makefile). Default target "all" is defined to compile all required classes. The second target - "start" will run the application (in fact calling java NetPlayer).

If you want the player to begin playing immediately after startup, pass the media URL to the application from command line, for example:

• java NetPlayer file:///usr/local/video/sample.mpg – start playing from specified file
• java NetPlayer http://iris.ics.agh.edu.pl/video/sample.mpg – start playing clip from web server
• java NetPlayer netdoc://224.0.0.100:54444/mpg- start playing stream from network multicast

There is a possibility to use JMFPlayer as an external application for playing MPEG-1 streams in SDR tool. Necessary plugin file, which name starts with sdr2.plugin… can be found together with the other sources of the application.

You should copy it to your /.plugin subdirectory in SDR main directory.

Then every time you will receive announcement containing MPEG-1 media you will be able to run JMFPlayer just by clicking the button.

JMFPlayer can be downloaded from ACC Meccano web site:

http://www.cs.agh.edu.pl/Meccano/software/jmfplayer/jmfplayer.zip

• www.javasoft.com/products/java-media/jmf – JMF main page
• www.javasoft.com/products/java-media/mail-archive/Framework/index.html – discussion list for JMF developers
• www.javasoft.com/products/java-media/jmf/forDevelopers/playerapi/packages.html – API docs for JMF library
• www.javasoft.com/products/jdk/1.1/docs/api/packages.html – API docs for JDK platform
• www.mpeg1.de/unix.html – other libraries for MPEG playback.

Below are listed all supported media types for playback from files. As for HTTP or network streams, there are more restrictions, so every stream format must be verified separately.
‘X’ stands for ability, ‘-‘ for disability