MeBox: Live TV with Time Shifting

The perfect media PC software must support time shifting with live television. MPlayer does not support this out-of-the-box, and so some modifications had to be made. The sanest approach is to make as few changes to MPlayer as possible, and instead move the real logic to a controlling application.

The design I outline below is coded and working, and you can download some test code now. I have tested this with my ivtv card over the past couple of days and it seems stable. I hope that some of these ideas and code will help the Freevo project in its quest to implement time shifting live TV.

There's nothing especially innovative about the design below. The general approach is used in apps like MythTV, and the bits that make this work with MPlayer are just hacks. Still, I've tried to minimize the ugliness added to MPlayer, and the overall results are pretty encouraging.

Design

The most obvious approach to implementing time shifting is to use a ring buffer: the recorder reads from the source and writes to the end of the buffer, looping back to the beginning once it reaches some configurable limit; the player can then play the video back from any point in the ring buffer.

How the ring buffer is implemented can vary, but the approach I took was to have a single file, and simply have the recorder loop to the beginning of the file once it reaches the buffer's size limit (which in my implementation is specified in time, not bytes, though it would be easy to change). Also, the container of the video must be streamable, and the most obvious choice there is MPEG. (This wasn't a matter for consideration in my case, since my ivtv card generates an MPEG-PS stream.) Since the player in MeBox's case is MPlayer, it must have support for the ring buffer: notably, it must automatically seek to the beginning of the file once it reaches the end.

MPlayer Support

Making MPlayer do this was fairly straightforward: I needed some way to tell MPlayer what the end of the buffer was (slave command), and then Mplayer would know to seek to the beginning of the file if it reached EOF but not End of Buffer. However, one fairly big obstacle was MPlayer's lack of precision when seeking in variable bitrate MPEG streams. Also, MPlayer does not support handling commands (such as seeking) while paused.

So, a number of changes had to be made to MPlayer's source:

A new slave command, RINGBUFFER_SET, was added. This command takes two arguments, the first specifying what value to set (0 = buffer end time, 1 = byte position in the buffer for the next seek command), and the second argument is the actual value.

When more data is added to the ring buffer, the controlling process will issue this command to MPlayer to indicate what the end timestamp of the buffer is. MPlayer uses this information to determine what to do when it reaches EOF: pause and wait for more data, or loop to the beginning of the file. In practice, we never want to let MPlayer reach the point where it needs to pause to wait for more data, but it should do so Just In Case.

The other purpose for this command is to help MPlayer out when seeking inside the buffer. MPlayer's accuracy is not good when seeking in MPEG streams: it estimates what byte in the file to seek to based on its bitrate, and then finds the next key frame from that point. For VBR streams, it can be several to tens of seconds off the mark. However, since the recorder is able to keep track of the timestamps for each block in the buffer, it passes this onto MPlayer before seeking, which makes seeking accurate to within 1-2 seconds.
The MPEG-PS demuxer code was modified to handle support for the ring buffer; specifically, it seeks to the beginning of the buffer when it reaches EOF. When I say seek here, I mean fseek(), not seeking to a timestamp. This means that looping in the buffer is seamless. The demuxer's seek code was also adjusted to use the byte position hint provided by the master process. When it finishes seeking, it outputs a parsable line that contains the actual seek position (since it won't be exactly what was requested) so that the user display can be updated more quickly to reflect the new position the buffer.
The pause loop was modified to allow certain commands to work while MPlayer is paused (specifically seeking, ring buffer hints, and loading a new file). Audio is not decoded when seeking while paused, so MPlayer will not blip and squeal. This allows a very smooth experience when seeking inside the buffer while paused. From what I understand, this is a frequently requested feature, so I separated this specific functionality into a separate patch and submitted it to mplayer-dev. Hopefully it gets merged as it's a nice feature and I believe it's correct.

All that sounds like a lot, but actually that description is much longer than the patch itself: the patch adds only about 70 lines of new code to MPlayer.

The Recorder / Player

A separate process is responsible for capturing the video stream from the TV card to the ring buffer, and controlling MPlayer. In my test implementation, this is the same process, but the actual design should have 3 processes: record server, master application (MeBox itself), and MPlayer.

How the record server gets the video is immaterial, but the end result must be an MPEG-PS stream with incrementing timestamps from the moment the capture was initiated. Thus, a buffer is represented by the 4-tuple (start epoch, buffer start, buffer end, current position), where:

start epoch: time (unix timestamp) indicating when the capture began. The buffer start and end times are relative to this time. For my ivtv card, the MPEG timestamps are reset to 0 when the device is first opened, so the start epoch remembers this time.
buffer start: time (unix timestamp) indicating the time at the beginning of the buffer. This isn't necessarily at the 0th byte of the buffer file because, of course, it's a ring buffer.
buffer end: time (unix timestamp) indicating the time at the end of the buffer.
current position: the current position in the buffer being played back. This time is not a unix timestamp, but is instead is an offset in seconds relative to the start epoch.

The recorder then straightforwardly loops, pulling bytes from the source (which can be either a file/device like /dev/video0, or a separate process like mp1e) and writes to the ring buffer, looping back to the beginning when necessary, and keeping track of the start/end times of the buffer, as well as the byte position for every second (to assist in seeking).

When seeking in the buffer, MPlayer's master will first issue a RINGBUFFER_SET 1 [position] command, and then a SEEK command. When the MPEG demuxer starts the seek, instead of calculating the byte position itself, it uses the one specified by the RINGBUFFER_SET command.

One important consideration is the minimum allowable margin between the player's current position and the start or end points of the ring buffer. If this margin is too short, it may happen that the player and recorder try to read/write at the same point at the same time resulting in a very ugly mess. If the margin is too long, changing channels becomes quite slow, because the player must be halted until the buffer is full enough. Experimentation has shown that a margin of 2 seconds is about as low as we can get before we run into the problems described above. It may be possible to lower this margin considerably, but it would require a deep kung-fu understanding of MPlayer's MPEG demuxer that I simply don't have.

Implementation

I've worked up a proof-of-concept implementation and results are very positive. It seems to be quite stable and performs well. You can obtain the code on the Downloads page.

One issue that must be addressed is how to behave when changing channels. I had originally thought to keep recording while changing the channel and simply skip ahead a few seconds once the change was complete but I haven't been able to make this work reliably with my ivtv card. Instead, the current approach resets the ring buffer completely and reloads the file in MPlayer (hence the necessity to load files in MPlayer while paused). This approach works reliably, but is also rather slow: it takes about 0.5 seconds to change the frequency of the tuner, about 1.5 seconds to start capturing from the ivtv device, and another 2 seconds of buffering, plus about 0.5 of miscellaneous overhead. The result is a 4-5 second delay while changing channels. This delay does not represent a fundamental problem in the design, but it's certainly one area I'd like to see improved.

What I have made available is a stripped down, console version to demonstrate the approach. The code I've done for MeBox makes use of the canvas system to present a nice progress bar as seen below:

This is a working design. (Well, the program title/info text is hard coded for now, but everything else is real.) And with the nice slide/fade in effect descibed on the canvas page, it looks very slick. The appearance is blatantly stolen from XP Media Centre (although my version is nicer because it has the pretty TV watermark on the lower left). I chose to represent the Live TV buffer start/end times as time of day, rather than duration (e.g. "20 minutes in the buffer") like TiVo. This made most sense to me, but the time shifting code is not tied into this. You can present the buffer to the user any way you'd like.