| [Back] | Home | About | Policy | Products | Contact | Links |

RealMedia Analyzer

v. 0.30.00 (beta) Released for testing purposes.

User's Manual.

============================================================================== [1.] Short description [2.] Limitations [3.] The technique behind [3.1] Other useful utilities you may need [4.] Command line options -c = check file integrity -f = try to fix broken file -d = dump some information about the file -i = rebuild index sections -j = join (multiplex) two RealMedia streams -l = strip operations log and RealMedia Analyzer mark -p = display RealMedia Analyzer use and (re)distribution policy -s = split (demultiplex) stream -u = discard changes according to specified undo file -x = extract part of stream [5.] How do I.. [5.1] Automatically fix broken realmedia file? [5.2] Synchronize video and audio in the realmedia file? [5.3] Extract fixed-rate video/audio from the SureStream file? [5.4] Replace soundtrack in the realmedia file? [5.5] Extract or cut out a clip from the realmedia file? [5.6] Make truncated realmedia file seekable? [5.7] Paste several realmedia files into one long single file? [5.8] Behave myself if the file is still broken and nothing helps?! [6.] Compiling the sources [7.] Contacting the author ============================================================================== [1.] Short description RealMedia Analyzer (rma) is a simple command-line driven utility designed to check integrity and optionally fix errors in the Real Networks' streaming media files. It also can rebuild index sections e.g. in order to make truncated files seekable, split multiplexed stream into separate audio/video streams, join two RealMedia streams into one multiplexed stream and extract or delete stream parts from the single-stream RealMedia files. ============================================================================== [2.] Limitations - rma has limited ability to work with SureStream files, some features will only work with the fixed bitrate files. - rma will not work with files containing multiple DATA sections. - rma will refuse to multiplex/demultiplex/cut/paste streams which contain packets larger than 64 k (16-bit version) or 256 k (32-bit version). - Since recovery method is very heuristic (not to say "barbaric") it is not guaranteed to work with all files. Actually, it is not guaranteed to work at all. Be sure you've made your backup copies. You have been warned. ============================================================================== [3.] The technique behind RealMedia (rm) is a file format developed by Real Networks to store and distribute streaming video and audio content. Being designed for real-time network broadcasting, these files however are quite often used for media storage and exchange. Unfortunately, the method used to store media streams in rm files is absolutely non-error-proof. As a result, you may've experienced glitches (unexpected pauses) when trying to playback files damaged during the network upload or download process. As far as I know, Real Networks do not offer any utilities to check and fix errors of this kind. However, it is possible to fix them. The following technique is used by rma to detect and eliminate broken stream parts. Typical rm file contains two interleaved media streams (audio and video). Each stream consists of a bunch of packets. Each packet has a header part, which contains stream number and packet size. Thus content of RealMedia file may be represented by the following figure: +--------+ +--------+ +--------+ +--------+ |stream 0| |stream 0| |stream 1| |stream 0| etc.. |packet 0| |packet 1| |packet 0| |packet 2| +--------+ +--------+ +--------+ +--------+ size00 size01 size10 size02 offset00 offset01= offset10= offset02= offset00+ offset01+ offset10+ size00 size01 size10 Offset of each packet may be calculated as (offset of previous packet)+(size of previous packet). When one or several packets are broken (usually just replaced by some baudbarf) this chain is destroyed, and the RealPlayer's file parser is no longer able to determine next packet disposition when broken packet is encountered. To exclude broken packets from the stream, rma tries to find next intact packet in the stream and then adjust size field of the last encountered intact packet header, so it will point to the next intact packet header. In previous versions packet size was the only criteria used to determine whether the packet is broken or not. Since v. 0.20.00 rma also optionally checks for correct packets' timestamps. Still this isn't something one could call a reliable method.. But it works. At least sometimes. [3.1] Other useful utilities you may need There's another RealMedia management utility that was available freely on the Net for some time, it is called rmfix. Now, however, its homepage is gone and I believe that the project is dead. There's still a chance to find the distribution archive via the web search engines though. Personally I find rma faster and more convenient, but you may like rmfix better. It has a menu-driven interface and some additional features which rma does not offer. rmfix is a DPMI program and thus requires a DOS-compatible environment and a DPMI server to run (in case if you don't know, Windows 9x meets both conditions). A command-line utility called rmedit is supplied with RealProducer. You may find trial version (and order a pro version) at http://www.real.com With this program you may edit CONT section and also paste several RealMedia files into one file. Probably rmedit's paste feature is more reliable than that of rma, but I'm not sure about it. ============================================================================== [4.] Command line options WARNING: *always* create backup copy of your file when running rma with the -f command. It won't hurt to make a backup copy when running with any other command though. ------------------------------------------------------------------------------ Note for Unix users: some of the rma commands require file names supplied as their arguments. The file names are enclosed in the slash characters ("/"). However, since in Unix-like environments slash is used as a directory delimiter in the file path, backslash ("\") should be used instead when running Unix version of rma. This only applies to the filenames; all other commands whose arguments contain slashes and backslashes should be left unchanged. Moreover, since most of Unix shells will treat backslash as a special escape character, you may have to repeat it twice for the shell to interpret it correctly. This applies to all of the commands with backslashes in their arguments. Examples: DOS or OS/2 or Win32 version: rma.exe -j/c:\tmp\audio.rm//c:\tmp\video.rm/ new.rm Unix version: rma -j\\~/tmp/audio.rm\\\\~/tmp/video.rm\\ new.rm ------------------------------------------------------------------------------ rma command line syntax is rma [command[arguments]] <filename> Command must start with a hyphen. All commands and their arguments are case-sensitive. Available commands are: -c[arguments] = check file integrity (default) With this command rma will check packets disposition in the given RealMedia file. When a broken packet is encountered, rma will stop and display broken packet offset (relative to file start). You may then explore the file manually with a binary editor. This command accepts five optional arguments: d<n> = set maximum allowed timestamp difference to n (0..32767) milliseconds, 0 means don't care, default is 2000 When performing packets timestamp checking, rma compares current packets' timestamp with a previous packets' timestamp. If the difference exceeds specified number of milliseconds, current packets' timestamp is considered invalid and error is reported. Default value is 2 seconds, but sometimes it may be necessary to increase or decrease it. s<n> = set maximum allowed packet size to n (1..32767) kilobytes, default is don't care With this option you may set maximum allowed packet size. Packets with larger values in the header will be considered broken and reported. By default any packet whose indicated size fits within DATA section bounds is treated as a normal packet. t = disable timestamps checking This option will disable checking for correct timestamps when searching for a broken packets. Note that invalid timestamps not always mean paused playback. It seems that sometimes timestamps may be incorrectly set during the encoding process (a bug in the RealProducer? or a feature? I don't know). If rma tells you about invalid timestamps but the file is processed normally by the player, just ignore these messages. v = lower verbosity level This argument will disable packets counter, so you may harmlessly redirect rma's output to a log file. z = stop on timestamp errors Since v. 0.24.00 rma does not stop on timestamp errors and continues to seek through the file until illegal packet size is encountered. If you want it to stop at the first encountered invalid timestamp (as in the previous versions), use 'z' option together with -c command. Note: do not confuse 't' and 'z' options. With 't' option turned on timestamp errors will not be reported at all. -f[arguments] = try to fix broken file When issued with this command, rma will not only check packets disposition but also will try to adjust size of the certain packets to eliminate broken stream parts. This command accepts following optional arguments: a<n> = exclude n (0..64) packets after first found intact packet, (but not more than the number used with 'c' argument), default is 0 If the fixed file still pauses at the same position, increasing this value may help. However it may not exceed minimum intact packets chain length. b<n> = exclude n (0..64) packets before last intact packet, default is 0 For the same purpose that the previous parameter. c<n> = use minimal size of n (2..32767) packets chain when searching for unbroken stream parts, default is 4 Set minimum length of the packets chain to search. Default value should suit in most cases, but sometimes it may help to increase it. d<n> = set maximum allowed timestamp difference to n (0..32767) milliseconds, 0 means don't care, default is 2000 When performing packets timestamp checking, rma compares current packets' timestamp with a previous packets' timestamp. If the difference exceeds specified number of milliseconds, current packets' timestamp is considered invalid. Default value is 2 seconds, but sometimes it may be necessary to increase or decrease it. m[suppression level] = suppress logging Since version 0.25.00 rma puts its signature at the end of processed RealMedia file, so one may determine whether this file has been altered by rma or not. Since v. 0.30.00 even more advanced logging system is implemented. When run with a command that may possibly alter the rm file contents, rma logs all performed operations and adds this information to the end of processed file. Operations log may be displayed with the -d command. There is a specific command to strip this information from the file as well. With 'm' option you can control the amount of information being logged. The 'suppression level' argument may take one of the following numeric values: 1 = do not update log. Will keep the log if it was present, but will not add any new info to it. 2 = mark only. Will strip operations log but leave the rma signature at the end of the file. 3 = suppress completely (default). Will strip both operations log and signature if present, and will not add any new ones. If 'm' option is not specified, full operation logging is performed. n<n> = only fix n (1..32767) broken packets, default is all Use this to ensure that the glitch was eliminated. E.g., use n1 for rma to try to fix only the first error, then try to playback the file. If it is fixed, try to fix next error, otherwise try to play around with 'a' and 'b' parameters. '-fb1' usually does the trick. p<n> = force maximum packet size (intact packet criterion) to n (16..32767) bytes This value is used to recognize intact packets among broken ones when searching for the intact packets chain. Do not confuse it with size set with the 's' option (see below). By default rma takes this value from file header, but if it is less than 1024 it is always set to 1024. s<n> = set maximum allowed packet size (broken packet criterion) to n (1..32767) kilobytes, default is don't care With this option you may set maximum allowed packet size. Packets with larger values in the header will be considered broken and eliminated. By default any packet whose indicated size fits within DATA section bounds is treated as normal packet. You should use 's' option if during the fixing process enormously large new packets' sizes are being displayed. t = disable timestamps checking It is recommended to fix errors in the file (if any) with this option turned on, and then when all the other errors are fixed try to fix the ones related to invalid timestamps. This kind of errors is usually hardest to fix, however invalid timestamps not always lead to a paused playback. v = lower verbosity level (see above) z = try to fix timestamp errors Since v. 0.24.00 rma does not try to fix timestamp errors by default, and continues to seek trough the file until illegal packet size is encountered. If you want it to try to fix timestamp errors as well (as in the previous versions), use 'z' option together with -f command. Note: do not confuse 't' and 'z' options. With 't' option turned on, timestamp errors will not be considered at all (i.e. not even be reported). /undo_filename/ = create an undo file with the specified pathname If undo filename is specified, all changes made to packet headers during the autofix process will be logged to the undo file so it is possible to discard them later (see -u command description). Note: undo feature is a "poor man's backup". If you have enough of free disk space, it is better to make a backup copy of the file being processed. -d[argument] = dump some information about the file When issued with this command, rma writes some information retrieved from rm file to the console (stdout). This command accepts one of the following (mutually exclusive) arguments: b = dump brief info about the file Only the average bitrate, file duration, audio and video bitrate, video frame size and approximate framerate are displayed. g = dump general information Will dump miscellaneous info retrieved from file header and the RealMedia Analyzer operations log if the latter is present. This is the default behaviour. l = display operations log only Will display the RealMedia Analyzer operations log, if any. i = dump file index Will verbosely dump rm file index sections. -i[arguments] = rebuild index sections With this command rma will try to rebuild index for each stream contained in the rm file. Use this command to make truncated file or file with broken index seekable. Note: You always have to use -f command before using -i command if you're not sure if the file has any errors. Available arguments for this command are: a = index all keyframe packets You may encounter a RealMedia file which contains a stream consisting of keyframes only (no diff frames). In this case usual algorithm used to build index will not work (only first frame will be indexed). With 'a' argument you may force indexing of each packet belonging to a keyframe. This will make any RealMedia file seekable, but you may experience some glitches when try to ffwd/rewind newly indexed file. m[suppression level] = suppress logging (see above) v = lower verbosity level (see above) -j[arguments] = join (multiplex) two RealMedia streams With this command you may multiplex or paste two RealMedia streams. Only single-stream files may be multiplexed or pasted together (but rma will not make any checks for number of streams, it simply takes stream 0 from each file). Normally, when multiplexing two files first file contains audio stream and second file contains video stream. When pasting two files they both must contain streams of similar nature. It is usually harmless to paste two audio streams, but trying to paste together two video streams rarely succeeds (it seems that RealPlayer is very sensitive to changes in packets' timestamps in the newly created stream). Available arguments for this command are: /firstfilename/ = pathname of the file containing first stream, default is stream00.rm /secondfilename/ = pathname of the file containing second stream, default is stream01.rm With these two arguments you may specify names of the files containing first and second stream. Default names are stream00.rm and stream01.rm in the current directory. Which stream will be numbered 0 in the resulting file is determined by order of appearance in the command line. i = build index after join With this option rma will automatically build index in the output file. Otherwise you will need to run rma another time with -i command explicitly. m[suppression level] = suppress logging (see above) p = paste instead of multiplex If this option is present, files will be pasted together. Otherwise they will be multiplexed. s = try to synchronize streams If this option is present, rma will try to equalize streams' durations. This is done by means of adjusting packets' timestamps in the stream with lesser duration. Note: it is dangerous to change timestamps in video stream. In most RealMedia files audio stream duration is slightly lesser than video stream duration. This is normal. In any case, you should avoid using 's' parameter unless absolutely necessary. Note: 's' parameter is ignored if used simultaneously with 'p' parameter. q[l]<time> = set relative stream 0 to stream 1 skew in milliseconds (0..10000, signed). ql means set logical skew, otherwise set physical skew With this argument you may set relative video-to-audio start delay. Negative number means first stream starts earlier. "Logical skew" means that only appropriate fields in the file header will be adjusted. However, it seems that these fields are usually ignored by the player. "Physical skew" means that packets' timestamps will be adjusted instead. If requested skew is negative second stream will be affected, otherwise adjustment will be applied to the first stream. Remember that it is dangerous to change timestamps in video stream. You should keep this in mind when using 'q' parameter. Note: 'q' parameter is ignored if used simultaneously with 'p' parameter. v = lower verbosity level (see above). -l = strip operations log and RealMedia Analyzer mark, if any With this command rma will strip operations log and signature from the end of processed RealMedia file, if they are present. -p = display RealMedia Analyzer use and (re)distribution policy With this command rma will type its use and (re)distribution policy text to the console (stdout). -s[arguments] = split (demultiplex) stream This command will demultiplex RealMedia stream to separated video and audio streams. By default stream 0 and stream 1 are extracted and written to the files with names equal to the input file name and extension replaced with a stream number (i. e. '000' and '001'). Available arguments: m[suppression level] = suppress logging (see above) p</directory prefix/> = use specified drive/directory for output files (default is current directory) Specified directory will be used to store extracted streams. By default files are written to the current directory. x<n> = extract n'th stream from the file Will extract specified stream instead of streams 0 and 1. This command is useful for converting SureStream files into conventional fixed-bitrate files. v = lower verbosity level (see above). -u[arguments] = discard changes according to specified undo file With this command you may restore original packet headers from the previously created undo file (see -f command description). This command is useful if autofix was ineffective and you would like to try -f command with a different parameters set. You should specify undo file name instead of RealMedia file name when using -u command. Example: rma -f/test.und/ broken.rm will create undo file named test.und in the current directory; rma -u test.und will restore original broken.rm. The only available argument for this command is: m[suppression level] = suppress logging (see above). -x[arguments] = extract part of stream With this command you may extract or delete parts of RealMedia stream. It only works with the single-stream files. If you want to extract or/and delete part of a multiplexed audio/video, you should demultiplex it first and use '-x' command on each stream, then use '-j' command to re-multiplex them. Again, rma will not check if the file does not contain multiple streams, it merely takes packets from the stream numbered 0. Resulting file (either extracted chunk or the file with a deleted chunk) is written to the file with the name equal to the input file name and extension replaced with 'cut'. Available arguments for this command are: /<start time> (default is 00:00:00.00) Set start time of the stream chunk to extract. Time record format is hh:mm:ss.hs. \<end time> (default is end of stream) Set end time of the stream chunk to extract. d = delete part of stream instead of extract If this option specified, stream chunk will be deleted instead of being extracted. k = ignore keyframes Normally rma will search for keyframes with the timestamps most close to specified start and end time. With this option, however, rma will consider any packet with the closest timestamp as a valid chunk start/end, doesn't matter whether it's a keyframe packet or not. Note: in some files there are only keyframes present (no diff frames). In audio streams all packets are also usually marked as a keyframe packets. To correctly extract chunk from such files you will need to use 'k' option with the -x command. m[suppression level] = suppress logging (see above) o = write old-style file header Since v. 0.30.00 rma uses new (V1) RealMedia file headers in the files it creates. With this option you may enforce old (V0) header to be used. I really don't see a reason for it though. p</directory prefix/> = use specified drive/directory for output file (default is current directory) Specified directory will be used to store extracted chunks. By default files are written to the current directory. t = ignore timestamps If this option specified, rma will not try to correct packets' timestamps in the extracted stream chunk or in the file with deleted stream chunk. Normally rma will adjust extracted chunk's packets' timestamps so that the first packet have timestamp 0. However, it may cause problems with video synchronization and/or playback. Leaving timestamps not fixed in the extracted chunk usually doesn't hurt. However, when deleting stream chunk, use of 't' parameter will lead to pause during playback. Pause duration is equal to deleted chunk's duration. Since it is quite dangerous to change timestamps in video streams, it is recommended to use this option whenever possible. ============================================================================== [5.] How do I.. This section contains command line examples for the most common tasks you may use rma for, with minimum explanations when necessary. [5.1] Automatically fix the broken realmedia file? rma -f broken.rm If it doesn't help, restore the backed up copy and try rma -fa1b2 broken.rm If something strange happens, try rma -ft broken.rm [5.2] Synchronize video and audio in the realmedia file? First, split the file into separate video/audio streams: rma -s broken.rm You will get two files: broken.000 and broken.001. Now try to play with video/audio relative time shift. E.g., set audio skew to 500 milliseconds (0.5 sec): rma -jql500/broken.000//broken.001/ fixed.rm If it's getting worse, try to change skew direction: rma -jql-500/broken.000//broken.001/ fixed.rm If nothing happens, try to force skew: rma -jq500/broken.000//broken.001/ fixed.rm or rma -jq-500/broken.000//broken.001/ fixed.rm If it still doesn't help, here's the last resort: rma -js/broken.000//broken.001/ fixed.rm Don't forget to reindex newly created file: rma -i fixed.rm Note: upgrading to a newer RealPlayer version may also help in this case. [5.3] Extract fixed-rate video/audio from the SureStream file? First, dump stream info into some file: rma -d surestream.rm > log.txt Browse log.txt with an ASCII text viewer. Find out numbers of streams with MIME type x-pn-realaudio and x-pn-realvideo. They are the fixed rate audio and video stream respectively. Say, they are stream 6 and stream 8. Extract these streams: rma -sx6 surestream.rm rma -sx8 surestream.rm You will get two files: surestream.006 and surestream.008. Now, multiplex them into the single file: rma -j/surestream.006//surestream.008/ fixed.rm Don't forget to reindex newly created file: rma -i fixed.rm [5.4] Replace soundtrack in the realmedia file? Extract video stream from the file. Usually it's stream 1. rma -sx1 broken.rm Encode new soundtrack into realaudio format. You will need RealProducer to accomplish this task. Now multiplex extracted video with the new sound: rma -j/newsound.ra//broken.001/ fixed.rm Don't forget to reindex newly created file: rma -i fixed.rm [5.5] Extract or cut out a clip from the realmedia file? Let's extract a 30-sec clip from the test.rm, starting at 50 sec mark and ending at 1:20. First, demultiplex test.rm: rma -s test.rm You will get test.000 and test.001. Now let's extract chunks from both streams: rma -xtk/00:00:50.00\00:01:20.00 test.000 ren test.cut audio.rm rma -xt/00:00:50.00\00:01:20.00 test.001 ren test.cut video.rm Note, we didn't forget to rename 'test.cut'! Now let's multiplex two clips: rma -ji/audio.rm//video.rm/ clip.rm [5.6] Make truncated realmedia file seekable? rma -i broken.rm If it doesn't work try rma -ia broken.rm [5.7] Paste several realmedia files into one long single file? Let's paste two RealAudio files 1.rm and 2.rm into one file 3.rm: rma -jp/1.rm//2.rm/ 3.rm rma -i 3.rm Note: trying to paste video usually doesn't give good results. [5.8] Behave myself if the file is still broken and nothing helps?! Fall on the floor and cry loudly. Trying to brake nearby furniture also may help. When finished, try this: 1) Locate broken packet: rma -c broken.rm rma will give you last good packet's timestamp. Write it down. Say it's e.g. 00:12:05.32. 2) Run autofix: rma -f broken.rm 3) Demultiplex the file: rma -s broken.rm 4) Delete broken part from the video stream: rma -xdt/00:12:05.30\00:12:05.35 broken.001 5) Multiplex video and audio: rma -ji/broken.000//broken.cut/ fixed.rm If it still doesn't help, try rmfix.exe ============================================================================== [6.] Compiling the sources Full source code is included with the distribution since v. 0.20.00. rma is written in a portable Pascal dialect and may be compiled with variety of Pascal compilers for various target platforms, namely: Borland Pascal - DOS, DPMI; Virtual Pascal - any of supported targets; TMT Pascal - any of supported targets; GNU Pascal - the same platform that the one gpc is hosted on. It will compile with Free Pascal (at least 1.0.6), but the resulting binary will be buggy. I'm not sure if it is caused by bugs in rma or in compiler. In any case it is not recommended to use fpc for release builds yet. With Pascal, unlike with C, you don't need any make files since the compiler resolves unit dependencies by itself. You just have to use respective compiler's option to enforce it to rebuild the sources. Some conditional symbols should be defined depending on the target platform in order to build rma correctly. These symbols are: __WIN32__ - for Win32 target MSDOS - for DOS target MSDOS and _32BIT - for 32-bit DPMI target DPMI - for 16-bit DPMI target __OS2__ - for OS/2 target __UNIX__ - for Unix target All of these symbols are either defined automatically or predefined in VERSION.INC for the above listed compilers. so if you are using one of these compilers you don't have to define them explicitly. Build date is taken from DATETIME.INC, so you have to provide this file containing the actual date and time. It may be done either by standard OS commands or by the DATETIME utility included along with the rma sources. Theoretically rma should compile and run on big-endian platforms as well, though it was not tested. In this case symbol __BYTES_BIG_ENDIAN__ should be defined at compilation time. It is automatically defined by gpc running on a big-endian platform. For fpc however you will need to define it explicitly. ============================================================================== [7.] Contacting the author You may contact me either by e-mail or by FTN netmail. Send a message to: e-mail - alexeysp@yahoo.com FTN - Alexey Pavluchenko, 2:463/416.256@fidonet Latest version of rma may be downloaded from http://www.i.com.ua/~alexeysp/rma/rma.html or its mirror http://www.anime.com.ua/misc/alexey/rma.htm or obtained from the author via e-mail. Comments, suggestions, bug reports are welcome. ============================================================================== - Asp //VRg 31.Oct.2000 Last updated 09.Jun.2003

 | [Back] | Home | About | Policy | Products | Contact | Links |