Sound and Video Codecs

[DanielGibson]

Thanks for releasing the source of Descent 3, this is really awesome!

---

Some proprietary sound and video libraries from Interplay have been stripped out (the ACM and MVE format). I have that code if someone wants to help make a converter so the old cutscenes work.

Open Source implementations of these codecs are available, so it might be feasible to use the original files without (external) conversion?

ACM

Information on the codec: https://wiki.multimedia.cx/index.php/Interplay_ACM
Decoder library (ISC license): https://github.com/markokr/libacm
FFmpeg's decoder, based on libacm: https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/interplayacm.c
Libfalltergeist (MIT license) supports ACM: https://github.com/falltergeist/libfalltergeist

Where is ACM used? Only as part of (MVE?) cutscene videos?

MVE

MVE seems to be supported by ffmpeg; though in my experience having a dependency on ffmpeg is a pain in the ass, a minimal lib (or directly integrating code from ffmpeg into the source port) is preferable.
(OTOH, if someone wants to go the "convert to better format" route, ffmpeg could probably be used for conversion)

Information on the codec:

• https://wiki.multimedia.cx/index.php/Interplay_MVE
• https://wiki.multimedia.cx/index.php/Interplay_Video
• https://multimedia.cx/interplay-mve.txt
  • sample code for parsing the structure (Public Domain): https://multimedia.cx/parse-ipmve.c

Implementations:

• In ScummVM (GPLv3): https://github.com/scummvm/scummvm/blob/master/video/mve_decoder.cpp https://github.com/scummvm/scummvm/blob/master/video/mve_decoder.h
• gstreamer (LGPLv2+): https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/tree/a1cfba43f649adc617fe41e265d5ed91b2813392/gst/mve
• ffmpeg (LGPLv2.1+): https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/interplayvideo.c
• In NBlood, based on ffmpeg (LGPLv2.1+): https://github.com/nukeykt/NBlood/blob/master/source/rr/src/playmve.cpp
• In FreeSpace2: http://svn.icculus.org/freespace2/trunk/src/movie/
  • The license is a bit unclear, but Velyn might have related testcode that's MIT license-friendly: https://cohost.org/rohit/post/5583600-descent-3-s-source-c#comment-fa05b37b-7995-4735-b9cc-df5e3ee7560d
• Libfalltergeist (MIT license) supports file formats used by Fallout 1 & 2, including MVE and ACM: https://github.com/falltergeist/libfalltergeist

[DanielGibson]

An example of a small library with usable interface that is based on ffmpeg code: libbinkdec from http://homepage.eircom.net/~duncandsl/avp/

Browsable source of libbinkdec: https://github.com/RobertBeckebans/RBDOOM-3-BFG/tree/master/neo/libs/libbinkdec

[leilei-]

Where is ACM used?

Interplay around this time (1997-2000) had used ACM generally for every sound effect and music stream. This included some Bioware games as well

[winterheart]

Implementations from falltergeist (Fallout 2 reimplementation):

https://github.com/falltergeist/falltergeist/tree/develop/src/Format/Acm
https://github.com/falltergeist/falltergeist/tree/develop/src/Format/Mve

ACM documentation available on TeamX site:
https://teamx.ru/site_arc/docs/acm_format_en.rar

[sirlemonhead]

I have an MVE playback implementation, based on FFmpeg that I used for Redneck Rampage Rides Again in the NBlood project: https://github.com/nukeykt/NBlood/blob/master/source/rr/src/playmve.cpp

http://homepage.eircom.net/~duncandsl/avp/ is also my site but the code repository is no longer accessible, I'll get that fixed! cool to see the code is used elsewhere though! :)

[DanielGibson]

@sirlemonhead Nice!
I integrated libbinkdev into rbdoom3bfg back in the day, IIRC integrating it it was pretty straightforward, much nicer than the code using FFPMEG they had before :)

[kevinbentley]

@burgerbecky (one of the original devs for the formats) told me last night that she has code she can release. Can someone verify if the mve files packaged with D3 work with ffmpeg?

[DanielGibson]

From the GOG version:

$ ls -lh movies/
total 517M
-rw-rw-r-- 1 caedes caedes 7,9M Nov 22  2017 dolby1.mv8
-rw-rw-r-- 1 caedes caedes 153M Nov 22  2017 End.mve
-rw-rw-r-- 1 caedes caedes 142M Nov 22  2017 intro.mve
-rw-rw-r-- 1 caedes caedes 149M Nov 22  2017 Level1.mve
-rw-rw-r-- 1 caedes caedes  31M Nov 22  2017 level5.mve
-rw-rw-r-- 1 caedes caedes 632K Nov 22  2017 mainmenu.mve
-rw-rw-r-- 1 caedes caedes 4,4M Nov 22  2017 ME.MVE
-rw-rw-r-- 1 caedes caedes  31M Nov 22  2017 MI.MVE

mpv (which uses ffmpeg/libav) can play all of those files (though it seems a bit confused about the length and seeking is glitchy)

But if Becky can release the original codec source that would be even better, of course :)

[chrominance]

Can also confirm ffmpeg can read the MVE files, I just tried converting the intro to an MP4 and it plays back just fine with video and audio intact:

D:\GOG Games\Descent 3\movies>ffmpeg -i intro.mve -c:v libx264 -preset slow -crf 20 -c:a aac -b:a 160k -vf format=yuv420p -movflags +faststart intro.mp4
ffmpeg version git-2020-08-09-6e951d0 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.1 (GCC) 20200805
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libgsm --enable-librav1e --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 58.100 / 56. 58.100
  libavcodec     58.100.100 / 58.100.100
  libavformat    58. 50.100 / 58. 50.100
  libavdevice    58. 11.101 / 58. 11.101
  libavfilter     7. 87.100 /  7. 87.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Input #0, ipmovie, from 'intro.mve':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: interplayvideo, rgb555le, 640x320, 29.98 fps, 29.98 tbr, 1000k tbn, 1000k tbc
    Stream #0:1: Audio: interplay_dpcm, 44100 Hz, stereo, s16, 705 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (interplayvideo (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (interplay_dpcm (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000015a405112c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000015a405112c0] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0000015a405112c0] 264 - core 161 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=5 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=8 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=10 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=50 rc=crf mbtree=1 crf=20.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'intro.mp4':
  Metadata:
    encoder         : Lavf58.50.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 640x320, q=-1--1, 29.98 fps, 12500 tbn, 29.98 tbc
    Metadata:
      encoder         : Lavc58.100.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 160 kb/s
    Metadata:
      encoder         : Lavc58.100.100 aac
[mp4 @ 0000015a4054c9c0] Starting second pass: moving the moov atom to the beginning of the file
frame= 7709 fps=274 q=-1.0 Lsize=   40087kB time=00:04:17.18 bitrate=1276.9kbits/s speed=9.13x
video:34795kB audio:5027kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.666721%
[libx264 @ 0000015a405112c0] frame I:71    Avg QP:16.12  size: 20194
[libx264 @ 0000015a405112c0] frame P:2885  Avg QP:20.30  size:  7360
[libx264 @ 0000015a405112c0] frame B:4753  Avg QP:23.74  size:  2727
[libx264 @ 0000015a405112c0] consecutive B-frames: 15.5%  5.4%  4.2% 74.9%
[libx264 @ 0000015a405112c0] mb I  I16..4: 23.8% 38.0% 38.3%
[libx264 @ 0000015a405112c0] mb P  I16..4:  1.7%  5.4%  2.0%  P16..4: 23.2% 17.9% 18.7%  0.0%  0.0%    skip:31.1%
[libx264 @ 0000015a405112c0] mb B  I16..4:  0.4%  1.2%  0.4%  B16..8: 28.6% 13.5%  7.2%  direct: 5.6%  skip:43.0%  L0:48.1% L1:44.8% BI: 7.1%
[libx264 @ 0000015a405112c0] 8x8 transform intra:55.4% inter:38.8%
[libx264 @ 0000015a405112c0] direct mvs  spatial:98.2% temporal:1.8%
[libx264 @ 0000015a405112c0] coded y,uvDC,uvAC intra: 51.7% 72.5% 49.7% inter: 15.2% 19.9% 4.7%
[libx264 @ 0000015a405112c0] i16 v,h,dc,p: 38% 28% 24% 10%
[libx264 @ 0000015a405112c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 12% 41%  5%  5%  5%  5%  5%  6%
[libx264 @ 0000015a405112c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 17% 25%  5%  6%  6%  6%  5%  8%
[libx264 @ 0000015a405112c0] i8c dc,h,v,p: 55% 23% 15%  7%
[libx264 @ 0000015a405112c0] Weighted P-Frames: Y:5.6% UV:5.0%
[libx264 @ 0000015a405112c0] ref P L0: 64.8%  9.6% 14.8%  5.8%  4.3%  0.7%  0.0%
[libx264 @ 0000015a405112c0] ref B L0: 82.7% 12.0%  4.2%  1.1%
[libx264 @ 0000015a405112c0] ref B L1: 94.8%  5.2%
[libx264 @ 0000015a405112c0] kb/s:1108.34
[aac @ 0000015a405131c0] Qavg: 327.527

[DanielGibson]

There might be MIT-license-friendly MVE decoding code, if Velyn can dig it up: https://cohost.org/rohit/post/5583600-descent-3-s-source-c#comment-fa05b37b-7995-4735-b9cc-df5e3ee7560d

Apparently it was written when porting FreeSpace2 to Linux (http://svn.icculus.org/freespace2/trunk/src/movie/ also has source, but AFAIK the source in that repo isn't under a proper open source license).
I also added those links to the list in the first post.

[667bdrm]

Implementations from falltergeist (Fallout 2 reimplementation):

https://github.com/falltergeist/falltergeist/tree/develop/src/Format/Acm https://github.com/falltergeist/falltergeist/tree/develop/src/Format/Mve

ACM documentation available on TeamX site: https://teamx.ru/site_arc/docs/acm_format_en.rar

And there are also libfalltergeist library with mve and acm code distributed under the MIT license

[DanielGibson]

And there are also libfalltergeist library with mve and acm code distributed under the MIT license

That's cool!
Added it to the list :)

[DanielGibson]

By the way, a nice side-effect of Descent3 being under GPLv3 now: Basically all the code linked above (except for the one from FreeSpace2) can be integrated without causing licensing problems :)

[icculus]

Some proprietary sound and video libraries from Interplay have been stripped out (the ACM and MVE format).

Is the MVE stuff not lib/win/Mvelibw.lib ?

And also, the .asm code in the repo, ./libmve/mvelibwa.asm is both the source code to that library and contains a copyright from Interplay and says "Confidential" on it.

This was not in the original source dump that Loki was working from, but it was added to Loki's tree later (under the assumption the repo was never going to be public), and I'm wondering if this is how it ended up in the public source tree.

Regardless, unless this is totally cool, I would strongly urge you to remove the .asm code at a minimum, and maybe force-push the repo so it isn't even recoverable from commit history.

But this might be totally cool as-is; I don't know all the details.

[JeodC]

@kevinbentley @midnite8177

[midnite8177]

I have memories of porting mvelibwa.asm to C/C++ for the initial Linux port before we handed it off to Loki. I do agree with Ryan's recommendations.

[JeodC]

Outright removing those files doesn't interfere with building: https://github.com/JeodC/Descent3/actions/runs/8867313091

This can be done immediately, but @kevinbentley may need to be the one to do it as repository owner. I can give it a shot though and see if git complains.

Edit: It worked on the branches, but main and d3-historical are protected so Kevin will need to intervene for it.

[winterheart]

Closing this issue as ACM already implemented, MVE development will be tracked in #253.