Add 'Cham' basic online live Videostream capability

[m-anish]

This PR adds basic live video streaming capability to the IIAB. It installs nginx from source alongside the mod-rtmp module. After this playbook is installed and running, a client can stream video to IIAB (through any rtmp streaming software like OBS - Open Broadcaster Studio), and users will be able to watch the stream at http://box.lan:8081 (which is the port used by nginx).

This playbook should run on all IIAB supported platforms (rpi, amd64-debian/ubuntu). Potential issues may be the names of the packages which are installed as dependencies which may break the compile of nginx. So please test. I have done so on ubuntu 18.04.2

[holta]

@m-anish wants us to consider this for inclusion within IIAB 7.0 during our "feature freeze" call starting in just over 12 h from now: http://minutes.iiab.io

FYI the web site for 'Cham' itself is emerging here: https://github.com/eka-foundation/cham

Refs: #1546, PR #1733

[georgejhunt]

I think we had a feature freeze a few weeks ago. I do not think our target users will have internet or mesh network. It seems to me that the risks of breaking something at this late in the release cycle are greater than the benefits. Once we have branched off 7.0, we can merge Cham, (perhaps after we know it works on rpi) I'd like to find out more about why nginx was installed from source, rather than using a package where someone else has ironed out the incompatibilities. I'm sorry to be a kill-joy. And I'm willing to be convinced otherwise at our meeting in less than 12 hours.

…

On Wed, Jun 19, 2019 at 7:29 PM A Holt ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In roles/cham/defaults/main.yml <#1743 (comment)>: > +# If nec, change them by editing /etc/iiab/local_vars.yml prior to installing! + +nginx_url: http://nginx.org/download/ +nginx_src_file: nginx-1.14.2.tar.gz +nginx_src_dir: nginx +nginx_rtmp_module_url: https://github.com/arut/nginx-rtmp-module/archive/ +nginx_rtmp_module_src_file: master.zip +nginx_rtmp_module_src_dir: nginx-rtmp-module-master +cham_url: https://github.com/eka-foundation/cham/archive/ +cham_src_file: v0.1.tar.gz +streaming_root: /library/streaming +hls_dir: hls +flv_dir: flv +recordings_dir: recordings + +# If true, recordings will also be made available at box.lan/content/recordings ⬇️ Suggested change -# If true, recordings will also be made available at box.lan/content/recordings +# If true, recordings will also be made available at box.lan/local_content/recordings — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#1743?email_source=notifications&email_token=AAOTQHGGFB2GXBHUR2L5QETP3LTQXA5CNFSM4HZGA63KYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB4DBGIY#pullrequestreview-252056355>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOTQHB2LHA6KVUT2YIBFPTP3LTQXANCNFSM4HZGA63A> .

[holta]

As to whenever this is ready to merge, @m-anish has a strong sentence he's provided here (for http://FAQ.IIAB.IO Item 9, IIAB's
showcase table of IIAB Apps) to illustrate Cham's purpose & possibilities:

Cham is a lightweight live video streaming platform with adaptive bitrates for Internet-in-a-Box.

[holta]

@m-anish

1. Can you explain this PR Add 'Cham' basic online live Videostream capability #1743's design for @georgejhunt, so he understands why you're compiling NGINX from source? This approach (can/often) make for incredibly-hard-to-maintain software & forward-going support paths.

   Also/separately: if NGINX were to become a mainline part of IIAB in the distant future, how might that be navigated/integrated? (In resource-tight environments like RPi especially!)

2. What does 'live video streaming' mean here?

   Are you saying that video files cannot be streamed? (i.e. that only live events can be streamed, with a video camera or similar?)

   See the bottom of http://FAQ.IIAB.IO Item 9. So that the explanation (of what Cham is, and what Cham does, with a 'Getting Started' for less technical operators if at all possible?) is crystal clear to as many people as possible, going a bit beyond https://github.com/m-anish/iiab/blob/videostream/roles/cham/README.rst, Thanks!

[holta]

@m-anish writes:

I rebased the video streaming PR as well on current master

if you feel like having a look

to george's question, most of the tutorials we saw for rtmp seemed to want to compile nginx from source and include the rtmp module in

but at a later stage it wont be much of an issue to use the system package

but we have no bandwidth to test that right now

this is something which is simple, small, works, doesn't disturb system files

[m-anish]

To be honest, the system packages for nginx should also probably work, but they bring extra clutter (read system bureaucracy) which starts to make sense when we use nginx more widely. Right now with this PR, nginx just serves this one web app.

And it's pretty impressive in what it does. Give it an rtmp stream, and it will give you an adaptive bitrate output hls stream.

Is it suitable for an rpi. It should work, but I would probably limit the transcoding defined in the conf file to maximum one stream, and I would also explore if the rpi offers any hardware encoding of videos. So, this can work on a pi, with a different nginx.conf file (which doesn't place a lot of load on system resources).

I don't mind bumping this to 7.1 at all! Lets make it more feature ready. Until then, I would love more testing, on more platforms.

Update: I also added support for VAAPI (hardware accelerated encoding) when this runs on platforms where that option is there (like intel kaby-lake based NUC's). This greatly reduces system load while streaming is happening, and now we start to get into territory of asking (okay how many parallel live streams can we do?)

[holta]

Thank you @m-anish

I agree this great work should be bumped to IIAB 7.1

But we can refine that decision during Thursday's community/team call (http://minutes.iiab.io)

[tim-moody]

default vars:

nginx_etc_dir: /etc/nginx
nginx_config_dir: {{ nginx_etc_dir }}/conf.d

[m-anish]

default vars:

nginx_etc_dir: /etc/nginx
nginx_config_dir: {{ nginx_etc_dir }}/conf.d

cool. I'll make the changes. Thanks for the tip :)

declare {{ nginx_etc_dir }}
put it in {{ nginx_etc_dir }}/protocols.d and include {{ nginx_etc_dir }}/protocols.d/*

this should probably happen in the role that installs nginx?

[m-anish]

default vars:

nginx_etc_dir: /etc/nginx
nginx_config_dir: {{ nginx_etc_dir }}/conf.d

cool. I'll make the changes. Thanks for the tip :)

declare {{ nginx_etc_dir }}
put it in {{ nginx_etc_dir }}/protocols.d and include {{ nginx_etc_dir }}/protocols.d/*

this should probably happen in the role that installs nginx?

[tim-moody]

I would use iiab/vars/default_vars.yml for the variables
I would create /etc/nginx/protocols.d in nginx role (or wherever other nginx dirs are created)

[m-anish]

@tim-moody done. please review :)

[m-anish]

these last two reverts were per @jvonau recommendation, to leave protocols.d dir creation in install.yml

[tim-moody]

protocol.d belongs with nginx. what if another app needs it too?

[m-anish]

protocol.d belongs with nginx. what if another app needs it too?

its part of the nginx role. it will get installed as part of a regular iiab install. me and jerry were just discussing where to best put it in the nginx playbook

[holta]

Cham (and this PR) were extensively discussed during today's Internet-in-a-Box (IIAB) call — please see the notes/minutes here: http://minutes.iiab.io

Refs: #2209 #2157 eka-foundation/home#31