apps/streamana
1
0
Fork 0
mirror of https://github.com/davedoesdev/streamana.git synced 2025-12-24 13:28:19 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into publish

Browse Source
This commit is contained in:
David Halls
2021-10-17 10:39:17 +01:00
parent ed05edeb9f 3edfb709da
commit f3ec11ae52
10 changed files with 564 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
README.adoc
View File

@@ -3,7 +3,7 @@
== Description
Streamana is a Web page which streams your camera and microphone to YouTube Live
(or any other HLS receiver). It uses https://github.com/davedoesdev/webm-muxer.js[webm-muxer.js] and
(or any other HLS or DASH receiver). It uses https://github.com/davedoesdev/webm-muxer.js[webm-muxer.js] and
https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js].
== Demo
@@ -15,7 +15,9 @@ Use Chrome 95 or later.
.. Click _CREATE_ and then select _Go Live_ from the drop-down menu.
.. Under _Select stream key_, select _Create new stream key_.
.. Give your key a name.
.. You must select _HLS_ as the streaming protocol.
.. You must select _HLS_ as the streaming protocol. Note: YouTube DASH ingestion is only available
by using the Youtube API. See https://developers.google.com/youtube/v3/live/guides/encoding-with-dash#url-structure[here]
for more details.
.. Click _CREATE_.
.. Make sure the key you created is selected.
.. Click _COPY_ next to _Stream URL_.
@@ -33,8 +35,9 @@ You can also change various options:
** Convert your camera's video to greyscale.
** Lock the camera to portrait mode (where available, e.g. mobile phones).
** Zoom the camera to fill the page.
** Switch between HLS and DASH encoding.
** Select a different version of https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js] to perform
the HLS encoding.
the HLS or DASH encoding.
== Customisation
@@ -46,24 +49,101 @@ so you can change this to add video effects or overlays. The shader already hand
resizing and rotating the video in `main()`. The optional greyscale conversion is in
the `tpix()` function.
The page's functionality is defined in link:site/streamana.js[] and link:site/hls.js[].
The page's functionality is defined in link:site/streamana.js[] and link:site/streamer.js[].
link:site/hls.js[] exports a class, `HLS`, which does the heavy lifting:
link:site/streamer.js[] exports a function, `get_default_config_from_url`, and a class,
`Streamer`, which does the heavy lifting.
You should first call `get_default_config_from_url`. It takes a single argument,
the URL of `ffmpeg-worker-hls.js` or `ffmpeg-worker-dash.js` in https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js].
This allows your application (or the end user if required) to supply its own version,
in accordance with LGPL. It can be a relative path (i.e. just `ffmpeg-worker-hls.js` or
`ffmpeg-worker-dash.js`).
`get_default_config_from_url` determines the streaming protocol (`hls` or `dash`) and returns
the default configuration for the protocol:
```js
{
ffmpeg_lib_url, // the URL you passed to `get_default_config_from_url`
protocol, // `hls` or `dash`
video: { // properies of the video you will be supplying
bitrate: 2500 * 1000,
framerate: 30
},
audio: { // properties of the audio you will be supplying
bitrate: 128 * 1000
},
media_recorder: { // default options for MediaRecorder if it ends up being used
video: {
codec: protocol === 'dash' ? 'vp9' : 'H264', // video codec
},
audio: {
codec: 'opus' // audio codec
},
webm: true, // container format
mp4: false // if true, requires ffmpeg-worker-hls.js or ffmpeg-worker-dash.js
// to be configured with MP4 support (which is not the default)
},
webcodecs: { // default options for WebCodecs if it ends up being used
video: {
// video codec and options
...(protocol === 'dash' ? {
codec: 'vp09.00.10.08.01'
} : {
codec: 'avc1.42E01E' /*'avc1.42001E'*/,
avc: { format: 'annexb' }
})
},
audio: {
codec: 'opus' /*'pcm'*/, // audio codec
},
webm_muxer: { // options for webm-muxer.js
video: {
codec: protocol === 'dash' ? 'V_VP9' : 'V_MPEG4/ISO/AVC'
},
audio: {
codec: 'A_OPUS',
bit_depth: 0 // 32 for pcm */
}
}
},
ffmpeg: { // desired ffmpeg output codecs
// Note: If the encoded stream already uses the desired codec then
// it will pass `copy` instead. For example, if your browser encodes
// your video to H.264 already then `copy` will be used instead of
// `libx264`. This means you can use `ffmpeg-worker-hls.js` or
// `ffmpeg-worker-dash.js` that doesn't contain a H.264 encoder.
video: {
codec: protocol === 'dash' ? 'libvpx-vp9' : 'libx264'
},
audio: {
codec: protocol === 'dash' ? 'libopus' : 'aac'
}
}
};
```
You application can modify the returned configuration before creating a `Streamer` object.
Use the `Streamer` class as follows:
* The constructor takes the following arguments:
** The https://developer.mozilla.org/en-US/docs/Web/API/MediaStream[`MediaStream`]
containing your video and audio tracks. Note that link:site/streamana.js[] supplies
blank video when the camera is hidden and silent audio when the microphone is muted.
** An https://developer.mozilla.org/en-US/docs/Web/API/AudioContext[AudioContext] instance.
This is used to create a persistent audio generator for triggering updates to
avoid browser timer throttling. If you don't already use one in your application,
you can just `new AudioContext()`.
** The ingestion URL.
** The URL of `ffmpeg-worker-hls.js` in https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js].
This allows your application (or the end user if required) to supply its own version,
in accordance with LGPL.
** The desired video frame rate.
** The configuration returned by calling `get_default_config_from_url` (see above),
optionally modified by your application.
** Whether the video is rotated.
* Call the `async start()` function to start streaming.
* Call the `end()` function to stop streaming.
`HLS` extends from https://developer.mozilla.org/en-US/docs/Web/API/EventTarget[`EventTarget`]
`Streamer` extends from https://developer.mozilla.org/en-US/docs/Web/API/EventTarget[`EventTarget`]
and dispatches the following events:
* `start` when streaming has started.
@@ -80,7 +160,7 @@ Note that https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js] is licensed under
Streamana runs it inside a Web Worker and communicates with it via message passing.
The end user can replace the version used by changing the URL in the user interface.
Note also that the https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js] HLS distribution
contains no H.264 or MP4 code. All encoding is done by the browser using
Note also that the https://github.com/davedoesdev/ffmpeg.js[ffmpeg.js] HLS and DASH
distributions contain no H.264 or MP4 code. All encoding is done by the browser using
https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder[`MediaRecorder`] or
https://www.w3.org/TR/webcodecs/[WebCodecs].

2
ffmpeg.js

Submodule ffmpeg.js updated: d23484b1ef...8b9ee57434

1
site/ffmpeg-worker-dash.js Symbolic link
View File

@@ -0,0 +1 @@
../ffmpeg.js/ffmpeg-worker-dash.js

1
site/ffmpeg-worker-dash.wasm Symbolic link
View File

@@ -0,0 +1 @@
../ffmpeg.js/ffmpeg-worker-dash.wasm

20
site/gl-canvas.js
View File

@@ -29,11 +29,12 @@ export class GlCanvas extends Canvas {
}
}), options);
this.update_limiter = new UpdateLimiter();
this.destroyed = false;
}
// Allow rendering loop to be driven externally (e.g. by the audio encoder)
// to avoid requestAnimationFrame (or indeed setInterval) throttling.
onLoop() {
if (this.update_limiter.check()) {
if (this.update_limiter.check() && !this.destroyed) {
const now = Date.now();
this.checkRender();
// Make sure we don't hog the main thread. Software rendering will take
@@ -50,14 +51,25 @@ export class GlCanvas extends Canvas {
}
// Prevent errors after destruction
destroy() {
super.destroy();
this.destroyed = true;
if (this.gl) {
super.destroy();
}
this.uniforms = {
createTexture() {
return {};
},
create() {}
create() {},
update() {}
};
this.textures = {
createOrUpdate() {
return {
then() {}
};
},
values: {}
};
this.textures = {};
this.buffers = {
values: {}
};

32
site/mux-receiver.js
View File

@@ -8,7 +8,7 @@ export class MuxReceiver extends EventTarget {
}, 0);
}
start({ ffmpeg_lib_url, ffmpeg_args, base_url }) {
start({ ffmpeg_lib_url, ffmpeg_args, base_url, protocol }) {
this.worker = new Worker(ffmpeg_lib_url);
this.worker.onerror = this.onerror.bind(this);
this.worker.onmessage = e => {
@@ -18,16 +18,24 @@ export class MuxReceiver extends EventTarget {
this.worker.postMessage({
type: 'run',
arguments: [
'-loglevel', 'debug',
'-seekable', '0',
...ffmpeg_args,
'-f', 'hls', // use hls encoder
'-hls_time', '2', // 2 second HLS chunks
'-hls_segment_type', 'mpegts', // MPEG2-TS muxer
'-hls_list_size', '2', // two chunks in the list at a time
'-hls_flags', 'split_by_time',
'/outbound/output.m3u8' // path to media playlist file in virtual FS,
// must be under /outbound
...(protocol === 'dash' ? [
'-f', 'dash', // use dash encoder
'-seg_duration', '2', // 2 second segments
'-window_size', '2', // two chunks in the list at a time
'-streaming', '1', // fragment data
'-dash_segment_type', 'webm', // container type
'/outbound/output.mpd'
] : [
'-f', 'hls', // use hls encoder
'-hls_time', '2', // 2 second HLS chunks
'-hls_segment_type', 'mpegts', // MPEG2-TS muxer
'-hls_list_size', '2', // two chunks in the list at a time
'-hls_flags', 'split_by_time', // if you don't have < 2s keyframes
'/outbound/output.m3u8' // path to media playlist file in virtual FS,
// must be under /outbound
])
],
MEMFS: [
{ name: 'stream1' },
@@ -48,7 +56,11 @@ export class MuxReceiver extends EventTarget {
case 'start-stream':
this.worker.postMessage({
type: 'base-url',
data: base_url
data: base_url,
protocol,
options: protocol === 'dash' ? {
//method: 'PUT'
} : {}
});
// falls through
case 'sending':

13
site/streamana.css
View File

@@ -32,3 +32,16 @@ canvas.zoom {
.camera-icon.off {
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 16'%3E%3Cpath fill-rule='evenodd' d='M10.961 12.365a1.99 1.99 0 0 0 .522-1.103l3.11 1.382A1 1 0 0 0 16 11.731V4.269a1 1 0 0 0-1.406-.913l-3.111 1.382A2 2 0 0 0 9.5 3H4.272l.714 1H9.5a1 1 0 0 1 1 1v6a1 1 0 0 1-.144.518l.605.847zM1.428 4.18A.999.999 0 0 0 1 5v6a1 1 0 0 0 1 1h5.014l.714 1H2a2 2 0 0 1-2-2V5c0-.675.334-1.272.847-1.634l.58.814zM15 11.73l-3.5-1.555v-4.35L15 4.269v7.462zm-4.407 3.56-10-14 .814-.58 10 14-.814.58z'/%3E%3C/svg%3E") !important;
}
#busy {
position: fixed;
top: 0;
left: 0;
width: 100%;
height: 100%;
z-index: 999;
}
.busy {
cursor: wait;
}

20
site/streamana.html
View File

@@ -1,5 +1,5 @@
<!DOCTYPE html>
<html>
<html class="busy">
<head>
<title>Streamana</title>
<meta charset="utf-8">
@@ -9,7 +9,8 @@
<link href="./streamana.css" rel="stylesheet">
<script type="module" src="./streamana.js"></script>
</head>
<body class="d-flex flex-column vh-100">
<body class="d-flex flex-column vh-100 d-none">
<div id="busy" class="busy"></div>
<nav id="nav" class="navbar navbar-light bg-light flex-grow-0">
<div class="container-fluid">
<div class="row gx-2 gy-2 gy-sm-0 w-100 mx-0 align-items-center">
@@ -65,9 +66,22 @@
<input id="zoom-video" class="form-check-input" type="checkbox">
<label for="zoom-video" class="form-check-label">Minimize vertical bars in local video display</label>
</div>
<div class="pt-4">
<label class="form-label">Ingestion protocol</label>
<div>
<div class="form-check form-check-inline">
<input id="protocol-hls" name="protocol" class="form-check-input" type="radio" value="ffmpeg-worker-hls.js">
<label for="protocol-hls" class="form-check-label">HLS (H.264)</label>
</div>
<div class="form-check form-check-inline">
<input id="protocol-dash" name="protocol" class="form-check-input" type="radio" value="ffmpeg-worker-dash.js">
<label for="protocol-dash" class="form-check-label">DASH (VP9)</label>
</div>
</div>
</div>
<div class="pt-4">
<label for="ffmpeg-lib-url" class="form-label">FFmpeg library URL</label>
<input id="ffmpeg-lib-url" class="form-control" placeholder="ffmpeg-worker-hls.js" type="text">
<input id="ffmpeg-lib-url" class="form-control" type="text">
</div>
</div>
<div id="error-alert" class="alert alert-danger alert-dismissible fade mb-0 flex-grow-0" role="alert">

218
site/streamana.js
View File

@@ -1,20 +1,14 @@
import { GlCanvas } from './gl-canvas.js';
import {
HLS,
video_encoder_codec,
videoBitsPerSecond
} from './hls.js';
get_default_config_from_url,
Streamer
} from './streamer.js';
import shader from './shader.js';
import {
supported_video_configs,
max_video_config,
} from './resolution.js';
const target_frame_rate = 30;
const ingestion_url_el = document.getElementById('ingestion-url');
ingestion_url_el.value = localStorage.getItem('streamana-ingestion-url');
const go_live_el = document.getElementById('go-live');
go_live_el.disabled = false;
go_live_el.addEventListener('click', function () {
@@ -39,25 +33,26 @@ const initial_ffmpeg_lib_url = (localStorage.getItem('streamana-ffmpeg-lib-url')
if (initial_ffmpeg_lib_url) {
ffmpeg_lib_url_el.value = initial_ffmpeg_lib_url;
}
ffmpeg_lib_url_el.addEventListener('input', function () {
localStorage.setItem('streamana-ffmpeg-lib-url', this.value);
ffmpeg_lib_url_el.addEventListener('change', function () {
localStorage.setItem('streamana-ffmpeg-lib-url', this.value.trim());
set_ingestion();
});
const zoom_video_el = document.getElementById('zoom-video');
zoom_video_el.checked = !!localStorage.getItem('streamana-zoom-video');
zoom_video_el.addEventListener('input', function () {
zoom_video_el.addEventListener('change', function () {
localStorage.setItem('streamana-zoom-video', this.checked ? 'true' : '');
});
const lock_portrait_el = document.getElementById('lock-portrait');
lock_portrait_el.checked = !!localStorage.getItem('streamana-lock-portrait');
lock_portrait_el.addEventListener('input', function () {
lock_portrait_el.addEventListener('change', function () {
localStorage.setItem('streamana-lock-portrait', this.checked ? 'true' : '');
});
const greyscale_el = document.getElementById('greyscale');
greyscale_el.checked = !!localStorage.getItem('streamana-greyscale');
greyscale_el.addEventListener('input', function () {
greyscale_el.addEventListener('change', function () {
localStorage.setItem('streamana-greyscale', this.checked ? 'true' : '');
});
@@ -106,37 +101,40 @@ camera_el.addEventListener('click', camera_save);
const camera_swap_el = document.getElementById('camera-swap');
let video_config;
let preferred_resolution = localStorage.getItem('streamana-resolution');
if (preferred_resolution) {
video_config = await max_video_config({
...JSON.parse(preferred_resolution),
codec: video_encoder_codec,
bitrate: videoBitsPerSecond
}, true);
}
if (!video_config) {
video_config = await max_video_config({
width: 1280,
height: 720,
ratio: 16/9,
codec: video_encoder_codec,
bitrate: videoBitsPerSecond
}, true);
}
const ingestion_url_el = document.getElementById('ingestion-url');
const protocol_hls_el = document.getElementById('protocol-hls');
const protocol_dash_el = document.getElementById('protocol-dash');
const resolution_el = document.getElementById('resolution');
let streamer_config;
let video_config;
const video_configs = new Map();
for (let config of (await supported_video_configs({
codec: video_encoder_codec,
bitrate: videoBitsPerSecond
}, true)).filter(c => c.ratio >= 1)) {
const option = document.createElement('option');
option.innerHTML = `${config.width}x${config.height} &mdash; ${config.label}`;
option.selected = config.label === video_config.label;
resolution_el.appendChild(option);
video_configs.set(option.innerText, config);
function set_ingestion_protocol(protocol) {
if (protocol === 'dash') {
protocol_hls_el.checked = false;
protocol_dash_el.checked = true;
ffmpeg_lib_url_el.placeholder = protocol_dash_el.value;
} else {
protocol_hls_el.checked = true;
protocol_dash_el.checked = false;
ffmpeg_lib_url_el.placeholder = protocol_hls_el.value;
}
}
resolution_el.addEventListener('change', function (ev) {
set_ingestion_protocol(localStorage.getItem('streamana-ingestion-protocol'));
protocol_hls_el.addEventListener('change', function () {
ffmpeg_lib_url_el.placeholder = protocol_hls_el.value;
set_ingestion();
});
protocol_dash_el.addEventListener('change', function () {
ffmpeg_lib_url_el.placeholder = protocol_dash_el.value;
set_ingestion();
});
resolution_el.addEventListener('change', function () {
video_config = video_configs.get(this.value);
localStorage.setItem('streamana-resolution', JSON.stringify({
width: video_config.width,
@@ -145,7 +143,85 @@ resolution_el.addEventListener('change', function (ev) {
}));
});
let hls;
const busy_el = document.getElementById('busy');
async function set_ingestion() {
busy_el.classList.remove('d-none');
try {
const ffmpeg_lib_url = ffmpeg_lib_url_el.value.trim() ||
ffmpeg_lib_url_el.placeholder.trim();
const protocol = streamer_config ? streamer_config.protocol : null;
streamer_config = get_default_config_from_url(ffmpeg_lib_url);
set_ingestion_protocol(streamer_config.protocol);
localStorage.setItem('streamana-ingestion-protocol', streamer_config.protocol);
if (ffmpeg_lib_url_el.value.trim()) {
protocol_hls_el.disabled = true;
protocol_dash_el.disabled = true;
} else {
protocol_hls_el.disabled = false;
protocol_dash_el.disabled = false;
}
if (streamer_config.protocol !== protocol) {
ingestion_url_el.value = (localStorage.getItem(
streamer_config.protocol === 'dash' ?
'streamana-dash-ingestion-url' :
'streamana-hls-ingestion-url') || '').trim();
}
video_config = null;
let preferred_resolution = localStorage.getItem('streamana-resolution');
if (preferred_resolution) {
video_config = await max_video_config({
...JSON.parse(preferred_resolution),
...streamer_config.video,
...streamer_config.webcodecs.video
}, true);
}
if (!video_config) {
video_config = await max_video_config({
width: 1280,
height: 720,
ratio: 16/9,
...streamer_config.video,
...streamer_config.webcodecs.video
}, true);
}
const configs = (await supported_video_configs({
...streamer_config.video,
...streamer_config.webcodecs.video
}, true)).filter(c => c.ratio >= 1);
resolution_el.innerHTML = '';
for (let config of configs) {
const option = document.createElement('option');
option.innerHTML = `${config.width}x${config.height} &mdash; ${config.label}`;
option.selected = video_config && (config.label === video_config.label);
resolution_el.appendChild(option);
video_configs.set(option.innerText, config);
}
return streamer_config;
} finally {
busy_el.classList.add('d-none');
}
}
await set_ingestion();
document.body.classList.remove('d-none');
document.documentElement.classList.remove('busy');
let streamer;
// ingestion url doesn't change when protocol changes due to ffmpeg_lib_url
// why does clicking on menu close the menu after change ffmpeg_lib_url?
async function start() {
const ingestion_url = ingestion_url_el.value.trim();
@@ -154,7 +230,10 @@ async function start() {
go_live_el.checked = false;
return;
}
localStorage.setItem('streamana-ingestion-url', ingestion_url);
localStorage.setItem(
streamer_config.protocol === 'dash' ? 'streamana-dash-ingestion-url' :
'streamana-hls-ingestion-url',
ingestion_url);
if (!video_config) {
console.error('No video config');
@@ -172,6 +251,8 @@ async function start() {
lock_portrait_el.disabled = true;
zoom_video_el.disabled = true;
resolution_el.disabled = true;
protocol_hls_el.disabled = true;
protocol_dash_el.disabled = true;
waiting_el.classList.remove('d-none');
mic_el.removeEventListener('click', mic_save);
camera_el.removeEventListener('click', camera_save);
@@ -198,7 +279,7 @@ async function start() {
function cleanup(err) {
if (err) {
console.error(err.toString());
console.error(err);
}
if (done) {
return;
@@ -218,7 +299,7 @@ async function start() {
camera_icon_el.classList.add('off');
}
camera_el.addEventListener('click', camera_save);
greyscale_el.removeEventListener('input', greyscale);
greyscale_el.removeEventListener('change', greyscale);
camera_swap_el.classList.add('d-none');
camera_swap_el.removeEventListener('click', about_face);
canvas_el_parent.classList.add('mx-auto');
@@ -267,8 +348,9 @@ async function start() {
track.stop();
}
}
if (hls) {
hls.end(!!err);
if (streamer) {
streamer.end(!!err);
streamer = null;
}
go_live_el.checked = false;
@@ -279,6 +361,8 @@ async function start() {
lock_portrait_el.disabled = false;
zoom_video_el.disabled = false;
resolution_el.disabled = false;
protocol_hls_el.disabled = ffmpeg_lib_url_el.value.trim();
protocol_dash_el.disabled = ffmpeg_lib_url_el.value.trim();;
waiting_el.classList.add('d-none');
canvas_el.classList.add('d-none');
}
@@ -356,7 +440,7 @@ async function start() {
camera_swap_el.removeEventListener('click', about_face);
async function finish() {
await hls.start();
await streamer.start();
mic_el.addEventListener('click', media_toggle);
camera_el.addEventListener('click', media_toggle);
camera_swap_el.addEventListener('click', about_face);
@@ -375,8 +459,8 @@ async function start() {
width: video_config.width,
height: video_config.height,
frameRate: {
ideal: target_frame_rate,
max: target_frame_rate
ideal: streamer_config.video.framerate,
max: streamer_config.video.framerate
},
facingMode: requested_facing_mode
};
@@ -389,7 +473,7 @@ async function start() {
});
} catch (ex) {
console.warn(`Failed to get user media (need_audio=${need_audio} need_video=${need_video})`);
console.error(ex.toString());
console.error(ex);
if (need_audio && need_video) {
console.warn("Retrying with only video");
try {
@@ -399,7 +483,7 @@ async function start() {
});
} catch (ex) {
console.warn('Failed to get user video, retrying with only audio');
console.error(ex.toString());
console.error(ex);
try {
media_stream = await navigator.mediaDevices.getUserMedia({
audio: true,
@@ -407,7 +491,7 @@ async function start() {
});
} catch (ex) {
console.warn('Failed to get user audio');
console.error(ex.toString());
console.error(ex);
}
}
}
@@ -534,7 +618,7 @@ async function start() {
// tell shader whether to greyscale
gl_canvas.setUniform('u_greyscale', greyscale_el.checked);
greyscale_el.addEventListener('input', greyscale);
greyscale_el.addEventListener('change', greyscale);
// tell shader camera hasn't started
gl_canvas.setUniform('u_active', false);
@@ -572,7 +656,7 @@ async function start() {
// capture video from the canvas
// Note: Safari on iOS doesn't get any data, might be related to
// https://bugs.webkit.org/show_bug.cgi?id=181663
canvas_stream = canvas_el.captureStream(target_frame_rate);
canvas_stream = canvas_el.captureStream(streamer_config.video.framerate);
// add audio to canvas stream
audio_dest = audio_context.createMediaStreamDestination();
@@ -593,11 +677,15 @@ async function start() {
audio_source = silence;
audio_source.connect(audio_dest);
// HLS from the canvas stream to the ingestion URL
hls = new HLS(canvas_stream, audio_context, ingestion_url, ffmpeg_lib_url, target_frame_rate, lock_portrait);
hls.addEventListener('run', () => console.log('HLS running'));
hls.addEventListener('exit', ev => {
const msg = `HLS exited with status ${ev.detail.code}`;
// Stream from the canvas stream to the ingestion URL
streamer = new Streamer(canvas_stream,
audio_context,
ingestion_url,
streamer_config,
lock_portrait);
streamer.addEventListener('run', () => console.log('Streamer running'));
streamer.addEventListener('exit', ev => {
const msg = `Streamer exited with status ${ev.detail.code}`;
if (ev.detail.code === 0) {
console.log(msg);
cleanup();
@@ -605,8 +693,8 @@ async function start() {
cleanup(msg);
}
});
hls.addEventListener('error', cleanup);
hls.addEventListener('start', function () {
streamer.addEventListener('error', cleanup);
streamer.addEventListener('start', function () {
if (done) {
this.end(true);
}
@@ -616,7 +704,7 @@ async function start() {
go_live_el.disabled = false;
update();
});
hls.addEventListener('update', update);
streamer.addEventListener('update', update);
await start_media(facing_mode);
} catch (ex) {
@@ -626,5 +714,5 @@ async function start() {
function stop() {
go_live_el.disabled = true;
hls.end();
streamer.end();
}

312
site/hls.js → site/streamer.js
View File

@@ -1,20 +1,71 @@
import { UpdateLimiter } from './update-limiter.js';
import { MuxReceiver } from './mux-receiver.js';
const audioBitsPerSecond = 128 * 1000;
export const videoBitsPerSecond = 2500 * 1000;
const key_frame_interval = 3;
export const video_encoder_codec = 'avc1.42E01E' /*'avc1.42001E'*/;
export function get_default_config_from_url(ffmpeg_lib_url) {
const protocol = ffmpeg_lib_url.indexOf('worker-dash') >= 0 ? 'dash' : 'hls';
return {
ffmpeg_lib_url,
protocol,
video: {
bitrate: 2500 * 1000,
framerate: 30
},
audio: {
bitrate: 128 * 1000
},
media_recorder: {
video: {
codec: protocol === 'dash' ? 'vp9' : 'H264',
},
audio: {
codec: 'opus'
},
webm: true,
mp4: false // requires ffmpeg-worker-hls.js or ffmpeg-worker-dash.js
// to be configured with MP4 support (which is not the default)
},
webcodecs: {
video: {
...(protocol === 'dash' ? {
codec: 'vp09.00.10.08.01'
} : {
codec: 'avc1.42E01E' /*'avc1.42001E'*/,
avc: { format: 'annexb' }
})
},
audio: {
codec: 'opus' /*'pcm'*/,
},
webm_muxer: {
video: {
codec: protocol === 'dash' ? 'V_VP9' : 'V_MPEG4/ISO/AVC'
},
audio: {
codec: 'A_OPUS',
bit_depth: 0 // 32 for pcm */
}
}
},
ffmpeg: {
video: {
codec: protocol === 'dash' ? 'libvpx-vp9' : 'libx264'
},
audio: {
codec: protocol === 'dash' ? 'libopus' : 'aac'
}
}
};
}
export class HLS extends EventTarget {
constructor(stream, audio_context, base_url, ffmpeg_lib_url, frame_rate, rotate) {
export class Streamer extends EventTarget {
constructor(stream, audio_context, base_url, config, rotate) {
super();
this.stream = stream;
this.audio_context = audio_context;
this.base_url = base_url;
this.ffmpeg_lib_url = ffmpeg_lib_url;
this.frame_rate = frame_rate;
this.config = config;
if (rotate) {
this.ffmpeg_metadata = ['-metadata:s:v:0', 'rotate=-90'];
} else {
@@ -30,25 +81,47 @@ export class HLS extends EventTarget {
return;
}
try {
// first try WebM/H264 MediaRecorder - this should work on Chrome Linux and Windows
await this.media_recorder('video/webm;codecs=H264');
console.log("Using MediaRecorder WebM/h264");
} catch (ex) {
console.warn(ex.toString());
try {
// next try WebCodecs - this should work on Chrome including Android
await this.webcodecs(video_encoder_codec,
'opus' /*'pcm'*/,
{ avc: { format: 'annexb' } });
console.log("Using WebCodecs");
} catch (ex) {
console.warn(ex.toString());
// finally try MP4 - this should work on Safari MacOS and iOS, producing H264
// this assumes ffmpeg-hls.js has been configured with MP4 support
const mrcfg = this.config.media_recorder;
const mp4 = async () => {
if (mfcrg.mp4) {
// try MediaRecorder MP4 - this should work on Safari MacOS and iOS,
// producing H.264 video and AAC audio
await this.media_recorder('video/mp4');
console.log("Using MediaRecorder MP4");
console.log("Using MediaRecorder MP4 (H264,aac)");
} else {
throw new Error('no supported encoding methods');
}
};
const webcodecs = async () => {
const wccfg = this.config.webcodecs;
if (wccfg) {
try {
// try WebCodecs - this should work on Chrome including Android
await this.webcodecs();
console.log("Using WebCodecs");
} catch (ex) {
console.warn(ex);
await mp4();
}
} else {
await mp4();
}
};
if (mrcfg.webm) {
try {
// try MediaRecorder WebM - this should work on Chrome Linux and Windows
const codecs = `${mrcfg.video.codec},${mrcfg.audio.codec}`;
await this.media_recorder(`video/webm;codecs=${codecs}`);
console.log(`Using MediaRecorder WebM (${codecs})`);
} catch (ex) {
console.warn(ex);
await webcodecs();
}
} else {
await webcodecs();
}
this.started = true;
@@ -59,7 +132,7 @@ export class HLS extends EventTarget {
await this.audio_context.audioWorklet.addModule('./dummy-worklet.js');
this.dummy_processor = new AudioWorkletNode(this.audio_context, 'dummy-processor', {
processorOptions: {
update_rate: this.frame_rate
update_rate: this.config.video.framerate
}
});
this.dummy_processor.onerror = onerror;
@@ -73,6 +146,31 @@ export class HLS extends EventTarget {
this.dummy_processor.disconnect();
}
receiver_args(video_codec, audio_codec) {
return {
ffmpeg_lib_url: this.config.ffmpeg_lib_url,
ffmpeg_args: [
//'-loglevel', 'debug',
'-i', '/work/stream1',
'-map', '0:v',
'-map', '0:a',
'-c:v', video_codec === this.config.ffmpeg.video.codec ||
video_codec === 'copy' ?
'copy' : // pass through the video data (no decoding or encoding)
this.config.ffmpeg.video.codec, // re-encode video
'-b:v', this.config.video.bitrate.toString(), // set video bitrate
...this.ffmpeg_metadata,
'-c:a', audio_codec === this.config.ffmpeg.audio.codec ||
audio_codec === 'copy' ?
'copy' : // pass through the audio data
this.config.ffmpeg.audio.codec, // re-encode audio
'-b:a', this.config.audio.bitrate.toString() // set audio bitrate
],
base_url: this.base_url,
protocol: this.config.protocol
};
}
async media_recorder(mimeType) {
const onerror = this.onerror.bind(this);
@@ -80,8 +178,8 @@ export class HLS extends EventTarget {
// note we don't start recording until ffmpeg has started (below)
const recorder = new MediaRecorder(this.stream, {
mimeType,
audioBitsPerSecond,
videoBitsPerSecond
videoBitsPerSecond: this.config.video.bitrate,
audioBitsPerSecond: this.config.audio.bitrate
});
recorder.onerror = onerror;
@@ -101,27 +199,71 @@ export class HLS extends EventTarget {
await this.start_dummy_processor();
let video_codec, audio_codec;
if (recorder.mimeType === 'video/mp4') {
video_codec = 'libx264';
audio_codec = 'aac';
} else {
switch (this.config.media_recorder.video.codec.toLowerCase()) {
case 'av1':
video_codec = 'libaom-av1';
break;
case 'h264':
video_codec = 'libx264';
break;
case 'vp8':
video_codec = 'libvpx';
break;
case 'vp9':
video_codec = 'libvpx-vp9';
break;
default:
video_codec = null;
break;
}
switch (this.config.media_recorder.audio.codec.toLowerCase()) {
case 'flac':
audio_codec = 'flac';
break;
case 'mp3':
audio_codec = 'libmp3lame';
break;
case 'opus':
audio_codec = 'libopus';
break;
case 'vorbis':
audio_codec = 'libvorbis';
break;
case 'pcm':
audio_codec = 'f32le';
break;
default:
if (audio_codes.startsWith('mp4a')) {
audio_codec = 'aac';
} else {
audio_codec = null;
}
break;
}
}
// start the ffmpeg worker
this.receiver = new MuxReceiver();
this.receiver.addEventListener('message', e => {
const msg = e.detail;
switch (msg.type) {
case 'ready':
this.receiver.start({
ffmpeg_lib_url: this.ffmpeg_lib_url,
ffmpeg_args: [
'-i', '/work/stream1',
'-map', '0:v',
'-map', '0:a',
'-c:v', 'copy', // pass through the video data (h264, no decoding or encoding)
...this.ffmpeg_metadata,
...(recorder.mimeType === 'video/mp4' ?
['-c:a', 'copy'] : // assume already AAC
['-c:a', 'aac', // re-encode audio as AAC-LC
'-b:a', audioBitsPerSecond.toString()]) // set audio bitrate
],
base_url: this.base_url
});
this.receiver.start(this.receiver_args(video_codec, audio_codec));
break;
case 'error':
@@ -153,7 +295,7 @@ export class HLS extends EventTarget {
});
}
async webcodecs(video_codec, audio_codec, video_config, audio_config) {
async webcodecs() {
const onerror = this.onerror.bind(this);
const video_track = this.stream.getVideoTracks()[0];
@@ -209,13 +351,11 @@ export class HLS extends EventTarget {
readable: video_readable,
key_frame_interval,
config: {
codec: video_codec,
bitrate: videoBitsPerSecond,
framerate: this.frame_rate,
...this.config.video,
...this.config.webcodecs.video,
latencyMode: 'realtime',
width: video_settings.width,
height: video_settings.height,
...video_config
},
}, [video_readable]);
@@ -224,11 +364,10 @@ export class HLS extends EventTarget {
audio: true,
readable: audio_readable,
config: {
codec: audio_codec,
bitrate: audioBitsPerSecond,
...this.config.audio,
...this.config.webcodecs.audio,
sampleRate: audio_settings.sampleRate,
numberOfChannels: audio_settings.channelCount,
...audio_config
},
}, [audio_readable]);
@@ -257,6 +396,60 @@ export class HLS extends EventTarget {
}
};
let video_codec;
switch (this.config.webcodecs.video.codec) {
case 'V_AV1':
video_codec = 'libaom-av1';
break;
case 'V_MPEG4/ISO/AVC':
video_codec = 'libx264';
break;
case 'V_VP8':
video_codec = 'libvpx';
break;
case 'V_VP9':
video_codec = 'libvpx-vp9';
break;
default:
video_codec = null;
break;
}
let audio_codec;
switch (this.config.webcodecs.audio.codec) {
case 'A_FLAC':
audio_codec = 'flac';
break;
case 'A_MPEG/L3':
audio_codec = 'libmp3lame';
break;
case 'A_OPUS':
audio_codec = 'libopus';
break;
case 'A_VORBIS':
audio_codec = 'libvorbis';
break;
case 'A_PCM/FLOAT/IEEE':
audio_codec = 'f32le';
break;
default:
if (audio_codes.startsWith('A_AAC')) {
audio_codec = 'aac';
} else {
audio_codec = null;
}
break;
}
this.worker.postMessage({
type: 'start',
webm_metadata: {
@@ -264,28 +457,19 @@ export class HLS extends EventTarget {
video: {
width: video_settings.width,
height: video_settings.height,
frame_rate: this.frame_rate,
codec_id: 'V_MPEG4/ISO/AVC'
frame_rate: this.config.video.framerate,
codec_id: this.config.webcodecs.webm_muxer.video.codec
},
audio: {
sample_rate: audio_settings.sampleRate,
channels: audio_settings.channelCount,
codec_id: 'A_OPUS'
bit_depth: this.config.webcodecs.webm_muxer.audio.bit_depth,
codec_id: this.config.webcodecs.webm_muxer.audio.codec
}
},
webm_receiver: './mux-receiver.js',
webm_receiver_data: { name: 'stream1' },
ffmpeg_lib_url: this.ffmpeg_lib_url,
base_url: this.base_url,
ffmpeg_args: [
'-i', '/work/stream1',
'-map', '0:v',
'-map', '0:a',
'-c:v', 'copy', // pass through the video data (h264, no decoding or encoding)
...this.ffmpeg_metadata,
'-c:a', 'aac', // re-encode audio as AAC-LC
'-b:a', audioBitsPerSecond.toString() // set audio bitrate
]
...this.receiver_args(video_codec, audio_codec)
});
}