jcarr
/
noVNC
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #78 from kasmtech/feature/KASM-5078_multi_monitor_display_ui 

KASM-5078 multi monitor display UI
This commit is contained in:
Richard Koliser 2023-12-18 23:40:45 -05:00 committed by GitHub
parent 68135beedd db2addddce
commit 860ead2fac
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
14 changed files with 2149 additions and 387 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -24,7 +24,7 @@ For support with KasmVNC, post on the [KasmVNC Project](https://github.com/kasmt
- Automatic mixing of webp and jpeg based on CPU availability on server - Automatic mixing of webp and jpeg based on CPU availability on server
- WebRTC UDP Transit - WebRTC UDP Transit
- Lossless QOI Image format for Local LAN - Lossless QOI Image format for Local LAN
- [Dynamic jpeg/webp image coompression](https://github.com/kasmtech/KasmVNC/wiki/Video-Rendering-Options#dynamic-image-quality) quality settings based on screen change rates - [Dynamic jpeg/webp image compression](https://github.com/kasmtech/KasmVNC/wiki/Video-Rendering-Options#dynamic-image-quality) quality settings based on screen change rates
- Seemless clipboard support (on Chromium based browsers) - Seemless clipboard support (on Chromium based browsers)
- Binary clipboard support for text, images, and formatted text (on Chromium based browsers) - Binary clipboard support for text, images, and formatted text (on Chromium based browsers)
- Allow client to set/change most configuration settings - Allow client to set/change most configuration settings

1
app/images/desktop-regular.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 576 512"><path fill="#fff" d="M512 48H64c-8.8 0-16 7.2-16 16V256H528V64c0-8.8-7.2-16-16-16zm64 208v48 48c0 35.3-28.7 64-64 64H364.3l8 48H424c13.3 0 24 10.7 24 24s-10.7 24-24 24H352 224 152c-13.3 0-24-10.7-24-24s10.7-24 24-24h51.7l8-48H64c-35.3 0-64-28.7-64-64V304 256 64C0 28.7 28.7 0 64 0H512c35.3 0 64 28.7 64 64V256zM48 304v48c0 8.8 7.2 16 16 16H239.5c.3 0 .6 0 .8 0h95.2c.3 0 .6 0 .8 0H512c8.8 0 16-7.2 16-16V304H48zM252.3 464h71.3l-8-48H260.3l-8 48z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 529 B

204
app/styles/base.css
View File

@ -897,7 +897,8 @@ select:active {
#noVNC_container { #noVNC_container {
width: 100%; width: 100%;
height: 100%; height: 100%;
background-image: url('../images/splash.jpg') background-image: url('../images/splash.jpg');
background-size: cover;
} }
#noVNC_keyboardinput { #noVNC_keyboardinput {
@ -1191,3 +1192,204 @@ a:link {
a:visited { a:visited {
color: white; color: white;
} }
/* ----------------------------------------
* Multi Display Arrangement
* ----------------------------------------
*/
#noVNC_connect_button {
background: rgba(0,0,0,0.7);
padding: 20px;
color: white;
font-size: 12px;
font-weight: 100;
display: flex;
align-items: center;
gap: 20px;
cursor: pointer;
border-radius: 8px;
}
#noVNC_connect_button .image {
color:#697ad6;
transition: all 0.3s;
position: relative;
overflow: hidden;
}
#noVNC_connect_button .text {
color: #ffffff91
}
#noVNC_connect_button .heading {
color: #fff;
font-size: 13px;
margin-bottom: 3px;
}
#noVNC_connect_button:hover .image {
color:#298453;
}
#noVNC_connect_button:hover .go {
transition: all 0.3s;
transform: translateX(5px);
}
#noVNC_connect_button .power {
transition: all 0.3s;
color: white!important;
transform: translateY(50px);
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
display: flex;
justify-content: center;
align-items: center;
}
#noVNC_connect_button:hover .power {
transform: translateY(0);
}
#noVNC_connect_button svg {
fill: currentColor;
}
#noVNC_identify_monitor {
position: fixed;
z-index: 999999;
background: rgba(0,0,0,0.7);
color: white;
font-size: 30px;
top: 0;
right: 0;
width: 100px;
height: 100px;
display: none;
justify-content: center;
align-items: center;
border-bottom-left-radius: 10px;
}
#noVNC_identify_monitor.show {
display: flex;
}
.flex {
display: flex;
}
.flex-col {
flex-direction: column;
}
.canvas {
background: rgb(225,225,226);
background: linear-gradient(0deg, rgba(225,225,226,1) 0%, rgba(237,237,238,1) 100%);
}
.row {
background: #e9e9ea;
border-radius: 8px;
border: 1px solid #dcdcdd;
padding: 6px 15px;
display: flex;
justify-content: space-between;
align-items: center;
}
.row input {
border: 1px solid #dcdcdd;
padding: 8px 2px;
text-align: right;
background-color: #ededee;
}
#noVNC_displays {
position: fixed;
display: none;
top: 0;
left: 0;
width: 100%;
height: 100%;
z-index: 100;
background: rgba(0,0,0,0.7);
cursor: pointer;
transition: 0.5s ease-in-out;
padding: 5px;
flex-direction: row;
justify-content: center;
align-items: center;
line-height: 25px;
word-wrap: break-word;
}
#noVNC_displays.noVNC_open {
display: flex;
transform: translateY(0);
visibility: visible;
opacity: 1;
}
#noVNC_displays .canvas {
display: flex;
width: 700px;
flex-direction: column;
border-radius: 15px;
background: #fdfdfd;
padding: 15px 20px;
box-shadow: 0 0 15px rgba(0,0,0,0.4);
position: relative;
}
#noVNC_displays .canvas-title {
font-size: 14px;
font-weight: bold;
padding: 0 10px;
line-height: 1;
line-height: 1.4;
}
#noVNC_displays .canvas-text {
font-size: 12px;
padding: 0 35px 0 10px;
opacity: 0.6;
line-height: 1.4;
margin-bottom: 15px;
}
#noVNC_displays .arrange-buttons {
margin-top: 15px;
display: flex;
justify-content: space-between;
}
#noVNC_displays .arrange-buttons button, #noVNC_displays .arrange-buttons .button {
border: none;
display: flex;
align-items: center;
padding: 4px 7px;
border-radius: 5px;
cursor: pointer;
background-color: #f1f1f1;
font-size: 13px;
border: 1px solid #e5e5e5;
line-height: 1;
}
#noVNC_setting_enable_hidpi_option {
display: none!important;
}
#noVNC_setting_enable_hidpi_option.show {
display: flex!important;
}
#noVNC_refreshMonitors {
position: absolute;
top: 20px;
right: 25px;
}
#noVNC_refreshMonitors_icon {
transition: all 0.3s;
transform-origin: center;
}
#noVNC_addMonitor {
background-color: #2196F3!important;
color: white;
border: none!important;
}
#noVNC_addMonitor svg {
margin-right: 5px;
}
#noVNC_displays .canvas canvas {
background: #f7f7f7;
border: 1px solid #ececec;
border-radius: 8px;
}

497
app/ui.js
View File

@ -13,13 +13,7 @@ window.addEventListener("load", function() {
loader.src = "vendor/browser-es-module-loader/dist/browser-es-module-loader.js"; loader.src = "vendor/browser-es-module-loader/dist/browser-es-module-loader.js";
document.head.appendChild(loader); document.head.appendChild(loader);
}); });
window.addEventListener("load", function() {
var connect_btn_el = document.getElementById("noVNC_connect_button");
if (typeof(connect_btn_el) != 'undefined' && connect_btn_el != null)
{
connect_btn_el.click();
}
});
window.updateSetting = (name, value) => { window.updateSetting = (name, value) => {
WebUtil.writeSetting(name, value); WebUtil.writeSetting(name, value);
@ -31,8 +25,8 @@ window.updateSetting = (name, value) => {
} }
} }
import "core-js/stable"; //import "core-js/stable";
import "regenerator-runtime/runtime"; //import "regenerator-runtime/runtime";
import * as Log from '../core/util/logging.js'; import * as Log from '../core/util/logging.js';
import _, { l10n } from './localization.js'; import _, { l10n } from './localization.js';
import { isTouchDevice, isSafari, hasScrollbarGutter, dragThreshold, supportsBinaryClipboard, isFirefox, isWindows, isIOS, supportsPointerLock } import { isTouchDevice, isSafari, hasScrollbarGutter, dragThreshold, supportsBinaryClipboard, isFirefox, isWindows, isIOS, supportsPointerLock }
@ -68,6 +62,12 @@ const UI = {
inhibitReconnect: true, inhibitReconnect: true,
reconnectCallback: null, reconnectCallback: null,
reconnectPassword: null, reconnectPassword: null,
monitors: [],
sortedMonitors: [],
selectedMonitor: null,
refreshRotation: 0,
supportsBroadcastChannel: (typeof BroadcastChannel !== "undefined"),
prime() { prime() {
return WebUtil.initSettings().then(() => { return WebUtil.initSettings().then(() => {
@ -131,11 +131,12 @@ const UI = {
UI.addConnectionControlHandlers(); UI.addConnectionControlHandlers();
UI.addClipboardHandlers(); UI.addClipboardHandlers();
UI.addSettingsHandlers(); UI.addSettingsHandlers();
UI.addDisplaysHandler();
// UI.addMultiMonitorAddHandler();
document.getElementById("noVNC_status") document.getElementById("noVNC_status")
.addEventListener('click', UI.hideStatus); .addEventListener('click', UI.hideStatus);
UI.openControlbar(); UI.openControlbar();
//
UI.updateVisualState('init'); UI.updateVisualState('init');
@ -166,7 +167,7 @@ const UI = {
} }
}); });
window.addEventListener("beforeunload", (e) => { window.addEventListener("unload", (e) => {
if (UI.rfb) { if (UI.rfb) {
UI.disconnect(); UI.disconnect();
} }
@ -402,6 +403,17 @@ const UI = {
} }
}, },
addConnectionControlHandlers() {
UI.addClickHandle('noVNC_disconnect_button', UI.disconnect);
var connect_btn_el = document.getElementById("noVNC_connect_button_2");
if (typeof(connect_btn_el) != 'undefined' && connect_btn_el != null)
{
connect_btn_el.addEventListener('click', UI.connect);
}
},
addTouchSpecificHandlers() { addTouchSpecificHandlers() {
document.getElementById("noVNC_keyboard_button") document.getElementById("noVNC_keyboard_button")
.addEventListener('click', UI.toggleVirtualKeyboard); .addEventListener('click', UI.toggleVirtualKeyboard);
@ -472,21 +484,6 @@ const UI = {
.addEventListener('click', () => UI.rfb.machineReset()); .addEventListener('click', () => UI.rfb.machineReset());
}, },
addConnectionControlHandlers() {
UI.addClickHandle('noVNC_disconnect_button', UI.disconnect);
var connect_btn_el = document.getElementById("noVNC_connect_button");
if (typeof(connect_btn_el) != 'undefined' && connect_btn_el != null)
{
connect_btn_el.addEventListener('click', UI.connect);
}
document.getElementById("noVNC_cancel_reconnect_button")
.addEventListener('click', UI.cancelReconnect);
document.getElementById("noVNC_credentials_button")
.addEventListener('click', UI.setCredentials);
},
addClipboardHandlers() { addClipboardHandlers() {
UI.addClickHandle('noVNC_clipboard_button', UI.toggleClipboardPanel); UI.addClickHandle('noVNC_clipboard_button', UI.toggleClipboardPanel);
@ -588,6 +585,24 @@ const UI = {
window.addEventListener('msfullscreenchange', UI.updateFullscreenButton); window.addEventListener('msfullscreenchange', UI.updateFullscreenButton);
}, },
addDisplaysHandler() {
if (UI.supportsBroadcastChannel) {
UI.showControlInput("noVNC_displays_button");
UI.addClickHandle('noVNC_displays_button', UI.openDisplays);
UI.addClickHandle('noVNC_close_displays', UI.closeDisplays);
UI.addClickHandle('noVNC_identify_monitors_button', UI._identify);
UI.addClickHandle('noVNC_addMonitor', UI.addSecondaryMonitor);
UI.addClickHandle('noVNC_refreshMonitors', UI.displaysRefresh);
}
},
/*addMultiMonitorAddHandler() {
if (UI.supportsBroadcastChannel) {
UI.addClickHandle('noVNC_addmonitor_button', UI.addSecondaryMonitor);
}
},*/
/* ------^------- /* ------^-------
* /EVENT HANDLERS * /EVENT HANDLERS
* ============== * ==============
@ -676,8 +691,6 @@ const UI = {
// State change closes dialogs as they may not be relevant // State change closes dialogs as they may not be relevant
// anymore // anymore
UI.closeAllPanels(); UI.closeAllPanels();
document.getElementById('noVNC_credentials_dlg')
.classList.remove('noVNC_open');
}, },
showStats() { showStats() {
@ -1380,9 +1393,12 @@ const UI = {
UI.rfb = new RFB(document.getElementById('noVNC_container'), UI.rfb = new RFB(document.getElementById('noVNC_container'),
document.getElementById('noVNC_keyboardinput'), document.getElementById('noVNC_keyboardinput'),
url, url,
{ shared: UI.getSetting('shared'), {
repeaterID: UI.getSetting('repeaterID'), shared: UI.getSetting('shared'),
credentials: { password: password } }); repeaterID: UI.getSetting('repeaterID'),
credentials: { password: password }
},
true );
UI.rfb.addEventListener("connect", UI.connectFinished); UI.rfb.addEventListener("connect", UI.connectFinished);
UI.rfb.addEventListener("disconnect", UI.disconnectFinished); UI.rfb.addEventListener("disconnect", UI.disconnectFinished);
UI.rfb.addEventListener("credentialsrequired", UI.credentials); UI.rfb.addEventListener("credentialsrequired", UI.credentials);
@ -1394,6 +1410,7 @@ const UI = {
UI.rfb.addEventListener("desktopname", UI.updateDesktopName); UI.rfb.addEventListener("desktopname", UI.updateDesktopName);
UI.rfb.addEventListener("inputlock", UI.inputLockChanged); UI.rfb.addEventListener("inputlock", UI.inputLockChanged);
UI.rfb.addEventListener("inputlockerror", UI.inputLockError); UI.rfb.addEventListener("inputlockerror", UI.inputLockError);
UI.rfb.addEventListener("screenregistered", UI.screenRegistered);
UI.rfb.translateShortcuts = UI.getSetting('translate_shortcuts'); UI.rfb.translateShortcuts = UI.getSetting('translate_shortcuts');
UI.rfb.clipViewport = UI.getSetting('view_clip'); UI.rfb.clipViewport = UI.getSetting('view_clip');
UI.rfb.scaleViewport = UI.getSetting('resize') === 'scale'; UI.rfb.scaleViewport = UI.getSetting('resize') === 'scale';
@ -1427,7 +1444,7 @@ const UI = {
UI.rfb.mouseButtonMapper = UI.initMouseButtonMapper(); UI.rfb.mouseButtonMapper = UI.initMouseButtonMapper();
if (UI.rfb.videoQuality === 5) { if (UI.rfb.videoQuality === 5) {
UI.rfb.enableQOI = true; UI.rfb.enableQOI = true;
} }
//Only explicitly request permission to clipboard on browsers that support binary clipboard access //Only explicitly request permission to clipboard on browsers that support binary clipboard access
if (supportsBinaryClipboard()) { if (supportsBinaryClipboard()) {
@ -1681,6 +1698,14 @@ const UI = {
UI.toggleIMEMode(); UI.toggleIMEMode();
} }
break; break;
case 'open_displays_mode':
if (UI.rfb) {
UI.openDisplays()
}
break;
case 'close_displays_mode':
UI.closeDisplays()
break;
case 'enable_webrtc': case 'enable_webrtc':
if (!UI.getSetting('enable_webrtc')) { if (!UI.getSetting('enable_webrtc')) {
UI.forceSetting('enable_webrtc', true, false); UI.forceSetting('enable_webrtc', true, false);
@ -1722,6 +1747,10 @@ const UI = {
UI.forceSetting('enable_hidpi', event.data.value, false); UI.forceSetting('enable_hidpi', event.data.value, false);
UI.enableHiDpi(); UI.enableHiDpi();
break; break;
case 'control_displays':
parent.postMessage({ action: 'can_control_displays', value: true}, '*' );
break;
} }
} }
}, },
@ -1746,57 +1775,6 @@ const UI = {
parent.postMessage({ action: 'clipboardrx', value: event.detail.text}, '*' ); //TODO fix star parent.postMessage({ action: 'clipboardrx', value: event.detail.text}, '*' ); //TODO fix star
}, },
/* ------^-------
* /CONNECTION
* ==============
* PASSWORD
* ------v------*/
credentials(e) {
// FIXME: handle more types
document.getElementById("noVNC_username_block").classList.remove("noVNC_hidden");
document.getElementById("noVNC_password_block").classList.remove("noVNC_hidden");
let inputFocus = "none";
if (e.detail.types.indexOf("username") === -1) {
document.getElementById("noVNC_username_block").classList.add("noVNC_hidden");
} else {
inputFocus = inputFocus === "none" ? "noVNC_username_input" : inputFocus;
}
if (e.detail.types.indexOf("password") === -1) {
document.getElementById("noVNC_password_block").classList.add("noVNC_hidden");
} else {
inputFocus = inputFocus === "none" ? "noVNC_password_input" : inputFocus;
}
document.getElementById('noVNC_credentials_dlg')
.classList.add('noVNC_open');
setTimeout(() => document
.getElementById(inputFocus).focus(), 100);
Log.Warn("Server asked for credentials");
UI.showStatus(_("Credentials are required"), "warning");
},
setCredentials(e) {
// Prevent actually submitting the form
e.preventDefault();
let inputElemUsername = document.getElementById('noVNC_username_input');
const username = inputElemUsername.value;
let inputElemPassword = document.getElementById('noVNC_password_input');
const password = inputElemPassword.value;
// Clear the input after reading the password
inputElemPassword.value = "";
UI.rfb.sendCredentials({ username: username, password: password });
UI.reconnectPassword = password;
document.getElementById('noVNC_credentials_dlg')
.classList.remove('noVNC_open');
},
/* ------^------- /* ------^-------
* /PASSWORD * /PASSWORD
* ============== * ==============
@ -1867,6 +1845,345 @@ const UI = {
UI.rfb.enableHiDpi = UI.getSetting('enable_hidpi'); UI.rfb.enableHiDpi = UI.getSetting('enable_hidpi');
}, },
/* ------^-------
* /MULTI-MONITOR SUPPORT
* ==============*/
_identify(e) {
UI.identify()
UI.rfb.identify(UI.monitors)
},
identify(data) {
document.getElementById('noVNC_identify_monitor').innerHTML = '1'
document.getElementById('noVNC_identify_monitor').classList.add("show")
setTimeout(() => {
document.getElementById('noVNC_identify_monitor').classList.remove("show")
}, 3500)
},
openDisplays() {
document.getElementById('noVNC_displays').classList.add("noVNC_open");
if (UI.monitors.length < 1 ) {
let screenPlan = UI.rfb.getScreenPlan();
UI.initMonitors(screenPlan)
}
UI.displayMonitors()
},
closeDisplays() {
document.getElementById('noVNC_displays').classList.remove("noVNC_open");
},
displaysRefresh() {
const rotation = UI.refreshRotation + 180;
let screenPlan = UI.rfb.getScreenPlan();
document.getElementById('noVNC_refreshMonitors_icon').style.transform = "rotate(" + rotation + "deg)"
UI.refreshRotation = rotation
UI.updateMonitors(screenPlan)
UI.recenter()
UI.draw()
},
addSecondaryMonitor() {
let new_display_path = window.location.pathname.replace(/[^/]*$/, '')
let new_display_url = `${window.location.protocol}//${window.location.host}${new_display_path}screen.html`;
Log.Debug(`Opening a secondary display ${new_display_url}`)
window.open(new_display_url);
},
initMonitors(screenPlan) {
const { scale } = UI.multiMonitorSettings()
let monitors = []
let showNativeResolution = false
let num = 1
screenPlan.screens.forEach(screen => {
if (parseFloat(screen.pixelRatio) != 1) {
showNativeResolution = true
}
monitors.push({
id: screen.screenID,
x: screen.x / scale,
y: screen.y / scale,
w: screen.serverWidth / scale,
h: screen.serverHeight / scale,
pixelRatio: screen.pixelRatio,
scale: 1,
fill: '#eeeeeecc',
isDragging: false,
num
})
num++
})
if (showNativeResolution) {
document.getElementById('noVNC_setting_enable_hidpi_option').classList.add("show");
} else {
document.getElementById('noVNC_setting_enable_hidpi_option').classList.remove("show");
}
UI.monitors = monitors
let deepCopyMonitors = JSON.parse(JSON.stringify(monitors))
UI.sortedMonitors = deepCopyMonitors.sort((a, b) => {
if (a.y >= b.y + (b.h / 2)) {
return 1
}
return a.x - b.x
})
},
updateMonitors(screenPlan) {
UI.initMonitors(screenPlan)
UI.recenter()
UI.draw()
},
multiMonitorSettings() {
const canvas = document.getElementById("noVNC_multiMonitorWidget")
return {
canvas,
ctx: canvas.getContext("2d"),
bb: canvas.getBoundingClientRect(),
scale: 12,
canvasWidth: 700,
canvasHeight: 230,
}
},
recenter() {
const monitors = UI.sortedMonitors
UI.removeSpaces()
const { startLeft, startTop } = UI.getSizes(monitors)
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
m.x += startLeft
m.y += startTop
}
UI.setScreenPlan()
},
removeSpaces() {
const monitors = UI.sortedMonitors
let prev = monitors[0]
if (monitors.length > 1) {
for (var i = 1; i < monitors.length; i++) {
var a = monitors[i];
let prevStart = prev.x + prev.w
let prevStartTop = prev.y + prev.h
if (a.x > prevStart) {
a.x = prevStart
}
if (a.x < prevStart) {
if (a.y < prevStartTop) {
a.x = prevStart
}
}
if (a.y > prevStartTop) {
if (a.x <= prevStart) {
a.y = prevStartTop
}
}
prev = monitors[i]
}
}
},
rect(ctx, x, y, w, h) {
ctx.beginPath();
ctx.roundRect(x, y, w, h, 5);
ctx.stroke();
ctx.closePath();
ctx.fill();
},
draw() {
const { ctx, canvasWidth, canvasHeight, scale } = UI.multiMonitorSettings()
const monitors = UI.sortedMonitors
ctx.clearRect(0, 0, canvasWidth, canvasHeight);
ctx.rect(0, 0, canvasWidth, canvasHeight);
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
ctx.fillStyle = m.fill;
ctx.lineWidth = 1;
ctx.lineJoin = "round";
ctx.strokeStyle = m === UI.selectedMonitor ? "#2196F3" : "#aaa";
UI.rect(ctx, m.x, m.y, (m.w / m.scale), (m.h / m.scale));
ctx.font = "13px sans-serif";
ctx.textAlign = "right";
ctx.textBaseline = "top";
ctx.fillStyle = "#000";
ctx.fillText((m.num), (m.x + m.w) - 4, m.y + 4);
ctx.font = "200 11px sans-serif";
ctx.textAlign = "center";
ctx.textBaseline = "middle";
ctx.fillText(m.w * scale + ' x ' + m.h * scale, m.x + (m.w / 2), m.y + (m.h / 2));
}
},
getSizes(monitors) {
const { canvasWidth, canvasHeight } = UI.multiMonitorSettings()
let top = monitors[0].y
let left = monitors[0].x
let width = monitors[0].w
let height = monitors[0].h
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
if (m.x < left) {
left = m.x
}
if (m.y < top) {
top = m.y
}
if(m.x + m.w > width) {
width = m.x + m.w
}
if(m.y + m.h > height) {
height = m.y + m.h
}
}
const startLeft = ((canvasWidth - width - left) / 2);
const startTop = ((canvasHeight - height - top) / 2);
return { top, left, width, height, startLeft, startTop }
},
setScreenPlan() {
let monitors = UI.monitors
let sortedMonitors = UI.sortedMonitors
const { scale } = UI.multiMonitorSettings()
const { top, left, width, height } = UI.getSizes(sortedMonitors)
const screens = []
for (var i = 0; i < monitors.length; i++) {
var monitor = monitors[i];
var a = sortedMonitors.find(el => el.id === monitor.id)
console.log(a)
screens.push({
screenID: a.id,
serverHeight: Math.round(a.h * scale),
serverWidth: Math.round(a.w * scale),
x: Math.round((a.x - left) * scale),
y: Math.round((a.y - top) * scale)
})
}
const screenPlan = {
serverHeight: Math.round(height * scale),
serverWidth: Math.round(width * scale),
screens
}
console.log('setScreenPlan')
console.log(screenPlan)
UI.rfb.applyScreenPlan(screenPlan);
},
displayMonitors() {
// const monitors = UI.sortedMonitors
let monitors = UI.sortedMonitors
const { canvas, ctx, bb, canvasWidth, canvasHeight, scale } = UI.multiMonitorSettings()
const { startLeft, startTop } = UI.getSizes(monitors)
let offsetX
let offsetY
let dragok = false
let startX;
let startY;
offsetX = bb.left
offsetY = bb.top
canvas.addEventListener("mousedown", myDown, false);
canvas.addEventListener("mouseup", myUp, false);
canvas.addEventListener("mousemove", myMove, false);
UI.recenter()
UI.draw()
function myDown(e) {
let monitors = UI.sortedMonitors
e.preventDefault();
e.stopPropagation();
let mx = parseInt(e.clientX - offsetX);
let my = parseInt(e.clientY - offsetY);
for (var i = 0; i < monitors.length; i++) {
var mon = monitors[i];
var monw = mon.w / mon.scale
var monh = mon.h / mon.scale
let monx = mon.x
let mony = mon.y
// Find the closest rect to drag
if (mx > monx && mx < (monx + monw) && my > mony && my < (mony + monh)) {
dragok = true;
mon.isDragging = true;
UI.selectedMonitor = mon
break // get out of the loop rather than dragging multiple
}
}
startX = mx;
startY = my;
UI.draw()
}
function myUp(e) {
let monitors = UI.sortedMonitors
e.preventDefault();
e.stopPropagation();
// clear all the dragging flags
dragok = false;
for (var i = 0; i < monitors.length; i++) {
monitors[i].isDragging = false;
}
monitors.sort((a, b) => {
if (a.y >= b.y + (b.h / 2)) {
return 1
}
return a.x - b.x
})
UI.recenter()
UI.draw()
}
function myMove(e) {
let monitors = UI.sortedMonitors
if (dragok) {
e.preventDefault();
e.stopPropagation();
// get the current mouse position
var mx = parseInt(e.clientX - offsetX);
var my = parseInt(e.clientY - offsetY);
// calculate the distance the mouse has moved
// since the last mousemove
var dx = mx - startX;
var dy = my - startY;
// move each rect that isDragging
// by the distance the mouse has moved
// since the last mousemove
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
if (m.isDragging) {
m.x += dx;
m.y += dy;
}
}
// redraw the scene with the new rect positions
UI.draw();
// reset the starting mouse position for the next mousemove
startX = mx;
startY = my;
}
}
},
/* ------^------- /* ------^-------
* /RESIZE * /RESIZE
* ============== * ==============
@ -2514,6 +2831,20 @@ const UI = {
} }
}, },
screenRegistered(e) {
console.log('screen registered')
// Get the current screen plan
// When a new display is added, it is defaulted to be placed to the far right relative to existing displays and to the top
if (UI.rfb) {
let screenPlan = UI.rfb.getScreenPlan();
console.log(screenPlan)
UI.updateMonitors(screenPlan)
UI._identify(UI.monitors)
}
},
//Helper to add options to dropdown. //Helper to add options to dropdown.
addOption(selectbox, text, value) { addOption(selectbox, text, value) {
const optn = document.createElement("OPTION"); const optn = document.createElement("OPTION");

333
app/ui_screen.js Normal file
View File

@ -0,0 +1,333 @@
import RFB from "../core/rfb.js";
import * as WebUtil from "./webutil.js";
import { isTouchDevice, isSafari, hasScrollbarGutter, dragThreshold, supportsBinaryClipboard, isFirefox, isWindows, isIOS, supportsPointerLock }
from '../core/util/browser.js';
import { MouseButtonMapper, XVNC_BUTTONS } from "../core/mousebuttonmapper.js";
import * as Log from '../core/util/logging.js';
const UI = {
connected: false,
screenID: null,
screen: {},
screens: [],
supportsBroadcastChannel: (typeof BroadcastChannel !== "undefined"),
controlChannel: null,
//Initial Loading of the UI
prime() {
console.log('prime')
this.start();
},
//Render default UI
start() {
console.log('start')
window.addEventListener("unload", (e) => {
if (UI.rfb) {
UI.disconnect();
}
});
// Settings with immediate effects
UI.initSetting('logging', 'warn');
UI.updateLogging();
UI.addDefaultHandlers();
UI.updateVisualState('disconnected');
},
addDefaultHandlers() {
document.getElementById('noVNC_connect_button').addEventListener('click', UI.connect);
},
getSetting(name, isBool, default_value) {
let val = WebUtil.readSetting(name);
if ((val === 'undefined' || val === null) && default_value !== 'undefined' && default_value !== null) {
val = default_value;
}
if (typeof val !== 'undefined' && val !== null && isBool) {
if (val.toString().toLowerCase() in {'0': 1, 'no': 1, 'false': 1}) {
val = false;
} else {
val = true;
}
}
return val;
},
connect() {
console.log('connect')
UI.rfb = new RFB(document.getElementById('noVNC_container'),
document.getElementById('noVNC_keyboardinput'),
"", //URL
{
shared: UI.getSetting('shared', true),
repeaterID: UI.getSetting('repeaterID', false),
credentials: { password: null }
},
false // Not a primary display
);
UI.rfb.addEventListener("connect", UI.connectFinished);
//UI.rfb.addEventListener("disconnect", UI.disconnectFinished);
UI.rfb.clipViewport = UI.getSetting('view_clip');
UI.rfb.scaleViewport = UI.getSetting('resize', false, 'remote') === 'scale';
UI.rfb.resizeSession = UI.getSetting('resize', false, 'remote') === 'remote';
UI.rfb.qualityLevel = parseInt(UI.getSetting('quality'));
UI.rfb.dynamicQualityMin = parseInt(UI.getSetting('dynamic_quality_min'));
UI.rfb.dynamicQualityMax = parseInt(UI.getSetting('dynamic_quality_max'));
UI.rfb.jpegVideoQuality = parseInt(UI.getSetting('jpeg_video_quality'));
UI.rfb.webpVideoQuality = parseInt(UI.getSetting('webp_video_quality'));
UI.rfb.videoArea = parseInt(UI.getSetting('video_area'));
UI.rfb.videoTime = parseInt(UI.getSetting('video_time'));
UI.rfb.videoOutTime = parseInt(UI.getSetting('video_out_time'));
UI.rfb.videoScaling = parseInt(UI.getSetting('video_scaling'));
UI.rfb.treatLossless = parseInt(UI.getSetting('treat_lossless'));
UI.rfb.maxVideoResolutionX = parseInt(UI.getSetting('max_video_resolution_x'));
UI.rfb.maxVideoResolutionY = parseInt(UI.getSetting('max_video_resolution_y'));
UI.rfb.frameRate = parseInt(UI.getSetting('framerate'));
UI.rfb.compressionLevel = parseInt(UI.getSetting('compression'));
UI.rfb.showDotCursor = UI.getSetting('show_dot', true);
UI.rfb.idleDisconnect = UI.getSetting('idle_disconnect');
UI.rfb.pointerRelative = UI.getSetting('pointer_relative');
UI.rfb.videoQuality = parseInt(UI.getSetting('video_quality'));
UI.rfb.antiAliasing = UI.getSetting('anti_aliasing');
UI.rfb.clipboardUp = UI.getSetting('clipboard_up', true, true);
UI.rfb.clipboardDown = UI.getSetting('clipboard_down', true, true);
UI.rfb.clipboardSeamless = UI.getSetting('clipboard_seamless', true, true);
UI.rfb.keyboard.enableIME = UI.getSetting('enable_ime', true, false);
UI.rfb.clipboardBinary = supportsBinaryClipboard() && UI.rfb.clipboardSeamless;
UI.rfb.enableWebRTC = UI.getSetting('enable_webrtc', true, false);
UI.rfb.enableHiDpi = UI.getSetting('enable_hidpi', true, false);
UI.rfb.mouseButtonMapper = UI.initMouseButtonMapper();
if (UI.rfb.videoQuality === 5) {
UI.rfb.enableQOI = true;
}
if (UI.supportsBroadcastChannel) {
console.log('add event listener')
UI.controlChannel = new BroadcastChannel("registrationChannel");
UI.controlChannel.addEventListener('message', UI.handleControlMessage)
}
//attach this secondary display to the primary display
if (UI.screenID === null) {
const screen = UI.rfb.attachSecondaryDisplay();
UI.screenID = screen.screenID
UI.screen = screen
} else {
console.log('else reattach screens')
console.log(UI.screen)
UI.rfb.reattachSecondaryDisplay(UI.screen);
}
document.querySelector('title').textContent = 'Display ' + UI.screenID
if (supportsBinaryClipboard()) {
// explicitly request permission to the clipboard
navigator.permissions.query({ name: "clipboard-read" }).then((result) => { Log.Debug('binary clipboard enabled') });
}
},
handleControlMessage(event) {
switch (event.data.eventType) {
case 'identify':
UI.identify(event.data)
break;
case 'secondarydisconnected':
UI.updateVisualState('disconnected');
break;
}
},
updateVisualState(state) {
document.documentElement.classList.remove("noVNC_connecting");
document.documentElement.classList.remove("noVNC_connected");
document.documentElement.classList.remove("noVNC_disconnecting");
document.documentElement.classList.remove("noVNC_reconnecting");
document.documentElement.classList.remove("noVNC_disconnected");
const transitionElem = document.getElementById("noVNC_transition_text");
if (WebUtil.isInsideKasmVDI())
{
parent.postMessage({ action: 'connection_state', value: state}, '*' );
}
let connect_el = document.getElementById('noVNC_connect_dlg');
switch (state) {
case 'init':
break;
case 'connecting':
transitionElem.textContent = _("Connecting...");
document.documentElement.classList.add("noVNC_connecting");
break;
case 'connected':
document.documentElement.classList.add("noVNC_connected");
if (!connect_el.classList.contains("noVNC_hidden")) {
connect_el.classList.add('noVNC_hidden');
}
break;
case 'disconnecting':
transitionElem.textContent = _("Disconnecting...");
document.documentElement.classList.add("noVNC_disconnecting");
break;
case 'disconnected':
console.log('disconnected')
document.documentElement.classList.add("noVNC_disconnected");
if (connect_el.classList.contains("noVNC_hidden")) {
connect_el.classList.remove('noVNC_hidden');
}
UI.disconnect()
break;
case 'reconnecting':
transitionElem.textContent = _("Reconnecting...");
document.documentElement.classList.add("noVNC_reconnecting");
break;
default:
Log.Error("Invalid visual state: " + state);
UI.showStatus(_("Internal error"), 'error');
return;
}
},
identify(data) {
UI.screens = data.screens
console.log('identify')
const screen = data.screens.find(el => el.id === UI.screenID)
if (screen) {
document.getElementById('noVNC_identify_monitor').innerHTML = screen.num
document.getElementById('noVNC_identify_monitor').classList.add("show")
document.querySelector('title').textContent = 'Display ' + screen.num + ' - ' + UI.screenID
setTimeout(() => {
document.getElementById('noVNC_identify_monitor').classList.remove("show")
}, 3500)
}
},
showStatus(text, statusType, time, kasm = false) {
// If inside the full Kasm CDI framework, don't show messages unless explicitly told to
if (WebUtil.isInsideKasmVDI() && !kasm) {
return;
}
const statusElem = document.getElementById('noVNC_status');
if (typeof statusType === 'undefined') {
statusType = 'normal';
}
// Don't overwrite more severe visible statuses and never
// errors. Only shows the first error.
if (statusElem.classList.contains("noVNC_open")) {
if (statusElem.classList.contains("noVNC_status_error")) {
return;
}
if (statusElem.classList.contains("noVNC_status_warn") &&
statusType === 'normal') {
return;
}
}
clearTimeout(UI.statusTimeout);
switch (statusType) {
case 'error':
statusElem.classList.remove("noVNC_status_warn");
statusElem.classList.remove("noVNC_status_normal");
statusElem.classList.add("noVNC_status_error");
break;
case 'warning':
case 'warn':
statusElem.classList.remove("noVNC_status_error");
statusElem.classList.remove("noVNC_status_normal");
statusElem.classList.add("noVNC_status_warn");
break;
case 'normal':
case 'info':
default:
statusElem.classList.remove("noVNC_status_error");
statusElem.classList.remove("noVNC_status_warn");
statusElem.classList.add("noVNC_status_normal");
break;
}
statusElem.textContent = text;
statusElem.classList.add("noVNC_open");
// If no time was specified, show the status for 1.5 seconds
if (typeof time === 'undefined') {
time = 1500;
}
// Error messages do not timeout
if (statusType !== 'error') {
UI.statusTimeout = window.setTimeout(UI.hideStatus, time);
}
},
hideStatus() {
clearTimeout(UI.statusTimeout);
document.getElementById('noVNC_status').classList.remove("noVNC_open");
},
disconnect() {
if (UI.rfb) {
UI.rfb.disconnect();
if (UI.supportsBroadcastChannel) {
console.log('remove event listeners')
UI.controlChannel.removeEventListener('message', UI.handleControlMessage);
UI.rfb.removeEventListener("connect", UI.connectFinished);
}
}
},
connectFinished(e) {
UI.connected = true;
UI.inhibitReconnect = false;
UI.showStatus("Secondary Screen Connected");
UI.updateVisualState('connected');
// Do this last because it can only be used on rendered elements
UI.rfb.focus();
},
initMouseButtonMapper() {
const mouseButtonMapper = new MouseButtonMapper();
const settings = WebUtil.readSetting("mouseButtonMapper");
if (settings) {
mouseButtonMapper.load(settings);
return mouseButtonMapper;
}
mouseButtonMapper.set(0, XVNC_BUTTONS.LEFT_BUTTON);
mouseButtonMapper.set(1, XVNC_BUTTONS.MIDDLE_BUTTON);
mouseButtonMapper.set(2, XVNC_BUTTONS.RIGHT_BUTTON);
mouseButtonMapper.set(3, XVNC_BUTTONS.BACK_BUTTON);
mouseButtonMapper.set(4, XVNC_BUTTONS.FORWARD_BUTTON);
WebUtil.writeSetting("mouseButtonMapper", mouseButtonMapper.dump());
return mouseButtonMapper;
},
updateLogging() {
WebUtil.initLogging(UI.getSetting('logging'));
},
// Initial page load read/initialization of settings
initSetting(name, defVal) {
// Check Query string followed by cookie
let val = WebUtil.getConfigVar(name);
if (val === null) {
val = WebUtil.readSetting(name, defVal);
}
WebUtil.setSetting(name, val);
return val;
},
}
UI.prime();
export default UI;

2
core/base64.js
View File

@ -58,7 +58,7 @@ export default {
/* Every four characters is 3 resulting numbers */ /* Every four characters is 3 resulting numbers */
const resultLength = (dataLength >> 2) * 3 + Math.floor((dataLength % 4) / 1.5); const resultLength = (dataLength >> 2) * 3 + Math.floor((dataLength % 4) / 1.5);
const result = new Array(resultLength); const result = new Uint8Array(resultLength);
// Convert one by one. // Convert one by one.

630
core/display.js
View File

@ -11,9 +11,10 @@ import * as Log from './util/logging.js';
import Base64 from "./base64.js"; import Base64 from "./base64.js";
import { toSigned32bit } from './util/int.js'; import { toSigned32bit } from './util/int.js';
import { isWindows } from './util/browser.js'; import { isWindows } from './util/browser.js';
import { uuidv4 } from './util/strings.js';
export default class Display { export default class Display {
constructor(target) { constructor(target, isPrimaryDisplay) {
Log.Debug(">> Display.constructor"); Log.Debug(">> Display.constructor");
/* /*
@ -30,6 +31,7 @@ export default class Display {
this._asyncFrameQueue = []; this._asyncFrameQueue = [];
this._maxAsyncFrameQueue = 3; this._maxAsyncFrameQueue = 3;
this._clearAsyncQueue(); this._clearAsyncQueue();
this._syncFrameQueue = [];
this._flushing = false; this._flushing = false;
@ -56,7 +58,7 @@ export default class Display {
this._targetCtx = this._target.getContext('2d'); this._targetCtx = this._target.getContext('2d');
// the visible canvas viewport (i.e. what actually gets seen) // the visible canvas viewport (i.e. what actually gets seen)
this._viewportLoc = { 'x': 0, 'y': 0, 'w': this._target.width, 'h': this._target.height }; //this._viewportLoc = { 'x': 0, 'y': 0, 'w': this._target.width, 'h': this._target.height };
Log.Debug("User Agent: " + navigator.userAgent); Log.Debug("User Agent: " + navigator.userAgent);
@ -80,23 +82,58 @@ export default class Display {
// ===== PROPERTIES ===== // ===== PROPERTIES =====
this._maxScreens = 4;
this._scale = 1.0; this._scale = 1.0;
this._clipViewport = false; this._clipViewport = false;
this._antiAliasing = 0; this._antiAliasing = 0;
this._fps = 0; this._fps = 0;
this._isPrimaryDisplay = isPrimaryDisplay;
this._screenID = uuidv4();
this._screens = [{
screenID: this._screenID,
screenIndex: 0,
width: this._target.width, //client
height: this._target.height, //client
serverWidth: 0, //calculated
serverHeight: 0, //calculated
x: 0,
y: 0,
relativePosition: 0, //left, right, up, down relative to primary display
relativePositionX: 0, //offset relative to primary monitor, always 0 for primary
relativePositionY: 0, //offset relative to primary monitor, always 0 for primary
pixelRatio: window.devicePixelRatio,
containerHeight: this._target.parentNode.offsetHeight,
containerWidth: this._target.parentNode.offsetWidth,
channel: null
}];
// ===== EVENT HANDLERS ===== // ===== EVENT HANDLERS =====
this.onflush = () => { }; // A flush request has finished this.onflush = () => { }; // A flush request has finished
// Use requestAnimationFrame to write to canvas, to match display refresh rate if (!this._isPrimaryDisplay) {
this._animationFrameID = window.requestAnimationFrame( () => { this._pushAsyncFrame(); }); this._screens[0].channel = new BroadcastChannel(`screen_${this._screenID}_channel`);
this._screens[0].channel.addEventListener('message', this._handleSecondaryDisplayMessage.bind(this));
} else {
//this._animationFrameID = window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
}
Log.Debug("<< Display.constructor"); Log.Debug("<< Display.constructor");
} }
// ===== PROPERTIES ===== // ===== PROPERTIES =====
get screens() { return this._screens; }
get screenId() { return this._screenID; }
get screenIndex() {
// A secondary screen should not have a screen index of 0, but it will be 0 until registration is complete
// returning a -1 lets the caller know the screen has not been registered yet
if (!this._isPrimaryDisplay && this._screens[0].screenIndex == 0) {
return -1;
}
return this._screens[0].screenIndex;
}
get antiAliasing() { return this._antiAliasing; } get antiAliasing() { return this._antiAliasing; }
set antiAliasing(value) { set antiAliasing(value) {
this._antiAliasing = value; this._antiAliasing = value;
@ -112,8 +149,8 @@ export default class Display {
set clipViewport(viewport) { set clipViewport(viewport) {
this._clipViewport = viewport; this._clipViewport = viewport;
// May need to readjust the viewport dimensions // May need to readjust the viewport dimensions
const vp = this._viewportLoc; const vp = this._screens[0];
this.viewportChangeSize(vp.w, vp.h); this.viewportChangeSize(vp.width, vp.height);
this.viewportChangePos(0, 0); this.viewportChangePos(0, 0);
} }
@ -136,18 +173,207 @@ export default class Display {
// ===== PUBLIC METHODS ===== // ===== PUBLIC METHODS =====
/*
Returns the screen index and relative coordinates given globally scoped coordinates
*/
getClientRelativeCoordinates(x, y) {
for (let i = 0; i < this._screens.length; i++) {
if (
(x >= this._screens[i].x && x <= this._screens[i].x + this._screens[i].serverWidth) &&
(y >= this._screens[i].y && y <= this._screens[i].y + this._screens[i].serverHeight)
)
{
return {
"screenIndex": i,
"x": x - this._screens[i].x,
"y": y - this._screens[i].y
}
}
}
}
/*
Returns coordinates that are server relative when multiple monitors are in use
*/
getServerRelativeCoordinates(screenIndex, x, y) {
if (screenIndex >= 0 && screenIndex < this._screens.length) {
x += this._screens[screenIndex].x;
y += this._screens[screenIndex].y;
}
return [x, y];
}
getScreenSize(resolutionQuality, max_width, max_height, hiDpi, disableLimit) {
let data = {
screens: null,
serverWidth: 0,
serverHeight: 0
}
//recalculate primary display container size
this._screens[0].containerHeight = this._target.parentNode.offsetHeight;
this._screens[0].containerWidth = this._target.parentNode.offsetWidth;
this._screens[0].width = this._target.parentNode.offsetWidth;
this._screens[0].height = this._target.parentNode.offsetHeight;
this._screens[0].pixelRatio = window.devicePixelRatio;
//this._screens[0].width = this._target.width;
//this._screens[0].height = this._target.height;
//calculate server-side resolution of each screen
for (let i=0; i<this._screens.length; i++) {
let width = max_width || this._screens[i].containerWidth;
let height = max_height || this._screens[i].containerHeight;
let scale = 0;
//max the resolution of a single screen to 1280
if (width > 1280 && !disableLimit && resolutionQuality == 1) {
height = 1280 * (height/width); //keeping the aspect ratio of original resolution, shrink y to match x
width = 1280;
}
//hard coded 720p
else if (resolutionQuality == 0 && !disableLimit) {
width = 1280;
height = 720;
}
//force full resolution on a high DPI monitor where the OS is scaling
else if (hiDpi) {
width = width * this._screens[i].pixelRatio;
height = height * this._screens[i].pixelRatio;
scale = 1 / this._screens[i].pixelRatio;
}
//physically small device with high DPI
else if (this._antiAliasing === 0 && this._screens[i].pixelRatio > 1 && width < 1000 & width > 0) {
Log.Info('Device Pixel ratio: ' + this._screens[i].pixelRatio + ' Reported Resolution: ' + width + 'x' + height);
let targetDevicePixelRatio = 1.5;
if (this._screens[i].pixelRatio > 2) { targetDevicePixelRatio = 2; }
let scaledWidth = (width * this._screens[i].pixelRatio) * (1 / targetDevicePixelRatio);
let scaleRatio = scaledWidth / width;
width = width * scaleRatio;
height = height * scaleRatio;
scale = 1 / scaleRatio;
Log.Info('Small device with hDPI screen detected, auto scaling at ' + scaleRatio + ' to ' + width + 'x' + height);
}
this._screens[i].serverWidth = width;
this._screens[i].serverHeight = height;
this._screens[i].scale = scale;
this._screens[i].x2 = this._screens[i].x + this._screens[i].serverWidth;
this._screens[i].y2 = this._screens[i].y + this._screens[i].serverHeight;
}
for (let i = 0; i < this._screens.length; i++) {
data.serverWidth = Math.max(data.serverWidth, this._screens[i].x + this._screens[i].serverWidth);
data.serverHeight = Math.max(data.serverHeight, this._screens[i].y + this._screens[i].serverHeight);
}
data.screens = this._screens;
return data;
}
applyScreenPlan(screenPlan) {
for (let i = 0; i < screenPlan.screens.length; i++) {
for (let z = 0; z < this._screens.length; z++) {
if (screenPlan.screens[i].screenID === this._screens[z].screenID) {
this._screens[z].x = screenPlan.screens[i].x;
this._screens[z].y = screenPlan.screens[i].y;
}
}
}
}
addScreen(screenID, width, height, pixelRatio, containerHeight, containerWidth) {
if (!this._isPrimaryDisplay) {
throw new Error("Cannot add a screen to a secondary display.");
}
let screenIdx = -1;
//Does the screen already exist?
for (let i = 0; i < this._screens.length; i++) {
if (this._screens[i].screenID === screenID) {
screenIdx = i;
}
}
if (screenIdx > 0) {
//existing screen, update
const screen = this._screens[screenIdx];
screen.width = width;
screen.height = height;
screen.containerHeight = containerHeight;
screen.containerWidth = containerWidth;
screen.pixelRatio = pixelRatio;
} else {
//New Screen, add to far right until user repositions it
let x = 0;
for (let i = 0; i < this._screens.length; i++) {
x = Math.max(x, this._screens[i].x + this._screens[i].serverWidth);
}
var new_screen = {
screenID: screenID,
screenIndex: this.screens.length,
width: width, //client
height: height, //client
serverWidth: 0, //calculated
serverHeight: 0, //calculated
x: x,
y: 0,
pixelRatio: pixelRatio,
containerHeight: containerHeight,
containerWidth: containerWidth,
channel: null,
scale: 0
}
new_screen.channel = new BroadcastChannel(`screen_${screenID}_channel`);
//new_screen.channel.message = this._handleSecondaryDisplayMessage().bind(this);
this._screens.push(new_screen);
new_screen.channel.postMessage({ eventType: "registered", screenIndex: new_screen.screenIndex });
}
}
removeScreen(screenID) {
let removed = false;
if (this._isPrimaryDisplay) {
for (let i=1; i<this._screens.length; i++) {
if (this._screens[i].screenID == screenID) {
//flush all rects on target screen
this._flushRectsScreen(i);
this._screens[i].channel.close();
this._screens.splice(i, 1);
removed = true;
break;
}
}
//recalculate indexes and update secondary displays
for (let i=1; i<this._screens.length; i++) {
this.screens[i].screenIndex = i;
if (i > 0) {
this._screens[i].channel.postMessage({ eventType: "registered", screenIndex: i });
}
}
return removed;
} else {
throw new Error("Secondary screens only allowed on primary display.")
}
}
viewportChangePos(deltaX, deltaY) { viewportChangePos(deltaX, deltaY) {
const vp = this._viewportLoc; const vp = this._screens[0];
deltaX = Math.floor(deltaX); deltaX = Math.floor(deltaX);
deltaY = Math.floor(deltaY); deltaY = Math.floor(deltaY);
if (!this._clipViewport) { if (!this._clipViewport) {
deltaX = -vp.w; // clamped later of out of bounds deltaX = -vp.width; // clamped later of out of bounds
deltaY = -vp.h; deltaY = -vp.height;
} }
const vx2 = vp.x + vp.w - 1; const vx2 = vp.x + vp.width - 1;
const vy2 = vp.y + vp.h - 1; const vy2 = vp.y + vp.height - 1;
// Position change // Position change
@ -173,7 +399,7 @@ export default class Display {
viewportChangeSize(width, height) { viewportChangeSize(width, height) {
if (!this._clipViewport || if ((!this._clipViewport && this._screens.length === 1 ) ||
typeof(width) === "undefined" || typeof(width) === "undefined" ||
typeof(height) === "undefined") { typeof(height) === "undefined") {
@ -192,10 +418,10 @@ export default class Display {
height = this._fbHeight; height = this._fbHeight;
} }
const vp = this._viewportLoc; const vp = this._screens[0];
if (vp.w !== width || vp.h !== height) { if (vp.serverWidth !== width || vp.serverHeight !== height) {
vp.w = width; vp.serverWidth = width;
vp.h = height; vp.serverHeight = height;
const canvas = this._target; const canvas = this._target;
canvas.width = width; canvas.width = width;
@ -213,14 +439,14 @@ export default class Display {
if (this._scale === 0) { if (this._scale === 0) {
return 0; return 0;
} }
return toSigned32bit(x / this._scale + this._viewportLoc.x); return toSigned32bit(x / this._scale + this._screens[0].x);
} }
absY(y) { absY(y) {
if (this._scale === 0) { if (this._scale === 0) {
return 0; return 0;
} }
return toSigned32bit(y / this._scale + this._viewportLoc.y); return toSigned32bit(y / this._scale + this._screens[0].y);
} }
resize(width, height) { resize(width, height) {
@ -231,6 +457,10 @@ export default class Display {
const canvas = this._target; const canvas = this._target;
if (canvas == undefined) { return; } if (canvas == undefined) { return; }
if (this._screens.length > 0) {
width = this._screens[0].serverWidth;
height = this._screens[0].serverHeight;
}
if (canvas.width !== width || canvas.height !== height) { if (canvas.width !== width || canvas.height !== height) {
// We have to save the canvas data since changing the size will clear it // We have to save the canvas data since changing the size will clear it
@ -253,8 +483,8 @@ export default class Display {
// Readjust the viewport as it may be incorrectly sized // Readjust the viewport as it may be incorrectly sized
// and positioned // and positioned
const vp = this._viewportLoc; const vp = this._screens[0];
this.viewportChangeSize(vp.w, vp.h); this.viewportChangeSize(vp.serverWidth, vp.serverHeight);
this.viewportChangePos(0, 0); this.viewportChangePos(0, 0);
} }
@ -267,7 +497,8 @@ export default class Display {
this._asyncRenderQPush({ this._asyncRenderQPush({
'type': 'flip', 'type': 'flip',
'frame_id': frame_id, 'frame_id': frame_id,
'rect_cnt': rect_cnt 'rect_cnt': rect_cnt,
'screenLocations': [ { screenIndex: 0, x: 0, y: 0 } ]
}); });
} }
@ -291,7 +522,7 @@ export default class Display {
this._asyncFrameComplete(0, true); this._asyncFrameComplete(0, true);
if (onflush_message) if (onflush_message)
this._flushing = true; this.onflush();
} }
/* /*
@ -307,21 +538,22 @@ export default class Display {
*/ */
dispose() { dispose() {
clearInterval(this._frameStatsInterval); clearInterval(this._frameStatsInterval);
cancelAnimationFrame(this._animationFrameID);
this.clear(); this.clear();
} }
fillRect(x, y, width, height, color, frame_id, fromQueue) { fillRect(x, y, width, height, color, frame_id, fromQueue) {
if (!fromQueue) { if (!fromQueue) {
this._asyncRenderQPush({ let rect = {
'type': 'fill', type: 'fill',
'x': x, x: x,
'y': y, y: y,
'width': width, width: width,
'height': height, height: height,
'color': color, color: color,
'frame_id': frame_id frame_id: frame_id
}); }
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else { } else {
this._setFillColor(color); this._setFillColor(color);
this._targetCtx.fillRect(x, y, width, height); this._targetCtx.fillRect(x, y, width, height);
@ -330,7 +562,7 @@ export default class Display {
copyImage(oldX, oldY, newX, newY, w, h, frame_id, fromQueue) { copyImage(oldX, oldY, newX, newY, w, h, frame_id, fromQueue) {
if (!fromQueue) { if (!fromQueue) {
this._asyncRenderQPush({ let rect = {
'type': 'copy', 'type': 'copy',
'oldX': oldX, 'oldX': oldX,
'oldY': oldY, 'oldY': oldY,
@ -339,7 +571,9 @@ export default class Display {
'width': w, 'width': w,
'height': h, 'height': h,
'frame_id': frame_id 'frame_id': frame_id
}); }
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else { } else {
// Due to this bug among others [1] we need to disable the image-smoothing to // Due to this bug among others [1] we need to disable the image-smoothing to
// avoid getting a blur effect when copying data. // avoid getting a blur effect when copying data.
@ -364,18 +598,31 @@ export default class Display {
if ((width === 0) || (height === 0)) { if ((width === 0) || (height === 0)) {
return; return;
} }
const img = new Image();
img.src = "data: " + mime + ";base64," + Base64.encode(arr);
this._asyncRenderQPush({ let rect = {
'type': 'img', 'type': 'img',
'img': img, 'img': null,
'x': x, 'x': x,
'y': y, 'y': y,
'width': width, 'width': width,
'height': height, 'height': height,
'frame_id': frame_id 'frame_id': frame_id
}); }
this._processRectScreens(rect);
if (rect.inPrimary) {
const img = new Image();
img.src = "data: " + mime + ";base64," + Base64.encode(arr);
rect.img = img;
} else {
rect.type = "_img";
}
if (rect.inSecondary) {
rect.mime = mime;
rect.src = "data: " + mime + ";base64," + Base64.encode(arr);
}
this._asyncRenderQPush(rect);
} }
transparentRect(x, y, width, height, img, frame_id) { transparentRect(x, y, width, height, img, frame_id) {
@ -393,12 +640,18 @@ export default class Display {
'height': height, 'height': height,
'frame_id': frame_id 'frame_id': frame_id
} }
this._processRectScreens(rect);
let imageBmpPromise = createImageBitmap(img); if (rect.inPrimary) {
imageBmpPromise.then( function(img) { let imageBmpPromise = createImageBitmap(img);
rect.img = img; imageBmpPromise.then( function(img) {
rect.img.complete = true; rect.img = img;
}.bind(rect) ); rect.img.complete = true;
}.bind(rect) );
}
if (rect.inSecondary) {
rect.arr = img;
}
this._asyncRenderQPush(rect); this._asyncRenderQPush(rect);
} }
@ -410,7 +663,7 @@ export default class Display {
// this probably isn't getting called *nearly* as much // this probably isn't getting called *nearly* as much
const newArr = new Uint8Array(width * height * 4); const newArr = new Uint8Array(width * height * 4);
newArr.set(new Uint8Array(arr.buffer, 0, newArr.length)); newArr.set(new Uint8Array(arr.buffer, 0, newArr.length));
this._asyncRenderQPush({ let rect = {
'type': 'blit', 'type': 'blit',
'data': newArr, 'data': newArr,
'x': x, 'x': x,
@ -418,7 +671,9 @@ export default class Display {
'width': width, 'width': width,
'height': height, 'height': height,
'frame_id': frame_id 'frame_id': frame_id
}); }
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else { } else {
// NB(directxman12): arr must be an Type Array view // NB(directxman12): arr must be an Type Array view
let data = new Uint8ClampedArray(arr.buffer, let data = new Uint8ClampedArray(arr.buffer,
@ -431,7 +686,7 @@ export default class Display {
blitQoi(x, y, width, height, arr, offset, frame_id, fromQueue) { blitQoi(x, y, width, height, arr, offset, frame_id, fromQueue) {
if (!fromQueue) { if (!fromQueue) {
this._asyncRenderQPush({ let rect = {
'type': 'blitQ', 'type': 'blitQ',
'data': arr, 'data': arr,
'x': x, 'x': x,
@ -439,7 +694,9 @@ export default class Display {
'width': width, 'width': width,
'height': height, 'height': height,
'frame_id': frame_id 'frame_id': frame_id
}); }
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else { } else {
this._targetCtx.putImageData(arr, x, y); this._targetCtx.putImageData(arr, x, y);
} }
@ -464,14 +721,14 @@ export default class Display {
} else if (scaleRatio === 0) { } else if (scaleRatio === 0) {
const vp = this._viewportLoc; const vp = this._screens[0];
const targetAspectRatio = containerWidth / containerHeight; const targetAspectRatio = containerWidth / containerHeight;
const fbAspectRatio = vp.w / vp.h; const fbAspectRatio = vp.width / vp.height;
if (fbAspectRatio >= targetAspectRatio) { if (fbAspectRatio >= targetAspectRatio) {
scaleRatio = containerWidth / vp.w; scaleRatio = containerWidth / vp.width;
} else { } else {
scaleRatio = containerHeight / vp.h; scaleRatio = containerHeight / vp.height;
} }
} }
@ -480,6 +737,112 @@ export default class Display {
// ===== PRIVATE METHODS ===== // ===== PRIVATE METHODS =====
_handleSecondaryDisplayMessage(event) {
if (!this._isPrimaryDisplay && event.data) {
switch (event.data.eventType) {
case 'rect':
let rect = event.data.rect;
//overwrite screen locations when received on the secondary display
rect.screenLocations = [ rect.screenLocations[event.data.screenLocationIndex] ]
rect.screenLocations[0].screenIndex = 0;
switch (rect.type) {
case 'img':
case '_img':
rect.img = new Image();
rect.img.src = rect.src;
rect.type = 'img';
break;
case 'transparent':
let imageBmpPromise = createImageBitmap(rect.arr);
imageBmpPromise.then(function(rect, img) {
rect.img.complete = true;
}).bind(this, rect);
break;
}
this._syncFrameQueue.push(rect);
break;
case 'frameComplete':
window.requestAnimationFrame( () => { this._pushSyncRects(); });
break;
case 'registered':
if (!this._isPrimaryDisplay) {
this._screens[0].screenIndex = event.data.screenIndex;
Log.Error(`Screen with index (${event.data.screenIndex}) successfully registered with the primary display.`);
}
break;
}
}
}
_pushSyncRects() {
whileLoop:
while (this._syncFrameQueue.length > 0) {
const a = this._syncFrameQueue[0];
const pos = a.screenLocations[0];
switch (a.type) {
case 'copy':
this.copyImage(pos.oldX, pos.oldY, pos.x, pos.y, a.width, a.height, a.frame_id, true);
break;
case 'fill':
this.fillRect(pos.x, pos.y, a.width, a.height, a.color, a.frame_id, true);
break;
case 'blit':
this.blitImage(pos.x, pos.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'blitQ':
this.blitQoi(pos.x, pos.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'img':
if (a.img.complete) {
this.drawImage(a.img, pos.x, pos.y, a.width, a.height);
} else {
if (this._syncFrameQueue.length > 1000) {
this._syncFrameQueue.shift();
this._droppedRects++;
} else {
break whileLoop;
}
}
break;
case 'transparent':
if (a.img.complete) {
this.drawImage(a.img, pos.x, pos.y, a.width, a.height);
} else {
if (this._syncFrameQueue.length > 1000) {
this._syncFrameQueue.shift();
this._droppedRects++;
} else {
break whileLoop;
}
}
break;
default:
Log.Warn(`Unknown rect type: ${rect}`);
}
this._syncFrameQueue.shift();
}
if (this._syncFrameQueue.length > 0) {
window.requestAnimationFrame( () => { this._pushSyncRects(); });
}
}
_flushRectsScreen(screenIndex) {
for (let i=0; i<this._asyncFrameQueue.length; i++) {
const frame = this._asyncFrameQueue[i];
for (let x=0; x < frame[2].length; x++) {
const rect = frame[2][x];
for (let y=0; y < rect.screenLocations.length; y++) {
if (rect.screenLocations[y].screenIndex === screenIndex) {
rect.screenLocations.splice(y, 1);
break;
}
}
}
}
}
/* /*
Process incoming rects into a frame buffer, assume rects are out of order due to either UDP or parallel processing of decoding Process incoming rects into a frame buffer, assume rects are out of order due to either UDP or parallel processing of decoding
*/ */
@ -504,19 +867,24 @@ export default class Display {
newestFrameID = Math.max(newestFrameID, this._asyncFrameQueue[i][0]); newestFrameID = Math.max(newestFrameID, this._asyncFrameQueue[i][0]);
} }
if (!this._firstRect) { //TODO: Remove this
this._firstRect = true;
Log.Info("First rect received.");
}
if (frameIx >= 0) { if (frameIx >= 0) {
if (rect.type == "flip") { if (rect.type == "flip") {
//flip rect contains the rect count for the frame //flip rect contains the rect count for the frame
if (this._asyncFrameQueue[frameIx][1] !== 0) { if (this._asyncFrameQueue[frameIx][1] !== 0) {
Log.Warn("Redundant flip rect, current rect_cnt: " + this._asyncFrameQueue[frameIx][1] + ", new rect_cnt: " + rect.rect_cnt ); Log.Warn("Redundant flip rect, current rect_cnt: " + this._asyncFrameQueue[frameIx][1] + ", new rect_cnt: " + rect.rect_cnt );
} }
this._asyncFrameQueue[frameIx][1] = rect.rect_cnt; this._asyncFrameQueue[frameIx][1] += rect.rect_cnt;
if (rect.rect_cnt == 0) { if (rect.rect_cnt == 0) {
Log.Warn("Invalid rect count"); Log.Warn("Invalid rect count");
} }
} }
if (this._asyncFrameQueue[frameIx][1] == this._asyncFrameQueue[frameIx][2].length) { if (this._asyncFrameQueue[frameIx][1] > 0 && this._asyncFrameQueue[frameIx][2].length >= this._asyncFrameQueue[frameIx][1]) {
//frame is complete //frame is complete
this._asyncFrameComplete(frameIx); this._asyncFrameComplete(frameIx);
} }
@ -527,11 +895,21 @@ export default class Display {
if (rect.type == "flip") { this._lateFlipRect++; } if (rect.type == "flip") { this._lateFlipRect++; }
return; return;
} else if (rect.frame_id > newestFrameID) { } else if (rect.frame_id > newestFrameID) {
//frame is newer than any frame in the queue, drop old frames //frame is newer than any frame in the queue, drop old frame
this._asyncFrameQueue.shift(); if (this._asyncFrameQueue[0][3] == true) {
Log.Warn("Forced frame to canvas");
this._pushAsyncFrame(true);
this._droppedFrames += (rect.frame_id - (newestFrameID + 1));
this._forcedFrameCnt++;
} else {
Log.Warn("Old frame dropped");
this._asyncFrameQueue.shift();
this._droppedFrames += (rect.frame_id - newestFrameID);
}
let rect_cnt = ((rect.type == "flip") ? rect.rect_cnt : 0); let rect_cnt = ((rect.type == "flip") ? rect.rect_cnt : 0);
this._asyncFrameQueue.push([ rect.frame_id, rect_cnt, [ rect ], (rect_cnt == 1), 0, 0 ]); this._asyncFrameQueue.push([ rect.frame_id, rect_cnt, [ rect ], (rect_cnt == 1), 0, 0 ]);
this._droppedFrames++;
} }
} }
@ -554,6 +932,10 @@ export default class Display {
If marked force, unloaded images will be skipped and the frame will be marked complete and ready for rendering If marked force, unloaded images will be skipped and the frame will be marked complete and ready for rendering
*/ */
_asyncFrameComplete(frameIx, force=false) { _asyncFrameComplete(frameIx, force=false) {
if (frameIx >= this._asyncFrameQueue.length) {
return;
}
let currentFrameRectIx = this._asyncFrameQueue[frameIx][4]; let currentFrameRectIx = this._asyncFrameQueue[frameIx][4];
if (force) { if (force) {
@ -566,10 +948,13 @@ export default class Display {
} }
} }
while (currentFrameRectIx < this._asyncFrameQueue[frameIx][2].length) { while (currentFrameRectIx < this._asyncFrameQueue[frameIx][2].length) {
if (this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type == 'img' && !this._asyncFrameQueue[frameIx][2][currentFrameRectIx].img.complete) { if (this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type == 'img') {
this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type = 'skip'; if (this._asyncFrameQueue[frameIx][2][currentFrameRectIx].img && !this._asyncFrameQueue[frameIx][2][currentFrameRectIx].img.complete) {
this._droppedRects++; this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type = 'skip';
this._droppedRects++;
}
} }
currentFrameRectIx++; currentFrameRectIx++;
} }
} else { } else {
@ -587,6 +972,12 @@ export default class Display {
} }
this._asyncFrameQueue[frameIx][4] = currentFrameRectIx; this._asyncFrameQueue[frameIx][4] = currentFrameRectIx;
this._asyncFrameQueue[frameIx][3] = true; this._asyncFrameQueue[frameIx][3] = true;
if (force && frameIx == 0) {
this._pushAsyncFrame(true);
} else {
window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
}
} }
/* /*
@ -594,45 +985,77 @@ export default class Display {
*/ */
_pushAsyncFrame(force=false) { _pushAsyncFrame(force=false) {
if (this._asyncFrameQueue[0][3] || force) { if (this._asyncFrameQueue[0][3] || force) {
let frame = this._asyncFrameQueue.shift()[2]; let frame = this._asyncFrameQueue[0][2];
let frameId = this._asyncFrameQueue.shift()[0];
if (this._asyncFrameQueue.length < this._maxAsyncFrameQueue) { if (this._asyncFrameQueue.length < this._maxAsyncFrameQueue) {
this._asyncFrameQueue.push([ 0, 0, [], false, 0, 0 ]); this._asyncFrameQueue.push([ 0, 0, [], false, 0, 0 ]);
} }
let transparent_rects = []; let transparent_rects = [];
let secondaryScreenRects = 0;
//render the selected frame //render the selected frame
for (let i = 0; i < frame.length; i++) { for (let i = 0; i < frame.length; i++) {
const a = frame[i]; const a = frame[i];
switch (a.type) {
case 'copy': for (let sI = 0; sI < a.screenLocations.length; sI++) {
this.copyImage(a.oldX, a.oldY, a.x, a.y, a.width, a.height, a.frame_id, true); let screenLocation = a.screenLocations[sI];
break; if (screenLocation.screenIndex == 0) {
case 'fill': switch (a.type) {
this.fillRect(a.x, a.y, a.width, a.height, a.color, a.frame_id, true); case 'copy':
break; this.copyImage(screenLocation.oldX, screenLocation.oldY, screenLocation.x, screenLocation.y, a.width, a.height, a.frame_id, true);
case 'blit': break;
this.blitImage(a.x, a.y, a.width, a.height, a.data, 0, a.frame_id, true); case 'fill':
break; this.fillRect(screenLocation.x, screenLocation.y, a.width, a.height, a.color, a.frame_id, true);
case 'blitQ': break;
this.blitQoi(a.x, a.y, a.width, a.height, a.data, 0, a.frame_id, true); case 'blit':
break; this.blitImage(screenLocation.x, screenLocation.y, a.width, a.height, a.data, 0, a.frame_id, true);
case 'img': break;
this.drawImage(a.img, a.x, a.y, a.width, a.height); case 'blitQ':
break; this.blitQoi(screenLocation.x, screenLocation.y, a.width, a.height, a.data, 0, a.frame_id, true);
case 'transparent': break;
transparent_rects.push(a); case 'img':
break; this.drawImage(a.img, screenLocation.x, screenLocation.y, a.width, a.height);
break;
case 'transparent':
transparent_rects.push(a);
break;
}
} else {
if (a.img) {
a.img = null;
}
if (a.type !== 'flip') {
secondaryScreenRects++;
this._screens[screenLocation.screenIndex].channel.postMessage({ eventType: 'rect', rect: a, screenLocationIndex: sI });
}
}
} }
} }
//rects with transparency get applied last //rects with transparency get applied last
for (let i = 0; i < transparent_rects.length; i++) { for (let i = 0; i < transparent_rects.length; i++) {
const a = transparent_rects[i]; const a = transparent_rects[i];
let screenIndexes = this._getRectScreenIndexes(a);
if (a.img) { for (let sI = 0; sI < screenLocations.length; sI++) {
this.drawImage(a.img, a.x, a.y, a.width, a.height); let screenLocation = a.screenLocations[sI];
if (sI == 0) {
if (a.img) {
this.drawImage(a.img, a.x, a.y, a.width, a.height);
}
} else {
secondaryScreenRects++;
this._screens[screenLocation.screenIndex].channel.postMessage({ eventType: 'rect', rect: a, screenLocationIndex: sI });
}
}
}
if (secondaryScreenRects > 0) {
for (let i = 1; i < this.screens.length; i++) {
this._screens[i].channel.postMessage({ eventType: 'frameComplete', frameId: frameId, rectCnt: secondaryScreenRects });
} }
} }
@ -650,22 +1073,51 @@ export default class Display {
this._pushAsyncFrame(true); this._pushAsyncFrame(true);
} }
} }
}
if (!force) { _processRectScreens(rect) {
window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
//find which screen this rect belongs to and adjust its x and y to be relative to the destination
let indexes = [];
rect.inPrimary = false;
rect.inSecondary = false;
for (let i=0; i < this._screens.length; i++) {
let screen = this._screens[i];
if (
!((rect.x > screen.x2 || screen.x > (rect.x + rect.width)) && (rect.y > screen.y2 || screen.y > (rect.y + rect.height)))
) {
let screenPosition = {
x: 0 - (screen.x - rect.x), //rect.x - screen.x,
y: 0 - (screen.y - rect.y), //rect.y - screen.y,
screenIndex: i
}
if (rect.type === 'copy') {
screenPosition.oldX = 0 - (screen.x - rect.oldX); //rect.oldX - screen.x;
screenPosition.oldY = 0 - (screen.y - rect.oldY); //rect.oldY - screen.y;
}
indexes.push(screenPosition);
if (i == 0) {
rect.inPrimary = true;
} else {
rect.inSecondary = true;
}
}
} }
rect.screenLocations = indexes;
} }
_rescale(factor) { _rescale(factor) {
this._scale = factor; this._scale = factor;
const vp = this._viewportLoc; const vp = this._screens[0];
// NB(directxman12): If you set the width directly, or set the // NB(directxman12): If you set the width directly, or set the
// style width to a number, the canvas is cleared. // style width to a number, the canvas is cleared.
// However, if you set the style width to a string // However, if you set the style width to a string
// ('NNNpx'), the canvas is scaled without clearing. // ('NNNpx'), the canvas is scaled without clearing.
const width = factor * vp.w + 'px'; const width = factor * vp.serverWidth + 'px';
const height = factor * vp.h + 'px'; const height = factor * vp.serverHeight + 'px';
if ((this._target.style.width !== width) || if ((this._target.style.width !== width) ||
(this._target.style.height !== height)) { (this._target.style.height !== height)) {
@ -673,12 +1125,12 @@ export default class Display {
this._target.style.height = height; this._target.style.height = height;
} }
Log.Info('Pixel Ratio: ' + window.devicePixelRatio + ', VNC Scale: ' + factor + 'VNC Res: ' + vp.w + 'x' + vp.h); Log.Info('Pixel Ratio: ' + window.devicePixelRatio + ', VNC Scale: ' + factor + 'VNC Res: ' + vp.serverWidth + 'x' + vp.serverHeight);
var pixR = Math.abs(Math.ceil(window.devicePixelRatio)); var pixR = Math.abs(Math.ceil(window.devicePixelRatio));
var isFirefox = navigator.userAgent.toLowerCase().indexOf('firefox') > -1; var isFirefox = navigator.userAgent.toLowerCase().indexOf('firefox') > -1;
if (this.antiAliasing === 2 || (this.antiAliasing === 0 && factor === 1 && this._target.style.imageRendering !== 'pixelated' && pixR === window.devicePixelRatio && vp.w > 0)) { if (this.antiAliasing === 2 || (this.antiAliasing === 0 && factor === 1 && this._target.style.imageRendering !== 'pixelated' && pixR === window.devicePixelRatio && vp.width > 0)) {
this._target.style.imageRendering = ((!isFirefox) ? 'pixelated' : 'crisp-edges' ); this._target.style.imageRendering = ((!isFirefox) ? 'pixelated' : 'crisp-edges' );
Log.Debug('Smoothing disabled'); Log.Debug('Smoothing disabled');
} else if (this.antiAliasing === 1 || (this.antiAliasing === 0 && factor !== 1 && this._target.style.imageRendering !== 'auto')) { } else if (this.antiAliasing === 1 || (this.antiAliasing === 0 && factor !== 1 && this._target.style.imageRendering !== 'auto')) {

7
core/input/keyboard.js
View File

@ -11,7 +11,6 @@ import KeyTable from "./keysym.js";
import keysyms from "./keysymdef.js"; import keysyms from "./keysymdef.js";
import imekeys from "./imekeys.js"; import imekeys from "./imekeys.js";
import * as browser from "../util/browser.js"; import * as browser from "../util/browser.js";
import UI from '../../app/ui.js';
import { isChromiumBased } from '../util/browser.js'; import { isChromiumBased } from '../util/browser.js';
// //
@ -46,6 +45,7 @@ export default class Keyboard {
this._lastKeyboardInput = null; this._lastKeyboardInput = null;
this._defaultKeyboardInputLen = 100; this._defaultKeyboardInputLen = 100;
this._keyboardInputReset(); this._keyboardInputReset();
this._translateShortcuts = true;
} }
// ===== PUBLIC METHODS ===== // ===== PUBLIC METHODS =====
@ -56,6 +56,9 @@ export default class Keyboard {
this.focus(); this.focus();
} }
get translateShortcuts() { return this._translateShortcuts; }
set translateShortcuts(value) { this._translateShortcuts = value; }
// ===== PRIVATE METHODS ===== // ===== PRIVATE METHODS =====
clearKeysDown(event) { clearKeysDown(event) {
@ -319,7 +322,7 @@ export default class Keyboard {
// Translate MacOs CMD based shortcuts to their CTRL based counterpart // Translate MacOs CMD based shortcuts to their CTRL based counterpart
if ( if (
browser.isMac() && browser.isMac() &&
UI.rfb && UI.rfb.translateShortcuts && this._translateShortcuts &&
code !== "MetaLeft" && code !== "MetaRight" && code !== "MetaLeft" && code !== "MetaRight" &&
e.metaKey && !e.ctrlKey && !e.altKey e.metaKey && !e.ctrlKey && !e.altKey
) { ) {

649
core/rfb.js
View File

@ -10,7 +10,7 @@
import { toUnsigned32bit, toSigned32bit } from './util/int.js'; import { toUnsigned32bit, toSigned32bit } from './util/int.js';
import * as Log from './util/logging.js'; import * as Log from './util/logging.js';
import { encodeUTF8, decodeUTF8 } from './util/strings.js'; import { encodeUTF8, decodeUTF8, uuidv4 } from './util/strings.js';
import { hashUInt8Array } from './util/int.js'; import { hashUInt8Array } from './util/int.js';
import { dragThreshold, supportsCursorURIs, isTouchDevice, isWindows, isMac, isIOS } from './util/browser.js'; import { dragThreshold, supportsCursorURIs, isTouchDevice, isWindows, isMac, isIOS } from './util/browser.js';
import { clientToElement } from './util/element.js'; import { clientToElement } from './util/element.js';
@ -76,11 +76,11 @@ const extendedClipboardActionNotify = 1 << 27;
const extendedClipboardActionProvide = 1 << 28; const extendedClipboardActionProvide = 1 << 28;
export default class RFB extends EventTargetMixin { export default class RFB extends EventTargetMixin {
constructor(target, touchInput, urlOrChannel, options) { constructor(target, touchInput, urlOrChannel, options, isPrimaryDisplay) {
if (!target) { if (!target) {
throw new Error("Must specify target"); throw new Error("Must specify target");
} }
if (!urlOrChannel) { if (!urlOrChannel && isPrimaryDisplay) {
throw new Error("Must specify URL, WebSocket or RTCDataChannel"); throw new Error("Must specify URL, WebSocket or RTCDataChannel");
} }
@ -101,6 +101,7 @@ export default class RFB extends EventTargetMixin {
this._shared = 'shared' in options ? !!options.shared : true; this._shared = 'shared' in options ? !!options.shared : true;
this._repeaterID = options.repeaterID || ''; this._repeaterID = options.repeaterID || '';
this._wsProtocols = options.wsProtocols || ['binary']; this._wsProtocols = options.wsProtocols || ['binary'];
this._isPrimaryDisplay = (isPrimaryDisplay !== false);
// Internal state // Internal state
this._rfbConnectionState = ''; this._rfbConnectionState = '';
@ -122,7 +123,8 @@ export default class RFB extends EventTargetMixin {
this._supportsContinuousUpdates = false; this._supportsContinuousUpdates = false;
this._enabledContinuousUpdates = false; this._enabledContinuousUpdates = false;
this._supportsSetDesktopSize = false; this._supportsSetDesktopSize = false;
this._screenID = 0; this._screenID = uuidv4();
this._screenIndex = 0;
this._screenFlags = 0; this._screenFlags = 0;
this._qemuExtKeyEventSupported = false; this._qemuExtKeyEventSupported = false;
@ -205,6 +207,7 @@ export default class RFB extends EventTargetMixin {
this._accumulatedWheelDeltaX = 0; this._accumulatedWheelDeltaX = 0;
this._accumulatedWheelDeltaY = 0; this._accumulatedWheelDeltaY = 0;
this.mouseButtonMapper = null; this.mouseButtonMapper = null;
this._mouseLastScreenIndex = -1;
// Gesture state // Gesture state
this._gestureLastTapTime = null; this._gestureLastTapTime = null;
@ -212,6 +215,19 @@ export default class RFB extends EventTargetMixin {
this._gestureLastMagnitudeX = 0; this._gestureLastMagnitudeX = 0;
this._gestureLastMagnitudeY = 0; this._gestureLastMagnitudeY = 0;
// Secondary Displays
this._secondaryDisplays = {};
this._supportsBroadcastChannel = (typeof BroadcastChannel !== "undefined");
if (this._supportsBroadcastChannel) {
this._controlChannel = new BroadcastChannel("registrationChannel");
this._controlChannel.addEventListener('message', this._handleControlMessage.bind(this));
Log.Debug("Attached to registrationChannel for secondary displays.")
}
if (!this._isPrimaryDisplay) {
this._screenIndex = 2;
}
// Bound event handlers // Bound event handlers
this._eventHandlers = { this._eventHandlers = {
updateHiddenKeyboard: this._updateHiddenKeyboard.bind(this), updateHiddenKeyboard: this._updateHiddenKeyboard.bind(this),
@ -223,6 +239,7 @@ export default class RFB extends EventTargetMixin {
handleWheel: this._handleWheel.bind(this), handleWheel: this._handleWheel.bind(this),
handleGesture: this._handleGesture.bind(this), handleGesture: this._handleGesture.bind(this),
handleFocusChange: this._handleFocusChange.bind(this), handleFocusChange: this._handleFocusChange.bind(this),
handleMouseOut: this._handleMouseOut.bind(this),
}; };
// main setup // main setup
@ -262,7 +279,7 @@ export default class RFB extends EventTargetMixin {
// NB: nothing that needs explicit teardown should be done // NB: nothing that needs explicit teardown should be done
// before this point, since this can throw an exception // before this point, since this can throw an exception
try { try {
this._display = new Display(this._canvas); this._display = new Display(this._canvas, this._isPrimaryDisplay);
} catch (exc) { } catch (exc) {
Log.Error("Display exception: " + exc); Log.Error("Display exception: " + exc);
throw exc; throw exc;
@ -283,68 +300,13 @@ export default class RFB extends EventTargetMixin {
this._gestures = new GestureHandler(); this._gestures = new GestureHandler();
this._sock = new Websock(); if (this._isPrimaryDisplay) {
this._sock.on('message', () => { this._setupWebSocket();
this._handleMessage(); }
});
this._sock.on('open', () => {
if ((this._rfbConnectionState === 'connecting') &&
(this._rfbInitState === '')) {
this._rfbInitState = 'ProtocolVersion';
Log.Debug("Starting VNC handshake");
} else {
this._fail("Unexpected server connection while " +
this._rfbConnectionState);
}
});
this._sock.on('close', (e) => {
Log.Debug("WebSocket on-close event");
let msg = "";
if (e.code) {
msg = "(code: " + e.code;
if (e.reason) {
msg += ", reason: " + e.reason;
}
msg += ")";
}
switch (this._rfbConnectionState) {
case 'connecting':
this._fail("Connection closed " + msg);
break;
case 'connected':
// Handle disconnects that were initiated server-side
this._updateConnectionState('disconnecting');
this._updateConnectionState('disconnected');
break;
case 'disconnecting':
// Normal disconnection path
this._updateConnectionState('disconnected');
break;
case 'disconnected':
this._fail("Unexpected server disconnect " +
"when already disconnected " + msg);
break;
default:
this._fail("Unexpected server disconnect before connecting " +
msg);
break;
}
this._sock.off('close');
// Delete reference to raw channel to allow cleanup.
this._rawChannel = null;
});
this._sock.on('error', e => Log.Warn("WebSocket on-error event"));
// Slight delay of the actual connection so that the caller has
// time to set up callbacks
setTimeout(this._updateConnectionState.bind(this, 'connecting'));
Log.Debug("<< RFB.constructor"); Log.Debug("<< RFB.constructor");
// ===== PROPERTIES ===== // ===== PROPERTIES =====
this.dragViewport = false; this.dragViewport = false;
this.focusOnClick = true; this.focusOnClick = true;
this.lastActiveAt = Date.now(); this.lastActiveAt = Date.now();
@ -367,6 +329,11 @@ export default class RFB extends EventTargetMixin {
// ===== PROPERTIES ===== // ===== PROPERTIES =====
get translateShortcuts() { return this._keyboard.translateShortcuts; }
set translateShortcuts(value) {
this._keyboard.translateShortcuts = value;
}
get pointerLock() { return this._pointerLock; } get pointerLock() { return this._pointerLock; }
set pointerLock(value) { set pointerLock(value) {
if (!this._pointerLock) { if (!this._pointerLock) {
@ -762,11 +729,87 @@ export default class RFB extends EventTargetMixin {
if (value !== this._hiDpi) { if (value !== this._hiDpi) {
this._hiDpi = value; this._hiDpi = value;
this._requestRemoteResize(); this._requestRemoteResize();
if (this._display.screens.length > 1) {
//force secondary displays to re-register and thus apply new hdpi setting
this._proxyRFBMessage('forceResize', [ value ]);
}
} }
} }
// ===== PUBLIC METHODS ===== // ===== PUBLIC METHODS =====
attachSecondaryDisplay() {
this._updateConnectionState('connecting');
const screen = this._registerSecondaryDisplay();
this._updateConnectionState('connected');
return screen
}
reattachSecondaryDisplay(screen) {
this._updateConnectionState('connecting');
this._registerSecondaryDisplay(screen);
this._updateConnectionState('connected');
return screen
}
applyScreenPlan(screenPlan) {
if (this._isPrimaryDisplay) {
let fullPlan = this._screenSize();
//check plan for validity
let minX = Number.MAX_SAFE_INTEGER, minY = Number.MAX_SAFE_INTEGER;
let numScreensFound = 0;
for (let i = 0; i < screenPlan.screens.length; i++) {
minX = Math.min(minX, screenPlan.screens[i].x);
minY = Math.min(minY, screenPlan.screens[i].y);
for (let z = 0; z < fullPlan.screens.length; z++) {
if (screenPlan.screens[i].screenID == fullPlan.screens[z].screenID) {
numScreensFound++;
}
}
}
if (minX !== 0 || minY !== 0) {
throw new Error("Screen plan invalid, improper coordinates provided.");
}
if (numScreensFound > fullPlan.screens.length) {
throw new Error("Screen plan contained more screens then there are registered.")
} else if (numScreensFound < fullPlan.screens.length) {
throw new Error("Screen plan contained fewer screens then there are registered.")
}
this._display.applyScreenPlan(screenPlan);
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._updateContinuousUpdates();
}
}
getScreenPlan() {
let fullPlan = this._screenSize();
let sanitizedPlan = {
screens: [],
serverWidth: fullPlan.serverWidth,
serverHeight: fullPlan.serverHeight
};
for (let i=0; i < fullPlan.screens.length; i++) {
sanitizedPlan.screens.push(
{
screenID: fullPlan.screens[i].screenID,
serverWidth: fullPlan.screens[i].serverWidth,
serverHeight: fullPlan.screens[i].serverHeight,
x: fullPlan.screens[i].x,
y: fullPlan.screens[i].y,
pixelRatio: fullPlan.screens[i].pixelRatio
}
)
}
return sanitizedPlan;
}
/* /*
This function must be called after changing any properties that effect rendering quality This function must be called after changing any properties that effect rendering quality
*/ */
@ -774,7 +817,9 @@ export default class RFB extends EventTargetMixin {
if (this._rfbConnectionState === 'connected') { if (this._rfbConnectionState === 'connected') {
if (this._pendingApplyVideoRes) { if (this._pendingApplyVideoRes) {
RFB.messages.setMaxVideoResolution(this._sock, this._maxVideoResolutionX, this._maxVideoResolutionY); if (this._isPrimaryDisplay){
RFB.messages.setMaxVideoResolution(this._sock, this._maxVideoResolutionX, this._maxVideoResolutionY);
}
} }
if (this._pendingApplyResolutionChange) { if (this._pendingApplyResolutionChange) {
@ -793,15 +838,16 @@ export default class RFB extends EventTargetMixin {
} }
disconnect() { disconnect() {
this._updateConnectionState('disconnecting'); if (this._isPrimaryDisplay) {
this._sock.off('error'); this._updateConnectionState('disconnecting');
this._sock.off('message'); this._sock.off('error');
this._sock.off('open'); this._sock.off('message');
} this._sock.off('open');
this._proxyRFBMessage('disconnect');
sendCredentials(creds) { } else {
this._rfbCredentials = creds; this._updateConnectionState('disconnecting');
setTimeout(this._initMsg.bind(this), 0); this._unregisterSecondaryDisplay();
}
} }
sendCtrlAltDel() { sendCtrlAltDel() {
@ -851,13 +897,21 @@ export default class RFB extends EventTargetMixin {
Log.Info("Sending key (" + (down ? "down" : "up") + "): keysym " + keysym + ", scancode " + scancode); Log.Info("Sending key (" + (down ? "down" : "up") + "): keysym " + keysym + ", scancode " + scancode);
RFB.messages.QEMUExtendedKeyEvent(this._sock, keysym, down, scancode); if (this._isPrimaryDisplay) {
RFB.messages.QEMUExtendedKeyEvent(this._sock, keysym, down, scancode);
} else {
this._proxyRFBMessage('QEMUExtendedKeyEvent', [ keysym, down, scancode ])
}
} else { } else {
if (!keysym) { if (!keysym) {
return; return;
} }
Log.Info("Sending keysym (" + (down ? "down" : "up") + "): " + keysym); Log.Info("Sending keysym (" + (down ? "down" : "up") + "): " + keysym);
RFB.messages.keyEvent(this._sock, keysym, down ? 1 : 0); if (this._isPrimaryDisplay) {
RFB.messages.keyEvent(this._sock, keysym, down ? 1 : 0);
} else {
this._proxyRFBMessage('keyEvent', [ keysym, down ? 1 : 0 ])
}
} }
} }
@ -909,7 +963,12 @@ export default class RFB extends EventTargetMixin {
let mimes = [ 'text/plain' ]; let mimes = [ 'text/plain' ];
dataset.push(data); dataset.push(data);
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes); if (this._isPrimaryDisplay) {
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes);
} else {
this._proxyRFBMessage('sendBinaryClipboard', [ dataset, mimes ]);
}
} }
async clipboardPasteDataFrom(clipdata) { async clipboardPasteDataFrom(clipdata) {
@ -973,23 +1032,33 @@ export default class RFB extends EventTargetMixin {
if (dataset.length > 0) { if (dataset.length > 0) {
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes); if (this._isPrimaryDisplay) {
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes);
} else {
this._proxyRFBMessage('sendBinaryClipboard', [ dataset, mimes ]);
}
} }
} }
requestBottleneckStats() { requestBottleneckStats() {
RFB.messages.requestStats(this._sock); if (this._isPrimaryDisplay) {
RFB.messages.requestStats(this._sock);
}
} }
subscribeUnixRelay(name, processRelayFn) { subscribeUnixRelay(name, processRelayFn) {
this._unixRelays = this._unixRelays || {}; if (this._isPrimaryDisplay){
this._unixRelays[name] = processRelayFn; this._unixRelays = this._unixRelays || {};
RFB.messages.sendSubscribeUnixRelay(this._sock, name); this._unixRelays[name] = processRelayFn;
RFB.messages.sendSubscribeUnixRelay(this._sock, name);
}
} }
sendUnixRelayData(name, payload) { sendUnixRelayData(name, payload) {
RFB.messages.sendUnixRelay(this._sock, name, payload); if (this._isPrimaryDisplay) {
RFB.messages.sendUnixRelay(this._sock, name, payload);
}
} }
// ===== PRIVATE METHODS ===== // ===== PRIVATE METHODS =====
@ -1003,10 +1072,68 @@ export default class RFB extends EventTargetMixin {
this._transitConnectionState = value; this._transitConnectionState = value;
} }
_setupWebSocket() {
this._sock = new Websock();
this._sock.on('message', () => {
this._handleMessage();
});
this._sock.on('open', () => {
if ((this._rfbConnectionState === 'connecting') &&
(this._rfbInitState === '')) {
this._rfbInitState = 'ProtocolVersion';
Log.Debug("Starting VNC handshake");
} else {
this._fail("Unexpected server connection while " +
this._rfbConnectionState);
}
});
this._sock.on('close', (e) => {
Log.Debug("WebSocket on-close event");
let msg = "";
if (e.code) {
msg = "(code: " + e.code;
if (e.reason) {
msg += ", reason: " + e.reason;
}
msg += ")";
}
switch (this._rfbConnectionState) {
case 'connecting':
this._fail("Connection closed " + msg);
break;
case 'connected':
// Handle disconnects that were initiated server-side
this._updateConnectionState('disconnecting');
this._updateConnectionState('disconnected');
break;
case 'disconnecting':
// Normal disconnection path
this._updateConnectionState('disconnected');
break;
case 'disconnected':
this._fail("Unexpected server disconnect " +
"when already disconnected " + msg);
break;
default:
this._fail("Unexpected server disconnect before connecting " +
msg);
break;
}
this._sock.off('close');
// Delete reference to raw channel to allow cleanup.
this._rawChannel = null;
});
this._sock.on('error', e => Log.Warn("WebSocket on-error event"));
// Slight delay of the actual connection so that the caller has
// time to set up callbacks
setTimeout(this._updateConnectionState.bind(this, 'connecting'));
}
_connect() { _connect() {
Log.Debug(">> RFB.connect"); Log.Debug(">> RFB.connect");
if (this._url) { if (this._url && this._isPrimaryDisplay) {
try { try {
Log.Info(`connecting to ${this._url}`); Log.Info(`connecting to ${this._url}`);
this._sock.open(this._url, this._wsProtocols); this._sock.open(this._url, this._wsProtocols);
@ -1018,7 +1145,7 @@ export default class RFB extends EventTargetMixin {
this._fail("Error when opening socket (" + e + ")"); this._fail("Error when opening socket (" + e + ")");
} }
} }
} else { } else if (this._isPrimaryDisplay) {
try { try {
Log.Info(`attaching ${this._rawChannel} to Websock`); Log.Info(`attaching ${this._rawChannel} to Websock`);
this._sock.attach(this._rawChannel); this._sock.attach(this._rawChannel);
@ -1046,6 +1173,9 @@ export default class RFB extends EventTargetMixin {
window.addEventListener("focus", this._eventHandlers.handleFocusChange); window.addEventListener("focus", this._eventHandlers.handleFocusChange);
window.addEventListener("blur", this._eventHandlers.handleFocusChange); window.addEventListener("blur", this._eventHandlers.handleFocusChange);
//User cursor moves outside of the window
window.addEventListener("mouseover", this._eventHandlers.handleMouseOut);
// In order for the keyboard to not occlude the input being edited // In order for the keyboard to not occlude the input being edited
// we move the hidden input we use for triggering the keyboard to the last click // we move the hidden input we use for triggering the keyboard to the last click
// position which should trigger a page being moved down enough // position which should trigger a page being moved down enough
@ -1085,7 +1215,7 @@ export default class RFB extends EventTargetMixin {
this._resendClipboardNextUserDrivenEvent = true; this._resendClipboardNextUserDrivenEvent = true;
// WebRTC UDP datachannel inits // WebRTC UDP datachannel inits
if (typeof RTCPeerConnection !== 'undefined') { if (typeof RTCPeerConnection !== 'undefined' && this._isPrimaryDisplay) {
this._udpBuffer = new Map(); this._udpBuffer = new Map();
this._udpPeer = new RTCPeerConnection({ this._udpPeer = new RTCPeerConnection({
@ -1190,7 +1320,7 @@ export default class RFB extends EventTargetMixin {
} }
} }
if (this._useUdp && typeof RTCPeerConnection !== 'undefined') { if (this._useUdp && typeof RTCPeerConnection !== 'undefined' && this._isPrimaryDisplay) {
setTimeout(function() { this._sendUdpUpgrade() }.bind(this), 3000); setTimeout(function() { this._sendUdpUpgrade() }.bind(this), 3000);
} }
@ -1224,7 +1354,17 @@ export default class RFB extends EventTargetMixin {
window.removeEventListener('focus', this._eventHandlers.handleFocusChange); window.removeEventListener('focus', this._eventHandlers.handleFocusChange);
this._keyboard.ungrab(); this._keyboard.ungrab();
this._gestures.detach(); this._gestures.detach();
this._sock.close(); if (this._isPrimaryDisplay) {
this._sock.close();
} else {
if (this._primaryDisplayChannel) {
this._primaryDisplayChannel.postMessage({eventType: 'unregister', screenID: this._screenID})
this._primaryDisplayChannel.removeEventListener('message', this._handleSecondaryDisplayMessage);
this._primaryDisplayChannel.close();
this._primaryDisplayChannel = null;
}
}
try { try {
this._target.removeChild(this._screen); this._target.removeChild(this._screen);
} catch (e) { } catch (e) {
@ -1325,7 +1465,7 @@ export default class RFB extends EventTargetMixin {
// When clipping is enabled, the screen is limited to // When clipping is enabled, the screen is limited to
// the size of the container. // the size of the container.
const size = this._screenSize(); const size = this._screenSize();
this._display.viewportChangeSize(size.w, size.h); this._display.viewportChangeSize(size.screens[0].serverWidth, size.screens[0].serverHeight);
this._fixScrollbars(); this._fixScrollbars();
} }
} }
@ -1335,7 +1475,7 @@ export default class RFB extends EventTargetMixin {
this._display.scale = 1.0; this._display.scale = 1.0;
} else { } else {
const size = this._screenSize(false); const size = this._screenSize(false);
this._display.autoscale(size.w, size.h, size.scale); this._display.autoscale(size.screens[0].serverWidth, size.screens[0].serverHeight, size.screens[0].scale);
} }
this._fixScrollbars(); this._fixScrollbars();
} }
@ -1346,60 +1486,29 @@ export default class RFB extends EventTargetMixin {
clearTimeout(this._resizeTimeout); clearTimeout(this._resizeTimeout);
this._resizeTimeout = null; this._resizeTimeout = null;
if (!this._resizeSession || this._viewOnly || if (this._isPrimaryDisplay) {
!this._supportsSetDesktopSize) { if (!this._resizeSession || this._viewOnly ||
return; !this._supportsSetDesktopSize) {
} return;
const size = this._screenSize(); }
RFB.messages.setDesktopSize(this._sock, const size = this._screenSize();
Math.floor(size.w), Math.floor(size.h), RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._screenID, this._screenFlags);
Log.Debug('Requested new desktop size: ' + Log.Debug('Requested new desktop size: ' +
size.w + 'x' + size.h); size.serverWidth + 'x' + size.serverHeight);
} else if (this._display.screenIndex > 0) {
//re-register the secondary display with new resolution
this._registerSecondaryDisplay();
}
if (this._display.screens.length > 1) {
this.dispatchEvent(new CustomEvent("screenregistered", {}));
}
} }
// Gets the the size of the available screen // Gets the the size of the available screen
_screenSize (limited) { _screenSize (limited) {
if (limited === undefined) { return this._display.getScreenSize(this.videoQuality, this.forcedResolutionX, this.forcedResolutionY, this._hiDpi, limited);
limited = true;
}
var x = this.forcedResolutionX || this._screen.offsetWidth;
var y = this.forcedResolutionY || this._screen.offsetHeight;
var scale = 0; // 0=auto
try {
if (x > 1280 && limited && this.videoQuality == 1) {
var ratio = y / x;
Log.Debug(ratio);
x = 1280;
y = x * ratio;
}
else if (limited && this.videoQuality == 0){
x = 1280;
y = 720;
} else if (this._hiDpi == true) {
x = x * window.devicePixelRatio;
y = y * window.devicePixelRatio;
scale = 1 / window.devicePixelRatio;
} else if (this._display.antiAliasing === 0 && window.devicePixelRatio > 1 && x < 1000 && x > 0) {
// small device with high resolution, browser is essentially zooming greater than 200%
Log.Info('Device Pixel ratio: ' + window.devicePixelRatio + ' Reported Resolution: ' + x + 'x' + y);
let targetDevicePixelRatio = 1.5;
if (window.devicePixelRatio > 2) { targetDevicePixelRatio = 2; }
let scaledWidth = (x * window.devicePixelRatio) * (1 / targetDevicePixelRatio);
let scaleRatio = scaledWidth / x;
x = x * scaleRatio;
y = y * scaleRatio;
scale = 1 / scaleRatio;
Log.Info('Small device with hDPI screen detected, auto scaling at ' + scaleRatio + ' to ' + x + 'x' + y);
}
} catch (err) {
Log.Debug(err);
}
return { w: x,
h: y,
scale: scale };
} }
_fixScrollbars() { _fixScrollbars() {
@ -1469,6 +1578,10 @@ export default class RFB extends EventTargetMixin {
} }
break; break;
case 'proxied':
//secondary display that needs to proxy messages through the broadcast channel
break;
default: default:
Log.Error("Unknown connection state: " + state); Log.Error("Unknown connection state: " + state);
return; return;
@ -1486,7 +1599,9 @@ export default class RFB extends EventTargetMixin {
this._disconnTimer = null; this._disconnTimer = null;
// make sure we don't get a double event // make sure we don't get a double event
this._sock.off('close'); if (this._isPrimaryDisplay) {
this._sock.off('close');
}
} }
switch (state) { switch (state) {
@ -1504,6 +1619,7 @@ export default class RFB extends EventTargetMixin {
this._disconnTimer = setTimeout(() => { this._disconnTimer = setTimeout(() => {
Log.Error("Disconnection timed out."); Log.Error("Disconnection timed out.");
this._updateConnectionState('disconnected'); this._updateConnectionState('disconnected');
this._proxyRFBMessage('secondarydisconnected')
}, DISCONNECT_TIMEOUT * 1000); }, DISCONNECT_TIMEOUT * 1000);
break; break;
@ -1550,6 +1666,146 @@ export default class RFB extends EventTargetMixin {
{ detail: { capabilities: this._capabilities } })); { detail: { capabilities: this._capabilities } }));
} }
_proxyRFBMessage(messageType, data) {
let message = {
eventType: messageType,
args: data,
screenId: this._display.screenId,
screenIndex: this._display.screenIndex,
mouseLastScreenIndex: this._mouseLastScreenIndex,
}
this._controlChannel.postMessage(message);
}
_handleControlMessage(event) {
if (this._isPrimaryDisplay) {
// Secondary to Primary screen message
switch (event.data.eventType) {
case 'register':
this._display.addScreen(event.data.screenID, event.data.width, event.data.height, event.data.pixelRatio, event.data.containerHeight, event.data.containerWidth);
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._sendEncodings();
this._updateContinuousUpdates();
this.dispatchEvent(new CustomEvent("screenregistered", {}));
Log.Info(`Secondary monitor (${event.data.screenID}) has been registered.`);
break;
case 'reattach':
console.log('reattach message')
console.log(event.data)
this._display.addScreen(event.data.screenID, event.data.width, event.data.height, event.data.pixelRatio, event.data.containerHeight, event.data.containerWidth);
this.dispatchEvent(new CustomEvent("screenregistered", {}));
Log.Info(`Secondary monitor (${event.data.screenID}) has been reattached.`);
break;
case 'unregister':
if (this._display.removeScreen(event.data.screenID)) {
this.dispatchEvent(new CustomEvent("screenregistered", {}));
Log.Info(`Secondary monitor (${event.data.screenID}) has been removed.`);
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._sendEncodings();
this._updateContinuousUpdates();
this.dispatchEvent(new CustomEvent("screenregistered", {}));
} else {
Log.Info(`Secondary monitor (${event.data.screenID}) not found.`);
}
break;
case 'pointerEvent':
let coords = this._display.getServerRelativeCoordinates(event.data.screenIndex, event.data.args[0], event.data.args[1]);
this._mouseLastScreenIndex = event.data.screenIndex;
event.data.args[0] = coords[0];
event.data.args[1] = coords[1];
RFB.messages.pointerEvent(this._sock, ...event.data.args);
break;
case 'keyEvent':
RFB.messages.keyEvent(this._sock, ...event.data.args);
break;
case 'sendBinaryClipboard':
RFB.messages.sendBinaryClipboard(this._sock, ...event.data.args);
break;
// The following are primary to secondary messages that should be ignored on the primary
case 'updateCursor':
break;
}
} else {
// Primary to secondary screen message
switch (event.data.eventType) {
case 'updateCursor':
if (event.data.mouseLastScreenIndex === this._display.screenIndex || this._mouseLastScreenIndex === -1) {
this._updateCursor(...event.data.args);
this._mouseLastScreenIndex = event.data.mouseLastScreenIndex;
}
break;
case 'disconnect':
this.disconnect();
break;
case 'forceResize':
this._hiDpi = event.data.args[0];
this._updateScale();
this._requestRemoteResize();
break;
}
}
}
_unregisterSecondaryDisplay() {
if (!this._isPrimaryDisplay){
let message = {
eventType: 'unregister',
screenID: this._display.screenId
}
this._controlChannel.postMessage(message);
}
}
_registerSecondaryDisplay(currentScreen = false) {
if (!this._isPrimaryDisplay) {
//let screen = this._screenSize().screens[0];
//
let size = this._screenSize();
this._display.resize(size.screens[0].containerWidth, size.screens[0].containerHeight);
this._display.autoscale(size.screens[0].containerWidth, size.screens[0].containerHeight, size.screens[0].scale);
screen = this._screenSize().screens[0];
const registertype = (currentScreen) ? 'reattach' : 'register'
let message = {
eventType: registertype,
screenID: screen.screenID,
width: screen.width,
height: screen.height,
x: currentScreen.x || 0,
y: currentScreen.y || 0,
pixelRatio: screen.pixelRatio,
containerWidth: screen.containerWidth,
containerHeight: screen.containerHeight,
channel: null
}
this._controlChannel.postMessage(message);
if (!this._viewOnly) { this._keyboard.grab(); }
// return screen.screenID
return screen
}
}
identify(screens) {
let message = {
eventType: 'identify',
screens
}
this._controlChannel.postMessage(message);
}
_handleSecondaryDisplayMessage(event) {
if (this._isPrimaryDisplay) {
}
}
_handleMessage() { _handleMessage() {
if (this._sock.rQlen === 0) { if (this._sock.rQlen === 0) {
Log.Warn("handleMessage called on an empty receive queue"); Log.Warn("handleMessage called on an empty receive queue");
@ -1583,6 +1839,18 @@ export default class RFB extends EventTargetMixin {
this.sendKey(keysym, code, down); this.sendKey(keysym, code, down);
} }
_handleMouseOut(ev) {
if (ev.toElement !== null && ev.relatedTarget === null) {
//mouse was outside of the window and just came in, this is our chance to do things
//Ensure the window was not moved to a different screen with a different pixel ratio
if (this._display.screens[0].pixelRatio !== window.devicePixelRatio) {
Log.Debug("Window moved to another screen with different pixel ratio, sending resize request.");
this._requestRemoteResize();
}
}
}
_handleMouse(ev) { _handleMouse(ev) {
/* /*
* We don't check connection status or viewOnly here as the * We don't check connection status or viewOnly here as the
@ -1638,6 +1906,7 @@ export default class RFB extends EventTargetMixin {
this._canvas); this._canvas);
} }
this._mouseLastScreenIndex = this._display.screenIndex;
this._setLastActive(); this._setLastActive();
const mappedButton = this.mouseButtonMapper.get(ev.button); const mappedButton = this.mouseButtonMapper.get(ev.button);
switch (ev.type) { switch (ev.type) {
@ -1666,6 +1935,9 @@ export default class RFB extends EventTargetMixin {
false, xvncButtonToMask(mappedButton)); false, xvncButtonToMask(mappedButton));
break; break;
case 'mousemove': case 'mousemove':
//when there are multiple screens
//This window can get mouse move events when the cursor is outside of the window, if the mouse is down
//when the cursor crosses the threshold of the window
this._handleMouseMove(pos.x, pos.y); this._handleMouseMove(pos.x, pos.y);
break; break;
} }
@ -1787,19 +2059,22 @@ export default class RFB extends EventTargetMixin {
var rel_16_x = toSignedRelative16bit(x - this._pointerLockPos.x); var rel_16_x = toSignedRelative16bit(x - this._pointerLockPos.x);
var rel_16_y = toSignedRelative16bit(y - this._pointerLockPos.y); var rel_16_y = toSignedRelative16bit(y - this._pointerLockPos.y);
//console.log("new_pos x" + x + ", y" + y); if (this._isPrimaryDisplay){
//console.log("lock x " + this._pointerLockPos.x + ", y " + this._pointerLockPos.y); RFB.messages.pointerEvent(this._sock, rel_16_x, rel_16_y, mask);
//console.log("rel x " + rel_16_x + ", y " + rel_16_y); } else {
this._proxyRFBMessage('pointerEvent', [ rel_16_x, rel_16_y, mask ]);
RFB.messages.pointerEvent(this._sock, rel_16_x, }
rel_16_y, mask);
// reset the cursor position to center // reset the cursor position to center
this._mousePos = { x: this._pointerLockPos.x , y: this._pointerLockPos.y }; this._mousePos = { x: this._pointerLockPos.x , y: this._pointerLockPos.y };
this._cursor.move(this._pointerLockPos.x, this._pointerLockPos.y); this._cursor.move(this._pointerLockPos.x, this._pointerLockPos.y);
} else { } else {
RFB.messages.pointerEvent(this._sock, this._display.absX(x), if (this._isPrimaryDisplay) {
this._display.absY(y), mask); RFB.messages.pointerEvent(this._sock, this._display.absX(x), this._display.absY(y), mask);
} else {
this._proxyRFBMessage('pointerEvent', [ this._display.absX(x), this._display.absY(y), mask ]);
}
} }
} }
@ -1808,7 +2083,12 @@ export default class RFB extends EventTargetMixin {
if (this._rfbConnectionState !== 'connected') { return; } if (this._rfbConnectionState !== 'connected') { return; }
if (this._viewOnly) { return; } // View only, skip mouse events if (this._viewOnly) { return; } // View only, skip mouse events
RFB.messages.pointerEvent(this._sock, this._display.absX(x), this._display.absY(y), 0, dX, dY); if (this._isPrimaryDisplay){
RFB.messages.pointerEvent(this._sock, this._display.absX(x), this._display.absY(y), 0, dX, dY);
} else {
this._proxyRFBMessage('pointerEvent', [ this._display.absX(x), this._display.absY(y), 0, dX, dY ]);
}
} }
_handleWheel(ev) { _handleWheel(ev) {
@ -2585,7 +2865,10 @@ export default class RFB extends EventTargetMixin {
const encs = []; const encs = [];
// In preference order // In preference order
encs.push(encodings.encodingCopyRect); // Disable copyrect when using multiple displays
if (this._display.screens.length === 1) {
encs.push(encodings.encodingCopyRect);
}
// Only supported with full depth support // Only supported with full depth support
if (this._fbDepth == 24) { if (this._fbDepth == 24) {
encs.push(encodings.encodingTight); encs.push(encodings.encodingTight);
@ -3244,9 +3527,9 @@ export default class RFB extends EventTargetMixin {
const payload = this._sock.rQshiftStr(len); const payload = this._sock.rQshiftStr(len);
if (status) { if (status) {
console.log("Unix relay subscription succeeded"); Log.Info("Unix relay subscription succeeded");
} else { } else {
console.log("Unix relay subscription failed, " + payload); Log.Warn("Unix relay subscription failed, " + payload);
} }
} }
@ -3570,7 +3853,7 @@ export default class RFB extends EventTargetMixin {
for (let i = 0; i < numberOfScreens; i += 1) { for (let i = 0; i < numberOfScreens; i += 1) {
// Save the id and flags of the first screen // Save the id and flags of the first screen
if (i === 0) { if (i === 0) {
this._screenID = this._sock.rQshiftBytes(4); // id this._screenIndex = this._sock.rQshiftBytes(4); // id
this._sock.rQskipBytes(2); // x-position this._sock.rQskipBytes(2); // x-position
this._sock.rQskipBytes(2); // y-position this._sock.rQskipBytes(2); // y-position
this._sock.rQskipBytes(2); // width this._sock.rQskipBytes(2); // width
@ -3687,7 +3970,12 @@ export default class RFB extends EventTargetMixin {
rgbaPixels: rgba, rgbaPixels: rgba,
hotx: hotx, hoty: hoty, w: w, h: h, hotx: hotx, hoty: hoty, w: w, h: h,
}; };
this._refreshCursor(); this._refreshCursor();
if (this._isPrimaryDisplay) {
this._proxyRFBMessage('updateCursor', [ rgba, hotx, hoty, w, h ]);
}
} }
_shouldShowDotCursor() { _shouldShowDotCursor() {
@ -4061,40 +4349,49 @@ RFB.messages = {
} }
}, },
setDesktopSize(sock, width, height, id, flags) { setDesktopSize(sock, size, flags) {
const buff = sock._sQ; const buff = sock._sQ;
const offset = sock._sQlen; const offset = sock._sQlen;
buff[offset] = 251; // msg-type buff[offset] = 251; // msg-type
buff[offset + 1] = 0; // padding buff[offset + 1] = 0; // padding
buff[offset + 2] = width >> 8; // width buff[offset + 2] = size.serverWidth >> 8; // width
buff[offset + 3] = width; buff[offset + 3] = size.serverWidth;
buff[offset + 4] = height >> 8; // height buff[offset + 4] = size.serverHeight >> 8; // height
buff[offset + 5] = height; buff[offset + 5] = size.serverHeight;
buff[offset + 6] = 1; // number-of-screens buff[offset + 6] = size.screens.length; // number-of-screens
buff[offset + 7] = 0; // padding buff[offset + 7] = 0; // padding
// screen array let i = 8;
buff[offset + 8] = id >> 24; // id for (let iS = 0; iS < size.screens.length; iS++) {
buff[offset + 9] = id >> 16; //screen id
buff[offset + 10] = id >> 8; buff[offset + i++] = iS >> 24;
buff[offset + 11] = id; buff[offset + i++] = iS >> 16;
buff[offset + 12] = 0; // x-position buff[offset + i++] = iS >> 8;
buff[offset + 13] = 0; buff[offset + i++] = iS;
buff[offset + 14] = 0; // y-position //screen x position
buff[offset + 15] = 0; buff[offset + i++] = size.screens[iS].x >> 8;
buff[offset + 16] = width >> 8; // width buff[offset + i++] = size.screens[iS].x;
buff[offset + 17] = width; //screen y position
buff[offset + 18] = height >> 8; // height buff[offset + i++] = size.screens[iS].y >> 8;
buff[offset + 19] = height; buff[offset + i++] = size.screens[iS].y;
buff[offset + 20] = flags >> 24; // flags //screen width
buff[offset + 21] = flags >> 16; buff[offset + i++] = size.screens[iS].serverWidth >> 8;
buff[offset + 22] = flags >> 8; buff[offset + i++] = size.screens[iS].serverWidth;
buff[offset + 23] = flags; //screen height
buff[offset + i++] = size.screens[iS].serverHeight >> 8;
buff[offset + i++] = size.screens[iS].serverHeight;
//flags
buff[offset + i++] = flags >> 24;
buff[offset + i++] = flags >> 16;
buff[offset + i++] = flags >> 8;
buff[offset + i++] = flags;
}
sock._sQlen += 24; sock._sQlen += i;
sock.flush(); sock.flush();
}, },
setMaxVideoResolution(sock, width, height) { setMaxVideoResolution(sock, width, height) {

16
core/util/element.js
View File

@ -28,5 +28,21 @@ export function clientToElement(x, y, elem) {
} else { } else {
pos.y = y - bounds.top; pos.y = y - bounds.top;
} }
//multiple KasmVNC screens, Window can still receive mouse events when cursor goes
//outside of the window if the mouse is down while the moving occurs
if (x > window.innerWidth) {
pos.x += (x - window.innerWidth);
}
else if (x < 0) {
pos.x = x + bounds.left;
}
if (y > window.innerHeight) {
pos.y += (y - window.innerHeight);
}
else if (y < 0) {
pos.y = y + bounds.top;
}
return pos; return pos;
} }

6
core/util/strings.js
View File

@ -26,3 +26,9 @@ export function decodeUTF8(utf8string, allowLatin1=false) {
export function encodeUTF8(DOMString) { export function encodeUTF8(DOMString) {
return unescape(encodeURIComponent(DOMString)); return unescape(encodeURIComponent(DOMString));
} }
export function uuidv4() {
return ([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g, c =>
(c ^ crypto.getRandomValues(new Uint8Array(1))[0] & 15 >> c / 4).toString(16)
);
}

110
screen.html Normal file
View File

@ -0,0 +1,110 @@
<!DOCTYPE html>
<html lang="en" class="noVNC_loading">
<head>
<!--
noVNC is licensed under the MPL 2.0 (see LICENSE.txt)
This file is licensed under the 2-Clause BSD license (see LICENSE.txt).
-->
<title>Additional Display</title>
<meta charset="utf-8">
<!-- Icons (see app/images/icons/Makefile for what the sizes are for) -->
<link rel="icon" sizes="16x16" type="image/png" href="app/images/icons/368_kasm_logo_only_16x16.png">
<link rel="icon" sizes="24x24" type="image/png" href="app/images/icons/368_kasm_logo_only_24x24.png">
<link rel="icon" sizes="32x32" type="image/png" href="app/images/icons/368_kasm_logo_only_32x32.png">
<link rel="icon" sizes="48x48" type="image/png" href="app/images/icons/368_kasm_logo_only_48x48.png">
<link rel="icon" sizes="60x60" type="image/png" href="app/images/icons/368_kasm_logo_only_60x60.png">
<link rel="icon" sizes="64x64" type="image/png" href="app/images/icons/368_kasm_logo_only_64x64.png">
<link rel="icon" sizes="72x72" type="image/png" href="app/images/icons/368_kasm_logo_only_72x72.png">
<link rel="icon" sizes="76x76" type="image/png" href="app/images/icons/368_kasm_logo_only_76x76.png">
<link rel="icon" sizes="96x96" type="image/png" href="app/images/icons/368_kasm_logo_only_96x96.png">
<link rel="icon" sizes="120x120" type="image/png" href="app/images/icons/368_kasm_logo_only_120x120.png">
<link rel="icon" sizes="144x144" type="image/png" href="app/images/icons/368_kasm_logo_only_144x144.png">
<link rel="icon" sizes="152x152" type="image/png" href="app/images/icons/368_kasm_logo_only_152x152.png">
<link rel="icon" sizes="192x192" type="image/png" href="app/images/icons/368_kasm_logo_only_192x192.png">
<!-- Firefox currently mishandles SVG, see #1419039
<link rel="icon" sizes="any" type="image/svg+xml" href="app/images/icons/kasm-icon.svg">
-->
<!-- Repeated last so that legacy handling will pick this -->
<link rel="icon" sizes="16x16" type="image/png" href="app/images/icons/368_kasm_logo_only_16x16.png">
<!-- Apple iOS Safari settings -->
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<!-- Home Screen Icons (favourites and bookmarks use the normal icons) -->
<link rel="apple-touch-icon" sizes="60x60" type="image/png" href="app/images/icons/368_kasm_logo_only_60x60.png">
<link rel="apple-touch-icon" sizes="76x76" type="image/png" href="app/images/icons/368_kasm_logo_only_76x76.png">
<link rel="apple-touch-icon" sizes="120x120" type="image/png" href="app/images/icons/368_kasm_logo_only_120x120.png">
<link rel="apple-touch-icon" sizes="152x152" type="image/png" href="app/images/icons/368_kasm_logo_only_152x152.png">
<script src="vendor/interact.min.js"></script>
<!-- Stylesheets -->
<link rel="stylesheet" href="app/styles/base.css">
<script src="app/error-handler.js"></script>
<script>
let isInsideKasmVDI = false;
try {
isInsideKasmVDI = (window.self !== window.top);
} catch (e) {
isInsideKasmVDI = true;
}
</script>
<script type="module" crossorigin="use-credentials" src="app/ui_screen.js"></script>
</head>
<body>
<div id="noVNC_fallback_error" class="noVNC_center">
<div>
<div id="noVNC_close_error" onclick="document.getElementById('noVNC_fallback_error').remove()"></div>
<div>KasmVNC encountered an error:</div>
<br>
<div id="noVNC_fallback_errormsg"></div>
</div>
</div>
<!-- Status Dialog -->
<div id="noVNC_status"></div>
<!-- Connect button -->
<div class="noVNC_center">
<div id="noVNC_connect_dlg">
<div id="noVNC_connect_button">
<div class="image display">
<svg xmlns="http://www.w3.org/2000/svg" width="45"
viewBox="0 0 576 512"><!--! Font Awesome Pro 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. -->
<path
d="M64 16C37.5 16 16 37.5 16 64V352c0 26.5 21.5 48 48 48H231.8h.3H343.9h.3H512c26.5 0 48-21.5 48-48V64c0-26.5-21.5-48-48-48H64zM222.6 416H64c-35.3 0-64-28.7-64-64V64C0 28.7 28.7 0 64 0H512c35.3 0 64 28.7 64 64V352c0 35.3-28.7 64-64 64H353.4l13.3 80H440c4.4 0 8 3.6 8 8s-3.6 8-8 8H360 216 136c-4.4 0-8-3.6-8-8s3.6-8 8-8h73.2l13.3-80zm16.2 0l-13.3 80H350.6l-13.3-80H238.8zM496 80H80l0 256H496V80zM80 64H496c8.8 0 16 7.2 16 16V336c0 8.8-7.2 16-16 16H80c-8.8 0-16-7.2-16-16V80c0-8.8 7.2-16 16-16zm165.7 77.7L179.3 208l66.3 66.3c3.1 3.1 3.1 8.2 0 11.3s-8.2 3.1-11.3 0l-72-72c-3.1-3.1-3.1-8.2 0-11.3l72-72c3.1-3.1 8.2-3.1 11.3 0s3.1 8.2 0 11.3zm96-11.3l72 72c3.1 3.1 3.1 8.2 0 11.3l-72 72c-3.1 3.1-8.2 3.1-11.3 0s-3.1-8.2 0-11.3L396.7 208l-66.3-66.3c-3.1-3.1-3.1-8.2 0-11.3s8.2-3.1 11.3 0z" />
</svg>
<div class="power">
<svg xmlns="http://www.w3.org/2000/svg" height="25px" viewBox="0 0 512 512"><!--! Font Awesome Pro 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. --><path d="M272 16c0-8.8-7.2-16-16-16s-16 7.2-16 16V272c0 8.8 7.2 16 16 16s16-7.2 16-16V16zM164.3 78.2c7.8-4.1 10.9-13.7 6.9-21.6s-13.7-10.9-21.6-6.9C75 88.3 24 166.2 24 256c0 128.1 103.9 232 232 232s232-103.9 232-232c0-89.8-51-167.7-125.6-206.2c-7.8-4.1-17.5-1-21.6 6.9s-1 17.5 6.9 21.6C412.1 111.5 456 178.6 456 256c0 110.5-89.5 200-200 200S56 366.5 56 256c0-77.4 43.9-144.5 108.3-177.8z"/></svg>
</div>
</div>
<div class="text">
<div class="heading">Additional Display</div>
<div>Click to connect this additional display</div>
</div>
<div class="image go">
<svg xmlns="http://www.w3.org/2000/svg" height="20px" viewBox="0 0 448 512"><!--! Font Awesome Pro 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. --><path d="M443.3 267.3c6.2-6.2 6.2-16.4 0-22.6l-176-176c-6.2-6.2-16.4-6.2-22.6 0s-6.2 16.4 0 22.6L393.4 240 16 240c-8.8 0-16 7.2-16 16s7.2 16 16 16l377.4 0L244.7 420.7c-6.2 6.2-6.2 16.4 0 22.6s16.4 6.2 22.6 0l176-176z"/></svg>
</div>
</div>
</div>
</div>
<!-- This is where the RFB elements will attach -->
<div id="noVNC_container">
<!-- Note that Google Chrome on Android doesn't respect any of these,
html attributes which attempt to disable text suggestions on the
on-screen keyboard. Let's hope Chrome implements the ime-mode
style for example -->
<textarea id="noVNC_keyboardinput" autocapitalize="off"
autocomplete="off" spellcheck="false" tabindex="-1"></textarea>
</div>
<div id="noVNC_identify_monitor">0</div>
</body>

2
tests/README.md
View File

@ -47,3 +47,5 @@ This table keeps track of performance of pre-defined recordings, defined in the
| losangeles.1 | 08233e6 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2272ms | | losangeles.1 | 08233e6 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2272ms |
| newyork.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2273ms | | newyork.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2273ms |
| losangeles.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 1847ms | | losangeles.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 1847ms |
| newyork.1 | 4a6aa73 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 119 | False | 2128ms |
| losangeles.1 | 4a6aa73 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 119 | False | 1766ms |

65
vnc.html
View File

@ -50,7 +50,7 @@
<script src="vendor/interact.min.js"></script> <script src="vendor/interact.min.js"></script>
<!-- Stylesheets --> <!-- Stylesheets -->
<!--link rel="stylesheet" href="app/styles/base.css"> <link rel="stylesheet" href="app/styles/base.css">
<script src="app/error-handler.js"></script> <script src="app/error-handler.js"></script>
@ -63,7 +63,7 @@
} }
</script> </script>
<script type="module" crossorigin="use-credentials" src="app/ui.js"></script--> <script type="module" crossorigin="use-credentials" src="app/ui.js"></script>
</head> </head>
<body> <body>
@ -185,6 +185,15 @@
Fullscreen Fullscreen
</div> </div>
<!-- Add Second Screen -->
<div class="noVNC_button_div noVNC_hidden" >
<input type="image" alt="Fullscreen" src="app/images/desktop-regular.svg"
id="noVNC_displays_button" class="noVNC_button"
style="margin: 10px 3px;"
title="Show Displays">
Displays
</div>
<!-- Toggle game mode --> <!-- Toggle game mode -->
<div class="noVNC_button_div noVNC_hidden noVNC_hide_on_disconnect" > <div class="noVNC_button_div noVNC_hidden noVNC_hide_on_disconnect" >
<input type="image" alt="Game Mode" src="app/images/gamepad.png" <input type="image" alt="Game Mode" src="app/images/gamepad.png"
@ -290,12 +299,6 @@
<span class="slider-label">Toggle Control Panel via Keystrokes</span> <span class="slider-label">Toggle Control Panel via Keystrokes</span>
</label> </label>
</li> </li>
<li>
<label class="switch"><input id="noVNC_setting_enable_hidpi" type="checkbox" />
<span class="slider round"></span>
<span class="slider-label">Render Native Resolution</span>
</label>
</li>
<li class="noVNC_hidden"> <li class="noVNC_hidden">
<label for="noVNC_setting_idle_disconnect">Idle Timeout:</label> <label for="noVNC_setting_idle_disconnect">Idle Timeout:</label>
<select id="noVNC_setting_idle_disconnect" name="vncIdleDisconnect"> <select id="noVNC_setting_idle_disconnect" name="vncIdleDisconnect">
@ -520,7 +523,7 @@
</div> </div>
<!-- Connection Controls --> <!-- Connection Controls -->
<div class="noVNC_button_div noVNC_hide_on_connect" id="noVNC_connect_button" > <div class="noVNC_button_div noVNC_hide_on_connect" id="noVNC_connect_button_2">
<input type="image" alt="Connect" src="app/images/connect.svg" <input type="image" alt="Connect" src="app/images/connect.svg"
class="noVNC_button" class="noVNC_button"
title="Connect"> title="Connect">
@ -537,6 +540,30 @@
<!-- Status Dialog --> <!-- Status Dialog -->
<div id="noVNC_status"></div> <div id="noVNC_status"></div>
<!-- Status Dialog -->
<div id="noVNC_displays">
<div class="canvas">
<div class="canvas-title">Arrange Displays</div>
<div class="canvas-text">Drag and drop to arrange displays, new monitors are added to the right hand side of the previous monitor.</div>
<div id="noVNC_refreshMonitors"><svg id="noVNC_refreshMonitors_icon" xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 512 512"><path class="fa-primary" d="M105.1 202.6c7.7-21.8 20.2-42.3 37.8-59.8c62.5-62.5 163.8-62.5 226.3 0L386.3 160H336c-17.7 0-32 14.3-32 32s14.3 32 32 32H463.5c0 0 0 0 0 0h.4c17.7 0 32-14.3 32-32V64c0-17.7-14.3-32-32-32s-32 14.3-32 32v51.2L414.4 97.6c-87.5-87.5-229.3-87.5-316.8 0C73.2 122 55.6 150.7 44.8 181.4c-5.9 16.7 2.9 34.9 19.5 40.8s34.9-2.9 40.8-19.5z"/><path class="fa-secondary" d="M80 396.9V448c0 17.7-14.3 32-32 32s-32-14.3-32-32V320c0-17.7 14.3-32 32-32H176c17.7 0 32 14.3 32 32s-14.3 32-32 32H125.6l17.2 17.1c62.5 62.5 163.8 62.5 226.3 0c17.5-17.5 30.1-38 37.8-59.8c5.9-16.7 24.2-25.4 40.8-19.5s25.4 24.2 19.5 40.8c-10.8 30.6-28.4 59.3-52.9 83.8c-87.5 87.5-229.3 87.5-316.7 0L80 396.9z"/></svg></div>
<canvas id="noVNC_multiMonitorWidget" class="" width="700" height="230"></canvas>
<div class="arrange-buttons">
<div style="gap: 10px;" class="flex">
<button id="noVNC_addMonitor">
<svg xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 448 512"><path fill="currentColor" d="M240 64c0-8.8-7.2-16-16-16s-16 7.2-16 16V240H32c-8.8 0-16 7.2-16 16s7.2 16 16 16H208V448c0 8.8 7.2 16 16 16s16-7.2 16-16V272H416c8.8 0 16-7.2 16-16s-7.2-16-16-16H240V64z"/></svg>
Add Monitor
</button>
<label id="noVNC_setting_enable_hidpi_option" class="button">
<input style="margin: 0 10px 0 3px;" id="noVNC_setting_enable_hidpi" type="checkbox" />
Native Resolution
</label>
<button id="noVNC_identify_monitors_button">Identify</button>
</div>
<button id="noVNC_close_displays">Done</button>
</div>
</div>
</div>
<!-- Connect button --> <!-- Connect button -->
<div class="noVNC_center"> <div class="noVNC_center">
<div id="noVNC_connect_dlg"> <div id="noVNC_connect_dlg">
@ -548,25 +575,6 @@
</div> </div>
</div> </div>
<!-- Password Dialog -->
<div class="noVNC_center noVNC_connect_layer">
<div id="noVNC_credentials_dlg" class="noVNC_panel"><form>
<ul>
<li id="noVNC_username_block">
<label>Username:</label>
<input id="noVNC_username_input">
</li>
<li id="noVNC_password_block">
<label>Password:</label>
<input id="noVNC_password_input" type="password">
</li>
<li>
<input id="noVNC_credentials_button" type="submit" value="Send Credentials" class="noVNC_submit">
</li>
</ul>
</form></div>
</div>
<!-- Transition Screens --> <!-- Transition Screens -->
<div id="noVNC_transition"> <div id="noVNC_transition">
<div id="noVNC_transition_text"></div> <div id="noVNC_transition_text"></div>
@ -603,5 +611,6 @@
<div id="noVNC_keyboard_control_handle" class="button keyboard handle"></div> <div id="noVNC_keyboard_control_handle" class="button keyboard handle"></div>
</div> </div>
<div id="noVNC_identify_monitor">0</div>
</body> </body>
</html> </html>