jcarr
/
noVNC
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #78 from kasmtech/feature/KASM-5078_multi_monitor_display_ui 

KASM-5078 multi monitor display UI
This commit is contained in:
Richard Koliser 2023-12-18 23:40:45 -05:00 committed by GitHub
parent 68135beedd db2addddce
commit 860ead2fac
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
14 changed files with 2149 additions and 387 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -24,7 +24,7 @@ For support with KasmVNC, post on the [KasmVNC Project](https://github.com/kasmt
- Automatic mixing of webp and jpeg based on CPU availability on server
- WebRTC UDP Transit
- Lossless QOI Image format for Local LAN
- [Dynamic jpeg/webp image coompression](https://github.com/kasmtech/KasmVNC/wiki/Video-Rendering-Options#dynamic-image-quality) quality settings based on screen change rates
- [Dynamic jpeg/webp image compression](https://github.com/kasmtech/KasmVNC/wiki/Video-Rendering-Options#dynamic-image-quality) quality settings based on screen change rates
- Seemless clipboard support (on Chromium based browsers)
- Binary clipboard support for text, images, and formatted text (on Chromium based browsers)
- Allow client to set/change most configuration settings

1
app/images/desktop-regular.svg Normal file
View File

@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 576 512"><path fill="#fff" d="M512 48H64c-8.8 0-16 7.2-16 16V256H528V64c0-8.8-7.2-16-16-16zm64 208v48 48c0 35.3-28.7 64-64 64H364.3l8 48H424c13.3 0 24 10.7 24 24s-10.7 24-24 24H352 224 152c-13.3 0-24-10.7-24-24s10.7-24 24-24h51.7l8-48H64c-35.3 0-64-28.7-64-64V304 256 64C0 28.7 28.7 0 64 0H512c35.3 0 64 28.7 64 64V256zM48 304v48c0 8.8 7.2 16 16 16H239.5c.3 0 .6 0 .8 0h95.2c.3 0 .6 0 .8 0H512c8.8 0 16-7.2 16-16V304H48zM252.3 464h71.3l-8-48H260.3l-8 48z"/></svg>
Side by Side

After

Width:  |  Height:  |  Size: 529 B

204
app/styles/base.css
View File

@ -897,7 +897,8 @@ select:active {
#noVNC_container {
width: 100%;
height: 100%;
background-image: url('../images/splash.jpg')
background-image: url('../images/splash.jpg');
background-size: cover;
}
#noVNC_keyboardinput {
@ -1191,3 +1192,204 @@ a:link {
a:visited {
color: white;
}
/* ----------------------------------------
* Multi Display Arrangement
* ----------------------------------------
*/
#noVNC_connect_button {
background: rgba(0,0,0,0.7);
padding: 20px;
color: white;
font-size: 12px;
font-weight: 100;
display: flex;
align-items: center;
gap: 20px;
cursor: pointer;
border-radius: 8px;
}
#noVNC_connect_button .image {
color:#697ad6;
transition: all 0.3s;
position: relative;
overflow: hidden;
}
#noVNC_connect_button .text {
color: #ffffff91
}
#noVNC_connect_button .heading {
color: #fff;
font-size: 13px;
margin-bottom: 3px;
}
#noVNC_connect_button:hover .image {
color:#298453;
}
#noVNC_connect_button:hover .go {
transition: all 0.3s;
transform: translateX(5px);
}
#noVNC_connect_button .power {
transition: all 0.3s;
color: white!important;
transform: translateY(50px);
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
display: flex;
justify-content: center;
align-items: center;
}
#noVNC_connect_button:hover .power {
transform: translateY(0);
}
#noVNC_connect_button svg {
fill: currentColor;
}
#noVNC_identify_monitor {
position: fixed;
z-index: 999999;
background: rgba(0,0,0,0.7);
color: white;
font-size: 30px;
top: 0;
right: 0;
width: 100px;
height: 100px;
display: none;
justify-content: center;
align-items: center;
border-bottom-left-radius: 10px;
}
#noVNC_identify_monitor.show {
display: flex;
}
.flex {
display: flex;
}
.flex-col {
flex-direction: column;
}
.canvas {
background: rgb(225,225,226);
background: linear-gradient(0deg, rgba(225,225,226,1) 0%, rgba(237,237,238,1) 100%);
}
.row {
background: #e9e9ea;
border-radius: 8px;
border: 1px solid #dcdcdd;
padding: 6px 15px;
display: flex;
justify-content: space-between;
align-items: center;
}
.row input {
border: 1px solid #dcdcdd;
padding: 8px 2px;
text-align: right;
background-color: #ededee;
}
#noVNC_displays {
position: fixed;
display: none;
top: 0;
left: 0;
width: 100%;
height: 100%;
z-index: 100;
background: rgba(0,0,0,0.7);
cursor: pointer;
transition: 0.5s ease-in-out;
padding: 5px;
flex-direction: row;
justify-content: center;
align-items: center;
line-height: 25px;
word-wrap: break-word;
}
#noVNC_displays.noVNC_open {
display: flex;
transform: translateY(0);
visibility: visible;
opacity: 1;
}
#noVNC_displays .canvas {
display: flex;
width: 700px;
flex-direction: column;
border-radius: 15px;
background: #fdfdfd;
padding: 15px 20px;
box-shadow: 0 0 15px rgba(0,0,0,0.4);
position: relative;
}
#noVNC_displays .canvas-title {
font-size: 14px;
font-weight: bold;
padding: 0 10px;
line-height: 1;
line-height: 1.4;
}
#noVNC_displays .canvas-text {
font-size: 12px;
padding: 0 35px 0 10px;
opacity: 0.6;
line-height: 1.4;
margin-bottom: 15px;
}
#noVNC_displays .arrange-buttons {
margin-top: 15px;
display: flex;
justify-content: space-between;
}
#noVNC_displays .arrange-buttons button, #noVNC_displays .arrange-buttons .button {
border: none;
display: flex;
align-items: center;
padding: 4px 7px;
border-radius: 5px;
cursor: pointer;
background-color: #f1f1f1;
font-size: 13px;
border: 1px solid #e5e5e5;
line-height: 1;
}
#noVNC_setting_enable_hidpi_option {
display: none!important;
}
#noVNC_setting_enable_hidpi_option.show {
display: flex!important;
}
#noVNC_refreshMonitors {
position: absolute;
top: 20px;
right: 25px;
}
#noVNC_refreshMonitors_icon {
transition: all 0.3s;
transform-origin: center;
}
#noVNC_addMonitor {
background-color: #2196F3!important;
color: white;
border: none!important;
}
#noVNC_addMonitor svg {
margin-right: 5px;
}
#noVNC_displays .canvas canvas {
background: #f7f7f7;
border: 1px solid #ececec;
border-radius: 8px;
}

499
app/ui.js
View File

@ -13,13 +13,7 @@ window.addEventListener("load", function() {
loader.src = "vendor/browser-es-module-loader/dist/browser-es-module-loader.js";
document.head.appendChild(loader);
});
window.addEventListener("load", function() {
var connect_btn_el = document.getElementById("noVNC_connect_button");
if (typeof(connect_btn_el) != 'undefined' && connect_btn_el != null)
{
connect_btn_el.click();
}
});
window.updateSetting = (name, value) => {
WebUtil.writeSetting(name, value);
@ -31,8 +25,8 @@ window.updateSetting = (name, value) => {
}
}
import "core-js/stable";
import "regenerator-runtime/runtime";
//import "core-js/stable";
//import "regenerator-runtime/runtime";
import * as Log from '../core/util/logging.js';
import _, { l10n } from './localization.js';
import { isTouchDevice, isSafari, hasScrollbarGutter, dragThreshold, supportsBinaryClipboard, isFirefox, isWindows, isIOS, supportsPointerLock }
@ -68,6 +62,12 @@ const UI = {
inhibitReconnect: true,
reconnectCallback: null,
reconnectPassword: null,
monitors: [],
sortedMonitors: [],
selectedMonitor: null,
refreshRotation: 0,
supportsBroadcastChannel: (typeof BroadcastChannel !== "undefined"),
prime() {
return WebUtil.initSettings().then(() => {
@ -131,11 +131,12 @@ const UI = {
UI.addConnectionControlHandlers();
UI.addClipboardHandlers();
UI.addSettingsHandlers();
UI.addDisplaysHandler();
// UI.addMultiMonitorAddHandler();
document.getElementById("noVNC_status")
.addEventListener('click', UI.hideStatus);
UI.openControlbar();
//
UI.updateVisualState('init');
@ -165,8 +166,8 @@ const UI = {
UI.hideKeyboardControls();
}
});
window.addEventListener("beforeunload", (e) => {
window.addEventListener("unload", (e) => {
if (UI.rfb) {
UI.disconnect();
}
@ -402,6 +403,17 @@ const UI = {
}
},
addConnectionControlHandlers() {
UI.addClickHandle('noVNC_disconnect_button', UI.disconnect);
var connect_btn_el = document.getElementById("noVNC_connect_button_2");
if (typeof(connect_btn_el) != 'undefined' && connect_btn_el != null)
{
connect_btn_el.addEventListener('click', UI.connect);
}
},
addTouchSpecificHandlers() {
document.getElementById("noVNC_keyboard_button")
.addEventListener('click', UI.toggleVirtualKeyboard);
@ -472,21 +484,6 @@ const UI = {
.addEventListener('click', () => UI.rfb.machineReset());
},
addConnectionControlHandlers() {
UI.addClickHandle('noVNC_disconnect_button', UI.disconnect);
var connect_btn_el = document.getElementById("noVNC_connect_button");
if (typeof(connect_btn_el) != 'undefined' && connect_btn_el != null)
{
connect_btn_el.addEventListener('click', UI.connect);
}
document.getElementById("noVNC_cancel_reconnect_button")
.addEventListener('click', UI.cancelReconnect);
document.getElementById("noVNC_credentials_button")
.addEventListener('click', UI.setCredentials);
},
addClipboardHandlers() {
UI.addClickHandle('noVNC_clipboard_button', UI.toggleClipboardPanel);
@ -588,6 +585,24 @@ const UI = {
window.addEventListener('msfullscreenchange', UI.updateFullscreenButton);
},
addDisplaysHandler() {
if (UI.supportsBroadcastChannel) {
UI.showControlInput("noVNC_displays_button");
UI.addClickHandle('noVNC_displays_button', UI.openDisplays);
UI.addClickHandle('noVNC_close_displays', UI.closeDisplays);
UI.addClickHandle('noVNC_identify_monitors_button', UI._identify);
UI.addClickHandle('noVNC_addMonitor', UI.addSecondaryMonitor);
UI.addClickHandle('noVNC_refreshMonitors', UI.displaysRefresh);
}
},
/*addMultiMonitorAddHandler() {
if (UI.supportsBroadcastChannel) {
UI.addClickHandle('noVNC_addmonitor_button', UI.addSecondaryMonitor);
}
},*/
/* ------^-------
* /EVENT HANDLERS
* ==============
@ -676,8 +691,6 @@ const UI = {
// State change closes dialogs as they may not be relevant
// anymore
UI.closeAllPanels();
document.getElementById('noVNC_credentials_dlg')
.classList.remove('noVNC_open');
},
showStats() {
@ -1380,9 +1393,12 @@ const UI = {
UI.rfb = new RFB(document.getElementById('noVNC_container'),
document.getElementById('noVNC_keyboardinput'),
url,
{ shared: UI.getSetting('shared'),
repeaterID: UI.getSetting('repeaterID'),
credentials: { password: password } });
{
shared: UI.getSetting('shared'),
repeaterID: UI.getSetting('repeaterID'),
credentials: { password: password }
},
true );
UI.rfb.addEventListener("connect", UI.connectFinished);
UI.rfb.addEventListener("disconnect", UI.disconnectFinished);
UI.rfb.addEventListener("credentialsrequired", UI.credentials);
@ -1394,6 +1410,7 @@ const UI = {
UI.rfb.addEventListener("desktopname", UI.updateDesktopName);
UI.rfb.addEventListener("inputlock", UI.inputLockChanged);
UI.rfb.addEventListener("inputlockerror", UI.inputLockError);
UI.rfb.addEventListener("screenregistered", UI.screenRegistered);
UI.rfb.translateShortcuts = UI.getSetting('translate_shortcuts');
UI.rfb.clipViewport = UI.getSetting('view_clip');
UI.rfb.scaleViewport = UI.getSetting('resize') === 'scale';
@ -1427,7 +1444,7 @@ const UI = {
UI.rfb.mouseButtonMapper = UI.initMouseButtonMapper();
if (UI.rfb.videoQuality === 5) {
UI.rfb.enableQOI = true;
}
}
//Only explicitly request permission to clipboard on browsers that support binary clipboard access
if (supportsBinaryClipboard()) {
@ -1681,6 +1698,14 @@ const UI = {
UI.toggleIMEMode();
}
break;
case 'open_displays_mode':
if (UI.rfb) {
UI.openDisplays()
}
break;
case 'close_displays_mode':
UI.closeDisplays()
break;
case 'enable_webrtc':
if (!UI.getSetting('enable_webrtc')) {
UI.forceSetting('enable_webrtc', true, false);
@ -1722,6 +1747,10 @@ const UI = {
UI.forceSetting('enable_hidpi', event.data.value, false);
UI.enableHiDpi();
break;
case 'control_displays':
parent.postMessage({ action: 'can_control_displays', value: true}, '*' );
break;
}
}
},
@ -1746,57 +1775,6 @@ const UI = {
parent.postMessage({ action: 'clipboardrx', value: event.detail.text}, '*' ); //TODO fix star
},
/* ------^-------
* /CONNECTION
* ==============
* PASSWORD
* ------v------*/
credentials(e) {
// FIXME: handle more types
document.getElementById("noVNC_username_block").classList.remove("noVNC_hidden");
document.getElementById("noVNC_password_block").classList.remove("noVNC_hidden");
let inputFocus = "none";
if (e.detail.types.indexOf("username") === -1) {
document.getElementById("noVNC_username_block").classList.add("noVNC_hidden");
} else {
inputFocus = inputFocus === "none" ? "noVNC_username_input" : inputFocus;
}
if (e.detail.types.indexOf("password") === -1) {
document.getElementById("noVNC_password_block").classList.add("noVNC_hidden");
} else {
inputFocus = inputFocus === "none" ? "noVNC_password_input" : inputFocus;
}
document.getElementById('noVNC_credentials_dlg')
.classList.add('noVNC_open');
setTimeout(() => document
.getElementById(inputFocus).focus(), 100);
Log.Warn("Server asked for credentials");
UI.showStatus(_("Credentials are required"), "warning");
},
setCredentials(e) {
// Prevent actually submitting the form
e.preventDefault();
let inputElemUsername = document.getElementById('noVNC_username_input');
const username = inputElemUsername.value;
let inputElemPassword = document.getElementById('noVNC_password_input');
const password = inputElemPassword.value;
// Clear the input after reading the password
inputElemPassword.value = "";
UI.rfb.sendCredentials({ username: username, password: password });
UI.reconnectPassword = password;
document.getElementById('noVNC_credentials_dlg')
.classList.remove('noVNC_open');
},
/* ------^-------
* /PASSWORD
* ==============
@ -1867,6 +1845,345 @@ const UI = {
UI.rfb.enableHiDpi = UI.getSetting('enable_hidpi');
},
/* ------^-------
* /MULTI-MONITOR SUPPORT
* ==============*/
_identify(e) {
UI.identify()
UI.rfb.identify(UI.monitors)
},
identify(data) {
document.getElementById('noVNC_identify_monitor').innerHTML = '1'
document.getElementById('noVNC_identify_monitor').classList.add("show")
setTimeout(() => {
document.getElementById('noVNC_identify_monitor').classList.remove("show")
}, 3500)
},
openDisplays() {
document.getElementById('noVNC_displays').classList.add("noVNC_open");
if (UI.monitors.length < 1 ) {
let screenPlan = UI.rfb.getScreenPlan();
UI.initMonitors(screenPlan)
}
UI.displayMonitors()
},
closeDisplays() {
document.getElementById('noVNC_displays').classList.remove("noVNC_open");
},
displaysRefresh() {
const rotation = UI.refreshRotation + 180;
let screenPlan = UI.rfb.getScreenPlan();
document.getElementById('noVNC_refreshMonitors_icon').style.transform = "rotate(" + rotation + "deg)"
UI.refreshRotation = rotation
UI.updateMonitors(screenPlan)
UI.recenter()
UI.draw()
},
addSecondaryMonitor() {
let new_display_path = window.location.pathname.replace(/[^/]*$/, '')
let new_display_url = `${window.location.protocol}//${window.location.host}${new_display_path}screen.html`;
Log.Debug(`Opening a secondary display ${new_display_url}`)
window.open(new_display_url);
},
initMonitors(screenPlan) {
const { scale } = UI.multiMonitorSettings()
let monitors = []
let showNativeResolution = false
let num = 1
screenPlan.screens.forEach(screen => {
if (parseFloat(screen.pixelRatio) != 1) {
showNativeResolution = true
}
monitors.push({
id: screen.screenID,
x: screen.x / scale,
y: screen.y / scale,
w: screen.serverWidth / scale,
h: screen.serverHeight / scale,
pixelRatio: screen.pixelRatio,
scale: 1,
fill: '#eeeeeecc',
isDragging: false,
num
})
num++
})
if (showNativeResolution) {
document.getElementById('noVNC_setting_enable_hidpi_option').classList.add("show");
} else {
document.getElementById('noVNC_setting_enable_hidpi_option').classList.remove("show");
}
UI.monitors = monitors
let deepCopyMonitors = JSON.parse(JSON.stringify(monitors))
UI.sortedMonitors = deepCopyMonitors.sort((a, b) => {
if (a.y >= b.y + (b.h / 2)) {
return 1
}
return a.x - b.x
})
},
updateMonitors(screenPlan) {
UI.initMonitors(screenPlan)
UI.recenter()
UI.draw()
},
multiMonitorSettings() {
const canvas = document.getElementById("noVNC_multiMonitorWidget")
return {
canvas,
ctx: canvas.getContext("2d"),
bb: canvas.getBoundingClientRect(),
scale: 12,
canvasWidth: 700,
canvasHeight: 230,
}
},
recenter() {
const monitors = UI.sortedMonitors
UI.removeSpaces()
const { startLeft, startTop } = UI.getSizes(monitors)
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
m.x += startLeft
m.y += startTop
}
UI.setScreenPlan()
},
removeSpaces() {
const monitors = UI.sortedMonitors
let prev = monitors[0]
if (monitors.length > 1) {
for (var i = 1; i < monitors.length; i++) {
var a = monitors[i];
let prevStart = prev.x + prev.w
let prevStartTop = prev.y + prev.h
if (a.x > prevStart) {
a.x = prevStart
}
if (a.x < prevStart) {
if (a.y < prevStartTop) {
a.x = prevStart
}
}
if (a.y > prevStartTop) {
if (a.x <= prevStart) {
a.y = prevStartTop
}
}
prev = monitors[i]
}
}
},
rect(ctx, x, y, w, h) {
ctx.beginPath();
ctx.roundRect(x, y, w, h, 5);
ctx.stroke();
ctx.closePath();
ctx.fill();
},
draw() {
const { ctx, canvasWidth, canvasHeight, scale } = UI.multiMonitorSettings()
const monitors = UI.sortedMonitors
ctx.clearRect(0, 0, canvasWidth, canvasHeight);
ctx.rect(0, 0, canvasWidth, canvasHeight);
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
ctx.fillStyle = m.fill;
ctx.lineWidth = 1;
ctx.lineJoin = "round";
ctx.strokeStyle = m === UI.selectedMonitor ? "#2196F3" : "#aaa";
UI.rect(ctx, m.x, m.y, (m.w / m.scale), (m.h / m.scale));
ctx.font = "13px sans-serif";
ctx.textAlign = "right";
ctx.textBaseline = "top";
ctx.fillStyle = "#000";
ctx.fillText((m.num), (m.x + m.w) - 4, m.y + 4);
ctx.font = "200 11px sans-serif";
ctx.textAlign = "center";
ctx.textBaseline = "middle";
ctx.fillText(m.w * scale + ' x ' + m.h * scale, m.x + (m.w / 2), m.y + (m.h / 2));
}
},
getSizes(monitors) {
const { canvasWidth, canvasHeight } = UI.multiMonitorSettings()
let top = monitors[0].y
let left = monitors[0].x
let width = monitors[0].w
let height = monitors[0].h
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
if (m.x < left) {
left = m.x
}
if (m.y < top) {
top = m.y
}
if(m.x + m.w > width) {
width = m.x + m.w
}
if(m.y + m.h > height) {
height = m.y + m.h
}
}
const startLeft = ((canvasWidth - width - left) / 2);
const startTop = ((canvasHeight - height - top) / 2);
return { top, left, width, height, startLeft, startTop }
},
setScreenPlan() {
let monitors = UI.monitors
let sortedMonitors = UI.sortedMonitors
const { scale } = UI.multiMonitorSettings()
const { top, left, width, height } = UI.getSizes(sortedMonitors)
const screens = []
for (var i = 0; i < monitors.length; i++) {
var monitor = monitors[i];
var a = sortedMonitors.find(el => el.id === monitor.id)
console.log(a)
screens.push({
screenID: a.id,
serverHeight: Math.round(a.h * scale),
serverWidth: Math.round(a.w * scale),
x: Math.round((a.x - left) * scale),
y: Math.round((a.y - top) * scale)
})
}
const screenPlan = {
serverHeight: Math.round(height * scale),
serverWidth: Math.round(width * scale),
screens
}
console.log('setScreenPlan')
console.log(screenPlan)
UI.rfb.applyScreenPlan(screenPlan);
},
displayMonitors() {
// const monitors = UI.sortedMonitors
let monitors = UI.sortedMonitors
const { canvas, ctx, bb, canvasWidth, canvasHeight, scale } = UI.multiMonitorSettings()
const { startLeft, startTop } = UI.getSizes(monitors)
let offsetX
let offsetY
let dragok = false
let startX;
let startY;
offsetX = bb.left
offsetY = bb.top
canvas.addEventListener("mousedown", myDown, false);
canvas.addEventListener("mouseup", myUp, false);
canvas.addEventListener("mousemove", myMove, false);
UI.recenter()
UI.draw()
function myDown(e) {
let monitors = UI.sortedMonitors
e.preventDefault();
e.stopPropagation();
let mx = parseInt(e.clientX - offsetX);
let my = parseInt(e.clientY - offsetY);
for (var i = 0; i < monitors.length; i++) {
var mon = monitors[i];
var monw = mon.w / mon.scale
var monh = mon.h / mon.scale
let monx = mon.x
let mony = mon.y
// Find the closest rect to drag
if (mx > monx && mx < (monx + monw) && my > mony && my < (mony + monh)) {
dragok = true;
mon.isDragging = true;
UI.selectedMonitor = mon
break // get out of the loop rather than dragging multiple
}
}
startX = mx;
startY = my;
UI.draw()
}
function myUp(e) {
let monitors = UI.sortedMonitors
e.preventDefault();
e.stopPropagation();
// clear all the dragging flags
dragok = false;
for (var i = 0; i < monitors.length; i++) {
monitors[i].isDragging = false;
}
monitors.sort((a, b) => {
if (a.y >= b.y + (b.h / 2)) {
return 1
}
return a.x - b.x
})
UI.recenter()
UI.draw()
}
function myMove(e) {
let monitors = UI.sortedMonitors
if (dragok) {
e.preventDefault();
e.stopPropagation();
// get the current mouse position
var mx = parseInt(e.clientX - offsetX);
var my = parseInt(e.clientY - offsetY);
// calculate the distance the mouse has moved
// since the last mousemove
var dx = mx - startX;
var dy = my - startY;
// move each rect that isDragging
// by the distance the mouse has moved
// since the last mousemove
for (var i = 0; i < monitors.length; i++) {
var m = monitors[i];
if (m.isDragging) {
m.x += dx;
m.y += dy;
}
}
// redraw the scene with the new rect positions
UI.draw();
// reset the starting mouse position for the next mousemove
startX = mx;
startY = my;
}
}
},
/* ------^-------
* /RESIZE
* ==============
@ -2514,6 +2831,20 @@ const UI = {
}
},
screenRegistered(e) {
console.log('screen registered')
// Get the current screen plan
// When a new display is added, it is defaulted to be placed to the far right relative to existing displays and to the top
if (UI.rfb) {
let screenPlan = UI.rfb.getScreenPlan();
console.log(screenPlan)
UI.updateMonitors(screenPlan)
UI._identify(UI.monitors)
}
},
//Helper to add options to dropdown.
addOption(selectbox, text, value) {
const optn = document.createElement("OPTION");

333
app/ui_screen.js Normal file
View File

@ -0,0 +1,333 @@
import RFB from "../core/rfb.js";
import * as WebUtil from "./webutil.js";
import { isTouchDevice, isSafari, hasScrollbarGutter, dragThreshold, supportsBinaryClipboard, isFirefox, isWindows, isIOS, supportsPointerLock }
from '../core/util/browser.js';
import { MouseButtonMapper, XVNC_BUTTONS } from "../core/mousebuttonmapper.js";
import * as Log from '../core/util/logging.js';
const UI = {
connected: false,
screenID: null,
screen: {},
screens: [],
supportsBroadcastChannel: (typeof BroadcastChannel !== "undefined"),
controlChannel: null,
//Initial Loading of the UI
prime() {
console.log('prime')
this.start();
},
//Render default UI
start() {
console.log('start')
window.addEventListener("unload", (e) => {
if (UI.rfb) {
UI.disconnect();
}
});
// Settings with immediate effects
UI.initSetting('logging', 'warn');
UI.updateLogging();
UI.addDefaultHandlers();
UI.updateVisualState('disconnected');
},
addDefaultHandlers() {
document.getElementById('noVNC_connect_button').addEventListener('click', UI.connect);
},
getSetting(name, isBool, default_value) {
let val = WebUtil.readSetting(name);
if ((val === 'undefined' || val === null) && default_value !== 'undefined' && default_value !== null) {
val = default_value;
}
if (typeof val !== 'undefined' && val !== null && isBool) {
if (val.toString().toLowerCase() in {'0': 1, 'no': 1, 'false': 1}) {
val = false;
} else {
val = true;
}
}
return val;
},
connect() {
console.log('connect')
UI.rfb = new RFB(document.getElementById('noVNC_container'),
document.getElementById('noVNC_keyboardinput'),
"", //URL
{
shared: UI.getSetting('shared', true),
repeaterID: UI.getSetting('repeaterID', false),
credentials: { password: null }
},
false // Not a primary display
);
UI.rfb.addEventListener("connect", UI.connectFinished);
//UI.rfb.addEventListener("disconnect", UI.disconnectFinished);
UI.rfb.clipViewport = UI.getSetting('view_clip');
UI.rfb.scaleViewport = UI.getSetting('resize', false, 'remote') === 'scale';
UI.rfb.resizeSession = UI.getSetting('resize', false, 'remote') === 'remote';
UI.rfb.qualityLevel = parseInt(UI.getSetting('quality'));
UI.rfb.dynamicQualityMin = parseInt(UI.getSetting('dynamic_quality_min'));
UI.rfb.dynamicQualityMax = parseInt(UI.getSetting('dynamic_quality_max'));
UI.rfb.jpegVideoQuality = parseInt(UI.getSetting('jpeg_video_quality'));
UI.rfb.webpVideoQuality = parseInt(UI.getSetting('webp_video_quality'));
UI.rfb.videoArea = parseInt(UI.getSetting('video_area'));
UI.rfb.videoTime = parseInt(UI.getSetting('video_time'));
UI.rfb.videoOutTime = parseInt(UI.getSetting('video_out_time'));
UI.rfb.videoScaling = parseInt(UI.getSetting('video_scaling'));
UI.rfb.treatLossless = parseInt(UI.getSetting('treat_lossless'));
UI.rfb.maxVideoResolutionX = parseInt(UI.getSetting('max_video_resolution_x'));
UI.rfb.maxVideoResolutionY = parseInt(UI.getSetting('max_video_resolution_y'));
UI.rfb.frameRate = parseInt(UI.getSetting('framerate'));
UI.rfb.compressionLevel = parseInt(UI.getSetting('compression'));
UI.rfb.showDotCursor = UI.getSetting('show_dot', true);
UI.rfb.idleDisconnect = UI.getSetting('idle_disconnect');
UI.rfb.pointerRelative = UI.getSetting('pointer_relative');
UI.rfb.videoQuality = parseInt(UI.getSetting('video_quality'));
UI.rfb.antiAliasing = UI.getSetting('anti_aliasing');
UI.rfb.clipboardUp = UI.getSetting('clipboard_up', true, true);
UI.rfb.clipboardDown = UI.getSetting('clipboard_down', true, true);
UI.rfb.clipboardSeamless = UI.getSetting('clipboard_seamless', true, true);
UI.rfb.keyboard.enableIME = UI.getSetting('enable_ime', true, false);
UI.rfb.clipboardBinary = supportsBinaryClipboard() && UI.rfb.clipboardSeamless;
UI.rfb.enableWebRTC = UI.getSetting('enable_webrtc', true, false);
UI.rfb.enableHiDpi = UI.getSetting('enable_hidpi', true, false);
UI.rfb.mouseButtonMapper = UI.initMouseButtonMapper();
if (UI.rfb.videoQuality === 5) {
UI.rfb.enableQOI = true;
}
if (UI.supportsBroadcastChannel) {
console.log('add event listener')
UI.controlChannel = new BroadcastChannel("registrationChannel");
UI.controlChannel.addEventListener('message', UI.handleControlMessage)
}
//attach this secondary display to the primary display
if (UI.screenID === null) {
const screen = UI.rfb.attachSecondaryDisplay();
UI.screenID = screen.screenID
UI.screen = screen
} else {
console.log('else reattach screens')
console.log(UI.screen)
UI.rfb.reattachSecondaryDisplay(UI.screen);
}
document.querySelector('title').textContent = 'Display ' + UI.screenID
if (supportsBinaryClipboard()) {
// explicitly request permission to the clipboard
navigator.permissions.query({ name: "clipboard-read" }).then((result) => { Log.Debug('binary clipboard enabled') });
}
},
handleControlMessage(event) {
switch (event.data.eventType) {
case 'identify':
UI.identify(event.data)
break;
case 'secondarydisconnected':
UI.updateVisualState('disconnected');
break;
}
},
updateVisualState(state) {
document.documentElement.classList.remove("noVNC_connecting");
document.documentElement.classList.remove("noVNC_connected");
document.documentElement.classList.remove("noVNC_disconnecting");
document.documentElement.classList.remove("noVNC_reconnecting");
document.documentElement.classList.remove("noVNC_disconnected");
const transitionElem = document.getElementById("noVNC_transition_text");
if (WebUtil.isInsideKasmVDI())
{
parent.postMessage({ action: 'connection_state', value: state}, '*' );
}
let connect_el = document.getElementById('noVNC_connect_dlg');
switch (state) {
case 'init':
break;
case 'connecting':
transitionElem.textContent = _("Connecting...");
document.documentElement.classList.add("noVNC_connecting");
break;
case 'connected':
document.documentElement.classList.add("noVNC_connected");
if (!connect_el.classList.contains("noVNC_hidden")) {
connect_el.classList.add('noVNC_hidden');
}
break;
case 'disconnecting':
transitionElem.textContent = _("Disconnecting...");
document.documentElement.classList.add("noVNC_disconnecting");
break;
case 'disconnected':
console.log('disconnected')
document.documentElement.classList.add("noVNC_disconnected");
if (connect_el.classList.contains("noVNC_hidden")) {
connect_el.classList.remove('noVNC_hidden');
}
UI.disconnect()
break;
case 'reconnecting':
transitionElem.textContent = _("Reconnecting...");
document.documentElement.classList.add("noVNC_reconnecting");
break;
default:
Log.Error("Invalid visual state: " + state);
UI.showStatus(_("Internal error"), 'error');
return;
}
},
identify(data) {
UI.screens = data.screens
console.log('identify')
const screen = data.screens.find(el => el.id === UI.screenID)
if (screen) {
document.getElementById('noVNC_identify_monitor').innerHTML = screen.num
document.getElementById('noVNC_identify_monitor').classList.add("show")
document.querySelector('title').textContent = 'Display ' + screen.num + ' - ' + UI.screenID
setTimeout(() => {
document.getElementById('noVNC_identify_monitor').classList.remove("show")
}, 3500)
}
},
showStatus(text, statusType, time, kasm = false) {
// If inside the full Kasm CDI framework, don't show messages unless explicitly told to
if (WebUtil.isInsideKasmVDI() && !kasm) {
return;
}
const statusElem = document.getElementById('noVNC_status');
if (typeof statusType === 'undefined') {
statusType = 'normal';
}
// Don't overwrite more severe visible statuses and never
// errors. Only shows the first error.
if (statusElem.classList.contains("noVNC_open")) {
if (statusElem.classList.contains("noVNC_status_error")) {
return;
}
if (statusElem.classList.contains("noVNC_status_warn") &&
statusType === 'normal') {
return;
}
}
clearTimeout(UI.statusTimeout);
switch (statusType) {
case 'error':
statusElem.classList.remove("noVNC_status_warn");
statusElem.classList.remove("noVNC_status_normal");
statusElem.classList.add("noVNC_status_error");
break;
case 'warning':
case 'warn':
statusElem.classList.remove("noVNC_status_error");
statusElem.classList.remove("noVNC_status_normal");
statusElem.classList.add("noVNC_status_warn");
break;
case 'normal':
case 'info':
default:
statusElem.classList.remove("noVNC_status_error");
statusElem.classList.remove("noVNC_status_warn");
statusElem.classList.add("noVNC_status_normal");
break;
}
statusElem.textContent = text;
statusElem.classList.add("noVNC_open");
// If no time was specified, show the status for 1.5 seconds
if (typeof time === 'undefined') {
time = 1500;
}
// Error messages do not timeout
if (statusType !== 'error') {
UI.statusTimeout = window.setTimeout(UI.hideStatus, time);
}
},
hideStatus() {
clearTimeout(UI.statusTimeout);
document.getElementById('noVNC_status').classList.remove("noVNC_open");
},
disconnect() {
if (UI.rfb) {
UI.rfb.disconnect();
if (UI.supportsBroadcastChannel) {
console.log('remove event listeners')
UI.controlChannel.removeEventListener('message', UI.handleControlMessage);
UI.rfb.removeEventListener("connect", UI.connectFinished);
}
}
},
connectFinished(e) {
UI.connected = true;
UI.inhibitReconnect = false;
UI.showStatus("Secondary Screen Connected");
UI.updateVisualState('connected');
// Do this last because it can only be used on rendered elements
UI.rfb.focus();
},
initMouseButtonMapper() {
const mouseButtonMapper = new MouseButtonMapper();
const settings = WebUtil.readSetting("mouseButtonMapper");
if (settings) {
mouseButtonMapper.load(settings);
return mouseButtonMapper;
}
mouseButtonMapper.set(0, XVNC_BUTTONS.LEFT_BUTTON);
mouseButtonMapper.set(1, XVNC_BUTTONS.MIDDLE_BUTTON);
mouseButtonMapper.set(2, XVNC_BUTTONS.RIGHT_BUTTON);
mouseButtonMapper.set(3, XVNC_BUTTONS.BACK_BUTTON);
mouseButtonMapper.set(4, XVNC_BUTTONS.FORWARD_BUTTON);
WebUtil.writeSetting("mouseButtonMapper", mouseButtonMapper.dump());
return mouseButtonMapper;
},
updateLogging() {
WebUtil.initLogging(UI.getSetting('logging'));
},
// Initial page load read/initialization of settings
initSetting(name, defVal) {
// Check Query string followed by cookie
let val = WebUtil.getConfigVar(name);
if (val === null) {
val = WebUtil.readSetting(name, defVal);
}
WebUtil.setSetting(name, val);
return val;
},
}
UI.prime();
export default UI;

2
core/base64.js
View File

@ -58,7 +58,7 @@ export default {
/* Every four characters is 3 resulting numbers */
const resultLength = (dataLength >> 2) * 3 + Math.floor((dataLength % 4) / 1.5);
const result = new Array(resultLength);
const result = new Uint8Array(resultLength);
// Convert one by one.

634
core/display.js
View File

@ -11,9 +11,10 @@ import * as Log from './util/logging.js';
import Base64 from "./base64.js";
import { toSigned32bit } from './util/int.js';
import { isWindows } from './util/browser.js';
import { uuidv4 } from './util/strings.js';
export default class Display {
constructor(target) {
constructor(target, isPrimaryDisplay) {
Log.Debug(">> Display.constructor");
/*
@ -30,6 +31,7 @@ export default class Display {
this._asyncFrameQueue = [];
this._maxAsyncFrameQueue = 3;
this._clearAsyncQueue();
this._syncFrameQueue = [];
this._flushing = false;
@ -56,7 +58,7 @@ export default class Display {
this._targetCtx = this._target.getContext('2d');
// the visible canvas viewport (i.e. what actually gets seen)
this._viewportLoc = { 'x': 0, 'y': 0, 'w': this._target.width, 'h': this._target.height };
//this._viewportLoc = { 'x': 0, 'y': 0, 'w': this._target.width, 'h': this._target.height };
Log.Debug("User Agent: " + navigator.userAgent);
@ -80,22 +82,57 @@ export default class Display {
// ===== PROPERTIES =====
this._maxScreens = 4;
this._scale = 1.0;
this._clipViewport = false;
this._antiAliasing = 0;
this._fps = 0;
this._isPrimaryDisplay = isPrimaryDisplay;
this._screenID = uuidv4();
this._screens = [{
screenID: this._screenID,
screenIndex: 0,
width: this._target.width, //client
height: this._target.height, //client
serverWidth: 0, //calculated
serverHeight: 0, //calculated
x: 0,
y: 0,
relativePosition: 0, //left, right, up, down relative to primary display
relativePositionX: 0, //offset relative to primary monitor, always 0 for primary
relativePositionY: 0, //offset relative to primary monitor, always 0 for primary
pixelRatio: window.devicePixelRatio,
containerHeight: this._target.parentNode.offsetHeight,
containerWidth: this._target.parentNode.offsetWidth,
channel: null
}];
// ===== EVENT HANDLERS =====
this.onflush = () => { }; // A flush request has finished
// Use requestAnimationFrame to write to canvas, to match display refresh rate
this._animationFrameID = window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
if (!this._isPrimaryDisplay) {
this._screens[0].channel = new BroadcastChannel(`screen_${this._screenID}_channel`);
this._screens[0].channel.addEventListener('message', this._handleSecondaryDisplayMessage.bind(this));
} else {
//this._animationFrameID = window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
}
Log.Debug("<< Display.constructor");
}
// ===== PROPERTIES =====
get screens() { return this._screens; }
get screenId() { return this._screenID; }
get screenIndex() {
// A secondary screen should not have a screen index of 0, but it will be 0 until registration is complete
// returning a -1 lets the caller know the screen has not been registered yet
if (!this._isPrimaryDisplay && this._screens[0].screenIndex == 0) {
return -1;
}
return this._screens[0].screenIndex;
}
get antiAliasing() { return this._antiAliasing; }
set antiAliasing(value) {
@ -112,8 +149,8 @@ export default class Display {
set clipViewport(viewport) {
this._clipViewport = viewport;
// May need to readjust the viewport dimensions
const vp = this._viewportLoc;
this.viewportChangeSize(vp.w, vp.h);
const vp = this._screens[0];
this.viewportChangeSize(vp.width, vp.height);
this.viewportChangePos(0, 0);
}
@ -136,18 +173,207 @@ export default class Display {
// ===== PUBLIC METHODS =====
/*
Returns the screen index and relative coordinates given globally scoped coordinates
*/
getClientRelativeCoordinates(x, y) {
for (let i = 0; i < this._screens.length; i++) {
if (
(x >= this._screens[i].x && x <= this._screens[i].x + this._screens[i].serverWidth) &&
(y >= this._screens[i].y && y <= this._screens[i].y + this._screens[i].serverHeight)
)
{
return {
"screenIndex": i,
"x": x - this._screens[i].x,
"y": y - this._screens[i].y
}
}
}
}
/*
Returns coordinates that are server relative when multiple monitors are in use
*/
getServerRelativeCoordinates(screenIndex, x, y) {
if (screenIndex >= 0 && screenIndex < this._screens.length) {
x += this._screens[screenIndex].x;
y += this._screens[screenIndex].y;
}
return [x, y];
}
getScreenSize(resolutionQuality, max_width, max_height, hiDpi, disableLimit) {
let data = {
screens: null,
serverWidth: 0,
serverHeight: 0
}
//recalculate primary display container size
this._screens[0].containerHeight = this._target.parentNode.offsetHeight;
this._screens[0].containerWidth = this._target.parentNode.offsetWidth;
this._screens[0].width = this._target.parentNode.offsetWidth;
this._screens[0].height = this._target.parentNode.offsetHeight;
this._screens[0].pixelRatio = window.devicePixelRatio;
//this._screens[0].width = this._target.width;
//this._screens[0].height = this._target.height;
//calculate server-side resolution of each screen
for (let i=0; i<this._screens.length; i++) {
let width = max_width || this._screens[i].containerWidth;
let height = max_height || this._screens[i].containerHeight;
let scale = 0;
//max the resolution of a single screen to 1280
if (width > 1280 && !disableLimit && resolutionQuality == 1) {
height = 1280 * (height/width); //keeping the aspect ratio of original resolution, shrink y to match x
width = 1280;
}
//hard coded 720p
else if (resolutionQuality == 0 && !disableLimit) {
width = 1280;
height = 720;
}
//force full resolution on a high DPI monitor where the OS is scaling
else if (hiDpi) {
width = width * this._screens[i].pixelRatio;
height = height * this._screens[i].pixelRatio;
scale = 1 / this._screens[i].pixelRatio;
}
//physically small device with high DPI
else if (this._antiAliasing === 0 && this._screens[i].pixelRatio > 1 && width < 1000 & width > 0) {
Log.Info('Device Pixel ratio: ' + this._screens[i].pixelRatio + ' Reported Resolution: ' + width + 'x' + height);
let targetDevicePixelRatio = 1.5;
if (this._screens[i].pixelRatio > 2) { targetDevicePixelRatio = 2; }
let scaledWidth = (width * this._screens[i].pixelRatio) * (1 / targetDevicePixelRatio);
let scaleRatio = scaledWidth / width;
width = width * scaleRatio;
height = height * scaleRatio;
scale = 1 / scaleRatio;
Log.Info('Small device with hDPI screen detected, auto scaling at ' + scaleRatio + ' to ' + width + 'x' + height);
}
this._screens[i].serverWidth = width;
this._screens[i].serverHeight = height;
this._screens[i].scale = scale;
this._screens[i].x2 = this._screens[i].x + this._screens[i].serverWidth;
this._screens[i].y2 = this._screens[i].y + this._screens[i].serverHeight;
}
for (let i = 0; i < this._screens.length; i++) {
data.serverWidth = Math.max(data.serverWidth, this._screens[i].x + this._screens[i].serverWidth);
data.serverHeight = Math.max(data.serverHeight, this._screens[i].y + this._screens[i].serverHeight);
}
data.screens = this._screens;
return data;
}
applyScreenPlan(screenPlan) {
for (let i = 0; i < screenPlan.screens.length; i++) {
for (let z = 0; z < this._screens.length; z++) {
if (screenPlan.screens[i].screenID === this._screens[z].screenID) {
this._screens[z].x = screenPlan.screens[i].x;
this._screens[z].y = screenPlan.screens[i].y;
}
}
}
}
addScreen(screenID, width, height, pixelRatio, containerHeight, containerWidth) {
if (!this._isPrimaryDisplay) {
throw new Error("Cannot add a screen to a secondary display.");
}
let screenIdx = -1;
//Does the screen already exist?
for (let i = 0; i < this._screens.length; i++) {
if (this._screens[i].screenID === screenID) {
screenIdx = i;
}
}
if (screenIdx > 0) {
//existing screen, update
const screen = this._screens[screenIdx];
screen.width = width;
screen.height = height;
screen.containerHeight = containerHeight;
screen.containerWidth = containerWidth;
screen.pixelRatio = pixelRatio;
} else {
//New Screen, add to far right until user repositions it
let x = 0;
for (let i = 0; i < this._screens.length; i++) {
x = Math.max(x, this._screens[i].x + this._screens[i].serverWidth);
}
var new_screen = {
screenID: screenID,
screenIndex: this.screens.length,
width: width, //client
height: height, //client
serverWidth: 0, //calculated
serverHeight: 0, //calculated
x: x,
y: 0,
pixelRatio: pixelRatio,
containerHeight: containerHeight,
containerWidth: containerWidth,
channel: null,
scale: 0
}
new_screen.channel = new BroadcastChannel(`screen_${screenID}_channel`);
//new_screen.channel.message = this._handleSecondaryDisplayMessage().bind(this);
this._screens.push(new_screen);
new_screen.channel.postMessage({ eventType: "registered", screenIndex: new_screen.screenIndex });
}
}
removeScreen(screenID) {
let removed = false;
if (this._isPrimaryDisplay) {
for (let i=1; i<this._screens.length; i++) {
if (this._screens[i].screenID == screenID) {
//flush all rects on target screen
this._flushRectsScreen(i);
this._screens[i].channel.close();
this._screens.splice(i, 1);
removed = true;
break;
}
}
//recalculate indexes and update secondary displays
for (let i=1; i<this._screens.length; i++) {
this.screens[i].screenIndex = i;
if (i > 0) {
this._screens[i].channel.postMessage({ eventType: "registered", screenIndex: i });
}
}
return removed;
} else {
throw new Error("Secondary screens only allowed on primary display.")
}
}
viewportChangePos(deltaX, deltaY) {
const vp = this._viewportLoc;
const vp = this._screens[0];
deltaX = Math.floor(deltaX);
deltaY = Math.floor(deltaY);
if (!this._clipViewport) {
deltaX = -vp.w; // clamped later of out of bounds
deltaY = -vp.h;
deltaX = -vp.width; // clamped later of out of bounds
deltaY = -vp.height;
}
const vx2 = vp.x + vp.w - 1;
const vy2 = vp.y + vp.h - 1;
const vx2 = vp.x + vp.width - 1;
const vy2 = vp.y + vp.height - 1;
// Position change
@ -173,7 +399,7 @@ export default class Display {
viewportChangeSize(width, height) {
if (!this._clipViewport ||
if ((!this._clipViewport && this._screens.length === 1 ) ||
typeof(width) === "undefined" ||
typeof(height) === "undefined") {
@ -192,10 +418,10 @@ export default class Display {
height = this._fbHeight;
}
const vp = this._viewportLoc;
if (vp.w !== width || vp.h !== height) {
vp.w = width;
vp.h = height;
const vp = this._screens[0];
if (vp.serverWidth !== width || vp.serverHeight !== height) {
vp.serverWidth = width;
vp.serverHeight = height;
const canvas = this._target;
canvas.width = width;
@ -213,14 +439,14 @@ export default class Display {
if (this._scale === 0) {
return 0;
}
return toSigned32bit(x / this._scale + this._viewportLoc.x);
return toSigned32bit(x / this._scale + this._screens[0].x);
}
absY(y) {
if (this._scale === 0) {
return 0;
}
return toSigned32bit(y / this._scale + this._viewportLoc.y);
return toSigned32bit(y / this._scale + this._screens[0].y);
}
resize(width, height) {
@ -231,6 +457,10 @@ export default class Display {
const canvas = this._target;
if (canvas == undefined) { return; }
if (this._screens.length > 0) {
width = this._screens[0].serverWidth;
height = this._screens[0].serverHeight;
}
if (canvas.width !== width || canvas.height !== height) {
// We have to save the canvas data since changing the size will clear it
@ -253,8 +483,8 @@ export default class Display {
// Readjust the viewport as it may be incorrectly sized
// and positioned
const vp = this._viewportLoc;
this.viewportChangeSize(vp.w, vp.h);
const vp = this._screens[0];
this.viewportChangeSize(vp.serverWidth, vp.serverHeight);
this.viewportChangePos(0, 0);
}
@ -267,7 +497,8 @@ export default class Display {
this._asyncRenderQPush({
'type': 'flip',
'frame_id': frame_id,
'rect_cnt': rect_cnt
'rect_cnt': rect_cnt,
'screenLocations': [ { screenIndex: 0, x: 0, y: 0 } ]
});
}
@ -291,7 +522,7 @@ export default class Display {
this._asyncFrameComplete(0, true);
if (onflush_message)
this._flushing = true;
this.onflush();
}
/*
@ -307,21 +538,22 @@ export default class Display {
*/
dispose() {
clearInterval(this._frameStatsInterval);
cancelAnimationFrame(this._animationFrameID);
this.clear();
}
fillRect(x, y, width, height, color, frame_id, fromQueue) {
if (!fromQueue) {
this._asyncRenderQPush({
'type': 'fill',
'x': x,
'y': y,
'width': width,
'height': height,
'color': color,
'frame_id': frame_id
});
let rect = {
type: 'fill',
x: x,
y: y,
width: width,
height: height,
color: color,
frame_id: frame_id
}
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else {
this._setFillColor(color);
this._targetCtx.fillRect(x, y, width, height);
@ -330,7 +562,7 @@ export default class Display {
copyImage(oldX, oldY, newX, newY, w, h, frame_id, fromQueue) {
if (!fromQueue) {
this._asyncRenderQPush({
let rect = {
'type': 'copy',
'oldX': oldX,
'oldY': oldY,
@ -339,7 +571,9 @@ export default class Display {
'width': w,
'height': h,
'frame_id': frame_id
});
}
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else {
// Due to this bug among others [1] we need to disable the image-smoothing to
// avoid getting a blur effect when copying data.
@ -364,18 +598,31 @@ export default class Display {
if ((width === 0) || (height === 0)) {
return;
}
const img = new Image();
img.src = "data: " + mime + ";base64," + Base64.encode(arr);
this._asyncRenderQPush({
let rect = {
'type': 'img',
'img': img,
'img': null,
'x': x,
'y': y,
'width': width,
'height': height,
'frame_id': frame_id
});
}
this._processRectScreens(rect);
if (rect.inPrimary) {
const img = new Image();
img.src = "data: " + mime + ";base64," + Base64.encode(arr);
rect.img = img;
} else {
rect.type = "_img";
}
if (rect.inSecondary) {
rect.mime = mime;
rect.src = "data: " + mime + ";base64," + Base64.encode(arr);
}
this._asyncRenderQPush(rect);
}
transparentRect(x, y, width, height, img, frame_id) {
@ -393,12 +640,18 @@ export default class Display {
'height': height,
'frame_id': frame_id
}
this._processRectScreens(rect);
let imageBmpPromise = createImageBitmap(img);
imageBmpPromise.then( function(img) {
rect.img = img;
rect.img.complete = true;
}.bind(rect) );
if (rect.inPrimary) {
let imageBmpPromise = createImageBitmap(img);
imageBmpPromise.then( function(img) {
rect.img = img;
rect.img.complete = true;
}.bind(rect) );
}
if (rect.inSecondary) {
rect.arr = img;
}
this._asyncRenderQPush(rect);
}
@ -410,7 +663,7 @@ export default class Display {
// this probably isn't getting called *nearly* as much
const newArr = new Uint8Array(width * height * 4);
newArr.set(new Uint8Array(arr.buffer, 0, newArr.length));
this._asyncRenderQPush({
let rect = {
'type': 'blit',
'data': newArr,
'x': x,
@ -418,7 +671,9 @@ export default class Display {
'width': width,
'height': height,
'frame_id': frame_id
});
}
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else {
// NB(directxman12): arr must be an Type Array view
let data = new Uint8ClampedArray(arr.buffer,
@ -431,7 +686,7 @@ export default class Display {
blitQoi(x, y, width, height, arr, offset, frame_id, fromQueue) {
if (!fromQueue) {
this._asyncRenderQPush({
let rect = {
'type': 'blitQ',
'data': arr,
'x': x,
@ -439,7 +694,9 @@ export default class Display {
'width': width,
'height': height,
'frame_id': frame_id
});
}
this._processRectScreens(rect);
this._asyncRenderQPush(rect);
} else {
this._targetCtx.putImageData(arr, x, y);
}
@ -464,14 +721,14 @@ export default class Display {
} else if (scaleRatio === 0) {
const vp = this._viewportLoc;
const vp = this._screens[0];
const targetAspectRatio = containerWidth / containerHeight;
const fbAspectRatio = vp.w / vp.h;
const fbAspectRatio = vp.width / vp.height;
if (fbAspectRatio >= targetAspectRatio) {
scaleRatio = containerWidth / vp.w;
scaleRatio = containerWidth / vp.width;
} else {
scaleRatio = containerHeight / vp.h;
scaleRatio = containerHeight / vp.height;
}
}
@ -480,6 +737,112 @@ export default class Display {
// ===== PRIVATE METHODS =====
_handleSecondaryDisplayMessage(event) {
if (!this._isPrimaryDisplay && event.data) {
switch (event.data.eventType) {
case 'rect':
let rect = event.data.rect;
//overwrite screen locations when received on the secondary display
rect.screenLocations = [ rect.screenLocations[event.data.screenLocationIndex] ]
rect.screenLocations[0].screenIndex = 0;
switch (rect.type) {
case 'img':
case '_img':
rect.img = new Image();
rect.img.src = rect.src;
rect.type = 'img';
break;
case 'transparent':
let imageBmpPromise = createImageBitmap(rect.arr);
imageBmpPromise.then(function(rect, img) {
rect.img.complete = true;
}).bind(this, rect);
break;
}
this._syncFrameQueue.push(rect);
break;
case 'frameComplete':
window.requestAnimationFrame( () => { this._pushSyncRects(); });
break;
case 'registered':
if (!this._isPrimaryDisplay) {
this._screens[0].screenIndex = event.data.screenIndex;
Log.Error(`Screen with index (${event.data.screenIndex}) successfully registered with the primary display.`);
}
break;
}
}
}
_pushSyncRects() {
whileLoop:
while (this._syncFrameQueue.length > 0) {
const a = this._syncFrameQueue[0];
const pos = a.screenLocations[0];
switch (a.type) {
case 'copy':
this.copyImage(pos.oldX, pos.oldY, pos.x, pos.y, a.width, a.height, a.frame_id, true);
break;
case 'fill':
this.fillRect(pos.x, pos.y, a.width, a.height, a.color, a.frame_id, true);
break;
case 'blit':
this.blitImage(pos.x, pos.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'blitQ':
this.blitQoi(pos.x, pos.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'img':
if (a.img.complete) {
this.drawImage(a.img, pos.x, pos.y, a.width, a.height);
} else {
if (this._syncFrameQueue.length > 1000) {
this._syncFrameQueue.shift();
this._droppedRects++;
} else {
break whileLoop;
}
}
break;
case 'transparent':
if (a.img.complete) {
this.drawImage(a.img, pos.x, pos.y, a.width, a.height);
} else {
if (this._syncFrameQueue.length > 1000) {
this._syncFrameQueue.shift();
this._droppedRects++;
} else {
break whileLoop;
}
}
break;
default:
Log.Warn(`Unknown rect type: ${rect}`);
}
this._syncFrameQueue.shift();
}
if (this._syncFrameQueue.length > 0) {
window.requestAnimationFrame( () => { this._pushSyncRects(); });
}
}
_flushRectsScreen(screenIndex) {
for (let i=0; i<this._asyncFrameQueue.length; i++) {
const frame = this._asyncFrameQueue[i];
for (let x=0; x < frame[2].length; x++) {
const rect = frame[2][x];
for (let y=0; y < rect.screenLocations.length; y++) {
if (rect.screenLocations[y].screenIndex === screenIndex) {
rect.screenLocations.splice(y, 1);
break;
}
}
}
}
}
/*
Process incoming rects into a frame buffer, assume rects are out of order due to either UDP or parallel processing of decoding
*/
@ -504,19 +867,24 @@ export default class Display {
newestFrameID = Math.max(newestFrameID, this._asyncFrameQueue[i][0]);
}
if (!this._firstRect) { //TODO: Remove this
this._firstRect = true;
Log.Info("First rect received.");
}
if (frameIx >= 0) {
if (rect.type == "flip") {
//flip rect contains the rect count for the frame
if (this._asyncFrameQueue[frameIx][1] !== 0) {
Log.Warn("Redundant flip rect, current rect_cnt: " + this._asyncFrameQueue[frameIx][1] + ", new rect_cnt: " + rect.rect_cnt );
}
this._asyncFrameQueue[frameIx][1] = rect.rect_cnt;
this._asyncFrameQueue[frameIx][1] += rect.rect_cnt;
if (rect.rect_cnt == 0) {
Log.Warn("Invalid rect count");
}
}
if (this._asyncFrameQueue[frameIx][1] == this._asyncFrameQueue[frameIx][2].length) {
if (this._asyncFrameQueue[frameIx][1] > 0 && this._asyncFrameQueue[frameIx][2].length >= this._asyncFrameQueue[frameIx][1]) {
//frame is complete
this._asyncFrameComplete(frameIx);
}
@ -527,11 +895,21 @@ export default class Display {
if (rect.type == "flip") { this._lateFlipRect++; }
return;
} else if (rect.frame_id > newestFrameID) {
//frame is newer than any frame in the queue, drop old frames
this._asyncFrameQueue.shift();
//frame is newer than any frame in the queue, drop old frame
if (this._asyncFrameQueue[0][3] == true) {
Log.Warn("Forced frame to canvas");
this._pushAsyncFrame(true);
this._droppedFrames += (rect.frame_id - (newestFrameID + 1));
this._forcedFrameCnt++;
} else {
Log.Warn("Old frame dropped");
this._asyncFrameQueue.shift();
this._droppedFrames += (rect.frame_id - newestFrameID);
}
let rect_cnt = ((rect.type == "flip") ? rect.rect_cnt : 0);
this._asyncFrameQueue.push([ rect.frame_id, rect_cnt, [ rect ], (rect_cnt == 1), 0, 0 ]);
this._droppedFrames++;
}
}
@ -554,6 +932,10 @@ export default class Display {
If marked force, unloaded images will be skipped and the frame will be marked complete and ready for rendering
*/
_asyncFrameComplete(frameIx, force=false) {
if (frameIx >= this._asyncFrameQueue.length) {
return;
}
let currentFrameRectIx = this._asyncFrameQueue[frameIx][4];
if (force) {
@ -566,10 +948,13 @@ export default class Display {
}
}
while (currentFrameRectIx < this._asyncFrameQueue[frameIx][2].length) {
if (this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type == 'img' && !this._asyncFrameQueue[frameIx][2][currentFrameRectIx].img.complete) {
this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type = 'skip';
this._droppedRects++;
if (this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type == 'img') {
if (this._asyncFrameQueue[frameIx][2][currentFrameRectIx].img && !this._asyncFrameQueue[frameIx][2][currentFrameRectIx].img.complete) {
this._asyncFrameQueue[frameIx][2][currentFrameRectIx].type = 'skip';
this._droppedRects++;
}
}
currentFrameRectIx++;
}
} else {
@ -587,6 +972,12 @@ export default class Display {
}
this._asyncFrameQueue[frameIx][4] = currentFrameRectIx;
this._asyncFrameQueue[frameIx][3] = true;
if (force && frameIx == 0) {
this._pushAsyncFrame(true);
} else {
window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
}
}
/*
@ -594,45 +985,77 @@ export default class Display {
*/
_pushAsyncFrame(force=false) {
if (this._asyncFrameQueue[0][3] || force) {
let frame = this._asyncFrameQueue.shift()[2];
let frame = this._asyncFrameQueue[0][2];
let frameId = this._asyncFrameQueue.shift()[0];
if (this._asyncFrameQueue.length < this._maxAsyncFrameQueue) {
this._asyncFrameQueue.push([ 0, 0, [], false, 0, 0 ]);
}
let transparent_rects = [];
let secondaryScreenRects = 0;
//render the selected frame
for (let i = 0; i < frame.length; i++) {
const a = frame[i];
switch (a.type) {
case 'copy':
this.copyImage(a.oldX, a.oldY, a.x, a.y, a.width, a.height, a.frame_id, true);
break;
case 'fill':
this.fillRect(a.x, a.y, a.width, a.height, a.color, a.frame_id, true);
break;
case 'blit':
this.blitImage(a.x, a.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'blitQ':
this.blitQoi(a.x, a.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'img':
this.drawImage(a.img, a.x, a.y, a.width, a.height);
break;
case 'transparent':
transparent_rects.push(a);
break;
for (let sI = 0; sI < a.screenLocations.length; sI++) {
let screenLocation = a.screenLocations[sI];
if (screenLocation.screenIndex == 0) {
switch (a.type) {
case 'copy':
this.copyImage(screenLocation.oldX, screenLocation.oldY, screenLocation.x, screenLocation.y, a.width, a.height, a.frame_id, true);
break;
case 'fill':
this.fillRect(screenLocation.x, screenLocation.y, a.width, a.height, a.color, a.frame_id, true);
break;
case 'blit':
this.blitImage(screenLocation.x, screenLocation.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'blitQ':
this.blitQoi(screenLocation.x, screenLocation.y, a.width, a.height, a.data, 0, a.frame_id, true);
break;
case 'img':
this.drawImage(a.img, screenLocation.x, screenLocation.y, a.width, a.height);
break;
case 'transparent':
transparent_rects.push(a);
break;
}
} else {
if (a.img) {
a.img = null;
}
if (a.type !== 'flip') {
secondaryScreenRects++;
this._screens[screenLocation.screenIndex].channel.postMessage({ eventType: 'rect', rect: a, screenLocationIndex: sI });
}
}
}
}
//rects with transparency get applied last
for (let i = 0; i < transparent_rects.length; i++) {
const a = transparent_rects[i];
if (a.img) {
this.drawImage(a.img, a.x, a.y, a.width, a.height);
let screenIndexes = this._getRectScreenIndexes(a);
for (let sI = 0; sI < screenLocations.length; sI++) {
let screenLocation = a.screenLocations[sI];
if (sI == 0) {
if (a.img) {
this.drawImage(a.img, a.x, a.y, a.width, a.height);
}
} else {
secondaryScreenRects++;
this._screens[screenLocation.screenIndex].channel.postMessage({ eventType: 'rect', rect: a, screenLocationIndex: sI });
}
}
}
if (secondaryScreenRects > 0) {
for (let i = 1; i < this.screens.length; i++) {
this._screens[i].channel.postMessage({ eventType: 'frameComplete', frameId: frameId, rectCnt: secondaryScreenRects });
}
}
@ -650,22 +1073,51 @@ export default class Display {
this._pushAsyncFrame(true);
}
}
}
if (!force) {
window.requestAnimationFrame( () => { this._pushAsyncFrame(); });
_processRectScreens(rect) {
//find which screen this rect belongs to and adjust its x and y to be relative to the destination
let indexes = [];
rect.inPrimary = false;
rect.inSecondary = false;
for (let i=0; i < this._screens.length; i++) {
let screen = this._screens[i];
if (
!((rect.x > screen.x2 || screen.x > (rect.x + rect.width)) && (rect.y > screen.y2 || screen.y > (rect.y + rect.height)))
) {
let screenPosition = {
x: 0 - (screen.x - rect.x), //rect.x - screen.x,
y: 0 - (screen.y - rect.y), //rect.y - screen.y,
screenIndex: i
}
if (rect.type === 'copy') {
screenPosition.oldX = 0 - (screen.x - rect.oldX); //rect.oldX - screen.x;
screenPosition.oldY = 0 - (screen.y - rect.oldY); //rect.oldY - screen.y;
}
indexes.push(screenPosition);
if (i == 0) {
rect.inPrimary = true;
} else {
rect.inSecondary = true;
}
}
}
rect.screenLocations = indexes;
}
_rescale(factor) {
this._scale = factor;
const vp = this._viewportLoc;
const vp = this._screens[0];
// NB(directxman12): If you set the width directly, or set the
// style width to a number, the canvas is cleared.
// However, if you set the style width to a string
// ('NNNpx'), the canvas is scaled without clearing.
const width = factor * vp.w + 'px';
const height = factor * vp.h + 'px';
const width = factor * vp.serverWidth + 'px';
const height = factor * vp.serverHeight + 'px';
if ((this._target.style.width !== width) ||
(this._target.style.height !== height)) {
@ -673,12 +1125,12 @@ export default class Display {
this._target.style.height = height;
}
Log.Info('Pixel Ratio: ' + window.devicePixelRatio + ', VNC Scale: ' + factor + 'VNC Res: ' + vp.w + 'x' + vp.h);
Log.Info('Pixel Ratio: ' + window.devicePixelRatio + ', VNC Scale: ' + factor + 'VNC Res: ' + vp.serverWidth + 'x' + vp.serverHeight);
var pixR = Math.abs(Math.ceil(window.devicePixelRatio));
var isFirefox = navigator.userAgent.toLowerCase().indexOf('firefox') > -1;
if (this.antiAliasing === 2 || (this.antiAliasing === 0 && factor === 1 && this._target.style.imageRendering !== 'pixelated' && pixR === window.devicePixelRatio && vp.w > 0)) {
if (this.antiAliasing === 2 || (this.antiAliasing === 0 && factor === 1 && this._target.style.imageRendering !== 'pixelated' && pixR === window.devicePixelRatio && vp.width > 0)) {
this._target.style.imageRendering = ((!isFirefox) ? 'pixelated' : 'crisp-edges' );
Log.Debug('Smoothing disabled');
} else if (this.antiAliasing === 1 || (this.antiAliasing === 0 && factor !== 1 && this._target.style.imageRendering !== 'auto')) {

7
core/input/keyboard.js
View File

@ -11,7 +11,6 @@ import KeyTable from "./keysym.js";
import keysyms from "./keysymdef.js";
import imekeys from "./imekeys.js";
import * as browser from "../util/browser.js";
import UI from '../../app/ui.js';
import { isChromiumBased } from '../util/browser.js';
//
@ -46,6 +45,7 @@ export default class Keyboard {
this._lastKeyboardInput = null;
this._defaultKeyboardInputLen = 100;
this._keyboardInputReset();
this._translateShortcuts = true;
}
// ===== PUBLIC METHODS =====
@ -56,6 +56,9 @@ export default class Keyboard {
this.focus();
}
get translateShortcuts() { return this._translateShortcuts; }
set translateShortcuts(value) { this._translateShortcuts = value; }
// ===== PRIVATE METHODS =====
clearKeysDown(event) {
@ -319,7 +322,7 @@ export default class Keyboard {
// Translate MacOs CMD based shortcuts to their CTRL based counterpart
if (
browser.isMac() &&
UI.rfb && UI.rfb.translateShortcuts &&
this._translateShortcuts &&
code !== "MetaLeft" && code !== "MetaRight" &&
e.metaKey && !e.ctrlKey && !e.altKey
) {

653
core/rfb.js
View File

@ -10,7 +10,7 @@
import { toUnsigned32bit, toSigned32bit } from './util/int.js';
import * as Log from './util/logging.js';
import { encodeUTF8, decodeUTF8 } from './util/strings.js';
import { encodeUTF8, decodeUTF8, uuidv4 } from './util/strings.js';
import { hashUInt8Array } from './util/int.js';
import { dragThreshold, supportsCursorURIs, isTouchDevice, isWindows, isMac, isIOS } from './util/browser.js';
import { clientToElement } from './util/element.js';
@ -76,11 +76,11 @@ const extendedClipboardActionNotify = 1 << 27;
const extendedClipboardActionProvide = 1 << 28;
export default class RFB extends EventTargetMixin {
constructor(target, touchInput, urlOrChannel, options) {
constructor(target, touchInput, urlOrChannel, options, isPrimaryDisplay) {
if (!target) {
throw new Error("Must specify target");
}
if (!urlOrChannel) {
if (!urlOrChannel && isPrimaryDisplay) {
throw new Error("Must specify URL, WebSocket or RTCDataChannel");
}
@ -101,6 +101,7 @@ export default class RFB extends EventTargetMixin {
this._shared = 'shared' in options ? !!options.shared : true;
this._repeaterID = options.repeaterID || '';
this._wsProtocols = options.wsProtocols || ['binary'];
this._isPrimaryDisplay = (isPrimaryDisplay !== false);
// Internal state
this._rfbConnectionState = '';
@ -122,7 +123,8 @@ export default class RFB extends EventTargetMixin {
this._supportsContinuousUpdates = false;
this._enabledContinuousUpdates = false;
this._supportsSetDesktopSize = false;
this._screenID = 0;
this._screenID = uuidv4();
this._screenIndex = 0;
this._screenFlags = 0;
this._qemuExtKeyEventSupported = false;
@ -205,6 +207,7 @@ export default class RFB extends EventTargetMixin {
this._accumulatedWheelDeltaX = 0;
this._accumulatedWheelDeltaY = 0;
this.mouseButtonMapper = null;
this._mouseLastScreenIndex = -1;
// Gesture state
this._gestureLastTapTime = null;
@ -212,6 +215,19 @@ export default class RFB extends EventTargetMixin {
this._gestureLastMagnitudeX = 0;
this._gestureLastMagnitudeY = 0;
// Secondary Displays
this._secondaryDisplays = {};
this._supportsBroadcastChannel = (typeof BroadcastChannel !== "undefined");
if (this._supportsBroadcastChannel) {
this._controlChannel = new BroadcastChannel("registrationChannel");
this._controlChannel.addEventListener('message', this._handleControlMessage.bind(this));
Log.Debug("Attached to registrationChannel for secondary displays.")
}
if (!this._isPrimaryDisplay) {
this._screenIndex = 2;
}
// Bound event handlers
this._eventHandlers = {
updateHiddenKeyboard: this._updateHiddenKeyboard.bind(this),
@ -223,6 +239,7 @@ export default class RFB extends EventTargetMixin {
handleWheel: this._handleWheel.bind(this),
handleGesture: this._handleGesture.bind(this),
handleFocusChange: this._handleFocusChange.bind(this),
handleMouseOut: this._handleMouseOut.bind(this),
};
// main setup
@ -262,7 +279,7 @@ export default class RFB extends EventTargetMixin {
// NB: nothing that needs explicit teardown should be done
// before this point, since this can throw an exception
try {
this._display = new Display(this._canvas);
this._display = new Display(this._canvas, this._isPrimaryDisplay);
} catch (exc) {
Log.Error("Display exception: " + exc);
throw exc;
@ -283,68 +300,13 @@ export default class RFB extends EventTargetMixin {
this._gestures = new GestureHandler();
this._sock = new Websock();
this._sock.on('message', () => {
this._handleMessage();
});
this._sock.on('open', () => {
if ((this._rfbConnectionState === 'connecting') &&
(this._rfbInitState === '')) {
this._rfbInitState = 'ProtocolVersion';
Log.Debug("Starting VNC handshake");
} else {
this._fail("Unexpected server connection while " +
this._rfbConnectionState);
}
});
this._sock.on('close', (e) => {
Log.Debug("WebSocket on-close event");
let msg = "";
if (e.code) {
msg = "(code: " + e.code;
if (e.reason) {
msg += ", reason: " + e.reason;
}
msg += ")";
}
switch (this._rfbConnectionState) {
case 'connecting':
this._fail("Connection closed " + msg);
break;
case 'connected':
// Handle disconnects that were initiated server-side
this._updateConnectionState('disconnecting');
this._updateConnectionState('disconnected');
break;
case 'disconnecting':
// Normal disconnection path
this._updateConnectionState('disconnected');
break;
case 'disconnected':
this._fail("Unexpected server disconnect " +
"when already disconnected " + msg);
break;
default:
this._fail("Unexpected server disconnect before connecting " +
msg);
break;
}
this._sock.off('close');
// Delete reference to raw channel to allow cleanup.
this._rawChannel = null;
});
this._sock.on('error', e => Log.Warn("WebSocket on-error event"));
// Slight delay of the actual connection so that the caller has
// time to set up callbacks
setTimeout(this._updateConnectionState.bind(this, 'connecting'));
if (this._isPrimaryDisplay) {
this._setupWebSocket();
}
Log.Debug("<< RFB.constructor");
// ===== PROPERTIES =====
this.dragViewport = false;
this.focusOnClick = true;
this.lastActiveAt = Date.now();
@ -367,6 +329,11 @@ export default class RFB extends EventTargetMixin {
// ===== PROPERTIES =====
get translateShortcuts() { return this._keyboard.translateShortcuts; }
set translateShortcuts(value) {
this._keyboard.translateShortcuts = value;
}
get pointerLock() { return this._pointerLock; }
set pointerLock(value) {
if (!this._pointerLock) {
@ -762,11 +729,87 @@ export default class RFB extends EventTargetMixin {
if (value !== this._hiDpi) {
this._hiDpi = value;
this._requestRemoteResize();
if (this._display.screens.length > 1) {
//force secondary displays to re-register and thus apply new hdpi setting
this._proxyRFBMessage('forceResize', [ value ]);
}
}
}
// ===== PUBLIC METHODS =====
attachSecondaryDisplay() {
this._updateConnectionState('connecting');
const screen = this._registerSecondaryDisplay();
this._updateConnectionState('connected');
return screen
}
reattachSecondaryDisplay(screen) {
this._updateConnectionState('connecting');
this._registerSecondaryDisplay(screen);
this._updateConnectionState('connected');
return screen
}
applyScreenPlan(screenPlan) {
if (this._isPrimaryDisplay) {
let fullPlan = this._screenSize();
//check plan for validity
let minX = Number.MAX_SAFE_INTEGER, minY = Number.MAX_SAFE_INTEGER;
let numScreensFound = 0;
for (let i = 0; i < screenPlan.screens.length; i++) {
minX = Math.min(minX, screenPlan.screens[i].x);
minY = Math.min(minY, screenPlan.screens[i].y);
for (let z = 0; z < fullPlan.screens.length; z++) {
if (screenPlan.screens[i].screenID == fullPlan.screens[z].screenID) {
numScreensFound++;
}
}
}
if (minX !== 0 || minY !== 0) {
throw new Error("Screen plan invalid, improper coordinates provided.");
}
if (numScreensFound > fullPlan.screens.length) {
throw new Error("Screen plan contained more screens then there are registered.")
} else if (numScreensFound < fullPlan.screens.length) {
throw new Error("Screen plan contained fewer screens then there are registered.")
}
this._display.applyScreenPlan(screenPlan);
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._updateContinuousUpdates();
}
}
getScreenPlan() {
let fullPlan = this._screenSize();
let sanitizedPlan = {
screens: [],
serverWidth: fullPlan.serverWidth,
serverHeight: fullPlan.serverHeight
};
for (let i=0; i < fullPlan.screens.length; i++) {
sanitizedPlan.screens.push(
{
screenID: fullPlan.screens[i].screenID,
serverWidth: fullPlan.screens[i].serverWidth,
serverHeight: fullPlan.screens[i].serverHeight,
x: fullPlan.screens[i].x,
y: fullPlan.screens[i].y,
pixelRatio: fullPlan.screens[i].pixelRatio
}
)
}
return sanitizedPlan;
}
/*
This function must be called after changing any properties that effect rendering quality
*/
@ -774,7 +817,9 @@ export default class RFB extends EventTargetMixin {
if (this._rfbConnectionState === 'connected') {
if (this._pendingApplyVideoRes) {
RFB.messages.setMaxVideoResolution(this._sock, this._maxVideoResolutionX, this._maxVideoResolutionY);
if (this._isPrimaryDisplay){
RFB.messages.setMaxVideoResolution(this._sock, this._maxVideoResolutionX, this._maxVideoResolutionY);
}
}
if (this._pendingApplyResolutionChange) {
@ -793,15 +838,16 @@ export default class RFB extends EventTargetMixin {
}
disconnect() {
this._updateConnectionState('disconnecting');
this._sock.off('error');
this._sock.off('message');
this._sock.off('open');
}
sendCredentials(creds) {
this._rfbCredentials = creds;
setTimeout(this._initMsg.bind(this), 0);
if (this._isPrimaryDisplay) {
this._updateConnectionState('disconnecting');
this._sock.off('error');
this._sock.off('message');
this._sock.off('open');
this._proxyRFBMessage('disconnect');
} else {
this._updateConnectionState('disconnecting');
this._unregisterSecondaryDisplay();
}
}
sendCtrlAltDel() {
@ -851,13 +897,21 @@ export default class RFB extends EventTargetMixin {
Log.Info("Sending key (" + (down ? "down" : "up") + "): keysym " + keysym + ", scancode " + scancode);
RFB.messages.QEMUExtendedKeyEvent(this._sock, keysym, down, scancode);
if (this._isPrimaryDisplay) {
RFB.messages.QEMUExtendedKeyEvent(this._sock, keysym, down, scancode);
} else {
this._proxyRFBMessage('QEMUExtendedKeyEvent', [ keysym, down, scancode ])
}
} else {
if (!keysym) {
return;
}
Log.Info("Sending keysym (" + (down ? "down" : "up") + "): " + keysym);
RFB.messages.keyEvent(this._sock, keysym, down ? 1 : 0);
if (this._isPrimaryDisplay) {
RFB.messages.keyEvent(this._sock, keysym, down ? 1 : 0);
} else {
this._proxyRFBMessage('keyEvent', [ keysym, down ? 1 : 0 ])
}
}
}
@ -909,7 +963,12 @@ export default class RFB extends EventTargetMixin {
let mimes = [ 'text/plain' ];
dataset.push(data);
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes);
if (this._isPrimaryDisplay) {
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes);
} else {
this._proxyRFBMessage('sendBinaryClipboard', [ dataset, mimes ]);
}
}
async clipboardPasteDataFrom(clipdata) {
@ -948,7 +1007,7 @@ export default class RFB extends EventTargetMixin {
continue;
}
mimes.push(mime);
mimes.push(mime);
dataset.push(data);
Log.Debug('Sending mime type: ' + mime);
break;
@ -973,23 +1032,33 @@ export default class RFB extends EventTargetMixin {
if (dataset.length > 0) {
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes);
if (this._isPrimaryDisplay) {
RFB.messages.sendBinaryClipboard(this._sock, dataset, mimes);
} else {
this._proxyRFBMessage('sendBinaryClipboard', [ dataset, mimes ]);
}
}
}
requestBottleneckStats() {
RFB.messages.requestStats(this._sock);
if (this._isPrimaryDisplay) {
RFB.messages.requestStats(this._sock);
}
}
subscribeUnixRelay(name, processRelayFn) {
this._unixRelays = this._unixRelays || {};
this._unixRelays[name] = processRelayFn;
RFB.messages.sendSubscribeUnixRelay(this._sock, name);
if (this._isPrimaryDisplay){
this._unixRelays = this._unixRelays || {};
this._unixRelays[name] = processRelayFn;
RFB.messages.sendSubscribeUnixRelay(this._sock, name);
}
}
sendUnixRelayData(name, payload) {
RFB.messages.sendUnixRelay(this._sock, name, payload);
if (this._isPrimaryDisplay) {
RFB.messages.sendUnixRelay(this._sock, name, payload);
}
}
// ===== PRIVATE METHODS =====
@ -1003,10 +1072,68 @@ export default class RFB extends EventTargetMixin {
this._transitConnectionState = value;
}
_setupWebSocket() {
this._sock = new Websock();
this._sock.on('message', () => {
this._handleMessage();
});
this._sock.on('open', () => {
if ((this._rfbConnectionState === 'connecting') &&
(this._rfbInitState === '')) {
this._rfbInitState = 'ProtocolVersion';
Log.Debug("Starting VNC handshake");
} else {
this._fail("Unexpected server connection while " +
this._rfbConnectionState);
}
});
this._sock.on('close', (e) => {
Log.Debug("WebSocket on-close event");
let msg = "";
if (e.code) {
msg = "(code: " + e.code;
if (e.reason) {
msg += ", reason: " + e.reason;
}
msg += ")";
}
switch (this._rfbConnectionState) {
case 'connecting':
this._fail("Connection closed " + msg);
break;
case 'connected':
// Handle disconnects that were initiated server-side
this._updateConnectionState('disconnecting');
this._updateConnectionState('disconnected');
break;
case 'disconnecting':
// Normal disconnection path
this._updateConnectionState('disconnected');
break;
case 'disconnected':
this._fail("Unexpected server disconnect " +
"when already disconnected " + msg);
break;
default:
this._fail("Unexpected server disconnect before connecting " +
msg);
break;
}
this._sock.off('close');
// Delete reference to raw channel to allow cleanup.
this._rawChannel = null;
});
this._sock.on('error', e => Log.Warn("WebSocket on-error event"));
// Slight delay of the actual connection so that the caller has
// time to set up callbacks
setTimeout(this._updateConnectionState.bind(this, 'connecting'));
}
_connect() {
Log.Debug(">> RFB.connect");
if (this._url) {
if (this._url && this._isPrimaryDisplay) {
try {
Log.Info(`connecting to ${this._url}`);
this._sock.open(this._url, this._wsProtocols);
@ -1018,7 +1145,7 @@ export default class RFB extends EventTargetMixin {
this._fail("Error when opening socket (" + e + ")");
}
}
} else {
} else if (this._isPrimaryDisplay) {
try {
Log.Info(`attaching ${this._rawChannel} to Websock`);
this._sock.attach(this._rawChannel);
@ -1046,6 +1173,9 @@ export default class RFB extends EventTargetMixin {
window.addEventListener("focus", this._eventHandlers.handleFocusChange);
window.addEventListener("blur", this._eventHandlers.handleFocusChange);
//User cursor moves outside of the window
window.addEventListener("mouseover", this._eventHandlers.handleMouseOut);
// In order for the keyboard to not occlude the input being edited
// we move the hidden input we use for triggering the keyboard to the last click
// position which should trigger a page being moved down enough
@ -1085,7 +1215,7 @@ export default class RFB extends EventTargetMixin {
this._resendClipboardNextUserDrivenEvent = true;
// WebRTC UDP datachannel inits
if (typeof RTCPeerConnection !== 'undefined') {
if (typeof RTCPeerConnection !== 'undefined' && this._isPrimaryDisplay) {
this._udpBuffer = new Map();
this._udpPeer = new RTCPeerConnection({
@ -1190,7 +1320,7 @@ export default class RFB extends EventTargetMixin {
}
}
if (this._useUdp && typeof RTCPeerConnection !== 'undefined') {
if (this._useUdp && typeof RTCPeerConnection !== 'undefined' && this._isPrimaryDisplay) {
setTimeout(function() { this._sendUdpUpgrade() }.bind(this), 3000);
}
@ -1224,7 +1354,17 @@ export default class RFB extends EventTargetMixin {
window.removeEventListener('focus', this._eventHandlers.handleFocusChange);
this._keyboard.ungrab();
this._gestures.detach();
this._sock.close();
if (this._isPrimaryDisplay) {
this._sock.close();
} else {
if (this._primaryDisplayChannel) {
this._primaryDisplayChannel.postMessage({eventType: 'unregister', screenID: this._screenID})
this._primaryDisplayChannel.removeEventListener('message', this._handleSecondaryDisplayMessage);
this._primaryDisplayChannel.close();
this._primaryDisplayChannel = null;
}
}
try {
this._target.removeChild(this._screen);
} catch (e) {
@ -1325,7 +1465,7 @@ export default class RFB extends EventTargetMixin {
// When clipping is enabled, the screen is limited to
// the size of the container.
const size = this._screenSize();
this._display.viewportChangeSize(size.w, size.h);
this._display.viewportChangeSize(size.screens[0].serverWidth, size.screens[0].serverHeight);
this._fixScrollbars();
}
}
@ -1335,7 +1475,7 @@ export default class RFB extends EventTargetMixin {
this._display.scale = 1.0;
} else {
const size = this._screenSize(false);
this._display.autoscale(size.w, size.h, size.scale);
this._display.autoscale(size.screens[0].serverWidth, size.screens[0].serverHeight, size.screens[0].scale);
}
this._fixScrollbars();
}
@ -1346,60 +1486,29 @@ export default class RFB extends EventTargetMixin {
clearTimeout(this._resizeTimeout);
this._resizeTimeout = null;
if (!this._resizeSession || this._viewOnly ||
!this._supportsSetDesktopSize) {
return;
}
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock,
Math.floor(size.w), Math.floor(size.h),
this._screenID, this._screenFlags);
if (this._isPrimaryDisplay) {
if (!this._resizeSession || this._viewOnly ||
!this._supportsSetDesktopSize) {
return;
}
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
Log.Debug('Requested new desktop size: ' +
size.w + 'x' + size.h);
Log.Debug('Requested new desktop size: ' +
size.serverWidth + 'x' + size.serverHeight);
} else if (this._display.screenIndex > 0) {
//re-register the secondary display with new resolution
this._registerSecondaryDisplay();
}
if (this._display.screens.length > 1) {
this.dispatchEvent(new CustomEvent("screenregistered", {}));
}
}
// Gets the the size of the available screen
_screenSize (limited) {
if (limited === undefined) {
limited = true;
}
var x = this.forcedResolutionX || this._screen.offsetWidth;
var y = this.forcedResolutionY || this._screen.offsetHeight;
var scale = 0; // 0=auto
try {
if (x > 1280 && limited && this.videoQuality == 1) {
var ratio = y / x;
Log.Debug(ratio);
x = 1280;
y = x * ratio;
}
else if (limited && this.videoQuality == 0){
x = 1280;
y = 720;
} else if (this._hiDpi == true) {
x = x * window.devicePixelRatio;
y = y * window.devicePixelRatio;
scale = 1 / window.devicePixelRatio;
} else if (this._display.antiAliasing === 0 && window.devicePixelRatio > 1 && x < 1000 && x > 0) {
// small device with high resolution, browser is essentially zooming greater than 200%
Log.Info('Device Pixel ratio: ' + window.devicePixelRatio + ' Reported Resolution: ' + x + 'x' + y);
let targetDevicePixelRatio = 1.5;
if (window.devicePixelRatio > 2) { targetDevicePixelRatio = 2; }
let scaledWidth = (x * window.devicePixelRatio) * (1 / targetDevicePixelRatio);
let scaleRatio = scaledWidth / x;
x = x * scaleRatio;
y = y * scaleRatio;
scale = 1 / scaleRatio;
Log.Info('Small device with hDPI screen detected, auto scaling at ' + scaleRatio + ' to ' + x + 'x' + y);
}
} catch (err) {
Log.Debug(err);
}
return { w: x,
h: y,
scale: scale };
return this._display.getScreenSize(this.videoQuality, this.forcedResolutionX, this.forcedResolutionY, this._hiDpi, limited);
}
_fixScrollbars() {
@ -1469,6 +1578,10 @@ export default class RFB extends EventTargetMixin {
}
break;
case 'proxied':
//secondary display that needs to proxy messages through the broadcast channel
break;
default:
Log.Error("Unknown connection state: " + state);
return;
@ -1486,9 +1599,11 @@ export default class RFB extends EventTargetMixin {
this._disconnTimer = null;
// make sure we don't get a double event
this._sock.off('close');
if (this._isPrimaryDisplay) {
this._sock.off('close');
}
}
switch (state) {
case 'connecting':
this._connect();
@ -1504,6 +1619,7 @@ export default class RFB extends EventTargetMixin {
this._disconnTimer = setTimeout(() => {
Log.Error("Disconnection timed out.");
this._updateConnectionState('disconnected');
this._proxyRFBMessage('secondarydisconnected')
}, DISCONNECT_TIMEOUT * 1000);
break;
@ -1550,6 +1666,146 @@ export default class RFB extends EventTargetMixin {
{ detail: { capabilities: this._capabilities } }));
}
_proxyRFBMessage(messageType, data) {
let message = {
eventType: messageType,
args: data,
screenId: this._display.screenId,
screenIndex: this._display.screenIndex,
mouseLastScreenIndex: this._mouseLastScreenIndex,
}
this._controlChannel.postMessage(message);
}
_handleControlMessage(event) {
if (this._isPrimaryDisplay) {
// Secondary to Primary screen message
switch (event.data.eventType) {
case 'register':
this._display.addScreen(event.data.screenID, event.data.width, event.data.height, event.data.pixelRatio, event.data.containerHeight, event.data.containerWidth);
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._sendEncodings();
this._updateContinuousUpdates();
this.dispatchEvent(new CustomEvent("screenregistered", {}));
Log.Info(`Secondary monitor (${event.data.screenID}) has been registered.`);
break;
case 'reattach':
console.log('reattach message')
console.log(event.data)
this._display.addScreen(event.data.screenID, event.data.width, event.data.height, event.data.pixelRatio, event.data.containerHeight, event.data.containerWidth);
this.dispatchEvent(new CustomEvent("screenregistered", {}));
Log.Info(`Secondary monitor (${event.data.screenID}) has been reattached.`);
break;
case 'unregister':
if (this._display.removeScreen(event.data.screenID)) {
this.dispatchEvent(new CustomEvent("screenregistered", {}));
Log.Info(`Secondary monitor (${event.data.screenID}) has been removed.`);
const size = this._screenSize();
RFB.messages.setDesktopSize(this._sock, size, this._screenFlags);
this._sendEncodings();
this._updateContinuousUpdates();
this.dispatchEvent(new CustomEvent("screenregistered", {}));
} else {
Log.Info(`Secondary monitor (${event.data.screenID}) not found.`);
}
break;
case 'pointerEvent':
let coords = this._display.getServerRelativeCoordinates(event.data.screenIndex, event.data.args[0], event.data.args[1]);
this._mouseLastScreenIndex = event.data.screenIndex;
event.data.args[0] = coords[0];
event.data.args[1] = coords[1];
RFB.messages.pointerEvent(this._sock, ...event.data.args);
break;
case 'keyEvent':
RFB.messages.keyEvent(this._sock, ...event.data.args);
break;
case 'sendBinaryClipboard':
RFB.messages.sendBinaryClipboard(this._sock, ...event.data.args);
break;
// The following are primary to secondary messages that should be ignored on the primary
case 'updateCursor':
break;
}
} else {
// Primary to secondary screen message
switch (event.data.eventType) {
case 'updateCursor':
if (event.data.mouseLastScreenIndex === this._display.screenIndex || this._mouseLastScreenIndex === -1) {
this._updateCursor(...event.data.args);
this._mouseLastScreenIndex = event.data.mouseLastScreenIndex;
}
break;
case 'disconnect':
this.disconnect();
break;
case 'forceResize':
this._hiDpi = event.data.args[0];
this._updateScale();
this._requestRemoteResize();
break;
}
}
}
_unregisterSecondaryDisplay() {
if (!this._isPrimaryDisplay){
let message = {
eventType: 'unregister',
screenID: this._display.screenId
}
this._controlChannel.postMessage(message);
}
}
_registerSecondaryDisplay(currentScreen = false) {
if (!this._isPrimaryDisplay) {
//let screen = this._screenSize().screens[0];
//
let size = this._screenSize();
this._display.resize(size.screens[0].containerWidth, size.screens[0].containerHeight);
this._display.autoscale(size.screens[0].containerWidth, size.screens[0].containerHeight, size.screens[0].scale);
screen = this._screenSize().screens[0];
const registertype = (currentScreen) ? 'reattach' : 'register'
let message = {
eventType: registertype,
screenID: screen.screenID,
width: screen.width,
height: screen.height,
x: currentScreen.x || 0,
y: currentScreen.y || 0,
pixelRatio: screen.pixelRatio,
containerWidth: screen.containerWidth,
containerHeight: screen.containerHeight,
channel: null
}
this._controlChannel.postMessage(message);
if (!this._viewOnly) { this._keyboard.grab(); }
// return screen.screenID
return screen
}
}
identify(screens) {
let message = {
eventType: 'identify',
screens
}
this._controlChannel.postMessage(message);
}
_handleSecondaryDisplayMessage(event) {
if (this._isPrimaryDisplay) {
}
}
_handleMessage() {
if (this._sock.rQlen === 0) {
Log.Warn("handleMessage called on an empty receive queue");
@ -1583,6 +1839,18 @@ export default class RFB extends EventTargetMixin {
this.sendKey(keysym, code, down);
}
_handleMouseOut(ev) {
if (ev.toElement !== null && ev.relatedTarget === null) {
//mouse was outside of the window and just came in, this is our chance to do things
//Ensure the window was not moved to a different screen with a different pixel ratio
if (this._display.screens[0].pixelRatio !== window.devicePixelRatio) {
Log.Debug("Window moved to another screen with different pixel ratio, sending resize request.");
this._requestRemoteResize();
}
}
}
_handleMouse(ev) {
/*
* We don't check connection status or viewOnly here as the
@ -1638,6 +1906,7 @@ export default class RFB extends EventTargetMixin {
this._canvas);
}
this._mouseLastScreenIndex = this._display.screenIndex;
this._setLastActive();
const mappedButton = this.mouseButtonMapper.get(ev.button);
switch (ev.type) {
@ -1666,6 +1935,9 @@ export default class RFB extends EventTargetMixin {
false, xvncButtonToMask(mappedButton));
break;
case 'mousemove':
//when there are multiple screens
//This window can get mouse move events when the cursor is outside of the window, if the mouse is down
//when the cursor crosses the threshold of the window
this._handleMouseMove(pos.x, pos.y);
break;
}
@ -1787,19 +2059,22 @@ export default class RFB extends EventTargetMixin {
var rel_16_x = toSignedRelative16bit(x - this._pointerLockPos.x);
var rel_16_y = toSignedRelative16bit(y - this._pointerLockPos.y);
//console.log("new_pos x" + x + ", y" + y);
//console.log("lock x " + this._pointerLockPos.x + ", y " + this._pointerLockPos.y);
//console.log("rel x " + rel_16_x + ", y " + rel_16_y);
RFB.messages.pointerEvent(this._sock, rel_16_x,
rel_16_y, mask);
if (this._isPrimaryDisplay){
RFB.messages.pointerEvent(this._sock, rel_16_x, rel_16_y, mask);
} else {
this._proxyRFBMessage('pointerEvent', [ rel_16_x, rel_16_y, mask ]);
}
// reset the cursor position to center
this._mousePos = { x: this._pointerLockPos.x , y: this._pointerLockPos.y };
this._cursor.move(this._pointerLockPos.x, this._pointerLockPos.y);
} else {
RFB.messages.pointerEvent(this._sock, this._display.absX(x),
this._display.absY(y), mask);
if (this._isPrimaryDisplay) {
RFB.messages.pointerEvent(this._sock, this._display.absX(x), this._display.absY(y), mask);
} else {
this._proxyRFBMessage('pointerEvent', [ this._display.absX(x), this._display.absY(y), mask ]);
}
}
}
@ -1808,7 +2083,12 @@ export default class RFB extends EventTargetMixin {
if (this._rfbConnectionState !== 'connected') { return; }
if (this._viewOnly) { return; } // View only, skip mouse events
RFB.messages.pointerEvent(this._sock, this._display.absX(x), this._display.absY(y), 0, dX, dY);
if (this._isPrimaryDisplay){
RFB.messages.pointerEvent(this._sock, this._display.absX(x), this._display.absY(y), 0, dX, dY);
} else {
this._proxyRFBMessage('pointerEvent', [ this._display.absX(x), this._display.absY(y), 0, dX, dY ]);
}
}
_handleWheel(ev) {
@ -2585,7 +2865,10 @@ export default class RFB extends EventTargetMixin {
const encs = [];
// In preference order
encs.push(encodings.encodingCopyRect);
// Disable copyrect when using multiple displays
if (this._display.screens.length === 1) {
encs.push(encodings.encodingCopyRect);
}
// Only supported with full depth support
if (this._fbDepth == 24) {
encs.push(encodings.encodingTight);
@ -3244,9 +3527,9 @@ export default class RFB extends EventTargetMixin {
const payload = this._sock.rQshiftStr(len);
if (status) {
console.log("Unix relay subscription succeeded");
Log.Info("Unix relay subscription succeeded");
} else {
console.log("Unix relay subscription failed, " + payload);
Log.Warn("Unix relay subscription failed, " + payload);
}
}
@ -3570,7 +3853,7 @@ export default class RFB extends EventTargetMixin {
for (let i = 0; i < numberOfScreens; i += 1) {
// Save the id and flags of the first screen
if (i === 0) {
this._screenID = this._sock.rQshiftBytes(4); // id
this._screenIndex = this._sock.rQshiftBytes(4); // id
this._sock.rQskipBytes(2); // x-position
this._sock.rQskipBytes(2); // y-position
this._sock.rQskipBytes(2); // width
@ -3687,7 +3970,12 @@ export default class RFB extends EventTargetMixin {
rgbaPixels: rgba,
hotx: hotx, hoty: hoty, w: w, h: h,
};
this._refreshCursor();
if (this._isPrimaryDisplay) {
this._proxyRFBMessage('updateCursor', [ rgba, hotx, hoty, w, h ]);
}
}
_shouldShowDotCursor() {
@ -4061,40 +4349,49 @@ RFB.messages = {
}
},
setDesktopSize(sock, width, height, id, flags) {
setDesktopSize(sock, size, flags) {
const buff = sock._sQ;
const offset = sock._sQlen;
buff[offset] = 251; // msg-type
buff[offset + 1] = 0; // padding
buff[offset + 2] = width >> 8; // width
buff[offset + 3] = width;
buff[offset + 4] = height >> 8; // height
buff[offset + 5] = height;
buff[offset + 2] = size.serverWidth >> 8; // width
buff[offset + 3] = size.serverWidth;
buff[offset + 4] = size.serverHeight >> 8; // height
buff[offset + 5] = size.serverHeight;
buff[offset + 6] = 1; // number-of-screens
buff[offset + 6] = size.screens.length; // number-of-screens
buff[offset + 7] = 0; // padding
// screen array
buff[offset + 8] = id >> 24; // id
buff[offset + 9] = id >> 16;
buff[offset + 10] = id >> 8;
buff[offset + 11] = id;
buff[offset + 12] = 0; // x-position
buff[offset + 13] = 0;
buff[offset + 14] = 0; // y-position
buff[offset + 15] = 0;
buff[offset + 16] = width >> 8; // width
buff[offset + 17] = width;
buff[offset + 18] = height >> 8; // height
buff[offset + 19] = height;
buff[offset + 20] = flags >> 24; // flags
buff[offset + 21] = flags >> 16;
buff[offset + 22] = flags >> 8;
buff[offset + 23] = flags;
let i = 8;
for (let iS = 0; iS < size.screens.length; iS++) {
//screen id
buff[offset + i++] = iS >> 24;
buff[offset + i++] = iS >> 16;
buff[offset + i++] = iS >> 8;
buff[offset + i++] = iS;
//screen x position
buff[offset + i++] = size.screens[iS].x >> 8;
buff[offset + i++] = size.screens[iS].x;
//screen y position
buff[offset + i++] = size.screens[iS].y >> 8;
buff[offset + i++] = size.screens[iS].y;
//screen width
buff[offset + i++] = size.screens[iS].serverWidth >> 8;
buff[offset + i++] = size.screens[iS].serverWidth;
//screen height
buff[offset + i++] = size.screens[iS].serverHeight >> 8;
buff[offset + i++] = size.screens[iS].serverHeight;
//flags
buff[offset + i++] = flags >> 24;
buff[offset + i++] = flags >> 16;
buff[offset + i++] = flags >> 8;
buff[offset + i++] = flags;
}
sock._sQlen += 24;
sock._sQlen += i;
sock.flush();
},
setMaxVideoResolution(sock, width, height) {

16
core/util/element.js
View File

@ -28,5 +28,21 @@ export function clientToElement(x, y, elem) {
} else {
pos.y = y - bounds.top;
}
//multiple KasmVNC screens, Window can still receive mouse events when cursor goes
//outside of the window if the mouse is down while the moving occurs
if (x > window.innerWidth) {
pos.x += (x - window.innerWidth);
}
else if (x < 0) {
pos.x = x + bounds.left;
}
if (y > window.innerHeight) {
pos.y += (y - window.innerHeight);
}
else if (y < 0) {
pos.y = y + bounds.top;
}
return pos;
}

6
core/util/strings.js
View File

@ -26,3 +26,9 @@ export function decodeUTF8(utf8string, allowLatin1=false) {
export function encodeUTF8(DOMString) {
return unescape(encodeURIComponent(DOMString));
}
export function uuidv4() {
return ([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g, c =>
(c ^ crypto.getRandomValues(new Uint8Array(1))[0] & 15 >> c / 4).toString(16)
);
}

110
screen.html Normal file
View File

@ -0,0 +1,110 @@
<!DOCTYPE html>
<html lang="en" class="noVNC_loading">
<head>
<!--
noVNC is licensed under the MPL 2.0 (see LICENSE.txt)
This file is licensed under the 2-Clause BSD license (see LICENSE.txt).
-->
<title>Additional Display</title>
<meta charset="utf-8">
<!-- Icons (see app/images/icons/Makefile for what the sizes are for) -->
<link rel="icon" sizes="16x16" type="image/png" href="app/images/icons/368_kasm_logo_only_16x16.png">
<link rel="icon" sizes="24x24" type="image/png" href="app/images/icons/368_kasm_logo_only_24x24.png">
<link rel="icon" sizes="32x32" type="image/png" href="app/images/icons/368_kasm_logo_only_32x32.png">
<link rel="icon" sizes="48x48" type="image/png" href="app/images/icons/368_kasm_logo_only_48x48.png">
<link rel="icon" sizes="60x60" type="image/png" href="app/images/icons/368_kasm_logo_only_60x60.png">
<link rel="icon" sizes="64x64" type="image/png" href="app/images/icons/368_kasm_logo_only_64x64.png">
<link rel="icon" sizes="72x72" type="image/png" href="app/images/icons/368_kasm_logo_only_72x72.png">
<link rel="icon" sizes="76x76" type="image/png" href="app/images/icons/368_kasm_logo_only_76x76.png">
<link rel="icon" sizes="96x96" type="image/png" href="app/images/icons/368_kasm_logo_only_96x96.png">
<link rel="icon" sizes="120x120" type="image/png" href="app/images/icons/368_kasm_logo_only_120x120.png">
<link rel="icon" sizes="144x144" type="image/png" href="app/images/icons/368_kasm_logo_only_144x144.png">
<link rel="icon" sizes="152x152" type="image/png" href="app/images/icons/368_kasm_logo_only_152x152.png">
<link rel="icon" sizes="192x192" type="image/png" href="app/images/icons/368_kasm_logo_only_192x192.png">
<!-- Firefox currently mishandles SVG, see #1419039
<link rel="icon" sizes="any" type="image/svg+xml" href="app/images/icons/kasm-icon.svg">
-->
<!-- Repeated last so that legacy handling will pick this -->
<link rel="icon" sizes="16x16" type="image/png" href="app/images/icons/368_kasm_logo_only_16x16.png">
<!-- Apple iOS Safari settings -->
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<!-- Home Screen Icons (favourites and bookmarks use the normal icons) -->
<link rel="apple-touch-icon" sizes="60x60" type="image/png" href="app/images/icons/368_kasm_logo_only_60x60.png">
<link rel="apple-touch-icon" sizes="76x76" type="image/png" href="app/images/icons/368_kasm_logo_only_76x76.png">
<link rel="apple-touch-icon" sizes="120x120" type="image/png" href="app/images/icons/368_kasm_logo_only_120x120.png">
<link rel="apple-touch-icon" sizes="152x152" type="image/png" href="app/images/icons/368_kasm_logo_only_152x152.png">
<script src="vendor/interact.min.js"></script>
<!-- Stylesheets -->
<link rel="stylesheet" href="app/styles/base.css">
<script src="app/error-handler.js"></script>
<script>
let isInsideKasmVDI = false;
try {
isInsideKasmVDI = (window.self !== window.top);
} catch (e) {
isInsideKasmVDI = true;
}
</script>
<script type="module" crossorigin="use-credentials" src="app/ui_screen.js"></script>
</head>
<body>
<div id="noVNC_fallback_error" class="noVNC_center">
<div>
<div id="noVNC_close_error" onclick="document.getElementById('noVNC_fallback_error').remove()"></div>
<div>KasmVNC encountered an error:</div>
<br>
<div id="noVNC_fallback_errormsg"></div>
</div>
</div>
<!-- Status Dialog -->
<div id="noVNC_status"></div>
<!-- Connect button -->
<div class="noVNC_center">
<div id="noVNC_connect_dlg">
<div id="noVNC_connect_button">
<div class="image display">
<svg xmlns="http://www.w3.org/2000/svg" width="45"
viewBox="0 0 576 512"><!--! Font Awesome Pro 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. -->
<path
d="M64 16C37.5 16 16 37.5 16 64V352c0 26.5 21.5 48 48 48H231.8h.3H343.9h.3H512c26.5 0 48-21.5 48-48V64c0-26.5-21.5-48-48-48H64zM222.6 416H64c-35.3 0-64-28.7-64-64V64C0 28.7 28.7 0 64 0H512c35.3 0 64 28.7 64 64V352c0 35.3-28.7 64-64 64H353.4l13.3 80H440c4.4 0 8 3.6 8 8s-3.6 8-8 8H360 216 136c-4.4 0-8-3.6-8-8s3.6-8 8-8h73.2l13.3-80zm16.2 0l-13.3 80H350.6l-13.3-80H238.8zM496 80H80l0 256H496V80zM80 64H496c8.8 0 16 7.2 16 16V336c0 8.8-7.2 16-16 16H80c-8.8 0-16-7.2-16-16V80c0-8.8 7.2-16 16-16zm165.7 77.7L179.3 208l66.3 66.3c3.1 3.1 3.1 8.2 0 11.3s-8.2 3.1-11.3 0l-72-72c-3.1-3.1-3.1-8.2 0-11.3l72-72c3.1-3.1 8.2-3.1 11.3 0s3.1 8.2 0 11.3zm96-11.3l72 72c3.1 3.1 3.1 8.2 0 11.3l-72 72c-3.1 3.1-8.2 3.1-11.3 0s-3.1-8.2 0-11.3L396.7 208l-66.3-66.3c-3.1-3.1-3.1-8.2 0-11.3s8.2-3.1 11.3 0z" />
</svg>
<div class="power">
<svg xmlns="http://www.w3.org/2000/svg" height="25px" viewBox="0 0 512 512"><!--! Font Awesome Pro 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. --><path d="M272 16c0-8.8-7.2-16-16-16s-16 7.2-16 16V272c0 8.8 7.2 16 16 16s16-7.2 16-16V16zM164.3 78.2c7.8-4.1 10.9-13.7 6.9-21.6s-13.7-10.9-21.6-6.9C75 88.3 24 166.2 24 256c0 128.1 103.9 232 232 232s232-103.9 232-232c0-89.8-51-167.7-125.6-206.2c-7.8-4.1-17.5-1-21.6 6.9s-1 17.5 6.9 21.6C412.1 111.5 456 178.6 456 256c0 110.5-89.5 200-200 200S56 366.5 56 256c0-77.4 43.9-144.5 108.3-177.8z"/></svg>
</div>
</div>
<div class="text">
<div class="heading">Additional Display</div>
<div>Click to connect this additional display</div>
</div>
<div class="image go">
<svg xmlns="http://www.w3.org/2000/svg" height="20px" viewBox="0 0 448 512"><!--! Font Awesome Pro 6.4.2 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2023 Fonticons, Inc. --><path d="M443.3 267.3c6.2-6.2 6.2-16.4 0-22.6l-176-176c-6.2-6.2-16.4-6.2-22.6 0s-6.2 16.4 0 22.6L393.4 240 16 240c-8.8 0-16 7.2-16 16s7.2 16 16 16l377.4 0L244.7 420.7c-6.2 6.2-6.2 16.4 0 22.6s16.4 6.2 22.6 0l176-176z"/></svg>
</div>
</div>
</div>
</div>
<!-- This is where the RFB elements will attach -->
<div id="noVNC_container">
<!-- Note that Google Chrome on Android doesn't respect any of these,
html attributes which attempt to disable text suggestions on the
on-screen keyboard. Let's hope Chrome implements the ime-mode
style for example -->
<textarea id="noVNC_keyboardinput" autocapitalize="off"
autocomplete="off" spellcheck="false" tabindex="-1"></textarea>
</div>
<div id="noVNC_identify_monitor">0</div>
</body>

4
tests/README.md
View File

@ -46,4 +46,6 @@ This table keeps track of performance of pre-defined recordings, defined in the
| newyork.1 | 08233e6 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2446ms |
| losangeles.1 | 08233e6 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2272ms |
| newyork.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 2273ms |
| losangeles.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 1847ms |
| losangeles.1 | base64opt | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 106 | False | 1847ms |
| newyork.1 | 4a6aa73 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 119 | False | 2128ms |
| losangeles.1 | 4a6aa73 | Macbook M1 Pro, 32GB RAM | macOS 12.2 | Chrome 119 | False | 1766ms |

65
vnc.html
View File

@ -50,7 +50,7 @@
<script src="vendor/interact.min.js"></script>
<!-- Stylesheets -->
<!--link rel="stylesheet" href="app/styles/base.css">
<link rel="stylesheet" href="app/styles/base.css">
<script src="app/error-handler.js"></script>
@ -63,7 +63,7 @@
}
</script>
<script type="module" crossorigin="use-credentials" src="app/ui.js"></script-->
<script type="module" crossorigin="use-credentials" src="app/ui.js"></script>
</head>
<body>
@ -185,6 +185,15 @@
Fullscreen
</div>
<!-- Add Second Screen -->
<div class="noVNC_button_div noVNC_hidden" >
<input type="image" alt="Fullscreen" src="app/images/desktop-regular.svg"
id="noVNC_displays_button" class="noVNC_button"
style="margin: 10px 3px;"
title="Show Displays">
Displays
</div>
<!-- Toggle game mode -->
<div class="noVNC_button_div noVNC_hidden noVNC_hide_on_disconnect" >
<input type="image" alt="Game Mode" src="app/images/gamepad.png"
@ -290,12 +299,6 @@
<span class="slider-label">Toggle Control Panel via Keystrokes</span>
</label>
</li>
<li>
<label class="switch"><input id="noVNC_setting_enable_hidpi" type="checkbox" />
<span class="slider round"></span>
<span class="slider-label">Render Native Resolution</span>
</label>
</li>
<li class="noVNC_hidden">
<label for="noVNC_setting_idle_disconnect">Idle Timeout:</label>
<select id="noVNC_setting_idle_disconnect" name="vncIdleDisconnect">
@ -520,7 +523,7 @@
</div>
<!-- Connection Controls -->
<div class="noVNC_button_div noVNC_hide_on_connect" id="noVNC_connect_button" >
<div class="noVNC_button_div noVNC_hide_on_connect" id="noVNC_connect_button_2">
<input type="image" alt="Connect" src="app/images/connect.svg"
class="noVNC_button"
title="Connect">
@ -537,6 +540,30 @@
<!-- Status Dialog -->
<div id="noVNC_status"></div>
<!-- Status Dialog -->
<div id="noVNC_displays">
<div class="canvas">
<div class="canvas-title">Arrange Displays</div>
<div class="canvas-text">Drag and drop to arrange displays, new monitors are added to the right hand side of the previous monitor.</div>
<div id="noVNC_refreshMonitors"><svg id="noVNC_refreshMonitors_icon" xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 512 512"><path class="fa-primary" d="M105.1 202.6c7.7-21.8 20.2-42.3 37.8-59.8c62.5-62.5 163.8-62.5 226.3 0L386.3 160H336c-17.7 0-32 14.3-32 32s14.3 32 32 32H463.5c0 0 0 0 0 0h.4c17.7 0 32-14.3 32-32V64c0-17.7-14.3-32-32-32s-32 14.3-32 32v51.2L414.4 97.6c-87.5-87.5-229.3-87.5-316.8 0C73.2 122 55.6 150.7 44.8 181.4c-5.9 16.7 2.9 34.9 19.5 40.8s34.9-2.9 40.8-19.5z"/><path class="fa-secondary" d="M80 396.9V448c0 17.7-14.3 32-32 32s-32-14.3-32-32V320c0-17.7 14.3-32 32-32H176c17.7 0 32 14.3 32 32s-14.3 32-32 32H125.6l17.2 17.1c62.5 62.5 163.8 62.5 226.3 0c17.5-17.5 30.1-38 37.8-59.8c5.9-16.7 24.2-25.4 40.8-19.5s25.4 24.2 19.5 40.8c-10.8 30.6-28.4 59.3-52.9 83.8c-87.5 87.5-229.3 87.5-316.7 0L80 396.9z"/></svg></div>
<canvas id="noVNC_multiMonitorWidget" class="" width="700" height="230"></canvas>
<div class="arrange-buttons">
<div style="gap: 10px;" class="flex">
<button id="noVNC_addMonitor">
<svg xmlns="http://www.w3.org/2000/svg" height="1em" viewBox="0 0 448 512"><path fill="currentColor" d="M240 64c0-8.8-7.2-16-16-16s-16 7.2-16 16V240H32c-8.8 0-16 7.2-16 16s7.2 16 16 16H208V448c0 8.8 7.2 16 16 16s16-7.2 16-16V272H416c8.8 0 16-7.2 16-16s-7.2-16-16-16H240V64z"/></svg>
Add Monitor
</button>
<label id="noVNC_setting_enable_hidpi_option" class="button">
<input style="margin: 0 10px 0 3px;" id="noVNC_setting_enable_hidpi" type="checkbox" />
Native Resolution
</label>
<button id="noVNC_identify_monitors_button">Identify</button>
</div>
<button id="noVNC_close_displays">Done</button>
</div>
</div>
</div>
<!-- Connect button -->
<div class="noVNC_center">
<div id="noVNC_connect_dlg">
@ -548,25 +575,6 @@
</div>
</div>
<!-- Password Dialog -->
<div class="noVNC_center noVNC_connect_layer">
<div id="noVNC_credentials_dlg" class="noVNC_panel"><form>
<ul>
<li id="noVNC_username_block">
<label>Username:</label>
<input id="noVNC_username_input">
</li>
<li id="noVNC_password_block">
<label>Password:</label>
<input id="noVNC_password_input" type="password">
</li>
<li>
<input id="noVNC_credentials_button" type="submit" value="Send Credentials" class="noVNC_submit">
</li>
</ul>
</form></div>
</div>
<!-- Transition Screens -->
<div id="noVNC_transition">
<div id="noVNC_transition_text"></div>
@ -603,5 +611,6 @@
<div id="noVNC_keyboard_control_handle" class="button keyboard handle"></div>
</div>
<div id="noVNC_identify_monitor">0</div>
</body>
</html>