Commit Graph

846 Commits

Author SHA1 Message Date
Chocobozzz 56f4783075
Support two factor authentication in backend 2022-10-07 10:51:16 +02:00
kontrollanten 6c5f0d3aeb
server: serve files from storage/well-known (#5214)
* server: serve files from storage/well-known

closes #5206

* well-known: add tests

* test: try to skip new tests

* test: another try

* fix(config/prod): well_known path

* test: fix broken tests

* Update misc-endpoints.ts

* Use getDirectoryPath for tests

* Fix tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-10-04 10:53:00 +02:00
Chocobozzz db312c7023
Fix old DB enum names 2022-09-13 15:22:02 +02:00
Chocobozzz 405c83f9af
Use worker thread to send HTTP requests
Compute HTTP signature could be CPU intensive
2022-09-08 12:27:22 +02:00
Chocobozzz 690bb8f9f3
Prefer using Object.values 2022-08-17 15:36:03 +02:00
Chocobozzz a3b472a12e
Add ability to list imports of a channel sync 2022-08-10 14:32:00 +02:00
Florent 2a491182e4
Channel sync (#5135)
* Add external channel URL for channel update / creation (#754)

* Disallow synchronisation if user has no video quota (#754)

* More constraints serverside (#754)

* Disable sync if server configuration does not allow HTTP import (#754)

* Working version synchronizing videos with a job (#754)

TODO: refactoring, too much code duplication

* More logs and try/catch (#754)

* Fix eslint error (#754)

* WIP: support synchronization time change (#754)

* New frontend #754

* WIP: Create sync front (#754)

* Enhance UI, sync creation form (#754)

* Warning message when HTTP upload is disallowed

* More consistent names (#754)

* Binding Front with API (#754)

* Add a /me API (#754)

* Improve list UI (#754)

* Implement creation and deletion routes (#754)

* Lint (#754)

* Lint again (#754)

* WIP: UI for triggering import existing videos (#754)

* Implement jobs for syncing and importing channels

* Don't sync videos before sync creation + avoid concurrency issue (#754)

* Cleanup (#754)

* Cleanup: OpenAPI + API rework (#754)

* Remove dead code (#754)

* Eslint (#754)

* Revert the mess with whitespaces in constants.ts (#754)

* Some fixes after rebase (#754)

* Several fixes after PR remarks (#754)

* Front + API: Rename video-channels-sync to video-channel-syncs (#754)

* Allow enabling channel sync through UI (#754)

* getChannelInfo (#754)

* Minor fixes: openapi + model + sql (#754)

* Simplified API validators (#754)

* Rename MChannelSync to MChannelSyncChannel (#754)

* Add command for VideoChannelSync (#754)

* Use synchronization.enabled config (#754)

* Check parameters test + some fixes (#754)

* Fix conflict mistake (#754)

* Restrict access to video channel sync list API (#754)

* Start adding unit test for synchronization (#754)

* Continue testing (#754)

* Tests finished + convertion of job to scheduler (#754)

* Add lastSyncAt field (#754)

* Fix externalRemoteUrl sort + creation date not well formatted (#754)

* Small fix (#754)

* Factorize addYoutubeDLImport and buildVideo (#754)

* Check duplicates on channel not on users (#754)

* factorize thumbnail generation (#754)

* Fetch error should return status 400 (#754)

* Separate video-channel-import and video-channel-sync-latest (#754)

* Bump DB migration version after rebase (#754)

* Prettier states in UI table (#754)

* Add DefaultScope in VideoChannelSyncModel (#754)

* Fix audit logs (#754)

* Ensure user can upload when importing channel + minor fixes (#754)

* Mark synchronization as failed on exception + typos (#754)

* Change REST API for importing videos into channel (#754)

* Add option for fully synchronize a chnanel (#754)

* Return a whole sync object on creation to avoid tricks in Front (#754)

* Various remarks (#754)

* Single quotes by default (#754)

* Rename synchronization to video_channel_synchronization

* Add check.latest_videos_count and max_per_user options (#754)

* Better channel rendering in list #754

* Allow sorting with channel name and state (#754)

* Add missing tests for channel imports (#754)

* Prefer using a parent job for channel sync

* Styling

* Client styling

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-08-10 09:53:39 +02:00
Chocobozzz cfb5edbd9e
Increase logs stack trace max size 2022-08-09 11:34:56 +02:00
Chocobozzz bd911b54b5
Use bullmq job dependency 2022-08-09 09:18:07 +02:00
Chocobozzz 5a921e7b74
Move to bullmq 2022-08-09 09:18:07 +02:00
Chocobozzz 84cae54e7a
Add option to not transcode original resolution 2022-08-09 09:18:07 +02:00
Chocobozzz c795e19663
Automatically rebuild native modules on ABI change 2022-08-03 15:08:36 +02:00
Chocobozzz 927fa4b11f
Add rejected state to follows
Prevent reprocessing already rejected follows
2022-07-27 13:52:13 +02:00
Florent 3efa4da1fe Fix password for root in dev mode
password should remain "test" in dev mode for root.
Fix regression of 9452d4fd33 which introduces NODE_ENV=dev
2022-07-26 10:00:17 +02:00
Chocobozzz c8fa571f32
Clearer live session
Get the save replay setting when the session started to prevent
inconsistent behaviour when the setting changed before the session was
processed by the live ending job

Display more information about the potential session replay in live
modal information
2022-07-22 15:22:21 +02:00
Chocobozzz 1efad362ef
Display a warning if using ffmpeg 4.4.0 2022-07-18 14:53:20 +02:00
Chocobozzz 42b4063699
Add ability for client to create server logs 2022-07-18 11:37:18 +02:00
Chocobozzz 5d7cb63ede
Add compat with openssl 3 2022-07-12 12:08:12 +02:00
Chocobozzz 9452d4fd33
/!\ Use a dedicated config file for development
It means you have to replace NODE_ENV=test to NODE_ENV=dev if you use it
npm run dev:* commands are already updated
2022-07-06 16:12:17 +02:00
Chocobozzz 630d0a1bf5
Introduce experimental telemetry 2022-07-06 15:13:55 +02:00
Chocobozzz a4152bed14
Add ability to disable unknown infohash log 2022-06-30 09:13:11 +02:00
Chocobozzz 3a54605d4e
Process images in a dedicated worker 2022-06-27 11:53:12 +02:00
Chocobozzz c53853ca1b
Introduce worker threads to process remote images 2022-06-24 16:31:48 +02:00
Chocobozzz 978489b64c
Add toki pona in video languages 2022-06-24 11:59:12 +02:00
Chocobozzz 92083e4228
Fix removed sha segments on fast restream 2022-06-24 09:57:55 +02:00
Chocobozzz 64324ac646
Merge branch 'release/4.2.0' into develop 2022-06-23 10:53:16 +02:00
Chocobozzz 53023be33a
Fix fast restream in saved permanent live 2022-06-23 10:30:48 +02:00
kontrollanten 2e401e8575
store uploaded video filename (#4885)
* store uploaded video filename

closes #4731

* dont crash if videos channel exist

* migration: use raw query

* video source: fixes after code review

* cleanup

* bump migration

* updates after code review

* refactor: use checkUserCanManageVideo

* videoSource: add openapi doc

* test(check-params/video-source): fix timeout

* Styling

* Correctly set original filename as source

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-06-21 15:31:25 +02:00
Chocobozzz 714e33a742
Merge branch 'release/4.2.0' into develop 2022-06-20 10:23:19 +02:00
Chocobozzz f53e1fcd99
Relax views federation 2022-06-20 10:04:52 +02:00
Chocobozzz aa2ce188d1
Optimize view endpoint 2022-06-17 14:56:16 +02:00
Chocobozzz fba911e2c8
Merge branch 'release/4.2.0' into develop 2022-06-17 14:17:06 +02:00
Chocobozzz f27b7a750f
Send views in a dedicated queue 2022-06-17 14:08:13 +02:00
Chocobozzz 42e6a7a747
Increase max watch sections 2022-06-17 09:04:45 +02:00
luz paz 7a4fd56ccd Fix various typos
Found via `codespell -q 3 -S ./CREDITS.md,./CHANGELOG.md,./client/src/locale,./yarn.lock,./client/yarn.lock -L doubleclick,followings,nd,ot,ro,serie,splitted,tread,truthy`
2022-06-07 16:41:21 +02:00
Chocobozzz 600befbd61
Check feeds config at startup 2022-05-24 15:57:02 +02:00
Chocobozzz db9d882c25
Add ability to set default feed items count 2022-05-24 15:56:23 +02:00
Chocobozzz 87a0cac618
Improve admin users list table
* Fix last login sort with null values
 * Remember last selected columns
 * Display last login date by default
2022-05-24 15:29:17 +02:00
Chocobozzz 3eba7ab815
Add ability to set a custom video import timeout 2022-05-24 14:55:07 +02:00
Chocobozzz 543affef08
Don't sign (created) when using rsa-sha256 2022-05-09 09:43:36 +02:00
Chocobozzz e08ec7a723
Handle HTTP signature draft 11 2022-05-06 15:13:59 +02:00
Chocobozzz f40712abbb
Add ability to filter overall video stats by date 2022-05-06 14:23:02 +02:00
Chocobozzz 26e3e98ff0
Support live session in server 2022-05-03 14:49:15 +02:00
Chocobozzz c74cd9feab
Fix removing directory content 2022-05-03 08:23:59 +02:00
Chocobozzz 21d70a7302
Support readonly tmp directory 2022-05-03 08:18:48 +02:00
Chocobozzz d05af385f4
Ensure rtmp public_hostname is set 2022-05-02 14:38:37 +02:00
Chocobozzz cfbe6be516
Add ability to provide rtmp listening hostname 2022-05-02 14:32:12 +02:00
Chocobozzz 4b7a20d88b
Add missing video mime types 2022-04-20 10:23:07 +02:00
Chocobozzz ac907dc7c1 Improve viewer counter
More precise, avoid weird decrease, reuse an id to federate viewers
2022-04-15 09:49:35 +02:00
Chocobozzz dfbcefc20d Improve views/viewers documentation 2022-04-15 09:49:35 +02:00
Chocobozzz b211106695 Support video views/viewers stats in server
* Add "currentTime" and "event" body params to view endpoint
 * Merge watching and view endpoints
 * Introduce WatchAction AP activity
 * Add tables to store viewer information of local videos
 * Add endpoints to fetch video views/viewers stats of local videos
 * Refactor views/viewers handlers
 * Support "views" and "viewers" counters for both VOD and live videos
2022-04-15 09:49:35 +02:00
Chocobozzz 92e66e04f7
Rename studio to editor 2022-03-22 17:24:32 +01:00
Chocobozzz 1808a1f8e4
Add video edition finished notification 2022-03-22 16:25:14 +01:00
Chocobozzz 348c2ce3ff
Remove old migration files 2022-03-21 14:54:29 +01:00
Chocobozzz 57e4e1c1a9
Don't store remote rates of remote videos
In the future we'll stop to expose all available rates to improve users
privacy
2022-03-18 11:21:50 +01:00
Chocobozzz f012319a64
Process video torrents in order
Prevent update before video torrent generation for example
2022-03-16 15:45:58 +01:00
Doug Luce f9915efa5e Make object storage ACL configurable
Override this value to allow uploads to non-public S3
buckets. Otherwise "AccessDenied: Access Denied" errors will end up in
the log.

Fixes #4850
2022-03-16 10:58:01 +01:00
kontrollanten 60233e90d2
add client.videos.upload.maxChunkSize config (#4857)
* add client.videos.upload.maxChunkSize config

* updates after review

* remove maxChunkSize from custom-config
2022-03-16 10:57:36 +01:00
Chocobozzz f443a74649 Add latency setting support 2022-03-09 09:23:10 +01:00
kontrollanten 2c102aac13
Display avatar in video miniatures (#4823)
* show avatar in video miniatures

closes #4040

* fix: videos for api, include avatars

* add config for display video miniature avatar

* Revert "fix: videos for api, include avatars"

This reverts commit 5520a3eda6.

* video-min: set display avatar to false as default

* remove empty line

* rename display_avatar > display_author_avatar

* fix renaming displayAvatar > displayAuthorAvatar
2022-03-08 13:39:27 +01:00
Chocobozzz cc45cc9ac1
Remove uneeded memoize 2022-03-08 11:39:56 +01:00
Chocobozzz d3d3deaa7a
Refactor user build and express file middlewares 2022-03-04 10:57:36 +01:00
Chocobozzz 90fbb78498
Node 12 is not supported anymore 2022-03-01 14:08:03 +01:00
kontrollanten 8d07888728
Possibility to set custom RTMP/RTMPS hostname (#4811)
* live: set custom RTMP/RTMPS hostname

closes #4786

* dont use webserver.hostname as default

* check that rtmp/s.hostname is set
2022-03-01 13:37:34 +01:00
Chocobozzz f7ac03ee94
Fix client html performance regression 2022-02-28 15:13:56 +01:00
Chocobozzz c729caf6cc Add basic video editor support 2022-02-28 10:42:19 +01:00
kontrollanten d0800f7661
Implement avatar miniatures (#4639)
* client: remove unused file

* refactor(client/my-actor-avatar): size from input

Read size from component input instead of scss, to make it possible to
use smaller avatar images when implemented.

* implement avatar miniatures

close #4560

* fix(test): max file size

* fix(search-index): normalize res acc to avatarMini

* refactor avatars to an array

* client/search: resize channel avatar to 120

* refactor(client/videos): remove unused function

* client(actor-avatar): set default size

* fix tests and avatars full result

When findOne is used only an array containting one avatar is returned.

* update migration version and version notations

* server/search: harmonize normalizing

* Cleanup avatar miniature PR

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-02-28 08:34:43 +01:00
Chocobozzz 5301e7762b
Fix video file storage column consistency 2022-02-22 14:49:23 +01:00
Chocobozzz 7630e1c893
Fix import timeout inconsistency 2022-02-09 11:54:18 +01:00
Chocobozzz 419b520ca4
Add ability to cancel & delete video imports 2022-01-19 14:31:05 +01:00
Chocobozzz 8f7d35fd0f
Add simplified/traditional chinese languages 2022-01-17 11:33:50 +01:00
Chocobozzz ce7c099f0e
Increase captions max size 2022-01-14 08:47:27 +01:00
Florent 22c7778630 Give the ability to define python path for youtube-dl / yt-dlp 2022-01-10 10:01:43 +01:00
Chocobozzz 0628157fe9
Move uuid stuff in extra utils
Since it requires an external dependency
2021-12-29 14:44:58 +01:00
Chocobozzz f1569117f9
Cleanup unavailable remote AP resource 2021-12-28 11:37:05 +01:00
Chocobozzz 4ff75a3b2c
Deprecate node 12 2021-12-24 13:52:32 +01:00
Chocobozzz 5a63748877
Node 10 is not supported anymore 2021-12-24 13:43:59 +01:00
Chocobozzz d17c7b4e8c
Fix shared imports 2021-12-24 10:14:47 +01:00
Chocobozzz dbd9fb44dd
Don't stuck state when move transcoding job failed 2021-12-23 11:13:06 +01:00
Chocobozzz 6def7d34e6
Fix migrations 2021-12-17 14:27:16 +01:00
Chocobozzz c55e3d7227
Move test functions outside extra-utils 2021-12-17 12:24:03 +01:00
Chocobozzz b65de1be4d
Use different p2p policy for embeds and webapp 2021-12-16 10:08:55 +01:00
Chocobozzz a9bfa85d2c
Add ability for admins to set default p2p policy 2021-12-16 10:08:55 +01:00
lutangar 06aad80165 chore(refactor): remove shared folder dependencies to the server
Many files from the `shared` folder were importing files from the `server` folder.
When attempting to use Typescript project references to describe dependencies,
it highlighted a circular dependency beetween `shared` <-> `server`.

The Typescript project forbid such usages.
Using project references greatly improve performance by rebuilding only
the updated project and not all source files.
> see https://www.typescriptlang.org/docs/handbook/project-references.html
2021-12-16 10:08:43 +01:00
Chocobozzz 3cf68b869d
Ability for admins to set default upload values 2021-12-14 17:17:01 +01:00
Florian CUNY 81547acb25
Added "zxx" (no linguistic content) in available video languages (#4631)
* Added "zxx" (no linguistic content) in available video languages

Implements https://github.com/Chocobozzz/PeerTube/issues/1489

* FIx lint

* Fix lint

* Put other languages below instance languages

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-12-13 15:32:14 +01:00
Chocobozzz 0bc53e2029
Add ability to redirect users on external auth 2021-12-03 17:04:47 +01:00
Chocobozzz 6d472b4046
Decrease time to cleanup resumable uploads 2021-12-03 14:01:38 +01:00
Chocobozzz ca87d95bcb
Fix plugin upgrade
Correctly decache all plugin paths
2021-12-03 10:14:17 +01:00
Chocobozzz 4c99953acd
Fix redundancy timeout 2021-11-29 15:45:02 +01:00
Chocobozzz 51353d9a03 Refactor video views
Introduce viewers attribute for live videos
Count views for live videos
Reduce delay to see the viewer update for lives
Add ability to configure video views buffer interval and view ip
expiration
2021-11-09 15:00:31 +01:00
kontrollanten 4e29f4fe23 video: add video stranscoding_failed state 2021-11-09 15:00:21 +01:00
Chocobozzz df1db951c5
Support RTMPS 2021-11-05 11:38:17 +01:00
Florian CUNY 8dd754c767
Added 144p encoding (#4492)
* Added 144p encoding

Implements https://github.com/Chocobozzz/PeerTube/issues/4428

* Fixed typo in core-utils

* Increased BitPerPixel for 144p

* Disabled 144p by default in test.yaml

* Another try at fixing tests

* Fixed test in video-transcoder (api-3)

* Fixed test in video-imports (api-4)

* Fixed test in live-constraints (api-2)

* Tried to fix tests in api-3 again

* Revert "Tried to fix tests in api-3 again"

This reverts commit 266e1143fa37f333d149c2c2791c7bd33621ac14.

* Fixed test in config.ts (api-2)

* Try to fix test in video-hls.ts (api-3)

* Fixed test in video-transcoder.ts (api-3)

* Fix tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-11-05 10:23:02 +01:00
Florian CUNY 754b6f5f41
Made the video channels limit (per user) server-wide configurable (#4491)
* Made the video channels limit (per user) server-wide configurable

Implements https://github.com/Chocobozzz/PeerTube/issues/3092

Also added a "quota bar" in the account's settings page

* Fixed lint errors

* Another pass at fixing lint errors

* Applied code suggestions

* Removed 'video channels quota'
2021-10-26 16:42:10 +02:00
Rigel Kent 276250f0a3
prevent multiple post-process triggering of upload-resumable (#4175)
* prevent multiple post-process triggering of upload-resumable

* switch from 409 to 503 for upload being processed

* Improve resumable upload check

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-10-25 17:42:20 +02:00
Chocobozzz 7fb2c60bbe
Remove sync call in logger
Winston already does it
2021-10-22 16:25:16 +02:00
Chocobozzz 7f91ecbde4
Completely remove torrent static path 2021-10-22 10:32:53 +02:00
Chocobozzz 619537426b
Constants consistency 2021-10-22 10:28:00 +02:00
Chocobozzz 62549e6c98
Rewrite youtube-dl import
Use python3 binary
Allows to use a custom youtube-dl release URL
Allows to use yt-dlp (youtube-dl fork)
Remove proxy config from configuration to use HTTP_PROXY and HTTPS_PROXY
env variables
2021-10-22 10:25:24 +02:00