Commit Graph

307 Commits

Author SHA1 Message Date
Chocobozzz 182082f52d
Update bullmq
Requires redis >= 6.2
2023-02-16 14:10:11 +01:00
Chocobozzz 20321f2049
Reencode the video on cut
Unfortunately copying audio/video is not precise enough and could lead
to inconsistencies
2023-02-14 14:06:34 +01:00
Chocobozzz 1c30b112b9
Fix lint 2023-01-09 14:42:52 +01:00
Chocobozzz 5949eca71f
Increase old jobs to delete per hour 2022-11-28 15:16:15 +01:00
Chocobozzz c3b21b68b5
Delete completed/failed jobs directly from bullmq 2022-11-23 17:13:01 +01:00
Chocobozzz b1dbb9fefc
Merge branch 'release/4.3.0' into develop 2022-11-16 14:40:34 +01:00
Chocobozzz f240fb4bea
Fix broadcasting in parallel views 2022-11-16 14:40:10 +01:00
Chocobozzz 99b757488c
Fix server lint 2022-11-15 15:00:23 +01:00
kontrollanten 564b9b5597
refactor(server): redis > ioredis (#5371)
* refactor(server): redis > ioredis

* refactor(JobQueue): reuse redis connection builder

* fix(redisio)

* fix(redis): setValue

* feat(redis): showFriendlyErrorStack

* feat(redis): auto pipelining

308017a6b9/README.md (autopipelining)

* dont use autopipelining for bullmq

* ioredis events
2022-11-14 16:26:20 +01:00
Chocobozzz 4efa5535cc
Fix infinite playlist import
Using an hard videos limit in config
2022-11-14 11:11:39 +01:00
Chocobozzz a32bf8cd20
Fix running again transcoding on a video only file 2022-11-07 10:25:24 +01:00
Chocobozzz 5fc4a1825c
Fix preview generation for imports 2022-10-31 14:29:48 +01:00
Chocobozzz 6740b6428b
Fix transcoding failure when importing a video 2022-10-31 13:30:39 +01:00
Chocobozzz 849f0fd3b2
Lock files to generate torrents/move files 2022-10-25 11:50:44 +02:00
Chocobozzz 3545e72c68 Put private videos under a specific subdirectory 2022-10-24 14:48:24 +02:00
Chocobozzz e37ca6cbc7
Prevent error log 2022-10-04 11:17:37 +02:00
Chocobozzz cfd57d2ca0
Live supports object storage
* Sync live files (segments, master playlist, resolution playlist,
   segment sha file) into object storage
 * Automatically delete them when the live ends
 * Segment sha file is now a file on disk, and not stored in memory
   anymore
2022-10-04 10:03:17 +02:00
Chocobozzz 97922ecf64
Handle sync failure in synchronizeChannel fn 2022-09-27 09:50:18 +02:00
Chocobozzz 3204f4d17d
Fix sync import of latest videos 2022-09-16 09:37:07 +02:00
Chocobozzz 4404a7c467
Prevent job queue to be started before plugins 2022-09-14 11:45:18 +02:00
Chocobozzz e4fc3697ac
Fix saved live master playlist bandwidth 2022-09-09 09:21:42 +02:00
Chocobozzz 405c83f9af
Use worker thread to send HTTP requests
Compute HTTP signature could be CPU intensive
2022-09-08 12:27:22 +02:00
Chocobozzz ab08ab4e28
Don't crash on redis connection error 2022-08-11 11:30:06 +02:00
Chocobozzz a3b472a12e
Add ability to list imports of a channel sync 2022-08-10 14:32:00 +02:00
Florent 2a491182e4
Channel sync (#5135)
* Add external channel URL for channel update / creation (#754)

* Disallow synchronisation if user has no video quota (#754)

* More constraints serverside (#754)

* Disable sync if server configuration does not allow HTTP import (#754)

* Working version synchronizing videos with a job (#754)

TODO: refactoring, too much code duplication

* More logs and try/catch (#754)

* Fix eslint error (#754)

* WIP: support synchronization time change (#754)

* New frontend #754

* WIP: Create sync front (#754)

* Enhance UI, sync creation form (#754)

* Warning message when HTTP upload is disallowed

* More consistent names (#754)

* Binding Front with API (#754)

* Add a /me API (#754)

* Improve list UI (#754)

* Implement creation and deletion routes (#754)

* Lint (#754)

* Lint again (#754)

* WIP: UI for triggering import existing videos (#754)

* Implement jobs for syncing and importing channels

* Don't sync videos before sync creation + avoid concurrency issue (#754)

* Cleanup (#754)

* Cleanup: OpenAPI + API rework (#754)

* Remove dead code (#754)

* Eslint (#754)

* Revert the mess with whitespaces in constants.ts (#754)

* Some fixes after rebase (#754)

* Several fixes after PR remarks (#754)

* Front + API: Rename video-channels-sync to video-channel-syncs (#754)

* Allow enabling channel sync through UI (#754)

* getChannelInfo (#754)

* Minor fixes: openapi + model + sql (#754)

* Simplified API validators (#754)

* Rename MChannelSync to MChannelSyncChannel (#754)

* Add command for VideoChannelSync (#754)

* Use synchronization.enabled config (#754)

* Check parameters test + some fixes (#754)

* Fix conflict mistake (#754)

* Restrict access to video channel sync list API (#754)

* Start adding unit test for synchronization (#754)

* Continue testing (#754)

* Tests finished + convertion of job to scheduler (#754)

* Add lastSyncAt field (#754)

* Fix externalRemoteUrl sort + creation date not well formatted (#754)

* Small fix (#754)

* Factorize addYoutubeDLImport and buildVideo (#754)

* Check duplicates on channel not on users (#754)

* factorize thumbnail generation (#754)

* Fetch error should return status 400 (#754)

* Separate video-channel-import and video-channel-sync-latest (#754)

* Bump DB migration version after rebase (#754)

* Prettier states in UI table (#754)

* Add DefaultScope in VideoChannelSyncModel (#754)

* Fix audit logs (#754)

* Ensure user can upload when importing channel + minor fixes (#754)

* Mark synchronization as failed on exception + typos (#754)

* Change REST API for importing videos into channel (#754)

* Add option for fully synchronize a chnanel (#754)

* Return a whole sync object on creation to avoid tricks in Front (#754)

* Various remarks (#754)

* Single quotes by default (#754)

* Rename synchronization to video_channel_synchronization

* Add check.latest_videos_count and max_per_user options (#754)

* Better channel rendering in list #754

* Allow sorting with channel name and state (#754)

* Add missing tests for channel imports (#754)

* Prefer using a parent job for channel sync

* Styling

* Client styling

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-08-10 09:53:39 +02:00
Chocobozzz 51335c72cf
Prevent job failure on concurrent HLS transcoding 2022-08-09 13:21:18 +02:00
Chocobozzz e2b2c726b1
Fix job queue tests 2022-08-09 11:39:31 +02:00
Chocobozzz b42c2c7e89
Avoid concurrency issue on transcoding 2022-08-09 09:32:17 +02:00
Chocobozzz bd911b54b5
Use bullmq job dependency 2022-08-09 09:18:07 +02:00
Chocobozzz 5a921e7b74
Move to bullmq 2022-08-09 09:18:07 +02:00
Chocobozzz 5e2afe4290
Limit import depending on transcoding resolutions 2022-08-09 09:18:07 +02:00
Chocobozzz 64fd6158fd
Fix resolution to transcode hook name 2022-08-09 09:18:07 +02:00
Chocobozzz 84cae54e7a
Add option to not transcode original resolution 2022-08-09 09:18:07 +02:00
Chocobozzz ebb9e53ada
Add plugin hook on transcoding resolutions building 2022-08-02 16:05:44 +02:00
Chocobozzz 22df69fdec
Add job queue hooks 2022-08-02 15:50:05 +02:00
Chocobozzz 1bb4c9ab2e
Add ability to delete a specific video file 2022-08-01 14:55:10 +02:00
Chocobozzz c8fa571f32
Clearer live session
Get the save replay setting when the session started to prevent
inconsistent behaviour when the setting changed before the session was
processed by the live ending job

Display more information about the potential session replay in live
modal information
2022-07-22 15:22:21 +02:00
Chocobozzz 9452d4fd33
/!\ Use a dedicated config file for development
It means you have to replace NODE_ENV=test to NODE_ENV=dev if you use it
npm run dev:* commands are already updated
2022-07-06 16:12:17 +02:00
Chocobozzz 630d0a1bf5
Introduce experimental telemetry 2022-07-06 15:13:55 +02:00
Chocobozzz 4fae2b1f30
Rename video full loading 2022-06-28 15:06:15 +02:00
Chocobozzz 92083e4228
Fix removed sha segments on fast restream 2022-06-24 09:57:55 +02:00
Chocobozzz 64324ac646
Merge branch 'release/4.2.0' into develop 2022-06-23 10:53:16 +02:00
Chocobozzz 53023be33a
Fix fast restream in saved permanent live 2022-06-23 10:30:48 +02:00
Chocobozzz fba911e2c8
Merge branch 'release/4.2.0' into develop 2022-06-17 14:17:06 +02:00
Chocobozzz f27b7a750f
Send views in a dedicated queue 2022-06-17 14:08:13 +02:00
Chocobozzz 3256771725
More robust move to object storage job failure 2022-06-16 14:29:53 +02:00
Chocobozzz cdd838168d
Fix live ending job that breaks new live session 2022-06-16 13:39:57 +02:00
Chocobozzz c8fdfab0e3
More robust live ending job 2022-05-25 15:18:29 +02:00
Chocobozzz 5333788c08
Fix saving permanent live replay on quick restream 2022-05-25 14:54:16 +02:00
Chocobozzz 941d28cc7f
Reduce error logs 2022-05-18 12:01:02 +02:00