Commit Graph

2543 Commits

Author SHA1 Message Date
Chocobozzz 819b656439
Merge branch 'release/3.2.0' into develop 2021-05-25 13:14:34 +02:00
Chocobozzz fc8f15d202
Increase plugin transcoding tests timeout 2021-05-25 13:14:05 +02:00
Chocobozzz c4d125527a
Avoid error when file has no torrent file 2021-05-25 11:35:29 +02:00
Chocobozzz d61fa154af
Fix duplicate ffmpeg preset option for live 2021-05-25 11:35:29 +02:00
Chocobozzz 31a911190b
Fix broadcast message log level 2021-05-25 11:35:29 +02:00
John Livingston fa263eeff7
Fix 4106: default boolean plugin setting for frontend 2021-05-25 11:35:29 +02:00
Chocobozzz be89e66895
Avoid error when file has no torrent file 2021-05-25 11:34:43 +02:00
Chocobozzz e318e0ebe1
Fix duplicate ffmpeg preset option for live 2021-05-25 11:34:43 +02:00
Chocobozzz 008af5c9bb
Fix broadcast message log level 2021-05-25 11:34:43 +02:00
John Livingston 8bb6e70be3 Fix 4106: default boolean plugin setting for frontend 2021-05-25 11:34:22 +02:00
Chocobozzz 5b1a6d45b5
Merge branch 'release/3.2.0' into develop 2021-05-14 14:18:13 +02:00
Chocobozzz cb4bab61c1
Fix logger warning level 2021-05-14 14:17:53 +02:00
Chocobozzz aea0b0e7cd
Inject server config in HTML 2021-05-14 13:37:23 +02:00
Chocobozzz c158a5faab
Refactor a little bit controllers 2021-05-12 16:57:31 +02:00
Chocobozzz 16c016e8b1
Stricter models typing 2021-05-12 14:12:14 +02:00
Chocobozzz 32985a0a77
Error if importing a torrent with multiple files 2021-05-11 14:56:30 +02:00
Chocobozzz a3c997b34c
Update README 2021-05-11 14:23:49 +02:00
Chocobozzz 428ccb8b7a
Reorganize plugin models 2021-05-11 13:33:11 +02:00
Chocobozzz 2b02c520e6
Cleanup shared models 2021-05-11 11:32:31 +02:00
Chocobozzz 7d9ba5c089
Cleanup models directory organization 2021-05-11 11:32:31 +02:00
Chocobozzz c07902b908
Move transcoding files in their own directory 2021-05-11 11:32:31 +02:00
Chocobozzz 1bcb03a100
Use a class for youtube-dl 2021-05-11 11:32:31 +02:00
Rigel Kent b8375da931
provide more schema examples in openapi spec 2021-05-11 01:50:42 +02:00
Chocobozzz 38ce2aa7a8
Increase tests timeout 2021-05-10 17:06:53 +02:00
Chocobozzz f67d1dca1a
Fix handle downs test 2021-05-10 16:28:26 +02:00
Chocobozzz 3ec535f72b
Fix E2E tests 2021-05-10 13:58:09 +02:00
Chocobozzz 649e812947
Optimize videos feed endpoint 2021-05-10 13:56:16 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
John Livingston 1e0741d165 Fix CLI plugins list command: use command parameters. 2021-05-10 08:03:20 +02:00
Chocobozzz 4076e2ef6b Increase test timeout 2021-05-07 15:42:23 +02:00
Chocobozzz 1ff9f1cda3
Fix ffmpeg version parsing 2021-05-07 14:23:43 +02:00
Chocobozzz a1bb73f9b5
Refactor a little bit live tests 2021-05-07 11:53:46 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz 60f1f61579
Fix ffmpeg version checker 2021-05-06 16:39:17 +02:00
kontrollanten 66f77f6343 server: fix HLS playlist format
Remove prefixed comma when there's no video codec.
2021-05-06 08:47:43 +02:00
Chocobozzz b31d72625d
Plugin user.getAuthUser is now async
So we can load the full user
2021-05-05 15:26:28 +02:00
Kimsible 08ac081b37 Add test for API actors route 2021-05-05 11:47:03 +02:00
Kimsible 1e37d32f4b Add server API actors route 2021-05-05 11:47:03 +02:00
Kimsible 9a911038d9 Add shorter URLs for accounts and channels server-side 2021-05-05 11:47:03 +02:00
Chocobozzz 36217ac537
Fix lint 2021-05-05 09:55:01 +02:00
Chocobozzz a6e37eebfb
Fix transcoding job priority
New resolution jobs are also important if waiting for transcoding is
enabled since we publish the video after the first resolution generation
2021-05-05 09:32:20 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz c9ae74d6bc
Try to fix bad timestamps in .srt 2021-04-27 09:00:16 +02:00
Chocobozzz f816da921f
Add migration to remove duplicated keys
Due to old/corrupted data, it can be problematic to restore a backup
2021-04-26 14:27:42 +02:00
Chocobozzz 185eabe1ef
Increase test timeout 2021-04-26 11:33:14 +02:00
Chocobozzz 6290699080
Fix plugin helpers tests 2021-04-22 15:16:35 +02:00
Chocobozzz 096231d00e
Add auth header in plugins guide 2021-04-22 11:43:03 +02:00
Chocobozzz 302eba0d89
Add data directory for plugins and some helpers 2021-04-22 10:55:28 +02:00
Chocobozzz b7a27f2860
Add sendmail log in server 2021-04-21 15:18:22 +02:00
Chocobozzz 08370f6244
Fix h265 video import using CLI 2021-04-21 15:18:22 +02:00
Chocobozzz 67baf6478a
Allow plugins to set short translation locale 2021-04-20 16:14:09 +02:00
Chocobozzz 24a792404c
Support async onSettingsChange 2021-04-20 16:02:15 +02:00
Chocobozzz 4024c44f90
Merge branch 'release/3.1.0' into develop 2021-04-20 15:01:54 +02:00
Chocobozzz 6bc672da81
Fix database ssl connection 2021-04-19 09:25:11 +02:00
Chocobozzz 46460547ca
Fix exception in markdown parser 2021-04-16 07:41:35 +02:00
Chocobozzz f650072985
More robust tests 2021-04-15 13:52:27 +02:00
Chocobozzz 2df6f943b8
Increase ffmpeg tests timeout 2021-04-15 11:31:40 +02:00
Rigel Kent a073c91270 modify tests to support current behaviour regarding plaintext description 2021-04-13 15:45:09 +02:00
Rigel Kent 84bced652c render markdown and plainify descriptions on previews 2021-04-13 15:45:09 +02:00
Chocobozzz 13fec08ba3
Ugrade tools dependencies 2021-04-13 09:42:13 +02:00
Chocobozzz 9172345484
Fix banner error in actor json 2021-04-13 09:40:20 +02:00
Chocobozzz 1a578165f9
Fix lint 2021-04-12 17:00:21 +02:00
Chocobozzz 5182473faf
Upgrade server dep' 2021-04-12 16:35:04 +02:00
Chocobozzz 8155db669b
Dissociate frameguard from csp 2021-04-12 15:33:54 +02:00
Chocobozzz c24822a8fd
Don't save other attributes when set as refreshed 2021-04-12 13:44:58 +02:00
Rigel Kent fe19f600da
add channel and playlist stats to server stats endpoint (#3747)
* add channel and playlist stats to nodeinfo

* add tests for active video channels stats

* fix tests for active channel stats
2021-04-12 11:19:07 +02:00
Chocobozzz 8280d0c227
Ensure to install supported plugins 2021-04-12 10:10:48 +02:00
Chocobozzz 78ec174cca
Don't retry failed transaction for refresh
It's just a refresh and could lead to some issues in our tests. If there
is a transaction conflict, it means that the same video is already
beeing updated so aborting should be fine
2021-04-09 16:29:38 +02:00
Chocobozzz 21c917b38b Fix print transcode command script 2021-04-09 15:53:18 +02:00
Chocobozzz a60696ab18 Add scale filter to documentation 2021-04-09 15:53:18 +02:00
Chocobozzz 3e03b961b8 Add ability for plugins to specify scale filter 2021-04-09 15:53:18 +02:00
Théo Le Calvar d2351bcfd4 add tests for inputOptions and videoFilters in trancode plugins 2021-04-09 15:53:18 +02:00
Théo Le Calvar 43f7a43ca4 add option for transcode plugins to add video filters and make all options optional 2021-04-09 15:53:18 +02:00
Théo Le Calvar 5fb7cfbac5 add support for inputOptions in trancode plugins 2021-04-09 15:53:18 +02:00
Chocobozzz 22820226e5
Add server plugin helpers 2021-04-09 15:17:43 +02:00
Chocobozzz bae9303726
Also decache config 2021-04-09 09:37:46 +02:00
John Livingston 09d535ef98
Fix #3940: unload all children from the plugin module on updates. 2021-04-09 09:32:16 +02:00
John Livingston 4113719209
Tests that show the bug. 2021-04-09 09:32:16 +02:00
Chocobozzz dc48fdbe68
Merge branch 'release/3.1.0' into develop 2021-04-08 15:16:01 +02:00
Chocobozzz 1afb3c474a
Fix outbox fetch with subtitled videos 2021-04-08 15:04:14 +02:00
Chocobozzz 083328eb27
Remove duplicate captions 2021-04-08 14:45:40 +02:00
Chocobozzz 4c7449d2ad
Fix subtitles import 2021-04-08 14:29:03 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz a0eeb45f14
Update data in DB when regenerate thumbnails 2021-04-08 13:38:04 +02:00
Chocobozzz 20373985c0
Fix static ffmpeg version check 2021-04-08 10:08:11 +02:00
Chocobozzz 02dd4f3c65 Display banner on channel page 2021-04-08 10:07:53 +02:00
Chocobozzz cdeddff142 Add ability to update the banner 2021-04-08 10:07:53 +02:00
Chocobozzz 282695e699 Add banner migrations 2021-04-08 10:07:53 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Chocobozzz f479685678 Agnostic actor image storage 2021-04-08 10:07:53 +02:00
Chocobozzz 93544419d2
Fix tests 2021-04-08 08:10:13 +02:00
Chocobozzz b70025bfdc
Remove libxmljs in favour of pure JS implem 2021-04-06 10:25:21 +02:00
Rigel Kent 8e286cdca4
add sort select to my videos, change default sort to -publishedAt
fixes #3779
2021-04-06 10:03:19 +02:00
Chocobozzz eff9f81810
Fix import tests 2021-04-06 09:52:53 +02:00
Chocobozzz 1bbc0270f6
Fix tests 2021-04-05 18:56:24 +02:00
Chocobozzz 4363ce0e3e
Increase thumbnail size on server 2021-04-05 11:44:09 +02:00
Thavarasa Prasanth 4097c6d66c
fix missing title attribute on <iframe> tag suggested for embedding (#3901)
* title attribute is missing on <iframe> tag suggested for embedding #3861

* fix #3901

* fix: escapeHTML #3901

* fix: playlist title instead of video title #3901

* fix #3901

* assign title directly #3901
2021-03-31 08:32:05 +02:00
Chocobozzz 2e257e36b7
Add client action hooks for pubish pages 2021-03-24 18:18:41 +01:00
Chocobozzz 74a4d53110
Add server hooks for search endpoint 2021-03-24 18:18:41 +01:00
Chocobozzz eebd9838f0
Add filter hook to forbid embed access 2021-03-24 18:18:41 +01:00
Chocobozzz 4bc45da342
Add hooks support for video download 2021-03-24 18:18:41 +01:00
Chocobozzz 97aeb3cc46
Fix external on logout hook 2021-03-24 18:18:41 +01:00
Chocobozzz c2bd7a6fcf
Add regenrate thumbnails scripts 2021-03-24 18:18:41 +01:00
Chocobozzz f43db2f46e
Refactor auth flow
Reimplement some node-oauth2-server methods to remove hacky code needed by our external
login workflow
2021-03-24 18:18:41 +01:00
Chocobozzz cae2df6bdc
Stricter email options typings 2021-03-24 18:18:41 +01:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz 829523cfa5
Reduce bundle size using a const enum 2021-03-24 18:18:40 +01:00
Chocobozzz ae71acca14
Check ffmepg version on startup 2021-03-24 18:18:40 +01:00
Chocobozzz e7053b1d9d
Fix AP security tests 2021-03-24 18:18:40 +01:00
Chocobozzz 266131e0ca
Don't skip all threads on AP fetcher error 2021-03-24 18:18:40 +01:00
Chocobozzz e5abb482c3
Fix index search 2021-03-24 18:18:40 +01:00
Chocobozzz 43b513660b
Fix AP flag validator 2021-03-24 18:18:40 +01:00
Chocobozzz 5a2c0f0c99
Bigger video thumbnails for feeds 2021-03-24 18:18:40 +01:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz b329abc2f0
Fix request body limit 2021-03-24 18:18:40 +01:00
Chocobozzz 18b24b2dc5
Add jobs tag to logger 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 67f87b66fe
Refactor a little bit AP validators 2021-03-24 18:18:40 +01:00
Chocobozzz 452b3bea08
Introduce tags to the logger
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
Chocobozzz 61fd98341f
Fix mention notification with deleted comment 2021-03-22 09:49:23 +01:00
Chocobozzz 42ec411bf8
Fix abuse tests 2021-03-09 16:10:52 +01:00
Chocobozzz a2eba50b8e
Fix moderation notification 2021-03-04 16:16:37 +01:00
Chocobozzz dbb15e371a
Increase timeouts 2021-03-04 09:40:17 +01:00
Chocobozzz fbd51e69f2
Upgrade server dependencies 2021-03-03 16:14:32 +01:00
Chocobozzz 5a22296bdb
Fix broken hot algorithm 2021-03-03 13:52:58 +01:00
Chocobozzz 534556052e
Optimize views sort sql query 2021-03-03 11:24:16 +01:00
Chocobozzz 6c9c3b7b14
Fix emptying video tags 2021-03-03 11:03:30 +01:00
Chocobozzz 1ef447bd83
Don't create another uneeded preview placeholder 2021-03-03 11:02:34 +01:00
Chocobozzz cd2c3dcdc4
Remove unnecessary transcoding job error 2021-03-03 10:23:05 +01:00
Chocobozzz dcd75f786c
Rename request timeout 2021-03-03 10:10:55 +01:00
Chocobozzz 67bd004d84
Increase broadcast concurrency 2021-03-03 10:09:34 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 095e225804
Add a test for refreshing remote keys 2021-02-26 14:44:34 +01:00
Chocobozzz 8795d6f254
Fix broken local actors
Some channels can't federate because they don't have public/private
keys, maybe because the generation failed for various reasons
2021-02-26 14:22:25 +01:00
Chocobozzz 92315d979c
More robust channel change federation 2021-02-26 11:50:18 +01:00
Chocobozzz 23ac334389
Increase jobs request timeout 2021-02-26 10:48:12 +01:00
Chocobozzz 543442a3be
Add more AP stats to stats endpoint
It will help to understand if the federation correctly works or not
2021-02-26 10:28:11 +01:00
Chocobozzz 494df94036
Fix abuse crash on deleted video 2021-02-26 09:28:50 +01:00
Chocobozzz 88cfa3e8fa
Fix preview upload with capitalized ext 2021-02-26 09:16:31 +01:00
Chocobozzz a77c7327a2
Fix process abuse transactions 2021-02-25 15:55:31 +01:00
Chocobozzz e7812bf091
Add refresh_token_expires_in to oauth res 2021-02-25 15:25:48 +01:00
Chocobozzz 031ea8efed
Use named chunk for embed on analyze 2021-02-25 15:01:29 +01:00
Chocobozzz d61893f723 Async torrent creation 2021-02-25 15:01:07 +01:00
Chocobozzz d7df188f23
Fix separate SQL query for video get 2021-02-25 11:18:11 +01:00
Chocobozzz 0221f8c9b1
Trim video name also on server 2021-02-25 10:01:33 +01:00
Chocobozzz e072fb1e16
Add m4a support 2021-02-25 09:14:24 +01:00
Chocobozzz 9556ce48e7
Fix async issues with channels list 2021-02-25 09:09:41 +01:00
Chocobozzz 70243d7a35
Correctly wait transcoding before federating 2021-02-22 10:35:37 +01:00