Commit Graph

1056 Commits

Author SHA1 Message Date
Chocobozzz f67d1dca1a
Fix handle downs test 2021-05-10 16:28:26 +02:00
Chocobozzz 3ec535f72b
Fix E2E tests 2021-05-10 13:58:09 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
Chocobozzz 4076e2ef6b Increase test timeout 2021-05-07 15:42:23 +02:00
Chocobozzz a1bb73f9b5
Refactor a little bit live tests 2021-05-07 11:53:46 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz b31d72625d
Plugin user.getAuthUser is now async
So we can load the full user
2021-05-05 15:26:28 +02:00
Kimsible 08ac081b37 Add test for API actors route 2021-05-05 11:47:03 +02:00
Kimsible 9a911038d9 Add shorter URLs for accounts and channels server-side 2021-05-05 11:47:03 +02:00
Chocobozzz a6e37eebfb
Fix transcoding job priority
New resolution jobs are also important if waiting for transcoding is
enabled since we publish the video after the first resolution generation
2021-05-05 09:32:20 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz 185eabe1ef
Increase test timeout 2021-04-26 11:33:14 +02:00
Chocobozzz 6290699080
Fix plugin helpers tests 2021-04-22 15:16:35 +02:00
Chocobozzz 302eba0d89
Add data directory for plugins and some helpers 2021-04-22 10:55:28 +02:00
Chocobozzz f650072985
More robust tests 2021-04-15 13:52:27 +02:00
Chocobozzz 2df6f943b8
Increase ffmpeg tests timeout 2021-04-15 11:31:40 +02:00
Rigel Kent a073c91270 modify tests to support current behaviour regarding plaintext description 2021-04-13 15:45:09 +02:00
Chocobozzz 8155db669b
Dissociate frameguard from csp 2021-04-12 15:33:54 +02:00
Rigel Kent fe19f600da
add channel and playlist stats to server stats endpoint (#3747)
* add channel and playlist stats to nodeinfo

* add tests for active video channels stats

* fix tests for active channel stats
2021-04-12 11:19:07 +02:00
Chocobozzz 21c917b38b Fix print transcode command script 2021-04-09 15:53:18 +02:00
Chocobozzz a60696ab18 Add scale filter to documentation 2021-04-09 15:53:18 +02:00
Chocobozzz 3e03b961b8 Add ability for plugins to specify scale filter 2021-04-09 15:53:18 +02:00
Théo Le Calvar d2351bcfd4 add tests for inputOptions and videoFilters in trancode plugins 2021-04-09 15:53:18 +02:00
Chocobozzz 22820226e5
Add server plugin helpers 2021-04-09 15:17:43 +02:00
John Livingston 4113719209
Tests that show the bug. 2021-04-09 09:32:16 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz a0eeb45f14
Update data in DB when regenerate thumbnails 2021-04-08 13:38:04 +02:00
Chocobozzz 20373985c0
Fix static ffmpeg version check 2021-04-08 10:08:11 +02:00
Chocobozzz cdeddff142 Add ability to update the banner 2021-04-08 10:07:53 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz b70025bfdc
Remove libxmljs in favour of pure JS implem 2021-04-06 10:25:21 +02:00
Chocobozzz eff9f81810
Fix import tests 2021-04-06 09:52:53 +02:00
Chocobozzz 1bbc0270f6
Fix tests 2021-04-05 18:56:24 +02:00
Thavarasa Prasanth 4097c6d66c
fix missing title attribute on <iframe> tag suggested for embedding (#3901)
* title attribute is missing on <iframe> tag suggested for embedding #3861

* fix #3901

* fix: escapeHTML #3901

* fix: playlist title instead of video title #3901

* fix #3901

* assign title directly #3901
2021-03-31 08:32:05 +02:00
Chocobozzz 2e257e36b7
Add client action hooks for pubish pages 2021-03-24 18:18:41 +01:00
Chocobozzz 74a4d53110
Add server hooks for search endpoint 2021-03-24 18:18:41 +01:00
Chocobozzz eebd9838f0
Add filter hook to forbid embed access 2021-03-24 18:18:41 +01:00
Chocobozzz 4bc45da342
Add hooks support for video download 2021-03-24 18:18:41 +01:00
Chocobozzz c2bd7a6fcf
Add regenrate thumbnails scripts 2021-03-24 18:18:41 +01:00
Chocobozzz f43db2f46e
Refactor auth flow
Reimplement some node-oauth2-server methods to remove hacky code needed by our external
login workflow
2021-03-24 18:18:41 +01:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz e7053b1d9d
Fix AP security tests 2021-03-24 18:18:40 +01:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 61fd98341f
Fix mention notification with deleted comment 2021-03-22 09:49:23 +01:00
Chocobozzz dbb15e371a
Increase timeouts 2021-03-04 09:40:17 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 095e225804
Add a test for refreshing remote keys 2021-02-26 14:44:34 +01:00
Chocobozzz 543442a3be
Add more AP stats to stats endpoint
It will help to understand if the federation correctly works or not
2021-02-26 10:28:11 +01:00
Chocobozzz 15bedeebd7
Fix loading things twice on trending page 2021-02-22 10:35:37 +01:00
Chocobozzz 94d721efdc Fix stats tests 2021-02-19 16:29:00 +01:00
Chocobozzz 300cb723eb
Fix tests 2021-02-19 11:27:38 +01:00
Chocobozzz 9d6b9d10ef
Fix video comments display with deleted comments 2021-02-19 10:06:52 +01:00
Chocobozzz fae6e4da8f Try to speed up server tests 2021-02-19 09:36:50 +01:00
Chocobozzz 53c06121a4
Fix import tests 2021-02-18 14:08:16 +01:00
Chocobozzz b3d5cb92b1 Add ability to update torrents cache in client 2021-02-18 13:38:09 +01:00
Chocobozzz 2451916e45 Add video files migration 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz 1664bc60eb Optimize remote image processing 2021-02-16 10:36:44 +01:00
Chocobozzz a35a22797c Remove previous thumbnail if needed 2021-02-16 10:36:44 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Chocobozzz ff9112ad21
Increase logs test timeout 2021-02-11 14:34:55 +01:00
Chocobozzz 6559da2821
Allow plugins to load a video by id 2021-02-11 10:23:52 +01:00
Chocobozzz 0c39df2536
Increase test timeout 2021-02-08 15:50:10 +01:00
Chocobozzz 9129b7694d Allow to specify transcoding and import jobs concurrency 2021-02-08 15:38:45 +01:00
Rigel Kent 3d4e112d16 add best trending strategy based on Reddit's best
inspired from https://www.reddit.com/r/changelog/comments/7spgg0/best_is_the_new_hotness/
this implementation only adds freshness, and doesn't personalize based
on subscribed communities yet.
2021-02-04 09:04:47 +01:00
Chocobozzz 657bba2b29
Fix hls tests 2021-02-02 10:07:46 +01:00
Chocobozzz 40930fda86
Transcode audio uploads to lower resolutions
Better consistency
2021-02-02 09:45:42 +01:00
Chocobozzz 5e8dd6e08a
Fix lint 2021-02-01 16:05:22 +01:00
Chocobozzz 6d989edc66
Add ability to update plugin auth 2021-02-01 15:39:13 +01:00
Chocobozzz 6949a1a111
Fix tests 2021-02-01 15:03:28 +01:00
Chocobozzz 7448551fe5
Fix redundancy with HLS only files 2021-02-01 11:23:12 +01:00
Chocobozzz 33c7131be5
Check banned status for external auths 2021-02-01 11:23:11 +01:00
Chocobozzz 7519c4a27b
Fix tests 2021-01-29 15:58:01 +01:00
Chocobozzz a8537c622e
Use veryfast preset for default transcoding profile 2021-01-29 15:31:31 +01:00
Chocobozzz b3eafc5fc5
Reduce transcoding overhead by disabling resolutions 2021-01-29 08:44:36 +01:00
Chocobozzz 1896bca09e
Support transcoding options/encoders by plugins 2021-01-28 15:55:39 +01:00
Rigel Kent 923d3d5ad5 add test and openapi for hot sort parameter 2021-01-28 15:55:34 +01:00
Rigel Kent ba5d4a849c move from trending routes to alg param 2021-01-28 15:55:34 +01:00
Rigel Kent 3da68f0a78 add default trending page choice, revert comments count for hot strategy 2021-01-28 15:55:34 +01:00
Chocobozzz e9cb361cf5
Fix tests 2021-01-26 11:37:31 +01:00
Chocobozzz 6939cbac48 Safely remove webtorrent files
Only remove them on max quality HLS playlist generation
2021-01-25 14:38:52 +01:00
Chocobozzz a4d2ca0715
Add user video list hooks 2021-01-20 15:28:34 +01:00
Rigel Kent 370240824e
Allow users/visitors to search through an account's videos (#3589)
* WIP: account search

* add search to account view

* add tests for account search
2021-01-19 13:43:33 +01:00
Chocobozzz 9f39c5466e
Fix user notification test 2021-01-19 08:35:44 +01:00
Chocobozzz 83befebe52
Increase test timeout 2021-01-18 09:52:05 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz d43c6b1ffc Implement remote interaction 2021-01-15 10:49:10 +01:00
Chocobozzz c9c43612d5
Fix lint 2021-01-13 10:15:38 +01:00
Chocobozzz f5961a8cdf
Add 4k transcoding test 2021-01-13 09:52:52 +01:00
Chocobozzz 78d62f4d18
Fix and add skipping ping log tests 2021-01-13 09:38:19 +01:00
Rigel Kent d8b34ee55b
Allow user to search through their watch history (#3576)
* allow user to search through their watch history

* add tests for search in watch history

* Update client/src/app/shared/shared-main/users/user-history.service.ts
2021-01-13 09:16:15 +01:00
Rigel Kent b7085c7132 add support for 1440p (Quad HD/QHD/WQHD) videos 2021-01-13 09:14:09 +01:00
Rigel Kent 7185dab3ff add test for script printing command 2021-01-13 09:13:31 +01:00
Chocobozzz 26171379d0 Use source sans 3 font 2021-01-13 09:05:22 +01:00
Chocobozzz 8ace68056d
Increase test timeout 2021-01-12 09:02:40 +01:00
Chocobozzz 55a5b0fd8d
Update translations 2021-01-11 16:05:10 +01:00
Chocobozzz a11a782f5c
Fix duplicate entry in job list page 2021-01-11 15:39:15 +01:00
Chocobozzz f42c21528b
Increase live test timeout 2021-01-08 12:11:45 +01:00
Chocobozzz adc8e7e19f
Fix ldap test port 2021-01-04 11:21:19 +01:00
Chocobozzz 2dbc170da3
Add missing audit log if the user deletes its account 2021-01-04 09:54:48 +01:00
Chocobozzz 47d8e266f1
Support activeMonth and activeHalfyear in nodeinfo 2020-12-30 11:41:36 +01:00
Chocobozzz a046453549
Increase notifications comment test timeout 2020-12-30 11:27:59 +01:00
Theodore R. Smith 8e76aa1d75
(#3520) [cli] Hardened `auth add`: No longer fails with extraneous characters.
**The Solution:**
I have hardened `auth add` by stripping out everything from the third '/' to the end of the instance URL.

**The Problem:**
When adding an authorization for the peertube-cli, before this commit you could not have anything after the domain_name:port.

For instance, if there was a trailing / in your instance URL, before this commit it will always fail with

    expected 200 "OK", got 404 "Not Found".

It took me over 20 minutes to figure out that this was the problem.

See Issue #3091.
2020-12-27 14:29:54 -06:00
Chocobozzz 9e454eba57
More robust server stats tests 2020-12-16 14:59:33 +01:00
Chocobozzz 1ab6024345
Fix live badge in videos list 2020-12-16 14:19:43 +01:00
Chocobozzz 54363cffa4
Fix stats tests 2020-12-16 10:36:15 +01:00
Chocobozzz 496b5a5bf6
Increase test timeout 2020-12-15 15:09:12 +01:00
Chocobozzz 99afa081bc
Add AP stats 2020-12-15 13:34:58 +01:00
Chocobozzz c9bc850e93
Use -1 for max live duration unlimited 2020-12-15 09:26:37 +01:00
Chocobozzz 113d4a3f73
Only link to crowdfunding pages in credits 2020-12-14 16:08:15 +01:00
Chocobozzz 402145b863
Refactor jobs state 2020-12-14 12:00:35 +01:00
Chocobozzz 59fd824cf3 Fix tests timeout 2020-12-14 09:11:27 +01:00
Chocobozzz 34caef7fc0 Add joblog at the end of ci 2020-12-14 09:11:27 +01:00
Chocobozzz e078371807 Parse log script can take files as args 2020-12-14 09:11:27 +01:00
Chocobozzz a800dbf345
Live views update 2020-12-09 16:31:42 +01:00
Chocobozzz 5cac83a78d
Refactor views component 2020-12-09 15:33:16 +01:00
Chocobozzz 0d8de2756f
Fix live tests 2020-12-09 14:45:47 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Rigel Kent f2eb23cd87
emit more specific status codes on video upload (#3423)
- reduce http status codes list to potentially useful codes
- convert more codes to typed ones
- factorize html generator for error responses
2020-12-08 21:16:10 +01:00
Chocobozzz 8e2392e512
Fix config test 2020-12-08 11:37:24 +01:00
Chocobozzz a59f210ff3 Refactor a little bit client canonical URL 2020-12-08 11:15:11 +01:00
Thomas Citharel e5024f5181 Add tests
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2020-12-08 11:15:11 +01:00
kontrollanten 2805cb7cb6 server/mw/oauth: res.loc.auth to true upon auth
Setting res.locals.authenticated explicity to
true makes it's more clear whether the req is
authenticated when using optionalAuthenticate
middlware.
2020-12-08 10:40:08 +01:00
kontrollanten f17faefb30 plugins: add optional authentication for routes 2020-12-08 10:40:08 +01:00
Chocobozzz c824e8a0c7
Fix hook test 2020-12-08 10:38:47 +01:00
Chocobozzz 38267c0c8a Use dedicated hooks for account/channel videos 2020-12-08 10:33:23 +01:00
Rigel Kent 2d53be0267
replace numbers with typed http status codes (#3409) 2020-12-07 14:32:36 +01:00
Chocobozzz 4ef9ea482f
Fix live transcoding tests 2020-12-04 16:41:03 +01:00
Chocobozzz e26dc0cdbd
Strict actor url comparison 2020-12-03 16:12:58 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Chocobozzz ffc12d3ad5
Increase live constraint test timeout 2020-12-02 16:58:45 +01:00
Chocobozzz 2650d6d489
Fix live replay duration glitch 2020-12-02 10:18:15 +01:00
Chocobozzz 9e3e4adc65
Add logs to fixture generations 2020-12-01 15:04:38 +01:00
Chocobozzz 0484ec9e41
Try to fix live tests 2020-11-30 09:16:41 +01:00
Chocobozzz e5a516e702
More robust live test 2020-11-27 15:46:49 +01:00
Chocobozzz 353f8bc0c7
Add registered setting CLI plugin install test 2020-11-27 11:41:38 +01:00
Chocobozzz 0e856b78e9
Try to fix live test 2020-11-26 15:16:30 +01:00
Chocobozzz 884d2c39ae
Fix live FPS limit 2020-11-26 11:30:18 +01:00
Chocobozzz 18490b0765 Fix migration and test 2020-11-25 11:07:56 +01:00
Rigel Kent 5beb89f223 refactor scoped token service 2020-11-25 11:07:56 +01:00
Rigel Kent afff310e50 allow private syndication feeds via a user feedToken 2020-11-25 11:07:56 +01:00
Chocobozzz f619de0e43 Add tests to gif resizer 2020-11-25 10:07:51 +01:00
Chocobozzz 6b67897e2e Add transcoding module comments 2020-11-25 10:07:51 +01:00
Chocobozzz 33ff70baa6 Fix audio encoding params 2020-11-25 10:07:51 +01:00
Chocobozzz ca5c612bfd Add live transcoding bit rate tests 2020-11-25 10:07:51 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Chocobozzz 47e363bbe9
Relax live fps test 2020-11-20 15:39:24 +01:00
Chocobozzz 74fd2643b4 Provide express request to onLogout call
+ pluginInfo related changes
2020-11-20 15:36:43 +01:00