Commit Graph

488 Commits

Author SHA1 Message Date
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz 7a22a0a56a
Add ability to search live videos 2021-05-05 12:10:00 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz 0f31933406
Add ability to hide plugin form fields 2021-04-22 14:35:49 +02:00
Chocobozzz 87e0b71d36
Add go-live example for plugin form fields 2021-04-22 12:00:23 +02:00
Chocobozzz 0ea9f463a9
Add action:admin-plugin-settings.init client hook 2021-04-22 11:18:13 +02:00
Chocobozzz 24a792404c
Support async onSettingsChange 2021-04-20 16:02:15 +02:00
Chocobozzz 1a578165f9
Fix lint 2021-04-12 17:00:21 +02:00
Chocobozzz 62bc0352d9
Add plugin placeholder elements support 2021-04-12 11:19:15 +02:00
Rigel Kent fe19f600da
add channel and playlist stats to server stats endpoint (#3747)
* add channel and playlist stats to nodeinfo

* add tests for active video channels stats

* fix tests for active channel stats
2021-04-12 11:19:07 +02:00
Chocobozzz 3e03b961b8 Add ability for plugins to specify scale filter 2021-04-09 15:53:18 +02:00
Théo Le Calvar 43f7a43ca4 add option for transcode plugins to add video filters and make all options optional 2021-04-09 15:53:18 +02:00
Théo Le Calvar 5fb7cfbac5 add support for inputOptions in trancode plugins 2021-04-09 15:53:18 +02:00
Chocobozzz d2466f0ac9
Update plugins doc 2021-04-09 15:21:06 +02:00
Chocobozzz 3c47fa3bc0
Add ability to hide plugin settings 2021-04-09 13:50:31 +02:00
Chocobozzz 32d13b203b
Plugins can register html/select settings 2021-04-09 11:19:14 +02:00
Chocobozzz cdeddff142 Add ability to update the banner 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Chocobozzz f479685678 Agnostic actor image storage 2021-04-08 10:07:53 +02:00
Chocobozzz faeec106ef
Add auth user client hook actions 2021-03-24 18:18:41 +01:00
Chocobozzz 2e257e36b7
Add client action hooks for pubish pages 2021-03-24 18:18:41 +01:00
Chocobozzz 74a4d53110
Add server hooks for search endpoint 2021-03-24 18:18:41 +01:00
Chocobozzz eebd9838f0
Add filter hook to forbid embed access 2021-03-24 18:18:41 +01:00
Chocobozzz 4bc45da342
Add hooks support for video download 2021-03-24 18:18:41 +01:00
Chocobozzz cae2df6bdc
Stricter email options typings 2021-03-24 18:18:41 +01:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz 829523cfa5
Reduce bundle size using a const enum 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 8795d6f254
Fix broken local actors
Some channels can't federate because they don't have public/private
keys, maybe because the generation failed for various reasons
2021-02-26 14:22:25 +01:00
Chocobozzz 543442a3be
Add more AP stats to stats endpoint
It will help to understand if the federation correctly works or not
2021-02-26 10:28:11 +01:00
Chocobozzz 9d6b9d10ef
Fix video comments display with deleted comments 2021-02-19 10:06:52 +01:00
Chocobozzz fae6e4da8f Try to speed up server tests 2021-02-19 09:36:50 +01:00
Chocobozzz f66db4d5c8 Fix feeds 2021-02-18 13:38:09 +01:00
Chocobozzz b3d5cb92b1 Add ability to update torrents cache in client 2021-02-18 13:38:09 +01:00
Chocobozzz d9a2a03196 Don't guess remote tracker URL 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz e3b4c084cd
Guess if we need to generate the thumbnail for imports 2021-02-12 09:37:01 +01:00
Chocobozzz 9129b7694d Allow to specify transcoding and import jobs concurrency 2021-02-08 15:38:45 +01:00
Rigel Kent 3d4e112d16 add best trending strategy based on Reddit's best
inspired from https://www.reddit.com/r/changelog/comments/7spgg0/best_is_the_new_hotness/
this implementation only adds freshness, and doesn't personalize based
on subscribed communities yet.
2021-02-04 09:04:47 +01:00
Chocobozzz 6d989edc66
Add ability to update plugin auth 2021-02-01 15:39:13 +01:00
Chocobozzz 789951460b
Try to support other codecs 2021-01-29 14:05:41 +01:00
Chocobozzz 2498aaead1
Add plugin transcoding profile guide 2021-01-29 14:05:41 +01:00
Chocobozzz 1896bca09e
Support transcoding options/encoders by plugins 2021-01-28 15:55:39 +01:00
Rigel Kent ba5d4a849c move from trending routes to alg param 2021-01-28 15:55:34 +01:00
Rigel Kent 3da68f0a78 add default trending page choice, revert comments count for hot strategy 2021-01-28 15:55:34 +01:00
Rigel Kent 5bcbcbe338 modularize abstract video list header and implement video hotness recommendation variant 2021-01-28 15:55:34 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
2021-01-25 14:38:52 +01:00