Commit Graph

2726 Commits

Author SHA1 Message Date
Chocobozzz 428ccb8b7a
Reorganize plugin models 2021-05-11 13:33:11 +02:00
Chocobozzz 2b02c520e6
Cleanup shared models 2021-05-11 11:32:31 +02:00
Chocobozzz 7d9ba5c089
Cleanup models directory organization 2021-05-11 11:32:31 +02:00
Chocobozzz c07902b908
Move transcoding files in their own directory 2021-05-11 11:32:31 +02:00
Chocobozzz 1bcb03a100
Use a class for youtube-dl 2021-05-11 11:32:31 +02:00
Rigel Kent b8375da931
provide more schema examples in openapi spec 2021-05-11 01:50:42 +02:00
Chocobozzz 38ce2aa7a8
Increase tests timeout 2021-05-10 17:06:53 +02:00
Chocobozzz f67d1dca1a
Fix handle downs test 2021-05-10 16:28:26 +02:00
Chocobozzz 3ec535f72b
Fix E2E tests 2021-05-10 13:58:09 +02:00
Chocobozzz 649e812947
Optimize videos feed endpoint 2021-05-10 13:56:16 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
John Livingston 1e0741d165 Fix CLI plugins list command: use command parameters. 2021-05-10 08:03:20 +02:00
Chocobozzz 4076e2ef6b Increase test timeout 2021-05-07 15:42:23 +02:00
Chocobozzz 1ff9f1cda3
Fix ffmpeg version parsing 2021-05-07 14:23:43 +02:00
Chocobozzz a1bb73f9b5
Refactor a little bit live tests 2021-05-07 11:53:46 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz 60f1f61579
Fix ffmpeg version checker 2021-05-06 16:39:17 +02:00
kontrollanten 66f77f6343 server: fix HLS playlist format
Remove prefixed comma when there's no video codec.
2021-05-06 08:47:43 +02:00
Chocobozzz b31d72625d
Plugin user.getAuthUser is now async
So we can load the full user
2021-05-05 15:26:28 +02:00
Kimsible 08ac081b37 Add test for API actors route 2021-05-05 11:47:03 +02:00
Kimsible 1e37d32f4b Add server API actors route 2021-05-05 11:47:03 +02:00
Kimsible 9a911038d9 Add shorter URLs for accounts and channels server-side 2021-05-05 11:47:03 +02:00
Chocobozzz 36217ac537
Fix lint 2021-05-05 09:55:01 +02:00
Chocobozzz a6e37eebfb
Fix transcoding job priority
New resolution jobs are also important if waiting for transcoding is
enabled since we publish the video after the first resolution generation
2021-05-05 09:32:20 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz c9ae74d6bc
Try to fix bad timestamps in .srt 2021-04-27 09:00:16 +02:00
Chocobozzz f816da921f
Add migration to remove duplicated keys
Due to old/corrupted data, it can be problematic to restore a backup
2021-04-26 14:27:42 +02:00
Chocobozzz 185eabe1ef
Increase test timeout 2021-04-26 11:33:14 +02:00
Chocobozzz 6290699080
Fix plugin helpers tests 2021-04-22 15:16:35 +02:00
Chocobozzz 096231d00e
Add auth header in plugins guide 2021-04-22 11:43:03 +02:00
Chocobozzz 302eba0d89
Add data directory for plugins and some helpers 2021-04-22 10:55:28 +02:00
Chocobozzz b7a27f2860
Add sendmail log in server 2021-04-21 15:18:22 +02:00
Chocobozzz 08370f6244
Fix h265 video import using CLI 2021-04-21 15:18:22 +02:00
Chocobozzz 67baf6478a
Allow plugins to set short translation locale 2021-04-20 16:14:09 +02:00
Chocobozzz 24a792404c
Support async onSettingsChange 2021-04-20 16:02:15 +02:00
Chocobozzz 4024c44f90
Merge branch 'release/3.1.0' into develop 2021-04-20 15:01:54 +02:00
Chocobozzz 6bc672da81
Fix database ssl connection 2021-04-19 09:25:11 +02:00
Chocobozzz 46460547ca
Fix exception in markdown parser 2021-04-16 07:41:35 +02:00
Chocobozzz f650072985
More robust tests 2021-04-15 13:52:27 +02:00
Chocobozzz 2df6f943b8
Increase ffmpeg tests timeout 2021-04-15 11:31:40 +02:00
Rigel Kent a073c91270 modify tests to support current behaviour regarding plaintext description 2021-04-13 15:45:09 +02:00
Rigel Kent 84bced652c render markdown and plainify descriptions on previews 2021-04-13 15:45:09 +02:00
Chocobozzz 13fec08ba3
Ugrade tools dependencies 2021-04-13 09:42:13 +02:00
Chocobozzz 9172345484
Fix banner error in actor json 2021-04-13 09:40:20 +02:00
Chocobozzz 1a578165f9
Fix lint 2021-04-12 17:00:21 +02:00
Chocobozzz 5182473faf
Upgrade server dep' 2021-04-12 16:35:04 +02:00
Chocobozzz 8155db669b
Dissociate frameguard from csp 2021-04-12 15:33:54 +02:00
Chocobozzz c24822a8fd
Don't save other attributes when set as refreshed 2021-04-12 13:44:58 +02:00
Rigel Kent fe19f600da
add channel and playlist stats to server stats endpoint (#3747)
* add channel and playlist stats to nodeinfo

* add tests for active video channels stats

* fix tests for active channel stats
2021-04-12 11:19:07 +02:00
Chocobozzz 8280d0c227
Ensure to install supported plugins 2021-04-12 10:10:48 +02:00
Chocobozzz 78ec174cca
Don't retry failed transaction for refresh
It's just a refresh and could lead to some issues in our tests. If there
is a transaction conflict, it means that the same video is already
beeing updated so aborting should be fine
2021-04-09 16:29:38 +02:00
Chocobozzz 21c917b38b Fix print transcode command script 2021-04-09 15:53:18 +02:00
Chocobozzz a60696ab18 Add scale filter to documentation 2021-04-09 15:53:18 +02:00
Chocobozzz 3e03b961b8 Add ability for plugins to specify scale filter 2021-04-09 15:53:18 +02:00
Théo Le Calvar d2351bcfd4 add tests for inputOptions and videoFilters in trancode plugins 2021-04-09 15:53:18 +02:00
Théo Le Calvar 43f7a43ca4 add option for transcode plugins to add video filters and make all options optional 2021-04-09 15:53:18 +02:00
Théo Le Calvar 5fb7cfbac5 add support for inputOptions in trancode plugins 2021-04-09 15:53:18 +02:00
Chocobozzz 22820226e5
Add server plugin helpers 2021-04-09 15:17:43 +02:00
Chocobozzz bae9303726
Also decache config 2021-04-09 09:37:46 +02:00
John Livingston 09d535ef98
Fix #3940: unload all children from the plugin module on updates. 2021-04-09 09:32:16 +02:00
John Livingston 4113719209
Tests that show the bug. 2021-04-09 09:32:16 +02:00
Chocobozzz dc48fdbe68
Merge branch 'release/3.1.0' into develop 2021-04-08 15:16:01 +02:00
Chocobozzz 1afb3c474a
Fix outbox fetch with subtitled videos 2021-04-08 15:04:14 +02:00
Chocobozzz 083328eb27
Remove duplicate captions 2021-04-08 14:45:40 +02:00
Chocobozzz 4c7449d2ad
Fix subtitles import 2021-04-08 14:29:03 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz a0eeb45f14
Update data in DB when regenerate thumbnails 2021-04-08 13:38:04 +02:00
Chocobozzz 20373985c0
Fix static ffmpeg version check 2021-04-08 10:08:11 +02:00
Chocobozzz 02dd4f3c65 Display banner on channel page 2021-04-08 10:07:53 +02:00
Chocobozzz cdeddff142 Add ability to update the banner 2021-04-08 10:07:53 +02:00
Chocobozzz 282695e699 Add banner migrations 2021-04-08 10:07:53 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Chocobozzz f479685678 Agnostic actor image storage 2021-04-08 10:07:53 +02:00
Chocobozzz 93544419d2
Fix tests 2021-04-08 08:10:13 +02:00
Chocobozzz b70025bfdc
Remove libxmljs in favour of pure JS implem 2021-04-06 10:25:21 +02:00
Rigel Kent 8e286cdca4
add sort select to my videos, change default sort to -publishedAt
fixes #3779
2021-04-06 10:03:19 +02:00
Chocobozzz eff9f81810
Fix import tests 2021-04-06 09:52:53 +02:00
Chocobozzz 1bbc0270f6
Fix tests 2021-04-05 18:56:24 +02:00
Chocobozzz 4363ce0e3e
Increase thumbnail size on server 2021-04-05 11:44:09 +02:00
Thavarasa Prasanth 4097c6d66c
fix missing title attribute on <iframe> tag suggested for embedding (#3901)
* title attribute is missing on <iframe> tag suggested for embedding #3861

* fix #3901

* fix: escapeHTML #3901

* fix: playlist title instead of video title #3901

* fix #3901

* assign title directly #3901
2021-03-31 08:32:05 +02:00
Chocobozzz 2e257e36b7
Add client action hooks for pubish pages 2021-03-24 18:18:41 +01:00
Chocobozzz 74a4d53110
Add server hooks for search endpoint 2021-03-24 18:18:41 +01:00
Chocobozzz eebd9838f0
Add filter hook to forbid embed access 2021-03-24 18:18:41 +01:00
Chocobozzz 4bc45da342
Add hooks support for video download 2021-03-24 18:18:41 +01:00
Chocobozzz 97aeb3cc46
Fix external on logout hook 2021-03-24 18:18:41 +01:00
Chocobozzz c2bd7a6fcf
Add regenrate thumbnails scripts 2021-03-24 18:18:41 +01:00
Chocobozzz f43db2f46e
Refactor auth flow
Reimplement some node-oauth2-server methods to remove hacky code needed by our external
login workflow
2021-03-24 18:18:41 +01:00
Chocobozzz cae2df6bdc
Stricter email options typings 2021-03-24 18:18:41 +01:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz 829523cfa5
Reduce bundle size using a const enum 2021-03-24 18:18:40 +01:00
Chocobozzz ae71acca14
Check ffmepg version on startup 2021-03-24 18:18:40 +01:00
Chocobozzz e7053b1d9d
Fix AP security tests 2021-03-24 18:18:40 +01:00
Chocobozzz 266131e0ca
Don't skip all threads on AP fetcher error 2021-03-24 18:18:40 +01:00
Chocobozzz e5abb482c3
Fix index search 2021-03-24 18:18:40 +01:00
Chocobozzz 43b513660b
Fix AP flag validator 2021-03-24 18:18:40 +01:00
Chocobozzz 5a2c0f0c99
Bigger video thumbnails for feeds 2021-03-24 18:18:40 +01:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz b329abc2f0
Fix request body limit 2021-03-24 18:18:40 +01:00
Chocobozzz 18b24b2dc5
Add jobs tag to logger 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 67f87b66fe
Refactor a little bit AP validators 2021-03-24 18:18:40 +01:00
Chocobozzz 452b3bea08
Introduce tags to the logger
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
Chocobozzz 61fd98341f
Fix mention notification with deleted comment 2021-03-22 09:49:23 +01:00
Chocobozzz 42ec411bf8
Fix abuse tests 2021-03-09 16:10:52 +01:00
Chocobozzz a2eba50b8e
Fix moderation notification 2021-03-04 16:16:37 +01:00
Chocobozzz dbb15e371a
Increase timeouts 2021-03-04 09:40:17 +01:00
Chocobozzz fbd51e69f2
Upgrade server dependencies 2021-03-03 16:14:32 +01:00
Chocobozzz 5a22296bdb
Fix broken hot algorithm 2021-03-03 13:52:58 +01:00
Chocobozzz 534556052e
Optimize views sort sql query 2021-03-03 11:24:16 +01:00
Chocobozzz 6c9c3b7b14
Fix emptying video tags 2021-03-03 11:03:30 +01:00
Chocobozzz 1ef447bd83
Don't create another uneeded preview placeholder 2021-03-03 11:02:34 +01:00
Chocobozzz cd2c3dcdc4
Remove unnecessary transcoding job error 2021-03-03 10:23:05 +01:00
Chocobozzz dcd75f786c
Rename request timeout 2021-03-03 10:10:55 +01:00
Chocobozzz 67bd004d84
Increase broadcast concurrency 2021-03-03 10:09:34 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 095e225804
Add a test for refreshing remote keys 2021-02-26 14:44:34 +01:00
Chocobozzz 8795d6f254
Fix broken local actors
Some channels can't federate because they don't have public/private
keys, maybe because the generation failed for various reasons
2021-02-26 14:22:25 +01:00
Chocobozzz 92315d979c
More robust channel change federation 2021-02-26 11:50:18 +01:00
Chocobozzz 23ac334389
Increase jobs request timeout 2021-02-26 10:48:12 +01:00
Chocobozzz 543442a3be
Add more AP stats to stats endpoint
It will help to understand if the federation correctly works or not
2021-02-26 10:28:11 +01:00
Chocobozzz 494df94036
Fix abuse crash on deleted video 2021-02-26 09:28:50 +01:00
Chocobozzz 88cfa3e8fa
Fix preview upload with capitalized ext 2021-02-26 09:16:31 +01:00
Chocobozzz a77c7327a2
Fix process abuse transactions 2021-02-25 15:55:31 +01:00
Chocobozzz e7812bf091
Add refresh_token_expires_in to oauth res 2021-02-25 15:25:48 +01:00
Chocobozzz 031ea8efed
Use named chunk for embed on analyze 2021-02-25 15:01:29 +01:00
Chocobozzz d61893f723 Async torrent creation 2021-02-25 15:01:07 +01:00
Chocobozzz d7df188f23
Fix separate SQL query for video get 2021-02-25 11:18:11 +01:00
Chocobozzz 0221f8c9b1
Trim video name also on server 2021-02-25 10:01:33 +01:00
Chocobozzz e072fb1e16
Add m4a support 2021-02-25 09:14:24 +01:00
Chocobozzz 9556ce48e7
Fix async issues with channels list 2021-02-25 09:09:41 +01:00
Chocobozzz 70243d7a35
Correctly wait transcoding before federating 2021-02-22 10:35:37 +01:00
Chocobozzz 15bedeebd7
Fix loading things twice on trending page 2021-02-22 10:35:37 +01:00
Chocobozzz 94d721efdc Fix stats tests 2021-02-19 16:29:00 +01:00
Chocobozzz d7764e2eb7
Fix import script when using the search index 2021-02-19 15:04:58 +01:00
Chocobozzz 4cbdcf4409
Fix sending accept to old follows 2021-02-19 13:55:00 +01:00
Chocobozzz 300cb723eb
Fix tests 2021-02-19 11:27:38 +01:00
Chocobozzz 2a021e6cb6
Fix thread replies API response 2021-02-19 10:52:31 +01:00
Chocobozzz 285981f4dd
Optimize rate endpoint 2021-02-19 10:26:58 +01:00
Chocobozzz 9d6b9d10ef
Fix video comments display with deleted comments 2021-02-19 10:06:52 +01:00
Chocobozzz fae6e4da8f Try to speed up server tests 2021-02-19 09:36:50 +01:00
Chocobozzz e33933e44c
Fix latest migrations
We made an historical mistake: we have a unique key on videoId,
resolution AND fps. But the filenames and torrent filenames do not have
fps in their name

Remove these duplicate entries
2021-02-19 08:41:32 +01:00
Chocobozzz 53c06121a4
Fix import tests 2021-02-18 14:08:16 +01:00
Chocobozzz 8efc27bf14 Cleanup 2021-02-18 13:38:09 +01:00
Chocobozzz f66db4d5c8 Fix feeds 2021-02-18 13:38:09 +01:00
Chocobozzz b3d5cb92b1 Add ability to update torrents cache in client 2021-02-18 13:38:09 +01:00
Chocobozzz d9a2a03196 Don't guess remote tracker URL 2021-02-18 13:38:09 +01:00
Chocobozzz 2451916e45 Add video files migration 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz 1664bc60eb Optimize remote image processing 2021-02-16 10:36:44 +01:00
Chocobozzz 374b725df5 Optimize remote thumbnail processing 2021-02-16 10:36:44 +01:00
Chocobozzz a35a22797c Remove previous thumbnail if needed 2021-02-16 10:36:44 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Chocobozzz a8b1b40485 Generate a name for thumbnails
Allows aggressive caching
2021-02-16 10:36:44 +01:00
Chocobozzz 0472d474fd Remove deprecated static routes 2021-02-16 10:36:44 +01:00
Chocobozzz c15aff666d
Remove old code
We consider Dislike as a classic activity now
2021-02-12 14:53:07 +01:00
Chocobozzz e3b4c084cd
Guess if we need to generate the thumbnail for imports 2021-02-12 09:37:01 +01:00
Chocobozzz ff9112ad21
Increase logs test timeout 2021-02-11 14:34:55 +01:00
Chocobozzz 6559da2821
Allow plugins to load a video by id 2021-02-11 10:23:52 +01:00
Chocobozzz 44d1f7f2e8
Painfully debug concurrent import jobs 2021-02-09 11:46:30 +01:00
Chocobozzz 3933a127d2
Ensure we don't receive things from local actors 2021-02-08 16:06:32 +01:00
Chocobozzz 0c39df2536
Increase test timeout 2021-02-08 15:50:10 +01:00
Chocobozzz 9129b7694d Allow to specify transcoding and import jobs concurrency 2021-02-08 15:38:45 +01:00
Chocobozzz 81b46cbc34 Optimize videos list API endpoint 2021-02-08 15:38:45 +01:00
Chocobozzz 9162476fe9
Support NodeJS 14 and 15, deprecate Node 10 2021-02-04 14:31:18 +01:00
Rigel Kent 3d4e112d16 add best trending strategy based on Reddit's best
inspired from https://www.reddit.com/r/changelog/comments/7spgg0/best_is_the_new_hotness/
this implementation only adds freshness, and doesn't personalize based
on subscribed communities yet.
2021-02-04 09:04:47 +01:00
Chocobozzz f6267b6101
Upgrade tools dependencies 2021-02-03 13:49:56 +01:00
Chocobozzz 2e1e4af03b
Fix get video from redudancy 2021-02-03 09:57:47 +01:00
Chocobozzz ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Chocobozzz 69eddafb17
Fix HLS generation after import script 2021-02-02 14:00:46 +01:00
Chocobozzz aaedadd538
Transcode HLS playlists in a tmp directory 2021-02-02 11:50:29 +01:00
Chocobozzz d8ba492119
Fix broken HLS playback with some audio streams 2021-02-02 11:19:46 +01:00
Chocobozzz 657bba2b29
Fix hls tests 2021-02-02 10:07:46 +01:00
Chocobozzz 40930fda86
Transcode audio uploads to lower resolutions
Better consistency
2021-02-02 09:45:42 +01:00
Chocobozzz 89613cb444
Purge entire video from redundancy 2021-02-02 08:50:30 +01:00
Chocobozzz 5e8dd6e08a
Fix lint 2021-02-01 16:05:22 +01:00
Chocobozzz 6d989edc66
Add ability to update plugin auth 2021-02-01 15:39:13 +01:00
Chocobozzz 6949a1a111
Fix tests 2021-02-01 15:03:28 +01:00
Chocobozzz 56df7815b5
Fix URI search config update 2021-02-01 11:30:01 +01:00
Chocobozzz 7e3592d70b
Correctly remove torrents with HLS only 2021-02-01 11:23:12 +01:00
Chocobozzz 7448551fe5
Fix redundancy with HLS only files 2021-02-01 11:23:12 +01:00
Chocobozzz 33c7131be5
Check banned status for external auths 2021-02-01 11:23:11 +01:00
Chocobozzz 7519c4a27b
Fix tests 2021-01-29 15:58:01 +01:00
Chocobozzz a8537c622e
Use veryfast preset for default transcoding profile 2021-01-29 15:31:31 +01:00
Chocobozzz 789951460b
Try to support other codecs 2021-01-29 14:05:41 +01:00
Chocobozzz ffd970faf9
Don't quick transcode if using a custom profile 2021-01-29 14:05:41 +01:00
Chocobozzz 2498aaead1
Add plugin transcoding profile guide 2021-01-29 14:05:41 +01:00
Chocobozzz b3eafc5fc5
Reduce transcoding overhead by disabling resolutions 2021-01-29 08:44:36 +01:00
Chocobozzz 80ac2e5548
Support transcoding profile in client admin 2021-01-28 16:10:09 +01:00
Chocobozzz 1896bca09e
Support transcoding options/encoders by plugins 2021-01-28 15:55:39 +01:00
Chocobozzz 529b37527c
Use a profile manager for transcoding 2021-01-28 15:55:39 +01:00
Rigel Kent 923d3d5ad5 add test and openapi for hot sort parameter 2021-01-28 15:55:34 +01:00
Rigel Kent ba5d4a849c move from trending routes to alg param 2021-01-28 15:55:34 +01:00
Rigel Kent 3da68f0a78 add default trending page choice, revert comments count for hot strategy 2021-01-28 15:55:34 +01:00
Rigel Kent 28eeb811c4 only count comments from people other than the video author 2021-01-28 15:55:34 +01:00
Rigel Kent 5bcbcbe338 modularize abstract video list header and implement video hotness recommendation variant 2021-01-28 15:55:34 +01:00
Chocobozzz 7a4994873c
Fix rss feed with HLS videos 2021-01-27 16:42:13 +01:00
Chocobozzz 00b87c5791
Fix live RAM usage when ffmpeg is too slow 2021-01-27 12:05:10 +01:00
Chocobozzz a4a8cd3971
Add missing niceness to ffmpeg thumbnail processes 2021-01-27 09:09:59 +01:00
Chocobozzz e9cb361cf5
Fix tests 2021-01-26 11:37:31 +01:00
Chocobozzz e08ff02a9f
Allow webp image uploads 2021-01-26 10:29:57 +01:00
Chocobozzz d223dca0cd
Reformat sql console output 2021-01-26 10:03:41 +01:00
Chocobozzz 1e743faafe
Disable sql prettifier by default
It adds too much lines, leading to difficulties when reading dev logs
2021-01-26 09:54:32 +01:00
Chocobozzz 448487a602
Fix sendmail emailer 2021-01-26 09:28:49 +01:00
Florent 82f5527fc3
add option for video-import CLI to wait between two video imports (#3310)
* peertube-import-videos: Add --wait-interval option

* peertube-import-videos: various code style improvements

Co-authored-by: Florent F <florent.fayolle69@gmail.com>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
2021-01-25 15:42:44 +01:00
Rigel Kent 2a6cf69cff
prettify SQL queries during debug (#3635)
* prettify SQL queries during debug

* Use sql-formatter

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-01-25 15:37:41 +01:00
Chocobozzz 6939cbac48 Safely remove webtorrent files
Only remove them on max quality HLS playlist generation
2021-01-25 14:38:52 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
2021-01-25 14:38:52 +01:00
Chocobozzz 24516aa26a
Refactor transcoding job handlers 2021-01-21 15:58:17 +01:00
Chocobozzz 3b01f4c0ac
Support progress for ffmpeg tasks 2021-01-21 14:42:43 +01:00
Chocobozzz a4d2ca0715
Add user video list hooks 2021-01-20 15:28:34 +01:00
Chocobozzz 70fdff3d4e
Serve AP objects for default account/channel pages 2021-01-20 11:48:59 +01:00
Chocobozzz 471251fa14
Fix error log 2021-01-20 09:50:22 +01:00
Chocobozzz 86f553fbdf
Fix raw file import 2021-01-20 08:45:21 +01:00
Chocobozzz 805b8619c1
Fix (again) youtube import 2021-01-19 16:36:21 +01:00
Rigel Kent 370240824e
Allow users/visitors to search through an account's videos (#3589)
* WIP: account search

* add search to account view

* add tests for account search
2021-01-19 13:43:33 +01:00
Chocobozzz 03aa518e54
Fix manifest 2021-01-19 10:47:04 +01:00
Chocobozzz 9f39c5466e
Fix user notification test 2021-01-19 08:35:44 +01:00
Chocobozzz 969e59d17d
Fix youtube import 2021-01-18 14:29:16 +01:00
Chocobozzz 17e6974409
Fix lint 2021-01-18 11:59:14 +01:00
Chocobozzz a8a63b1ffe
Fix youtube-dl format fallback 2021-01-18 11:40:31 +01:00
Chocobozzz 1d22d2514f
Fix service worker 2021-01-18 11:16:37 +01:00
Chocobozzz 83befebe52
Increase test timeout 2021-01-18 09:52:05 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz d43c6b1ffc Implement remote interaction 2021-01-15 10:49:10 +01:00
Chocobozzz dd9c7929e6
Display debug ffmpeg stdout/stderr 2021-01-15 10:23:40 +01:00
Chocobozzz 53d43fc914
Remove unused tracker param 2021-01-15 10:23:40 +01:00
Chocobozzz c9c43612d5
Fix lint 2021-01-13 10:15:38 +01:00
Chocobozzz f5961a8cdf
Add 4k transcoding test 2021-01-13 09:52:52 +01:00
Chocobozzz 78d62f4d18
Fix and add skipping ping log tests 2021-01-13 09:38:19 +01:00
Rigel Kent 7abb6060a9 specific niceness for live transcoding 2021-01-13 09:16:44 +01:00
Rigel Kent d8b34ee55b
Allow user to search through their watch history (#3576)
* allow user to search through their watch history

* add tests for search in watch history

* Update client/src/app/shared/shared-main/users/user-history.service.ts
2021-01-13 09:16:15 +01:00
Johan Fleury 12c1e38df2 feat: allow administrator to disable logging of ping requests 2021-01-13 09:14:43 +01:00
Rigel Kent b7085c7132 add support for 1440p (Quad HD/QHD/WQHD) videos 2021-01-13 09:14:09 +01:00
Rigel Kent 7185dab3ff add test for script printing command 2021-01-13 09:13:31 +01:00
Rigel Kent 0984960345 add script printing command to generate a resolution for a given file 2021-01-13 09:13:31 +01:00
Rigel Kent 1ea7da819e
add ability to remove one's avatar for account and channels (#3467)
* add ability to remove one's avatar for account and channels

* add ability to remove one's avatar for account and channels

* only display avatar edition options after input change
2021-01-13 09:12:55 +01:00
Chocobozzz 26171379d0 Use source sans 3 font 2021-01-13 09:05:22 +01:00
Chocobozzz 8ace68056d
Increase test timeout 2021-01-12 09:02:40 +01:00
Chocobozzz 55a5b0fd8d
Update translations 2021-01-11 16:05:10 +01:00
Chocobozzz b14e8e461f
Fix tracker IP with reverse proxy 2021-01-11 15:45:04 +01:00
Chocobozzz a11a782f5c
Fix duplicate entry in job list page 2021-01-11 15:39:15 +01:00
Chocobozzz 1acb947504
Retrieve user by id instead of username 2021-01-11 13:53:08 +01:00
Chocobozzz b3fa96eea0
Don't reject accounts with empty description 2021-01-11 11:06:11 +01:00
Chocobozzz c2b823826d
Log RTMP server error (address already in use) 2021-01-11 09:33:14 +01:00
Chocobozzz 4f20856ed4
Add more hints regarding live port 2021-01-11 09:20:44 +01:00
Chocobozzz f42c21528b
Increase live test timeout 2021-01-08 12:11:45 +01:00
Chocobozzz 66dbfa1f30
Reduce inbox stats updated interval 2021-01-08 11:59:42 +01:00
Chocobozzz adc8e7e19f
Fix ldap test port 2021-01-04 11:21:19 +01:00
Chocobozzz f0484f7a16
Fix autoblacklist notification issue 2021-01-04 10:50:47 +01:00
Chocobozzz 027e308099
Don't crash on youtube-dl update write error 2021-01-04 10:05:47 +01:00
Chocobozzz 2dbc170da3
Add missing audit log if the user deletes its account 2021-01-04 09:54:48 +01:00
Chocobozzz cc49be3e35
Fix views processing hour 0 2020-12-30 14:29:17 +01:00
Chocobozzz 47d8e266f1
Support activeMonth and activeHalfyear in nodeinfo 2020-12-30 11:41:36 +01:00
Chocobozzz a046453549
Increase notifications comment test timeout 2020-12-30 11:27:59 +01:00
Chocobozzz 24198e976f
Styling 2020-12-30 11:26:24 +01:00
Theodore R. Smith 8e76aa1d75
(#3520) [cli] Hardened `auth add`: No longer fails with extraneous characters.
**The Solution:**
I have hardened `auth add` by stripping out everything from the third '/' to the end of the instance URL.

**The Problem:**
When adding an authorization for the peertube-cli, before this commit you could not have anything after the domain_name:port.

For instance, if there was a trailing / in your instance URL, before this commit it will always fail with

    expected 200 "OK", got 404 "Not Found".

It took me over 20 minutes to figure out that this was the problem.

See Issue #3091.
2020-12-27 14:29:54 -06:00
Chocobozzz f88453e233
Fix build 2020-12-22 16:49:17 +01:00
Chocobozzz 236841a1d7
Force HLS transcoding
Fix weird behaviour with some web browsers. Don't really know if it's a
ffmpeg bug, a safari bug or a peertube bug, but forcing transcoding
seems to fix this playback bug
2020-12-22 16:04:42 +01:00
Chocobozzz 9cfeb3cf98
Fix redundancy federation in some cases 2020-12-17 09:23:57 +01:00
Chocobozzz 9e454eba57
More robust server stats tests 2020-12-16 14:59:33 +01:00
Chocobozzz 1ab6024345
Fix live badge in videos list 2020-12-16 14:19:43 +01:00
Chocobozzz 54363cffa4
Fix stats tests 2020-12-16 10:36:15 +01:00
Chocobozzz 496b5a5bf6
Increase test timeout 2020-12-15 15:09:12 +01:00
Chocobozzz 99afa081bc
Add AP stats 2020-12-15 13:34:58 +01:00
Chocobozzz c9bc850e93
Use -1 for max live duration unlimited 2020-12-15 09:26:37 +01:00
Chocobozzz 113d4a3f73
Only link to crowdfunding pages in credits 2020-12-14 16:08:15 +01:00
kontrollanten 40c52969f7 emails: remove hardcoded PeerTube names 2020-12-14 15:37:29 +01:00
kontrollanten 2e4b8ae485 emailer: use instance name instead of hostname 2020-12-14 15:37:29 +01:00
kontrollanten 915e2bbb97 video embed: use instance name as html title 2020-12-14 15:25:51 +01:00
kontrollanten 2564d97e0d video-embed: inject custom CSS
closes #3420
2020-12-14 15:25:51 +01:00
Chocobozzz 402145b863
Refactor jobs state 2020-12-14 12:00:35 +01:00
Chocobozzz 5721fd8374
Fix channels import with a channel with playlists 2020-12-14 09:55:30 +01:00
Chocobozzz d8794cf855
More robust import script when using since/until 2020-12-14 09:55:30 +01:00
Chocobozzz f98c395295 Keep scope in notifier 2020-12-14 09:34:47 +01:00
kontrollanten b1771455af fix(notifier): notifyModeratorsOfNewAbuse
closes #3445
2020-12-14 09:34:47 +01:00
Chocobozzz 59fd824cf3 Fix tests timeout 2020-12-14 09:11:27 +01:00
Chocobozzz 34caef7fc0 Add joblog at the end of ci 2020-12-14 09:11:27 +01:00
Chocobozzz e078371807 Parse log script can take files as args 2020-12-14 09:11:27 +01:00
Rigel Kent 040d6896a3
add display of logs matching any state 2020-12-13 19:27:25 +01:00
Girish Ramakrishnan 75594f474a do not crash if SMTP server is down
just log a warning if the SMTP server is down on startup time

fixes #3457
2020-12-12 08:50:59 +01:00
Chocobozzz a800dbf345
Live views update 2020-12-09 16:31:42 +01:00
Chocobozzz 5cac83a78d
Refactor views component 2020-12-09 15:33:16 +01:00
Chocobozzz 0d8de2756f
Fix live tests 2020-12-09 14:45:47 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Rigel Kent f2eb23cd87
emit more specific status codes on video upload (#3423)
- reduce http status codes list to potentially useful codes
- convert more codes to typed ones
- factorize html generator for error responses
2020-12-08 21:16:10 +01:00
Chocobozzz 8e2392e512
Fix config test 2020-12-08 11:37:24 +01:00
Chocobozzz a59f210ff3 Refactor a little bit client canonical URL 2020-12-08 11:15:11 +01:00
Thomas Citharel e5024f5181 Add tests
Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2020-12-08 11:15:11 +01:00
Thomas Citharel 106fa2249d Set canonical link to original video/playlist url
The local URL was used instead of the original url

Signed-off-by: Thomas Citharel <tcit@tcit.fr>
2020-12-08 11:15:11 +01:00
kontrollanten 2805cb7cb6 server/mw/oauth: res.loc.auth to true upon auth
Setting res.locals.authenticated explicity to
true makes it's more clear whether the req is
authenticated when using optionalAuthenticate
middlware.
2020-12-08 10:40:08 +01:00
kontrollanten f17faefb30 plugins: add optional authentication for routes 2020-12-08 10:40:08 +01:00
Chocobozzz c824e8a0c7
Fix hook test 2020-12-08 10:38:47 +01:00
Chocobozzz 38267c0c8a Use dedicated hooks for account/channel videos 2020-12-08 10:33:23 +01:00
kontrollanten 1bfc07e4cc server accounts/videos: trigger filter hook 2020-12-08 10:33:23 +01:00
Chocobozzz 17b7d4b315
Don't allow manual redundancy of live videos 2020-12-07 16:07:52 +01:00
Chocobozzz 5b9b403a20
Fix replay last segments loop 2020-12-07 16:07:00 +01:00
Rigel Kent 2d53be0267
replace numbers with typed http status codes (#3409) 2020-12-07 14:32:36 +01:00
Chocobozzz 4ef9ea482f
Fix live transcoding tests 2020-12-04 16:41:03 +01:00
Chocobozzz aa5ee5017a
Fix lint 2020-12-04 15:58:29 +01:00
Chocobozzz e772bdf14c
Load video in permanent live after last one ended 2020-12-04 15:38:32 +01:00
Chocobozzz 3851e732c4
Fix audio issues with live replay 2020-12-04 15:10:13 +01:00
Chocobozzz 49bcdb0d66
Try to fix live freeze 2020-12-04 10:40:53 +01:00
Chocobozzz 55223d6574
Fix live transcoding threads config 2020-12-04 10:02:09 +01:00
Chocobozzz e26dc0cdbd
Strict actor url comparison 2020-12-03 16:12:58 +01:00
Chocobozzz 5147a6d945
Try to fix non HTTPS remote accounts 2020-12-03 15:21:16 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Rigel Kent d4132d3f56 more explicit error messages for file uploads 2020-12-03 10:15:49 +01:00
Chocobozzz 4a54a93941
Fix live invalid save replay duration 2020-12-03 10:05:10 +01:00
Chocobozzz ffc12d3ad5
Increase live constraint test timeout 2020-12-02 16:58:45 +01:00
Chocobozzz 3a735f7df7
Prepare changelog 2020-12-02 14:24:07 +01:00
Chocobozzz 2650d6d489
Fix live replay duration glitch 2020-12-02 10:18:15 +01:00
Chocobozzz 49cff3a4c9
Keep displayName as valid playlist sort 2020-12-01 15:08:59 +01:00
Chocobozzz d9550daf05
Add local actor follow url migration 2020-12-01 15:04:38 +01:00
Chocobozzz 9e3e4adc65
Add logs to fixture generations 2020-12-01 15:04:38 +01:00
Arman 117c643e45 Fix sort fields on Video Playlists
displayName does not exist in VideoPlaylistModel, the correct field is `name`
2020-12-01 14:58:35 +01:00
Chocobozzz 8c666c44ab
Fix live saving after a server restart 2020-11-30 17:03:13 +01:00
Chocobozzz 6cddd97d1d
Fix lint 2020-11-30 16:02:34 +01:00
Chocobozzz 937581b8f6
Fix high CPU with long live when save replay is true 2020-11-30 15:59:22 +01:00
Chocobozzz bd2e2f11d0
Add inspect to test script 2020-11-30 15:06:25 +01:00
Chocobozzz 0484ec9e41
Try to fix live tests 2020-11-30 09:16:41 +01:00
Chocobozzz 4a7f90248b
Fix cors on sha segment endpoint 2020-11-27 17:09:36 +01:00
Chocobozzz e5a516e702
More robust live test 2020-11-27 15:46:49 +01:00
Chocobozzz 9b337d8c0f
Fix 500 on unknown thread 2020-11-27 11:48:20 +01:00
Chocobozzz 353f8bc0c7
Add registered setting CLI plugin install test 2020-11-27 11:41:38 +01:00
Chocobozzz 0e856b78e9
Try to fix live test 2020-11-26 15:16:30 +01:00
Chocobozzz ce4a50b99b
Fix "Too many packets buffered for output stream"
And move encoder global options in a global wrapper
2020-11-26 14:45:48 +01:00
Chocobozzz fb4b3f91dc
Count only active lives for account limit 2020-11-26 14:08:31 +01:00
Chocobozzz 884d2c39ae
Fix live FPS limit 2020-11-26 11:30:18 +01:00
Chocobozzz d07b2944db
Fix comment account external URL
It did not work, and I think it's better to target the local account
page anyway
2020-11-25 15:11:57 +01:00
Chocobozzz a9d4c3c8ca
Avoid too large cert error messages 2020-11-25 14:43:43 +01:00
Chocobozzz 18490b0765 Fix migration and test 2020-11-25 11:07:56 +01:00
Rigel Kent 5beb89f223 refactor scoped token service 2020-11-25 11:07:56 +01:00
Rigel Kent afff310e50 allow private syndication feeds via a user feedToken 2020-11-25 11:07:56 +01:00
Chocobozzz f619de0e43 Add tests to gif resizer 2020-11-25 10:07:51 +01:00
Chocobozzz 6b67897e2e Add transcoding module comments 2020-11-25 10:07:51 +01:00
Chocobozzz 33ff70baa6 Fix audio encoding params 2020-11-25 10:07:51 +01:00
Chocobozzz ca5c612bfd Add live transcoding bit rate tests 2020-11-25 10:07:51 +01:00
Chocobozzz 5a547f69d5 Support encoding profiles 2020-11-25 10:07:51 +01:00
Chocobozzz 9252a33d11 Export encoders options in a dedicated struct 2020-11-25 10:07:51 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Kimsible 123f619336
Add pixel size to tooltip and gif support with FFmpeg for avatar upload (#3329)
* Add avatar pixel size upload in tooltip

* Add gif support for avatar

* Add ffmpeg GIF process

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-11-25 09:26:31 +01:00
Chocobozzz 47e363bbe9
Relax live fps test 2020-11-20 15:39:24 +01:00
Chocobozzz 74fd2643b4 Provide express request to onLogout call
+ pluginInfo related changes
2020-11-20 15:36:43 +01:00
Chocobozzz 8f3ad70874
Fix transcoder test 2020-11-20 15:14:23 +01:00
Chocobozzz d218e7de94
Do not transcode to an higher bitrate
Thanks bkil https://github.com/bkil
2020-11-20 15:11:31 +01:00
Chocobozzz 4749078b8a
Add actor follow url migration 2020-11-20 12:09:14 +01:00
Chocobozzz 55d5fbc583
Fix lint 2020-11-20 11:35:33 +01:00
Rigel Kent 64b5c2476a
remove outdated badges, add authorization to caption routes 2020-11-20 11:29:37 +01:00