Commit Graph

335 Commits

Author SHA1 Message Date
Chocobozzz 99cb53fd5a
Don't count deleted comment for replies 2021-06-08 13:48:58 +02:00
Chocobozzz 2b02c520e6
Cleanup shared models 2021-05-11 11:32:31 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
Chocobozzz a6e37eebfb
Fix transcoding job priority
New resolution jobs are also important if waiting for transcoding is
enabled since we publish the video after the first resolution generation
2021-05-05 09:32:20 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz 185eabe1ef
Increase test timeout 2021-04-26 11:33:14 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz cdeddff142 Add ability to update the banner 2021-04-08 10:07:53 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz 9d6b9d10ef
Fix video comments display with deleted comments 2021-02-19 10:06:52 +01:00
Chocobozzz 53c06121a4
Fix import tests 2021-02-18 14:08:16 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Rigel Kent 3d4e112d16 add best trending strategy based on Reddit's best
inspired from https://www.reddit.com/r/changelog/comments/7spgg0/best_is_the_new_hotness/
this implementation only adds freshness, and doesn't personalize based
on subscribed communities yet.
2021-02-04 09:04:47 +01:00
Chocobozzz 657bba2b29
Fix hls tests 2021-02-02 10:07:46 +01:00
Chocobozzz 40930fda86
Transcode audio uploads to lower resolutions
Better consistency
2021-02-02 09:45:42 +01:00
Chocobozzz 7519c4a27b
Fix tests 2021-01-29 15:58:01 +01:00
Chocobozzz 1896bca09e
Support transcoding options/encoders by plugins 2021-01-28 15:55:39 +01:00
Rigel Kent 923d3d5ad5 add test and openapi for hot sort parameter 2021-01-28 15:55:34 +01:00
Chocobozzz 6939cbac48 Safely remove webtorrent files
Only remove them on max quality HLS playlist generation
2021-01-25 14:38:52 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz f5961a8cdf
Add 4k transcoding test 2021-01-13 09:52:52 +01:00
Rigel Kent d8b34ee55b
Allow user to search through their watch history (#3576)
* allow user to search through their watch history

* add tests for search in watch history

* Update client/src/app/shared/shared-main/users/user-history.service.ts
2021-01-13 09:16:15 +01:00
Rigel Kent b7085c7132 add support for 1440p (Quad HD/QHD/WQHD) videos 2021-01-13 09:14:09 +01:00
Chocobozzz a11a782f5c
Fix duplicate entry in job list page 2021-01-11 15:39:15 +01:00
Chocobozzz 59fd824cf3 Fix tests timeout 2020-12-14 09:11:27 +01:00
Rigel Kent f2eb23cd87
emit more specific status codes on video upload (#3423)
- reduce http status codes list to potentially useful codes
- convert more codes to typed ones
- factorize html generator for error responses
2020-12-08 21:16:10 +01:00
Rigel Kent 2d53be0267
replace numbers with typed http status codes (#3409) 2020-12-07 14:32:36 +01:00
Chocobozzz 9e3e4adc65
Add logs to fixture generations 2020-12-01 15:04:38 +01:00
Chocobozzz 33ff70baa6 Fix audio encoding params 2020-11-25 10:07:51 +01:00
Chocobozzz ca5c612bfd Add live transcoding bit rate tests 2020-11-25 10:07:51 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Chocobozzz 8f3ad70874
Fix transcoder test 2020-11-20 15:14:23 +01:00
Chocobozzz d218e7de94
Do not transcode to an higher bitrate
Thanks bkil https://github.com/bkil
2020-11-20 15:11:31 +01:00
Chocobozzz 0aa52e1707
Add ability to display all channel/account videos 2020-11-18 15:29:38 +01:00
Chocobozzz 021c42657f
Try to fix multiple server tests 2020-11-18 14:15:25 +01:00
Chocobozzz 7f88a58e3f
Try to fix playlist tests 2020-11-18 13:41:27 +01:00
Chocobozzz 19149d45b8
Merge branch 'feature/admin-comments' into develop 2020-11-16 13:57:14 +01:00
Chocobozzz f127331459
Add admin view to manage comments 2020-11-16 13:48:58 +01:00
Chocobozzz 52a350a15c
Fix muted accounts on overviews 2020-11-10 11:06:36 +01:00
Chocobozzz c655c9ef6f Update ffmpeg static version for tests 2020-11-09 15:33:04 +01:00
Chocobozzz bd54ad1953 Add live notification tests 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Chocobozzz 25ae21107c
Fix tests 2020-08-26 15:22:26 +02:00
Chocobozzz 20dcfd7498
Deprecate old static routes 2020-08-26 15:03:00 +02:00
Chocobozzz 07491f4b98 Add fixtures cache to github actions 2020-08-24 18:54:37 +02:00
Chocobozzz 8b6f0fd53d
Reorganize a little bit tests 2020-08-24 13:53:45 +02:00
Chocobozzz 371906639e Add ability to a video multiple times in a playlist 2020-08-19 11:30:21 +02:00
Chocobozzz 951b582f52
Add ability to share playlists in modal 2020-08-07 13:43:48 +02:00
Chocobozzz 6fad8e51c4 Handle playlist oembed 2020-08-07 08:58:29 +02:00
Chocobozzz edbc932546 Add server API to abuse messages 2020-07-31 11:35:19 +02:00
Chocobozzz b488ba1e26 Don't rely on youtube for tests
Use another import URL when possible, and disable import tests when we
want to do a youtube import test
2020-07-31 11:32:04 +02:00
Rigel Kent 7b3909644d test search for subscriptions and video-channels 2020-07-29 18:15:53 +02:00
Chocobozzz 17aa80ed01
Merge branch 'release/2.3.0' into develop 2020-07-10 15:23:31 +02:00
Chocobozzz 1dee8d68cb
Fix video import tests 2020-07-10 14:56:17 +02:00
Chocobozzz 4f32032fed Add migrations 2020-07-10 14:02:41 +02:00
Chocobozzz d95d155988 Use 3 tables to represent abuses 2020-07-10 14:02:41 +02:00
Rigel Kent 1ebddadd07
predefined report reasons & improved reporter UI (#2842)
- added `startAt` and `endAt` optional timestamps to help pin down reported sections of a video
- added predefined report reasons
- added video player with report modal
2020-06-22 13:00:39 +02:00
Chocobozzz 8eb07b0130
Split notification tests 2020-06-16 15:55:42 +02:00
Chocobozzz 1ba471c55f
Add videos count in channels list 2020-06-16 14:13:01 +02:00
Chocobozzz 5a61ffbb7c
Optimize views per day in video channels 2020-06-12 16:27:41 +02:00
Rigel Kent 3487330d30
preserve original variable names server-side 2020-06-10 21:12:09 +02:00
Rigel Kent 5baee5fca4
rename blacklist to block/blocklist, merge block and auto-block views
- also replace whitelist with allowlist
- add advanced filters for video-block-list view
- move icons in video-block-list and video-abuse-list to left side
for visibility
- add robot icon to depict automated nature of a block in
video-block-list

resolves #2790
2020-06-10 21:12:05 +02:00
Chocobozzz f9b6d51f48
Fix tests 2020-06-10 15:35:20 +02:00
Levi Bard 3092e9bbb0
Make federation of unlisted videos an instance-level server preference (#2802)
* Add preference for federating unlisted videos

* Connect unlisted video federation with new preference

* Apply pull request feedback

* Fix lint issues

* Remove preference for federating unlisted videos from web admin interface
2020-06-03 09:42:07 +02:00
Chocobozzz 696d83fd13 Block comments from muted accounts/servers
Add better control for users of comments displayed on their videos:

 * Do not forward comments from muted remote accounts/servers (muted by the current server or by the video owner)
 * Do not list threads and hide replies (with their children) of accounts/servers muted by the video owner
 * Hide from RSS comments of muted accounts/servers by video owners

Use case:
  * Try to limit spam propagation in the federation
  * Add ability for users to automatically hide comments on their videos from undesirable accounts/servers (the comment section belongs to videomakers, so they choose what's posted there)
2020-05-29 09:32:20 +02:00
Chocobozzz 9a7fd9600b
Fix external auth email/password update
Also check if an actor does not already exist when creating the user
2020-05-20 10:17:27 +02:00
Chocobozzz feb34f6b6b Use video abuse filters on client side 2020-05-07 08:33:34 +02:00
Rigel Kent d1261d9a3c Add links from user-edit 2020-05-01 16:41:02 +02:00
Rigel Kent 197876ea87 Add tests for video-abuse persistence after video deletion 2020-05-01 16:41:02 +02:00
Chocobozzz 2fd59d7d89
Add ability to sort by originallyPublishedAt 2020-04-20 14:05:52 +02:00
Kim b1770a0af4
Add thumbnail / preview generation from url on the fly (#2646)
* Add thumbnails generation on the fly to URL import

* Display generated preview to import first edit

* Use ternary to get type inference

* Move preview/thumbnail test just after import

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-04-20 10:28:38 +02:00
Chocobozzz 652c64165b
Fix import captions test 2020-04-15 14:15:44 +02:00
kimsible ba6e9e8f1d Add unit tests for captions via URL import 2020-04-14 17:23:01 +02:00
Rigel Kent 3d527ba173 Use inner join and document code for viewr stats for channels 2020-03-31 10:29:24 +02:00
Rigel Kent 714bfcc556 Tests for viewsPerDay 2020-03-31 10:29:24 +02:00
Chocobozzz 764a965778
Implement pagination for overviews endpoint 2020-03-11 15:02:20 +01:00
Chocobozzz 7b81edc854
Video file metadata PR cleanup 2020-03-10 16:18:29 +01:00
Rigel Kent 8319d6ae72
Add video file metadata to download modal, via ffprobe (#2411)
* Add video file metadata via ffprobe

* Federate video file metadata

* Add tests for file metadata generation

* Complete tests for videoFile metadata federation

* Lint migration and video-file for metadata

* Objectify metadata from getter in ffmpeg-utils

* Add metadataUrl to all videoFiles

* Simplify metadata API middleware

* Load playlist in videoFile when requesting metadata
2020-03-10 14:39:40 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz c7f36e4f48
Fix transcoding 2020-01-29 16:54:52 +01:00
Rigel Kent 837666fe48 Add tests for video downscale framerate matching 2020-01-29 13:42:15 +01:00
Chocobozzz ffc65cbd2a
Remove HLS torrents 2020-01-24 16:48:05 +01:00
Chocobozzz fe98765624
Add ability to skip count query 2020-01-08 14:15:16 +01:00
Rigel Kent 32c68d67d9
Tests for totalRepliesFromVideoAuthor 2020-01-03 16:25:31 +01:00
Chocobozzz 822c7e610d
Fix playlist search 2019-12-27 09:04:04 +01:00
Chocobozzz 22a73cb879
Add internal privacy mode 2019-12-12 16:51:59 +01:00
Chocobozzz 941c5eac17
Add missing hotkeys to the watch page 2019-12-06 10:05:49 +01:00
Chocobozzz c883db6d03
Move deleted comment on new follow tests 2019-12-04 11:48:12 +01:00
Julien Maulny b5206dfc45 Fix retrieving of deleted comments when subscribing to a new instance 2019-12-04 09:36:45 +01:00
Julien Maulny 69222afac8 Soft delete video comments instead of detroy 2019-12-04 09:36:45 +01:00
Chocobozzz 52201311e1
Add codec information in HLS playlist 2019-11-26 16:25:36 +01:00
Julien Maulny 66a3674004 Fix lint 2019-11-25 16:59:09 +01:00
Chocobozzz 3a149e9f8b
Add audio only transcoding tests 2019-11-25 10:59:49 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Chocobozzz 210709a907
Fix tests 2019-09-18 17:20:37 +02:00
Chocobozzz f0a47bc92a
Hide video abuses from muted accounts 2019-08-29 14:31:04 +02:00
Chocobozzz f92e7f76d4
Fix ownership change 2019-08-21 08:57:00 +02:00
Chocobozzz 0b353d1d8a
Fix redundancy exceeding the limit 2019-08-12 08:46:46 +02:00
Chocobozzz 511765c9f8
Remove comment federation by video owner 2019-08-07 15:35:29 +02:00
Chocobozzz 65af03a241
Automatically update playlist thumbnails 2019-08-01 16:54:24 +02:00
Chocobozzz bfbd912886 Fix broken playlist api 2019-08-01 09:11:04 +02:00
Chocobozzz 2284f20207
Add gitlab ci support 2019-07-29 13:40:39 +02:00
Chocobozzz 24163420dd Add retries to travis mocha 2019-07-24 10:58:16 +02:00
Chocobozzz 109d893ff5 Fix lint 2019-07-24 10:58:16 +02:00
Chocobozzz 1f20622f2b
Improve registration
* Add ability to set the user display name
 * Use display name to guess the username/channel name
 * Add explanations about what is the purpose of a username/channel name
 * Add a loader at the "done" step
2019-06-07 17:05:42 +02:00
Chocobozzz b91bc1d1f3
Merge branch 'release/v1.3.0' into develop 2019-06-07 15:04:41 +02:00
Chocobozzz 03371ad9d0
Fix video import if autoblacklist is enabled 2019-06-07 14:35:53 +02:00
Chocobozzz 1b319b7aa6
Add ability to unpublish video/playlist 2019-06-06 17:29:15 +02:00
Chocobozzz 46a6db245f
Add ability to set to private a public/unlisted video 2019-06-06 16:44:02 +02:00
Chocobozzz 7d14d4d2ca
Server: Bulk update videos support field 2019-05-31 16:44:46 +02:00
Chocobozzz 57cfff7885
Remove unused actor uuid field 2019-05-31 14:49:31 +02:00
Chocobozzz 91b6631984
Add pagination to account video channels endpoint 2019-05-29 15:09:38 +02:00
Chocobozzz 4d09cfba78
Fix playlist get for classic users 2019-05-28 10:16:59 +02:00
Chocobozzz 964298de4e
Fix playlist get for classic users 2019-05-28 09:50:27 +02:00
Chocobozzz 80b8ad2a70
Update travis ffmpeg patch version 2019-05-17 16:50:52 +02:00
Chocobozzz b345a8047b
Add audio upload tests 2019-05-17 14:04:54 +02:00
Chocobozzz 1600235a2f
Merge remote-tracking branch 'origin/pr/1785' into develop 2019-05-16 08:58:39 +02:00
Chocobozzz 48f07b4a40
All API tests in parallel 2019-05-15 15:14:21 +02:00
Chocobozzz 7243f84db0
Redundancy and search tests in parallel too 2019-05-14 15:37:43 +02:00
Felix Ableitner 7ed2c1a46f fixed formatting, added test case 2019-05-04 13:06:43 +02:00
Chocobozzz 8519cc9234
Fix overview endpoint 2019-04-24 17:29:19 +02:00
Chocobozzz 7c3b79768b
Use test wrapper exit function 2019-04-24 16:26:25 +02:00
Chocobozzz 42e1ec25ec
Try to fix travis 2019-04-24 16:26:24 +02:00
Chocobozzz 210feb6cc4
Cleanup tests 2019-04-24 16:26:23 +02:00
Chocobozzz 94565d52bb
Shared utils -> extra-utils
Because they need dev dependencies
2019-04-24 16:25:52 +02:00
Chocobozzz 1eddc9a74f
Add user adminFlags 2019-04-15 14:39:52 +02:00
Chocobozzz cda03765fe
Add ability to delete old remote views 2019-04-12 08:31:06 +02:00
Chocobozzz 8f0bc73d7d
Add ability to limit videos history size 2019-04-11 15:38:53 +02:00
Josh Morel 7ccddd7b52 add quarantine videos feature (#1637)
* add quarantine videos feature

* increase Notification settings test timeout

to 20000ms. was completing 7000 locally but timing out
after 10000 on travis

* fix quarantine video test issues

-propagate misspelling
-remove skip from server/tests/client.ts

* WIP use blacklist for moderator video approval

instead of video.quarantine boolean

* finish auto-blacklist feature
2019-04-02 11:26:47 +02:00
Chocobozzz 2ba9287131
Cleanup invalid rates/comments/shares 2019-03-19 16:23:02 +01:00
Chocobozzz 397d78fb3e Fix unit tests 2019-03-18 11:17:59 +01:00
Chocobozzz 2a10aab3d7 Add playlist updatedAt tests 2019-03-18 11:17:59 +01:00
Chocobozzz 0b16f5f220 Add videos playlist exist tests 2019-03-18 11:17:59 +01:00
Chocobozzz d4c9f45b31 Add server migrations 2019-03-18 11:17:59 +01:00
Chocobozzz 09979f8959 Refactor video playlist middlewares 2019-03-18 11:17:59 +01:00
Chocobozzz df0b219d36 Add playlist rest tests 2019-03-18 11:17:59 +01:00
Chocobozzz 418d092afa Playlist server API 2019-03-18 11:17:59 +01:00
Chocobozzz 17036be5bc
Try to fix travis tests 2019-02-15 14:04:34 +01:00
Chocobozzz c74c9be934
Refractor published date on video import 2019-02-12 11:47:23 +01:00
Chocobozzz 7519127b5c
Add originallyPublishedAt unit tests 2019-02-11 15:16:39 +01:00
Chocobozzz 88108880bb
Merge branch 'develop' into pr/1217 2019-02-11 11:52:34 +01:00
Chocobozzz 53a94c7cfa
Add federation tests on download enabled 2019-02-11 11:48:56 +01:00
Chocobozzz 4c280004ce Use a single file instead of segments for HLS 2019-02-11 09:13:02 +01:00
Chocobozzz 0920929696 Add hls support on server 2019-02-11 09:13:02 +01:00
Chocobozzz 5abb9fbbd1
Add ability to unfederate a local video (on blacklist) 2019-01-10 15:39:51 +01:00
Chocobozzz 8b9a525a18
Add history on server side
Add ability to disable, clear and list user videos history
2018-12-18 11:35:50 +01:00
Chocobozzz 14e2014acc Support additional video extensions 2018-12-11 15:11:09 +01:00
Chocobozzz 8923187455
Add test regarding tmp directory 2018-12-11 09:18:02 +01:00