Commit Graph

369 Commits

Author SHA1 Message Date
Chocobozzz 1d43c3a613
Use separate queries for video files 2021-06-11 09:31:59 +02:00
Chocobozzz ea54cd04c1
Fix video upload with a capitalized ext 2021-06-08 10:01:50 +02:00
Rigel Kent c756bae079
add video upload types, add doc middleware to more routes 2021-06-06 10:21:06 +02:00
Chocobozzz 91f8f8db97
createThumbnail -> updateThumbnail 2021-06-04 15:45:44 +02:00
Rigel Kent 1333ab1f2d
add operationId doc middleware to so API endpoints 2021-06-04 09:21:46 +02:00
Rigel Kent 1c627fd8d2
refactor docMiddleware to support operationId-only form 2021-06-04 09:21:46 +02:00
Chocobozzz e030bfb59d
Refactor server errors handler 2021-06-02 18:15:41 +02:00
Chocobozzz 304a84d59c
Refactor getOrCreateAPVideo 2021-06-02 16:57:53 +02:00
Rigel Kent 3866ea02d4 correct error codes and backward compat 2021-06-02 16:57:07 +02:00
Rigel Kent 76148b27f7 refactor API errors to standard error format 2021-06-02 16:57:07 +02:00
Chocobozzz 2539932e16
Instance homepage support (#4007)
* Prepare homepage parsers

* Add ability to update instance hompage

* Add ability to set homepage as landing page

* Add homepage preview in admin

* Dynamically update left menu for homepage

* Inject home content in homepage

* Add videos list and channel miniature custom markup

* Remove unused elements in markup service
2021-05-27 15:59:55 +02:00
Chocobozzz c158a5faab
Refactor a little bit controllers 2021-05-12 16:57:31 +02:00
Chocobozzz 16c016e8b1
Stricter models typing 2021-05-12 14:12:14 +02:00
Chocobozzz 32985a0a77
Error if importing a torrent with multiple files 2021-05-11 14:56:30 +02:00
Chocobozzz 2b02c520e6
Cleanup shared models 2021-05-11 11:32:31 +02:00
Chocobozzz 7d9ba5c089
Cleanup models directory organization 2021-05-11 11:32:31 +02:00
Chocobozzz 1bcb03a100
Use a class for youtube-dl 2021-05-11 11:32:31 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Chocobozzz 452b3bea08
Introduce tags to the logger
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
Chocobozzz 6c9c3b7b14
Fix emptying video tags 2021-03-03 11:03:30 +01:00
Chocobozzz d61893f723 Async torrent creation 2021-02-25 15:01:07 +01:00
Chocobozzz 300cb723eb
Fix tests 2021-02-19 11:27:38 +01:00
Chocobozzz 2a021e6cb6
Fix thread replies API response 2021-02-19 10:52:31 +01:00
Chocobozzz 285981f4dd
Optimize rate endpoint 2021-02-19 10:26:58 +01:00
Chocobozzz 9d6b9d10ef
Fix video comments display with deleted comments 2021-02-19 10:06:52 +01:00
Chocobozzz 8efc27bf14 Cleanup 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz a35a22797c Remove previous thumbnail if needed 2021-02-16 10:36:44 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Chocobozzz e3b4c084cd
Guess if we need to generate the thumbnail for imports 2021-02-12 09:37:01 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
2021-01-25 14:38:52 +01:00
Chocobozzz 805b8619c1
Fix (again) youtube import 2021-01-19 16:36:21 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Rigel Kent 2d53be0267
replace numbers with typed http status codes (#3409) 2020-12-07 14:32:36 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Rigel Kent d4132d3f56 more explicit error messages for file uploads 2020-12-03 10:15:49 +01:00
Chocobozzz 9b337d8c0f
Fix 500 on unknown thread 2020-11-27 11:48:20 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Chocobozzz de94ac86a2
Fix incorrect IDs in AP federation 2020-11-20 11:21:08 +01:00
Chocobozzz 0f8d00e314
Implement video comment list in admin 2020-11-13 16:38:23 +01:00
Chocobozzz 7a4ea93246
Remove deprecated abuse api 2020-11-10 14:50:16 +01:00
Chocobozzz e4bf785617 Handle views for live videos 2020-11-09 15:33:04 +01:00
Chocobozzz 3cabf3532b Add live server hooks 2020-11-09 15:33:04 +01:00
Chocobozzz 053aed43fb Regenerate miniature on live save 2020-11-09 15:33:04 +01:00
Chocobozzz af4ae64f6f Begin live tests 2020-11-09 15:33:04 +01:00
Chocobozzz b5b687550d Add ability to save live replay 2020-11-09 15:33:04 +01:00
Chocobozzz de6310b2fc Handle live federation 2020-11-09 15:33:04 +01:00
Chocobozzz 1ef65f4c03 Refactor video creation 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Chocobozzz 7294aab0c8 Add ability to set custom field to video form 2020-08-21 15:39:51 +02:00
Chocobozzz 8054669f11
Optimize default sort when listing videos
createdAt does not have the appropriate index
2020-08-20 11:05:45 +02:00
Chocobozzz 583eb04b54 Upgrade to angular 10 2020-08-07 08:28:14 +02:00
Chocobozzz edbc932546 Add server API to abuse messages 2020-07-31 11:35:19 +02:00
Chocobozzz d95d155988 Use 3 tables to represent abuses 2020-07-10 14:02:41 +02:00
Rigel Kent 1ebddadd07
predefined report reasons & improved reporter UI (#2842)
- added `startAt` and `endAt` optional timestamps to help pin down reported sections of a video
- added predefined report reasons
- added video player with report modal
2020-06-22 13:00:39 +02:00
Chocobozzz 26d6bf6533
Split types and typings 2020-06-18 10:46:27 +02:00
Chocobozzz faa9d434b4
Update server dependencies 2020-06-17 11:42:50 +02:00
Chocobozzz 696d83fd13 Block comments from muted accounts/servers
Add better control for users of comments displayed on their videos:

 * Do not forward comments from muted remote accounts/servers (muted by the current server or by the video owner)
 * Do not list threads and hide replies (with their children) of accounts/servers muted by the video owner
 * Hide from RSS comments of muted accounts/servers by video owners

Use case:
  * Try to limit spam propagation in the federation
  * Add ability for users to automatically hide comments on their videos from undesirable accounts/servers (the comment section belongs to videomakers, so they choose what's posted there)
2020-05-29 09:32:20 +02:00
Chocobozzz 8adf0a767f Fix comments deleted display 2020-05-29 09:21:26 +02:00
Chocobozzz 444c0a0e01 Add ability to bulk delete comments 2020-05-29 09:21:26 +02:00
Rigel Kent 1d26d05fd9 Align follower handle display on instance display in tables 2020-05-08 15:31:51 +02:00
Rigel Kent df4c603dea Switch emails to pug templates and provide richer html/text-only versions 2020-05-08 15:31:51 +02:00
Chocobozzz 80fdaf0645
Add moderation helpers to plugins 2020-05-07 15:07:01 +02:00
Chocobozzz feb34f6b6b Use video abuse filters on client side 2020-05-07 08:33:34 +02:00
Chocobozzz 8dc8a34ee8 Avoir some circular dependencies 2020-05-04 16:21:39 +02:00
Rigel Kent e0a929179a Add filter inputs for blacklisted videos and muted accounts/servers 2020-05-01 16:41:02 +02:00
Rigel Kent 844db39ee5 Add search for video, reporter and channel name fields 2020-05-01 16:41:02 +02:00
Chocobozzz 86ad0cde9e
Add licence and language support to y-dl imports 2020-04-20 10:41:43 +02:00
Kim b1770a0af4
Add thumbnail / preview generation from url on the fly (#2646)
* Add thumbnails generation on the fly to URL import

* Display generated preview to import first edit

* Use ternary to get type inference

* Move preview/thumbnail test just after import

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-04-20 10:28:38 +02:00
Chocobozzz 8a86e5dc4f
Use originallyPublishedAt from body on import if it exists 2020-04-17 14:35:13 +02:00
Chocobozzz 652c64165b
Fix import captions test 2020-04-15 14:15:44 +02:00
kimsible 50ad0a1c16 Add getSubs to YoutubeDL video import 2020-04-14 14:39:30 +02:00
Chocobozzz d57d1d83c6
Support audio files import 2020-04-03 15:41:39 +02:00
Rigel Kent 8319d6ae72
Add video file metadata to download modal, via ffprobe (#2411)
* Add video file metadata via ffprobe

* Federate video file metadata

* Add tests for file metadata generation

* Complete tests for videoFile metadata federation

* Lint migration and video-file for metadata

* Objectify metadata from getter in ffmpeg-utils

* Add metadataUrl to all videoFiles

* Simplify metadata API middleware

* Load playlist in videoFile when requesting metadata
2020-03-10 14:39:40 +01:00
Chocobozzz 2c8776fc31
Optimize view endpoint 2020-02-04 16:18:01 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz ca6d36227a
Add url field in caption and use it for thumbnails 2020-01-30 11:53:38 +01:00
Rigel Kent 06bcfbd9f7 Downsample to the closest divisor standard framerate 2020-01-29 13:42:15 +01:00
Chocobozzz fe98765624
Add ability to skip count query 2020-01-08 14:15:16 +01:00
Chocobozzz 22a73cb879
Add internal privacy mode 2019-12-12 16:51:59 +01:00
Julien Maulny 69222afac8 Soft delete video comments instead of detroy 2019-12-04 09:36:45 +01:00
Chocobozzz 3f6b7aa1cf
Don't move the caption if it has the right name 2019-12-03 10:10:14 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Frank Sträter 46aaefa9b8 Update default value for commentsEnabled 2019-09-18 15:25:12 +02:00
Chocobozzz f0a47bc92a
Hide video abuses from muted accounts 2019-08-29 14:31:04 +02:00
Chocobozzz 3155c8606c
Fix default true values when uploding videos 2019-08-28 16:03:26 +02:00
Frank Sträter b1b7f7160b Bug fix logical and/or boolean selector 2019-08-28 12:58:47 +02:00
Chocobozzz 1ca9f7c3f7
Type toFormattedJSON 2019-08-20 19:23:10 +02:00
Chocobozzz 0283eaac2a
Cleanup model types 2019-08-20 14:53:42 +02:00
Chocobozzz 96ca24f00e
Fix tests 2019-08-20 10:22:05 +02:00
Chocobozzz 453e83ea5d
Stronger model typings 2019-08-19 17:26:35 +02:00
Chocobozzz 511765c9f8
Remove comment federation by video owner 2019-08-07 15:35:29 +02:00
Chocobozzz 65af03a241
Automatically update playlist thumbnails 2019-08-01 16:54:24 +02:00
Chocobozzz 2284f20207
Add gitlab ci support 2019-07-29 13:40:39 +02:00
Chocobozzz 5b77537ce5 Correctly notify on auto blacklist 2019-07-24 10:58:16 +02:00
Chocobozzz a8b666e9f1 Add plugin static files cache 2019-07-24 10:58:16 +02:00
Chocobozzz 6691c52280 Add filter hooks tests 2019-07-24 10:58:16 +02:00
Chocobozzz 89cd127560 Add hook filters tests 2019-07-24 10:58:16 +02:00
Chocobozzz b4055e1c23 Add server hooks 2019-07-24 10:58:16 +02:00
Chocobozzz b91bc1d1f3
Merge branch 'release/v1.3.0' into develop 2019-06-07 15:04:41 +02:00
Chocobozzz 03371ad9d0
Fix video import if autoblacklist is enabled 2019-06-07 14:35:53 +02:00
Chocobozzz 46a6db245f
Add ability to set to private a public/unlisted video 2019-06-06 16:44:02 +02:00
Chocobozzz ad3405d087
Improve 4K video quality after transcoding 2019-06-06 14:45:57 +02:00
Chocobozzz b345a8047b
Add audio upload tests 2019-05-17 14:04:54 +02:00
Chocobozzz 536598cfaf
Add audio support in upload 2019-05-16 17:16:47 +02:00
Chocobozzz 2b4dd7e26d
Fix optional privacy in upload endpoint 2019-04-25 13:55:28 +02:00
Chocobozzz b876eaf11a
Fix ownership changes 2019-04-24 16:26:21 +02:00
Chocobozzz 3acc508440
Upgrade sequelize 2019-04-24 16:26:21 +02:00
Chocobozzz 1735c82572
Update sequelize 2019-04-24 16:26:20 +02:00
Chocobozzz e8bafea35b
Create a dedicated table to track video thumbnails 2019-04-24 16:25:52 +02:00
Chocobozzz 74dc3bca2b
Don't expose constants directly in initializers/ 2019-04-11 14:26:41 +02:00
Chocobozzz 6dd9de95df
Move config in its own file 2019-04-11 13:45:39 +02:00
Josh Morel 7ccddd7b52 add quarantine videos feature (#1637)
* add quarantine videos feature

* increase Notification settings test timeout

to 20000ms. was completing 7000 locally but timing out
after 10000 on travis

* fix quarantine video test issues

-propagate misspelling
-remove skip from server/tests/client.ts

* WIP use blacklist for moderator video approval

instead of video.quarantine boolean

* finish auto-blacklist feature
2019-04-02 11:26:47 +02:00
Chocobozzz a0327eedb0
Rename video-file job to video-transcoding 2019-03-19 17:00:08 +01:00
Chocobozzz 2ba9287131
Cleanup invalid rates/comments/shares 2019-03-19 16:23:02 +01:00
Chocobozzz dae86118ed
Cleanup express locals typings 2019-03-19 10:35:15 +01:00
Chocobozzz 0f6acda116
Does exist 2019-03-19 09:26:50 +01:00
Chocobozzz 418d092afa Playlist server API 2019-03-18 11:17:59 +01:00
Andrés Maldonado 4e553a41fd
Import original publication date (web UI)
Import original publication date when importing a video from YouTube in the web UI
2019-02-12 11:19:19 +01:00
Chocobozzz b718fd2237
Merge branch 'develop' into pr/1285 2019-02-11 14:09:23 +01:00
Chocobozzz 88108880bb
Merge branch 'develop' into pr/1217 2019-02-11 11:52:34 +01:00
Chocobozzz 28f3d1b36a
Merge branch 'feature/correctly-send-activities' into develop 2019-02-11 11:00:09 +01:00
Chocobozzz 0920929696 Add hls support on server 2019-02-11 09:13:02 +01:00
Chocobozzz 1e7eb25f6c
Correctly send Flag/Dislike/View activities 2019-01-15 14:59:40 +01:00
Chocobozzz 744d0eca19
Refresh remote actors on GET enpoints 2019-01-14 11:30:15 +01:00
clementbrizard 1e74f19a21 Enable video upload and edit 2019-01-12 13:45:23 +00:00
Chocobozzz 5abb9fbbd1
Add ability to unfederate a local video (on blacklist) 2019-01-10 15:39:51 +01:00
Chocobozzz e8d246d526 Add notification settings migration 2019-01-09 11:15:15 +01:00
Chocobozzz cef534ed53 Add user notification base code 2019-01-09 11:15:15 +01:00
Chocobozzz f481c4f9f3
Use move instead rename
To avoid EXDEV errors
2018-12-11 15:12:38 +01:00
Chocobozzz 14e2014acc Support additional video extensions 2018-12-11 15:11:09 +01:00
Chocobozzz 6040f87d14
Add tmp and redundancy directories 2018-12-04 16:04:15 +01:00
Chocobozzz dbe6aa698e
Fix trending page 2018-12-03 09:14:56 +01:00
Chocobozzz 04b8c3fba6
Delete invalid or deleted remote videos 2018-11-20 10:44:48 +01:00
Chocobozzz 8d4273463f
Check follow constraints when getting a video 2018-11-16 15:49:16 +01:00
Chocobozzz 030177d246
Don't forward view, send updates instead
To avoid inconsistencies in the federation, now the origin server will
tell other instances what is the correct number of views
2018-11-15 16:38:37 +01:00
Chocobozzz 5c6d985fae
Check activities host 2018-11-14 16:32:27 +01:00
Chocobozzz 7ad9b9846c Add ability for users to block an account/instance on server side 2018-10-16 16:41:36 +02:00
Lucas Declercq 7f2cfe3a79 Rename downloadingEnabled property to downloadEnabled 2018-10-08 14:45:22 +02:00
Lucas Declercq 4ffdcfc63b Fix some defaults values + indentation 2018-10-08 14:42:55 +02:00
Lucas Declercq 156c50af30 Add downloadingEnabled property to video model 2018-10-06 19:17:39 +02:00
Chocobozzz 6e46de095d
Add user history and resume videos 2018-10-05 11:22:38 +02:00
Chocobozzz 8b917537af
Increase timeout on upload endpoint 2018-10-01 09:14:00 +02:00
Chocobozzz 91411dba92
Limit associations fetch when loading token 2018-09-20 11:45:59 +02:00
Chocobozzz 993cef4b6e
Refractor audit user identifier 2018-09-19 17:02:27 +02:00
Chocobozzz 627621c1e8
Optimize SQL requests of watch page API endpoints 2018-09-19 09:54:37 +02:00