Commit Graph

520 Commits

Author SHA1 Message Date
Chocobozzz 60520c7c85
Increase fetcher jobs concurrency 2021-06-15 15:31:08 +02:00
Chocobozzz a13df19d57
Increase max image/caption/torrent upload size 2021-06-08 10:01:50 +02:00
Chocobozzz 2539932e16
Instance homepage support (#4007)
* Prepare homepage parsers

* Add ability to update instance hompage

* Add ability to set homepage as landing page

* Add homepage preview in admin

* Dynamically update left menu for homepage

* Inject home content in homepage

* Add videos list and channel miniature custom markup

* Remove unused elements in markup service
2021-05-27 15:59:55 +02:00
Chocobozzz aea0b0e7cd
Inject server config in HTML 2021-05-14 13:37:23 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz a6e37eebfb
Fix transcoding job priority
New resolution jobs are also important if waiting for transcoding is
enabled since we publish the video after the first resolution generation
2021-05-05 09:32:20 +02:00
Chocobozzz f816da921f
Add migration to remove duplicated keys
Due to old/corrupted data, it can be problematic to restore a backup
2021-04-26 14:27:42 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz 02dd4f3c65 Display banner on channel page 2021-04-08 10:07:53 +02:00
Chocobozzz 282695e699 Add banner migrations 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Chocobozzz f479685678 Agnostic actor image storage 2021-04-08 10:07:53 +02:00
Chocobozzz 4363ce0e3e
Increase thumbnail size on server 2021-04-05 11:44:09 +02:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 534556052e
Optimize views sort sql query 2021-03-03 11:24:16 +01:00
Chocobozzz dcd75f786c
Rename request timeout 2021-03-03 10:10:55 +01:00
Chocobozzz 67bd004d84
Increase broadcast concurrency 2021-03-03 10:09:34 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 8795d6f254
Fix broken local actors
Some channels can't federate because they don't have public/private
keys, maybe because the generation failed for various reasons
2021-02-26 14:22:25 +01:00
Chocobozzz 23ac334389
Increase jobs request timeout 2021-02-26 10:48:12 +01:00
Chocobozzz e072fb1e16
Add m4a support 2021-02-25 09:14:24 +01:00
Chocobozzz 9556ce48e7
Fix async issues with channels list 2021-02-25 09:09:41 +01:00
Chocobozzz e33933e44c
Fix latest migrations
We made an historical mistake: we have a unique key on videoId,
resolution AND fps. But the filenames and torrent filenames do not have
fps in their name

Remove these duplicate entries
2021-02-19 08:41:32 +01:00
Chocobozzz d9a2a03196 Don't guess remote tracker URL 2021-02-18 13:38:09 +01:00
Chocobozzz 2451916e45 Add video files migration 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Chocobozzz a8b1b40485 Generate a name for thumbnails
Allows aggressive caching
2021-02-16 10:36:44 +01:00
Chocobozzz 9129b7694d Allow to specify transcoding and import jobs concurrency 2021-02-08 15:38:45 +01:00
Rigel Kent 3d4e112d16 add best trending strategy based on Reddit's best
inspired from https://www.reddit.com/r/changelog/comments/7spgg0/best_is_the_new_hotness/
this implementation only adds freshness, and doesn't personalize based
on subscribed communities yet.
2021-02-04 09:04:47 +01:00
Chocobozzz 529b37527c
Use a profile manager for transcoding 2021-01-28 15:55:39 +01:00
Rigel Kent 5bcbcbe338 modularize abstract video list header and implement video hotness recommendation variant 2021-01-28 15:55:34 +01:00
Chocobozzz 00b87c5791
Fix live RAM usage when ffmpeg is too slow 2021-01-27 12:05:10 +01:00
Chocobozzz a4a8cd3971
Add missing niceness to ffmpeg thumbnail processes 2021-01-27 09:09:59 +01:00
Chocobozzz e9cb361cf5
Fix tests 2021-01-26 11:37:31 +01:00
Chocobozzz e08ff02a9f
Allow webp image uploads 2021-01-26 10:29:57 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
2021-01-25 14:38:52 +01:00
Rigel Kent 7abb6060a9 specific niceness for live transcoding 2021-01-13 09:16:44 +01:00
Chocobozzz 66dbfa1f30
Reduce inbox stats updated interval 2021-01-08 11:59:42 +01:00
Chocobozzz 99afa081bc
Add AP stats 2020-12-15 13:34:58 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Chocobozzz 49cff3a4c9
Keep displayName as valid playlist sort 2020-12-01 15:08:59 +01:00
Chocobozzz d9550daf05
Add local actor follow url migration 2020-12-01 15:04:38 +01:00
Arman 117c643e45 Fix sort fields on Video Playlists
displayName does not exist in VideoPlaylistModel, the correct field is `name`
2020-12-01 14:58:35 +01:00
Chocobozzz 937581b8f6
Fix high CPU with long live when save replay is true 2020-11-30 15:59:22 +01:00
Chocobozzz 18490b0765 Fix migration and test 2020-11-25 11:07:56 +01:00
Chocobozzz 9252a33d11 Export encoders options in a dedicated struct 2020-11-25 10:07:51 +01:00
Kimsible 123f619336
Add pixel size to tooltip and gif support with FFmpeg for avatar upload (#3329)
* Add avatar pixel size upload in tooltip

* Add gif support for avatar

* Add ffmpeg GIF process

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-11-25 09:26:31 +01:00
Chocobozzz 4749078b8a
Add actor follow url migration 2020-11-20 12:09:14 +01:00
Chocobozzz e14de000f9
Update live settings edge delay 2020-11-18 11:32:57 +01:00
Chocobozzz 1648dcb509
Increase live delay 2020-11-17 14:05:15 +01:00
Chocobozzz 0f8d00e314
Implement video comment list in admin 2020-11-13 16:38:23 +01:00
Chocobozzz 501af82d99
Delay notification when waiting for a live 2020-11-12 17:13:32 +01:00
Chocobozzz 797d05bdd9
Force signed headers in http signatures
Thanks Roger
2020-11-12 16:29:32 +01:00
Chocobozzz adbf1ed8c4
Use our implementation of randomInt 2020-11-10 17:04:09 +01:00
Chocobozzz 2af337c839
Cleanup follows of orphean actors 2020-11-10 16:29:35 +01:00
Chocobozzz 0223291c45
Delay video views scheduled job
To not overload the federation
2020-11-10 15:04:08 +01:00
Chocobozzz 998440b6ec
Fix live migrations 2020-11-10 10:45:20 +01:00
Chocobozzz c655c9ef6f Update ffmpeg static version for tests 2020-11-09 15:33:04 +01:00
Chocobozzz e4bf785617 Handle views for live videos 2020-11-09 15:33:04 +01:00
Chocobozzz 529f037294 Revert some mistakes 2020-11-09 15:33:04 +01:00
Chocobozzz ba881f0e3f Reduce P2P blacklist in tracker 2020-11-09 15:33:04 +01:00
Chocobozzz 68e70a745b Add save replay live tests 2020-11-09 15:33:04 +01:00
Chocobozzz 97969c4edf Add check constraints live tests 2020-11-09 15:33:04 +01:00
Chocobozzz 31c82cd914 Fix replay saving 2020-11-09 15:33:04 +01:00
Chocobozzz fb7194043d Check live duration and size 2020-11-09 15:33:04 +01:00
Chocobozzz a5cf76afa3 Add watch messages if live has not started 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Chocobozzz 20dcfd7498
Deprecate old static routes 2020-08-26 15:03:00 +02:00
Chocobozzz cbb513e737 Add server migration for playlists 2020-08-19 11:30:21 +02:00
Rigel Kent b056afe0ac
add .ac3, .aac audio and more video extensions 2020-08-18 09:54:05 +02:00
Chocobozzz 07c01b3154
Add kabile to supported languages 2020-08-11 09:41:57 +02:00
Chocobozzz 30bc55c88b Refactor video extensions logic in server 2020-08-11 07:57:20 +02:00
Rigel Kent fccbbc1a77 add and document new additional video extensions supported at upload 2020-08-10 09:46:59 +02:00
Chocobozzz d573926e9b Add migrations for abuse messages 2020-07-31 11:35:19 +02:00
Chocobozzz edbc932546 Add server API to abuse messages 2020-07-31 11:35:19 +02:00
Rigel Kent 654a188f80 allow sorting notifications 2020-07-29 18:15:53 +02:00
Chocobozzz 4f32032fed Add migrations 2020-07-10 14:02:41 +02:00
Chocobozzz d95d155988 Use 3 tables to represent abuses 2020-07-10 14:02:41 +02:00
Kim caf2aaf4f9
Add ability to override client assets : logo - favicon - PWA icons - PWA manifest name and description (#2897)
* Add client-overrides storage to config

* Add static-serve for client overrides

* Move backgroun-image logo from bundle to css tag for runtime content hash

* Add dynamic JSON manifest

* Add content hash for manifest, favicon and logo

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-07-10 10:20:11 +02:00
Chocobozzz db48de8597
Block infohash spammers from tracker 2020-06-25 16:28:07 +02:00
Rigel Kent 1ebddadd07
predefined report reasons & improved reporter UI (#2842)
- added `startAt` and `endAt` optional timestamps to help pin down reported sections of a video
- added predefined report reasons
- added video player with report modal
2020-06-22 13:00:39 +02:00
Chocobozzz 5fb2e2888c First implem global search 2020-06-10 14:02:41 +02:00
Chocobozzz d2a5c4e150
Fix broken migration introduced in 2.2.0-rc.1 2020-05-25 08:59:59 +02:00
Rigel Kent 2158ac9034
Add server plugin filter hooks for import with torrent and url (#2621)
* Add server plugin filter hooks for import with torrent and url

* WIP: pre and post-import filter hooks

* Rebased

* Cleanup filters to accept imports

Co-authored-by: Chocobozzz <me@florianbigard.com>
2020-05-14 11:10:26 +02:00
Chocobozzz 2ca154da93
Fix auto follow index URL 2020-05-12 09:37:39 +02:00
Chocobozzz 3cc665f48f
Add last login date to users 2020-05-07 10:39:09 +02:00
Chocobozzz 298b3fd315
Increase video comments limit 2020-05-05 16:48:30 +02:00
Chocobozzz 4c576aca1c
Fix playlist description length 2020-05-05 16:34:50 +02:00
Rigel Kent 5140edafeb
Fix plugin-auth migration 2020-05-05 13:52:10 +02:00
Chocobozzz 9107d791e2 Add external login tests 2020-05-04 16:21:39 +02:00
Rigel Kent 68d19a0ace Make sure a report doesn't get deleted upon the deletion of its video 2020-05-01 16:41:02 +02:00
Chocobozzz 2fd59d7d89
Add ability to sort by originallyPublishedAt 2020-04-20 14:05:52 +02:00
Chocobozzz d694b19344
Increase fetcher job ttl 2020-04-11 09:07:15 +02:00
Chocobozzz 4d9ae8f7cf
Increase max page value 2020-04-10 09:46:01 +02:00
Chocobozzz 764a965778
Implement pagination for overviews endpoint 2020-03-11 15:02:20 +01:00
Rigel Kent 8319d6ae72
Add video file metadata to download modal, via ffprobe (#2411)
* Add video file metadata via ffprobe

* Federate video file metadata

* Add tests for file metadata generation

* Complete tests for videoFile metadata federation

* Lint migration and video-file for metadata

* Objectify metadata from getter in ffmpeg-utils

* Add metadataUrl to all videoFiles

* Simplify metadata API middleware

* Load playlist in videoFile when requesting metadata
2020-03-10 14:39:40 +01:00
John Livingston 45f1bd72a0
Creating a user with an empty password will send an email to let him set his password (#2479)
* Creating a user with an empty password will send an email to let him set his password

* Consideration of Chocobozzz's comments

* Tips for optional password

* API documentation

* Fix circular imports

* Tests
2020-02-17 10:16:52 +01:00
Chocobozzz c6de3a85ee
Fix remote avatar without AP mediatype field 2020-02-12 10:23:24 +01:00
Chocobozzz 338eb9d33a
Merge branch 'release/2.1.0' into develop 2020-02-07 09:10:59 +01:00
Chocobozzz 26469f9ed8
Correctly fix octet stream fallback for video ext 2020-02-07 09:02:18 +01:00
Chocobozzz 99d362de0c
Support wma and wav audio files 2020-02-07 09:02:04 +01:00
Chocobozzz bc053fb303
Remove octet stream fallback for video extension
It could be a FLAC file for example
2020-02-06 17:22:32 +01:00
Chocobozzz cef4e38f1c
Remove octet stream fallback for video extension
It could be a FLAC file for example
2020-02-06 17:07:29 +01:00
Chocobozzz 4fdeef8125
Fix language list build 2020-02-06 17:06:11 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz ca6d36227a
Add url field in caption and use it for thumbnails 2020-01-30 11:53:38 +01:00
Rigel Kent 06bcfbd9f7 Downsample to the closest divisor standard framerate 2020-01-29 13:42:15 +01:00
Chocobozzz b764380ac2 Add ability to list redundancies 2020-01-28 11:35:26 +01:00
Chocobozzz ab4d4db44a
Avoid follow SQL conflicts 2020-01-10 13:34:20 +01:00
Chocobozzz 6fd8c4da26
Cleanup DB indexes 2020-01-10 13:33:22 +01:00
Chocobozzz e0b56b7495
Return an error on invalid count pagination 2020-01-09 09:36:31 +01:00
Chocobozzz 35f28e94c7
Add infohash cache 2020-01-03 13:47:45 +01:00
Rigel Kent c1125bcadc
Add comment filtering by reply count 2019-12-27 17:02:44 +01:00
Nassim Bounouas f88ee4a952 Feature/password reset link expiration (#2305)
* #1928 Add a sentence indicating the reset time limit on form and email

* #1928 Customizable password reset lifetime

* #1928 Add a route to verify reset link and call it on reset form init

* Revert "#1928 Customizable password reset lifetime"

This reverts commit 0ed97453f8.

* #1928 Reset password lifetime hardcoded to one hour

* Remove useless modifications for #1928
2019-12-18 09:59:00 +01:00
Chocobozzz 2735a154dc
Fix video import with long thumbnail url 2019-12-17 09:55:33 +01:00
Rigel Kent bee29df8a9 autoplay next video support for playlists 2019-12-13 09:13:43 +01:00
Chocobozzz 22a73cb879
Add internal privacy mode 2019-12-12 16:51:59 +01:00
Chocobozzz 566c125d6e
Serve audit logs to client 2019-12-11 14:14:01 +01:00
Chocobozzz c37a52b046
Support m4v videos 2019-12-05 11:07:57 +01:00
Chocobozzz fee47735bd
Fix migrations 2019-12-05 08:38:32 +01:00
Chocobozzz a3ce4ae847
Limit channel numbers
We can't load too much channels in selects and it helps to prevent actor
name squatting
2019-11-29 16:35:27 +01:00
Chocobozzz cb5ce4cb13
Add more filters to admin follows table 2019-11-29 11:16:43 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Chocobozzz f67d757452
Increase clock skew for HTTP signatures 2019-10-25 14:54:32 +02:00
Chocobozzz 47581df073
Fix federation with some actors
That don't have a shared inbox, or a URL
2019-10-23 11:33:53 +02:00
Chocobozzz fff2183df6
Fix audio uploading without preview 2019-10-18 12:03:32 +02:00
Chocobozzz 2f26030dd2
Support text/plain caption files 2019-10-18 10:36:32 +02:00
Chocobozzz 85c1df6ae8
Add auto play next video migration 2019-09-24 10:35:36 +02:00
Chocobozzz 5faed16653
Fix migrations 2019-09-06 08:59:02 +02:00
Chocobozzz e1b49ee534 Implement auto follow in client 2019-09-04 16:24:58 +02:00
Chocobozzz 6f1b4fa417 Add auto follow instances index support 2019-09-04 16:24:58 +02:00
Chocobozzz 0b5c385b45
Handle reports from mastodon 2019-08-30 09:40:32 +02:00
Chocobozzz 557b13ae24
Lazy load avatars 2019-08-09 11:32:40 +02:00
Chocobozzz 65af03a241
Automatically update playlist thumbnails 2019-08-01 16:54:24 +02:00
Chocobozzz bfbd912886 Fix broken playlist api 2019-08-01 09:11:04 +02:00
Chocobozzz cd4cb177e6
Improve static files cache 2019-07-29 15:20:36 +02:00
Chocobozzz aa3796bda5
Sort on videoQuotaUsed in backend 2019-07-25 11:15:03 +02:00
Chocobozzz 587568e1cc Add plugin table migration table 2019-07-24 10:58:16 +02:00
Chocobozzz 9fa6ca160a Some plugins fixes and doc enhancements 2019-07-24 10:58:16 +02:00
Chocobozzz 66170ca8c6 Use process.env.npm_package_version 2019-07-24 10:58:16 +02:00
Chocobozzz d133f38582 Check latest plugins version with config interval 2019-07-24 10:58:16 +02:00
Chocobozzz e0ce715a1d Check latest plugins version 2019-07-24 10:58:16 +02:00
Chocobozzz 6702a1b2cc Add ability to search available plugins 2019-07-24 10:58:16 +02:00
Chocobozzz 503c6f440a Add peertube plugin index website models 2019-07-24 10:58:16 +02:00
Chocobozzz ad91e7006e WIP plugins: plugin settings on server side 2019-07-24 10:58:16 +02:00
Chocobozzz ffb321bedc WIP plugins: load theme on client side 2019-07-24 10:58:16 +02:00
Chocobozzz 7cd4d2ba10 WIP plugins: add theme support 2019-07-24 10:58:16 +02:00