Commit Graph

267 Commits

Author SHA1 Message Date
Chocobozzz a6e37eebfb
Fix transcoding job priority
New resolution jobs are also important if waiting for transcoding is
enabled since we publish the video after the first resolution generation
2021-05-05 09:32:20 +02:00
Chocobozzz e7053b1d9d
Fix AP security tests 2021-03-24 18:18:40 +01:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 67f87b66fe
Refactor a little bit AP validators 2021-03-24 18:18:40 +01:00
Chocobozzz dcd75f786c
Rename request timeout 2021-03-03 10:10:55 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 8795d6f254
Fix broken local actors
Some channels can't federate because they don't have public/private
keys, maybe because the generation failed for various reasons
2021-02-26 14:22:25 +01:00
Chocobozzz 70243d7a35
Correctly wait transcoding before federating 2021-02-22 10:35:37 +01:00
Chocobozzz 8efc27bf14 Cleanup 2021-02-18 13:38:09 +01:00
Chocobozzz 2451916e45 Add video files migration 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz a35a22797c Remove previous thumbnail if needed 2021-02-16 10:36:44 +01:00
Chocobozzz e3b4c084cd
Guess if we need to generate the thumbnail for imports 2021-02-12 09:37:01 +01:00
Chocobozzz 44d1f7f2e8
Painfully debug concurrent import jobs 2021-02-09 11:46:30 +01:00
Chocobozzz 9129b7694d Allow to specify transcoding and import jobs concurrency 2021-02-08 15:38:45 +01:00
Chocobozzz 69eddafb17
Fix HLS generation after import script 2021-02-02 14:00:46 +01:00
Chocobozzz 40930fda86
Transcode audio uploads to lower resolutions
Better consistency
2021-02-02 09:45:42 +01:00
Chocobozzz 7e3592d70b
Correctly remove torrents with HLS only 2021-02-01 11:23:12 +01:00
Chocobozzz 6939cbac48 Safely remove webtorrent files
Only remove them on max quality HLS playlist generation
2021-01-25 14:38:52 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
2021-01-25 14:38:52 +01:00
Chocobozzz 24516aa26a
Refactor transcoding job handlers 2021-01-21 15:58:17 +01:00
Chocobozzz 3b01f4c0ac
Support progress for ffmpeg tasks 2021-01-21 14:42:43 +01:00
Chocobozzz 805b8619c1
Fix (again) youtube import 2021-01-19 16:36:21 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz f88453e233
Fix build 2020-12-22 16:49:17 +01:00
Chocobozzz 236841a1d7
Force HLS transcoding
Fix weird behaviour with some web browsers. Don't really know if it's a
ffmpeg bug, a safari bug or a peertube bug, but forcing transcoding
seems to fix this playback bug
2020-12-22 16:04:42 +01:00
Chocobozzz a800dbf345
Live views update 2020-12-09 16:31:42 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Chocobozzz 5b9b403a20
Fix replay last segments loop 2020-12-07 16:07:00 +01:00
Chocobozzz aa5ee5017a
Fix lint 2020-12-04 15:58:29 +01:00
Chocobozzz e772bdf14c
Load video in permanent live after last one ended 2020-12-04 15:38:32 +01:00
Chocobozzz 3851e732c4
Fix audio issues with live replay 2020-12-04 15:10:13 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Chocobozzz 4a54a93941
Fix live invalid save replay duration 2020-12-03 10:05:10 +01:00
Chocobozzz 2650d6d489
Fix live replay duration glitch 2020-12-02 10:18:15 +01:00
Chocobozzz 8c666c44ab
Fix live saving after a server restart 2020-11-30 17:03:13 +01:00
Chocobozzz 937581b8f6
Fix high CPU with long live when save replay is true 2020-11-30 15:59:22 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Chocobozzz de94ac86a2
Fix incorrect IDs in AP federation 2020-11-20 11:21:08 +01:00
Chocobozzz e4bf785617 Handle views for live videos 2020-11-09 15:33:04 +01:00
Chocobozzz 9e2b2e76ba Don't add live videos in redundancy 2020-11-09 15:33:04 +01:00
Chocobozzz 053aed43fb Regenerate miniature on live save 2020-11-09 15:33:04 +01:00
Chocobozzz 3bc68dfd61 Fix audio sync after saving replay
hls.js seems to not correctly handle audio gaps with fragmented mp4
(but can with a ts playlist)
2020-11-09 15:33:04 +01:00
Chocobozzz 68e70a745b Add save replay live tests 2020-11-09 15:33:04 +01:00
Chocobozzz 97969c4edf Add check constraints live tests 2020-11-09 15:33:04 +01:00
Chocobozzz d846d99c6c Add modal to display live information 2020-11-09 15:33:04 +01:00
Chocobozzz 31c82cd914 Fix replay saving 2020-11-09 15:33:04 +01:00
Chocobozzz b5b687550d Add ability to save live replay 2020-11-09 15:33:04 +01:00
Chocobozzz fb7194043d Check live duration and size 2020-11-09 15:33:04 +01:00
Chocobozzz a5cf76afa3 Add watch messages if live has not started 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Chocobozzz dca0fe12ec
Fix resolution for portrait videos 2020-08-03 16:03:52 +02:00
Chocobozzz 26d6bf6533
Split types and typings 2020-06-18 10:46:27 +02:00
Chocobozzz fd261a8de9
Fix video view filename 2020-06-15 15:18:54 +02:00
Rigel Kent 2158ac9034
Add server plugin filter hooks for import with torrent and url (#2621)
* Add server plugin filter hooks for import with torrent and url

* WIP: pre and post-import filter hooks

* Rebased

* Cleanup filters to accept imports

Co-authored-by: Chocobozzz <me@florianbigard.com>
2020-05-14 11:10:26 +02:00
Chocobozzz 80fdaf0645
Add moderation helpers to plugins 2020-05-07 15:07:01 +02:00
Chocobozzz 1c32067367
Fix HLS audio only transcoding 2020-05-05 16:27:46 +02:00
Chocobozzz e307e4fce3 Add ability for auth plugins to hook tokens validity 2020-05-04 16:21:39 +02:00
Chocobozzz 8dc8a34ee8 Avoir some circular dependencies 2020-05-04 16:21:39 +02:00
Kim b1770a0af4
Add thumbnail / preview generation from url on the fly (#2646)
* Add thumbnails generation on the fly to URL import

* Display generated preview to import first edit

* Use ternary to get type inference

* Move preview/thumbnail test just after import

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-04-20 10:28:38 +02:00
Chocobozzz 0dd57e4d43
Fix tests 2020-04-20 09:04:38 +02:00
Chocobozzz 2c0ccd4b3f
Do not support subscriptions to accounts 2020-04-16 16:37:25 +02:00
Chocobozzz d57d1d83c6
Support audio files import 2020-04-03 15:41:39 +02:00
Chocobozzz 84ebcf345c
Fix federation with pleroma 2020-03-12 14:14:11 +01:00
Chocobozzz 598edb8af1
Reduce AP context size on specific activities 2020-02-03 11:31:23 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz ca6d36227a
Add url field in caption and use it for thumbnails 2020-01-30 11:53:38 +01:00
Chocobozzz b764380ac2 Add ability to list redundancies 2020-01-28 11:35:26 +01:00
Chocobozzz 99849607a6
Remove debug messages 2020-01-24 11:38:35 +01:00
Chocobozzz e92269053e
Update http signature 2020-01-07 15:56:09 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Lukas Winkler 32d7f2b754 fix a few typos (#2141)
* fix a few typos

* apply changes to original files instead

* additional correction
2019-09-23 08:17:42 +02:00
Chocobozzz 8424c4026a Add auto follow back support for instances 2019-09-04 16:24:58 +02:00
Chocobozzz 0283eaac2a
Cleanup model types 2019-08-20 14:53:42 +02:00
Chocobozzz 453e83ea5d
Stronger model typings 2019-08-19 17:26:35 +02:00
Chocobozzz 6b9c966f64
Automatically remove bad followings 2019-08-06 17:26:51 +02:00
Chocobozzz 1198edf4bb
Fix user notifications on new follow 2019-08-02 10:53:36 +02:00
Chocobozzz 2284f20207
Add gitlab ci support 2019-07-29 13:40:39 +02:00
Chocobozzz 5b77537ce5 Correctly notify on auto blacklist 2019-07-24 10:58:16 +02:00
Chocobozzz 21d141c296
Merge branch 'release/v1.3.0' into develop 2019-05-21 13:32:00 +02:00
Chocobozzz 63dc589865
Fix video views 2019-05-21 13:14:27 +02:00
Chocobozzz 536598cfaf
Add audio support in upload 2019-05-16 17:16:47 +02:00
Chocobozzz c28bcdd10a
Fix pleroma follow 2019-04-25 15:20:30 +02:00
Chocobozzz 3acc508440
Upgrade sequelize 2019-04-24 16:26:21 +02:00
Chocobozzz e8bafea35b
Create a dedicated table to track video thumbnails 2019-04-24 16:25:52 +02:00
Chocobozzz 74dc3bca2b
Don't expose constants directly in initializers/ 2019-04-11 14:26:41 +02:00
Chocobozzz 6dd9de95df
Move config in its own file 2019-04-11 13:45:39 +02:00
Chocobozzz 883993c81e
Add notification on new instance follower (server side) 2019-04-08 17:30:48 +02:00
Josh Morel 7ccddd7b52 add quarantine videos feature (#1637)
* add quarantine videos feature

* increase Notification settings test timeout

to 20000ms. was completing 7000 locally but timing out
after 10000 on travis

* fix quarantine video test issues

-propagate misspelling
-remove skip from server/tests/client.ts

* WIP use blacklist for moderator video approval

instead of video.quarantine boolean

* finish auto-blacklist feature
2019-04-02 11:26:47 +02:00
Chocobozzz 308421283a
Move video file import in its own file 2019-03-19 17:10:53 +01:00
Chocobozzz a0327eedb0
Rename video-file job to video-transcoding 2019-03-19 17:00:08 +01:00
Chocobozzz 2ba9287131
Cleanup invalid rates/comments/shares 2019-03-19 16:23:02 +01:00
Chocobozzz d74d29ad9e
Limit user tokens cache 2019-03-19 14:30:43 +01:00
Chocobozzz 9f79ade627
Refresh playlists 2019-03-19 14:13:53 +01:00
Chocobozzz 418d092afa Playlist server API 2019-03-18 11:17:59 +01:00
Chocobozzz 374c1db98c
Upgrade server dependencies 2019-02-21 14:22:39 +01:00
Chocobozzz dee77e767f
Create send email type
Instead of using email job handler payload type
2019-02-20 15:54:32 +01:00
Chocobozzz 4759fedc61
Fix from header in contact form 2019-02-15 09:43:38 +01:00
Chocobozzz 597a9266d4 Add player mode in watch/embed urls 2019-02-11 09:13:02 +01:00
Chocobozzz 0920929696 Add hls support on server 2019-02-11 09:13:02 +01:00
Josh Morel f7effe8dc7 don't notify prior to scheduled update
also increase timeouts on user-notification test
2019-02-07 14:49:28 +01:00
Chocobozzz 744d0eca19
Refresh remote actors on GET enpoints 2019-01-14 11:30:15 +01:00
Chocobozzz a4101923e6
Implement contact form on server side 2019-01-10 11:32:37 +01:00
Chocobozzz f7cc67b455 Add new follow, mention and user registered notifs 2019-01-09 11:15:15 +01:00
Chocobozzz dc13348070 Add import finished and video published notifs 2019-01-09 11:15:15 +01:00
Chocobozzz e8d246d526 Add notification settings migration 2019-01-09 11:15:15 +01:00
Chocobozzz cef534ed53 Add user notification base code 2019-01-09 11:15:15 +01:00
Chocobozzz 2f5c6b2fc6
Optimize actor follow scores modifications 2018-12-20 14:31:11 +01:00
Chocobozzz 56b13bd193
Fix federation of some videos
If we don't transcode additional resolutions, and user decided to wait
transcoding before publishing the video
2018-12-19 13:41:47 +01:00
Chocobozzz f481c4f9f3
Use move instead rename
To avoid EXDEV errors
2018-12-11 15:12:38 +01:00
Chocobozzz 6040f87d14
Add tmp and redundancy directories 2018-12-04 16:04:15 +01:00
Chocobozzz 745778256c
Fix thumbnail processing 2018-12-04 16:04:14 +01:00
Chocobozzz f9a971c671
Update dependencies 2018-12-04 10:08:45 +01:00
Chocobozzz dbe6aa698e
Fix trending page 2018-12-03 09:14:56 +01:00
Chocobozzz 04b8c3fba6
Delete invalid or deleted remote videos 2018-11-20 10:44:48 +01:00
Chocobozzz 58d515e32f
Fix images size when downloading them 2018-11-16 16:48:17 +01:00
Chocobozzz 6385c0cb7f
Fix embed video id parsing 2018-11-15 16:57:59 +01:00
Chocobozzz 030177d246
Don't forward view, send updates instead
To avoid inconsistencies in the federation, now the origin server will
tell other instances what is the correct number of views
2018-11-15 16:38:37 +01:00
Chocobozzz 5c6d985fae
Check activities host 2018-11-14 16:32:27 +01:00
Chocobozzz df66d81583
Add compatibility with other Linked Signature algorithms 2018-11-14 16:32:27 +01:00
Chocobozzz 41f2ebae4f
Add HTTP signature check before linked signature
It's faster, and will allow us to use RSA signature 2018 (with upstream
jsonld-signature module) without too much incompatibilities in the
peertube federation
2018-10-19 12:26:37 +02:00
Chocobozzz 71e318b4fe
Merge branch 'release/v1.0.0' into develop 2018-10-11 09:52:16 +02:00
Chocobozzz 729bb18481
Add more headers to broadcast/unicast 2018-10-10 08:51:58 +02:00
Felix Ableitner edb4ffc7e0 Set bitrate limits for transcoding (fixes #638) (#1135)
* Set bitrate limits for transcoding (fixes #638)

* added optimization script and test, changed stuff

* fix test, improve docs

* re-add optimize-old-videos script

* added documentation

* Don't optimize videos without valid UUID, or redundancy videos

* move getUUIDFromFilename

* fix tests?

* update torrent and file size, some more fixes/improvements

* use higher bitrate for high fps video, adjust bitrates

* add test video

* don't throw error if resolution is undefined

* generate test fixture on the fly

* use random noise video for bitrate test, add promise

* shorten test video to avoid timeout

* use existing function to optimize video

* various fixes

* increase test timeout

* limit test fixture size, add link

* test fixes

* add await

* more test fixes, add -b:v parameter

* replace ffmpeg wiki link

* fix ffmpeg params

* fix unit test

* add test fixture to .gitgnore

* add video transcoding fps model

* add missing file
2018-10-08 16:26:04 +02:00
Chocobozzz cf9166cf2f
Add timeout on youtube dl to cleaup files 2018-10-01 12:00:05 +02:00
Chocobozzz e95e0463d8
Correctly delete files on failed import 2018-10-01 10:44:16 +02:00
Chocobozzz 4157cdb137
Refractor videos AP functions 2018-09-19 11:16:23 +02:00
Chocobozzz 627621c1e8
Optimize SQL requests of watch page API endpoints 2018-09-19 09:54:37 +02:00
Chocobozzz 098eb37797
Reduce video.ts file size by moving some methods in other files 2018-09-19 09:54:35 +02:00
Chocobozzz 6f0c46be8c
Process last hour views instead of current hour 2018-09-13 14:14:15 +02:00
Chocobozzz 9431cabf8a
Fix video views increment 2018-09-13 10:13:25 +02:00
Chocobozzz 6b6168606b
Bufferize videos views in redis 2018-08-30 15:03:18 +02:00
Chocobozzz 62689b942b
Correctly migrate to fs-extra 2018-08-27 16:23:34 +02:00
Chocobozzz f6eebcb336 Add ability to search a video with an URL 2018-08-27 09:41:54 +02:00
Chocobozzz 22a16e36f6 Add local user subscriptions 2018-08-27 09:41:54 +02:00
Chocobozzz 06a05d5f47 Add subscriptions endpoints to REST API 2018-08-27 09:41:54 +02:00
Chocobozzz 06215f15e0
Cleanup utils helper 2018-08-14 15:28:30 +02:00
Chocobozzz 541006e355 Correct webtorrent download cleanup 2018-08-08 09:30:31 +02:00
Chocobozzz 3e17515e29 Add torrent tests 2018-08-08 09:30:31 +02:00
Chocobozzz a84b8fa5cf Add import.video.torrent configuration 2018-08-08 09:30:31 +02:00
Chocobozzz 990b6a0b0c Import torrents with webtorrent 2018-08-08 09:30:31 +02:00
Chocobozzz ce33919c24 Import magnets with webtorrent 2018-08-08 09:30:31 +02:00
Chocobozzz 590fb50690 Add tests regarding video import 2018-08-06 11:19:16 +02:00
Chocobozzz 516df59b3b Remove ability to delete video imports
Users should remove the linked video instead
2018-08-06 11:19:16 +02:00
Chocobozzz d7f83948a1 Correctly handle video import errors 2018-08-06 11:19:16 +02:00
Chocobozzz ed31c05985 Add ability to list video imports 2018-08-06 11:19:16 +02:00
Chocobozzz fbad87b047 Add ability to import video with youtube-dl 2018-08-06 11:19:16 +02:00
Chocobozzz 6104adc3e9
Don't send follow request if the follow is already accepted 2018-07-31 11:15:59 +02:00
Rigel Kent c1e791bad0
expliciting type checks and predicates (server only) 2018-07-25 22:01:25 +02:00