Commit Graph

267 Commits

Author SHA1 Message Date
Chocobozzz 6939cbac48 Safely remove webtorrent files
Only remove them on max quality HLS playlist generation
2021-01-25 14:38:52 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
2021-01-25 14:38:52 +01:00
Chocobozzz 24516aa26a
Refactor transcoding job handlers 2021-01-21 15:58:17 +01:00
Chocobozzz 3b01f4c0ac
Support progress for ffmpeg tasks 2021-01-21 14:42:43 +01:00
Chocobozzz 805b8619c1
Fix (again) youtube import 2021-01-19 16:36:21 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz f88453e233
Fix build 2020-12-22 16:49:17 +01:00
Chocobozzz 236841a1d7
Force HLS transcoding
Fix weird behaviour with some web browsers. Don't really know if it's a
ffmpeg bug, a safari bug or a peertube bug, but forcing transcoding
seems to fix this playback bug
2020-12-22 16:04:42 +01:00
Chocobozzz 402145b863
Refactor jobs state 2020-12-14 12:00:35 +01:00
Rigel Kent 040d6896a3
add display of logs matching any state 2020-12-13 19:27:25 +01:00
Chocobozzz a800dbf345
Live views update 2020-12-09 16:31:42 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Chocobozzz 5b9b403a20
Fix replay last segments loop 2020-12-07 16:07:00 +01:00
Chocobozzz aa5ee5017a
Fix lint 2020-12-04 15:58:29 +01:00
Chocobozzz e772bdf14c
Load video in permanent live after last one ended 2020-12-04 15:38:32 +01:00
Chocobozzz 3851e732c4
Fix audio issues with live replay 2020-12-04 15:10:13 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Chocobozzz 4a54a93941
Fix live invalid save replay duration 2020-12-03 10:05:10 +01:00
Chocobozzz 2650d6d489
Fix live replay duration glitch 2020-12-02 10:18:15 +01:00
Chocobozzz 8c666c44ab
Fix live saving after a server restart 2020-11-30 17:03:13 +01:00
Chocobozzz 937581b8f6
Fix high CPU with long live when save replay is true 2020-11-30 15:59:22 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Chocobozzz de94ac86a2
Fix incorrect IDs in AP federation 2020-11-20 11:21:08 +01:00
Chocobozzz e4bf785617 Handle views for live videos 2020-11-09 15:33:04 +01:00
Chocobozzz 9e2b2e76ba Don't add live videos in redundancy 2020-11-09 15:33:04 +01:00
Chocobozzz 053aed43fb Regenerate miniature on live save 2020-11-09 15:33:04 +01:00
Chocobozzz 3bc68dfd61 Fix audio sync after saving replay
hls.js seems to not correctly handle audio gaps with fragmented mp4
(but can with a ts playlist)
2020-11-09 15:33:04 +01:00
Chocobozzz 68e70a745b Add save replay live tests 2020-11-09 15:33:04 +01:00
Chocobozzz 97969c4edf Add check constraints live tests 2020-11-09 15:33:04 +01:00
Chocobozzz d846d99c6c Add modal to display live information 2020-11-09 15:33:04 +01:00
Chocobozzz 31c82cd914 Fix replay saving 2020-11-09 15:33:04 +01:00
Chocobozzz b5b687550d Add ability to save live replay 2020-11-09 15:33:04 +01:00
Chocobozzz fb7194043d Check live duration and size 2020-11-09 15:33:04 +01:00
Chocobozzz a5cf76afa3 Add watch messages if live has not started 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Chocobozzz dca0fe12ec
Fix resolution for portrait videos 2020-08-03 16:03:52 +02:00
Chocobozzz 26d6bf6533
Split types and typings 2020-06-18 10:46:27 +02:00
Chocobozzz fd261a8de9
Fix video view filename 2020-06-15 15:18:54 +02:00
Rigel Kent 2158ac9034
Add server plugin filter hooks for import with torrent and url (#2621)
* Add server plugin filter hooks for import with torrent and url

* WIP: pre and post-import filter hooks

* Rebased

* Cleanup filters to accept imports

Co-authored-by: Chocobozzz <me@florianbigard.com>
2020-05-14 11:10:26 +02:00
Chocobozzz 80fdaf0645
Add moderation helpers to plugins 2020-05-07 15:07:01 +02:00
Chocobozzz 1c32067367
Fix HLS audio only transcoding 2020-05-05 16:27:46 +02:00
Chocobozzz e307e4fce3 Add ability for auth plugins to hook tokens validity 2020-05-04 16:21:39 +02:00
Chocobozzz e1c5503114 Support logout and add id and pass tests 2020-05-04 16:21:39 +02:00
Chocobozzz 8dc8a34ee8 Avoir some circular dependencies 2020-05-04 16:21:39 +02:00
Kim b1770a0af4
Add thumbnail / preview generation from url on the fly (#2646)
* Add thumbnails generation on the fly to URL import

* Display generated preview to import first edit

* Use ternary to get type inference

* Move preview/thumbnail test just after import

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-04-20 10:28:38 +02:00
Chocobozzz 0dd57e4d43
Fix tests 2020-04-20 09:04:38 +02:00
Chocobozzz 2c0ccd4b3f
Do not support subscriptions to accounts 2020-04-16 16:37:25 +02:00
Chocobozzz d57d1d83c6
Support audio files import 2020-04-03 15:41:39 +02:00
Chocobozzz 84ebcf345c
Fix federation with pleroma 2020-03-12 14:14:11 +01:00
Chocobozzz 598edb8af1
Reduce AP context size on specific activities 2020-02-03 11:31:23 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz ca6d36227a
Add url field in caption and use it for thumbnails 2020-01-30 11:53:38 +01:00
Chocobozzz 0374b6b5cd
Cleanup server fixme 2020-01-28 14:45:17 +01:00
Chocobozzz b764380ac2 Add ability to list redundancies 2020-01-28 11:35:26 +01:00
Chocobozzz 99849607a6
Remove debug messages 2020-01-24 11:38:35 +01:00
Chocobozzz e92269053e
Update http signature 2020-01-07 15:56:09 +01:00
Chocobozzz 1061c73fde
Add ability to filter per job type 2019-12-04 14:49:59 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Lukas Winkler 32d7f2b754 fix a few typos (#2141)
* fix a few typos

* apply changes to original files instead

* additional correction
2019-09-23 08:17:42 +02:00
Chocobozzz 8424c4026a Add auto follow back support for instances 2019-09-04 16:24:58 +02:00
Chocobozzz 0283eaac2a
Cleanup model types 2019-08-20 14:53:42 +02:00
Chocobozzz 453e83ea5d
Stronger model typings 2019-08-19 17:26:35 +02:00
Chocobozzz 6b9c966f64
Automatically remove bad followings 2019-08-06 17:26:51 +02:00
Chocobozzz 1198edf4bb
Fix user notifications on new follow 2019-08-02 10:53:36 +02:00
Chocobozzz 2284f20207
Add gitlab ci support 2019-07-29 13:40:39 +02:00
Chocobozzz 5b77537ce5 Correctly notify on auto blacklist 2019-07-24 10:58:16 +02:00
Chocobozzz 21d141c296
Merge branch 'release/v1.3.0' into develop 2019-05-21 13:32:00 +02:00
Chocobozzz 63dc589865
Fix video views 2019-05-21 13:14:27 +02:00
Chocobozzz 536598cfaf
Add audio support in upload 2019-05-16 17:16:47 +02:00
Chocobozzz c28bcdd10a
Fix pleroma follow 2019-04-25 15:20:30 +02:00
Chocobozzz 47f6409bb8
Use apicache instead of our broken implementation 2019-04-25 11:27:13 +02:00
Chocobozzz 3acc508440
Upgrade sequelize 2019-04-24 16:26:21 +02:00
Chocobozzz e8bafea35b
Create a dedicated table to track video thumbnails 2019-04-24 16:25:52 +02:00
Chocobozzz 74dc3bca2b
Don't expose constants directly in initializers/ 2019-04-11 14:26:41 +02:00
Chocobozzz 6dd9de95df
Move config in its own file 2019-04-11 13:45:39 +02:00
Chocobozzz 883993c81e
Add notification on new instance follower (server side) 2019-04-08 17:30:48 +02:00
Josh Morel 7ccddd7b52 add quarantine videos feature (#1637)
* add quarantine videos feature

* increase Notification settings test timeout

to 20000ms. was completing 7000 locally but timing out
after 10000 on travis

* fix quarantine video test issues

-propagate misspelling
-remove skip from server/tests/client.ts

* WIP use blacklist for moderator video approval

instead of video.quarantine boolean

* finish auto-blacklist feature
2019-04-02 11:26:47 +02:00
Chocobozzz 308421283a
Move video file import in its own file 2019-03-19 17:10:53 +01:00
Chocobozzz a0327eedb0
Rename video-file job to video-transcoding 2019-03-19 17:00:08 +01:00
Chocobozzz 2ba9287131
Cleanup invalid rates/comments/shares 2019-03-19 16:23:02 +01:00
Chocobozzz d74d29ad9e
Limit user tokens cache 2019-03-19 14:30:43 +01:00
Chocobozzz 9f79ade627
Refresh playlists 2019-03-19 14:13:53 +01:00
Chocobozzz 418d092afa Playlist server API 2019-03-18 11:17:59 +01:00
Chocobozzz 374c1db98c
Upgrade server dependencies 2019-02-21 14:22:39 +01:00
Chocobozzz dee77e767f
Create send email type
Instead of using email job handler payload type
2019-02-20 15:54:32 +01:00
Chocobozzz 4759fedc61
Fix from header in contact form 2019-02-15 09:43:38 +01:00
Chocobozzz 597a9266d4 Add player mode in watch/embed urls 2019-02-11 09:13:02 +01:00
Chocobozzz 0920929696 Add hls support on server 2019-02-11 09:13:02 +01:00
Josh Morel f7effe8dc7 don't notify prior to scheduled update
also increase timeouts on user-notification test
2019-02-07 14:49:28 +01:00
Chocobozzz 744d0eca19
Refresh remote actors on GET enpoints 2019-01-14 11:30:15 +01:00
Chocobozzz a4101923e6
Implement contact form on server side 2019-01-10 11:32:37 +01:00
Chocobozzz f7cc67b455 Add new follow, mention and user registered notifs 2019-01-09 11:15:15 +01:00
Chocobozzz dc13348070 Add import finished and video published notifs 2019-01-09 11:15:15 +01:00
Chocobozzz e8d246d526 Add notification settings migration 2019-01-09 11:15:15 +01:00
Chocobozzz cef534ed53 Add user notification base code 2019-01-09 11:15:15 +01:00
Chocobozzz 2f5c6b2fc6
Optimize actor follow scores modifications 2018-12-20 14:31:11 +01:00
Chocobozzz 56b13bd193
Fix federation of some videos
If we don't transcode additional resolutions, and user decided to wait
transcoding before publishing the video
2018-12-19 13:41:47 +01:00
Chocobozzz f481c4f9f3
Use move instead rename
To avoid EXDEV errors
2018-12-11 15:12:38 +01:00
Chocobozzz 9f8ca79284
Don't quit on queue error 2018-12-11 09:18:01 +01:00
Chocobozzz 6040f87d14
Add tmp and redundancy directories 2018-12-04 16:04:15 +01:00
Chocobozzz 745778256c
Fix thumbnail processing 2018-12-04 16:04:14 +01:00
Chocobozzz f9a971c671
Update dependencies 2018-12-04 10:08:45 +01:00
Chocobozzz dbe6aa698e
Fix trending page 2018-12-03 09:14:56 +01:00
Chocobozzz 04b8c3fba6
Delete invalid or deleted remote videos 2018-11-20 10:44:48 +01:00
Chocobozzz 58d515e32f
Fix images size when downloading them 2018-11-16 16:48:17 +01:00
Chocobozzz 6385c0cb7f
Fix embed video id parsing 2018-11-15 16:57:59 +01:00
Chocobozzz 030177d246
Don't forward view, send updates instead
To avoid inconsistencies in the federation, now the origin server will
tell other instances what is the correct number of views
2018-11-15 16:38:37 +01:00
Chocobozzz 5c6d985fae
Check activities host 2018-11-14 16:32:27 +01:00
Chocobozzz df66d81583
Add compatibility with other Linked Signature algorithms 2018-11-14 16:32:27 +01:00
Chocobozzz 41f2ebae4f
Add HTTP signature check before linked signature
It's faster, and will allow us to use RSA signature 2018 (with upstream
jsonld-signature module) without too much incompatibilities in the
peertube federation
2018-10-19 12:26:37 +02:00
Chocobozzz 71e318b4fe
Merge branch 'release/v1.0.0' into develop 2018-10-11 09:52:16 +02:00
Chocobozzz 729bb18481
Add more headers to broadcast/unicast 2018-10-10 08:51:58 +02:00
Felix Ableitner edb4ffc7e0 Set bitrate limits for transcoding (fixes #638) (#1135)
* Set bitrate limits for transcoding (fixes #638)

* added optimization script and test, changed stuff

* fix test, improve docs

* re-add optimize-old-videos script

* added documentation

* Don't optimize videos without valid UUID, or redundancy videos

* move getUUIDFromFilename

* fix tests?

* update torrent and file size, some more fixes/improvements

* use higher bitrate for high fps video, adjust bitrates

* add test video

* don't throw error if resolution is undefined

* generate test fixture on the fly

* use random noise video for bitrate test, add promise

* shorten test video to avoid timeout

* use existing function to optimize video

* various fixes

* increase test timeout

* limit test fixture size, add link

* test fixes

* add await

* more test fixes, add -b:v parameter

* replace ffmpeg wiki link

* fix ffmpeg params

* fix unit test

* add test fixture to .gitgnore

* add video transcoding fps model

* add missing file
2018-10-08 16:26:04 +02:00
Chocobozzz cf9166cf2f
Add timeout on youtube dl to cleaup files 2018-10-01 12:00:05 +02:00
Chocobozzz e95e0463d8
Correctly delete files on failed import 2018-10-01 10:44:16 +02:00
Chocobozzz 4157cdb137
Refractor videos AP functions 2018-09-19 11:16:23 +02:00
Chocobozzz 627621c1e8
Optimize SQL requests of watch page API endpoints 2018-09-19 09:54:37 +02:00
Chocobozzz 098eb37797
Reduce video.ts file size by moving some methods in other files 2018-09-19 09:54:35 +02:00
Chocobozzz 6f0c46be8c
Process last hour views instead of current hour 2018-09-13 14:14:15 +02:00
Chocobozzz 9431cabf8a
Fix video views increment 2018-09-13 10:13:25 +02:00
Chocobozzz 6b6168606b
Bufferize videos views in redis 2018-08-30 15:03:18 +02:00
Chocobozzz 62689b942b
Correctly migrate to fs-extra 2018-08-27 16:23:34 +02:00
Chocobozzz f6eebcb336 Add ability to search a video with an URL 2018-08-27 09:41:54 +02:00
Chocobozzz 22a16e36f6 Add local user subscriptions 2018-08-27 09:41:54 +02:00
Chocobozzz 06a05d5f47 Add subscriptions endpoints to REST API 2018-08-27 09:41:54 +02:00
Chocobozzz 06215f15e0
Cleanup utils helper 2018-08-14 15:28:30 +02:00
Chocobozzz 541006e355 Correct webtorrent download cleanup 2018-08-08 09:30:31 +02:00
Chocobozzz 3e17515e29 Add torrent tests 2018-08-08 09:30:31 +02:00
Chocobozzz a84b8fa5cf Add import.video.torrent configuration 2018-08-08 09:30:31 +02:00
Chocobozzz 990b6a0b0c Import torrents with webtorrent 2018-08-08 09:30:31 +02:00
Chocobozzz ce33919c24 Import magnets with webtorrent 2018-08-08 09:30:31 +02:00
Chocobozzz 590fb50690 Add tests regarding video import 2018-08-06 11:19:16 +02:00
Chocobozzz 2b86fe7274 Add job ttl 2018-08-06 11:19:16 +02:00
Chocobozzz 516df59b3b Remove ability to delete video imports
Users should remove the linked video instead
2018-08-06 11:19:16 +02:00
Chocobozzz d7f83948a1 Correctly handle video import errors 2018-08-06 11:19:16 +02:00
Chocobozzz ed31c05985 Add ability to list video imports 2018-08-06 11:19:16 +02:00
Chocobozzz fbad87b047 Add ability to import video with youtube-dl 2018-08-06 11:19:16 +02:00
Chocobozzz 6104adc3e9
Don't send follow request if the follow is already accepted 2018-07-31 11:15:59 +02:00
Chocobozzz 4a9e71c2b1
Increase max stalled count in job queue 2018-07-30 19:18:01 +02:00
Chocobozzz 14f2b3ad11
Graceful job queue shutdown 2018-07-30 18:49:54 +02:00
Rigel Kent c1e791bad0
expliciting type checks and predicates (server only) 2018-07-25 22:01:25 +02:00
Rigel Kent 19f7b248d8 adding redis unix connection 2018-07-14 15:00:56 +02:00
Chocobozzz 94831479f5 Migrate to bull 2018-07-11 14:00:17 +02:00
Chocobozzz 1b952dd426
Reset published date on publish 2018-06-28 16:58:15 +02:00
Chocobozzz 90d4bb8125
Refractor retry transaction function 2018-06-13 14:27:40 +02:00
Chocobozzz 2186386cca
Add concept of video state, and add ability to wait transcoding before
publishing a video
2018-06-12 20:37:51 +02:00
Chocobozzz 8eb8bc201e
Pick average video file insead of max quality 2018-06-07 17:57:11 +02:00
Chocobozzz 28be89161a
Improve create import file job
Fix federation of .ogv videos
2018-06-07 09:43:18 +02:00
Florent Fayolle 0138af9237 Add create-import-video-file-job command 2018-06-07 08:57:48 +02:00
Chocobozzz 0c948c1659
Add ability to manually run transcoding job 2018-05-30 10:49:56 +02:00