0305db28c9
Add support for saving video files to object storage ( #4290 )
...
* Add support for saving video files to object storage
* Add support for custom url generation on s3 stored files
Uses two config keys to support url generation that doesn't directly go
to (compatible s3). Can be used to generate urls to any cache server or
CDN.
* Upload files to s3 concurrently and delete originals afterwards
* Only publish after move to object storage is complete
* Use base url instead of url template
* Fix mistyped config field
* Add rudenmentary way to download before transcode
* Implement Chocobozzz suggestions
https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478
The remarks in question:
Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names
Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket
Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET)
I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs)
https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer
I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs
* Import correct function
* Support multipart upload
* Remove import of node 15.0 module stream/promises
* Extend maximum upload job length
Using the same value as for redundancy downloading seems logical
* Use dynamic part size for really large uploads
Also adds very small part size for local testing
* Fix decreasePendingMove query
* Resolve various PR comments
* Move to object storage after optimize
* Make upload size configurable and increase default
* Prune webtorrent files that are stored in object storage
* Move files after transcoding jobs
* Fix federation
* Add video path manager
* Support move to external storage job in client
* Fix live object storage tests
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-17 08:26:20 +02:00
27db78400c
Fix backend channel name validator consistency
2021-08-05 13:54:35 +02:00
9e87894973
Merge branch 'release/3.3.0' into develop
2021-07-29 17:18:28 +02:00
cd162f25d2
Fix video format utils
2021-07-29 17:18:09 +02:00
9c9a236b54
Refactor options in models
2021-07-29 14:17:03 +02:00
d688602710
Refactor search query options
2021-07-29 11:54:38 +02:00
b033851fb5
Search channels against handles and not names
2021-07-29 10:27:24 +02:00
fbd67e7f38
Add ability to search by uuids/actor names
2021-07-28 16:40:21 +02:00
fa47956ecf
Filter host for channels and playlists search
2021-07-28 10:32:40 +02:00
29837f8885
Add ability to search by host in server
2021-07-27 17:10:52 +02:00
15a7eafb89
Refactor video links builders
2021-07-26 15:04:37 +02:00
764b1a14fc
Use random names for VOD HLS playlists
2021-07-26 11:29:31 +02:00
4d029ef8ec
Add ability for instances to follow any actor
2021-07-21 13:35:31 +02:00
2a408c40a9
change audio label, 0p > Audio
...
Changes the audio label in the Download modal.
2021-07-12 15:34:32 +02:00
06c2759338
Fix hooks definition
2021-07-01 16:47:14 +02:00
693c6586cb
Fix missing delete cascade video -> channel
2021-07-01 14:53:37 +02:00
c17ee0903e
Don't check enum for video model
...
Licence, language and category could be extended by a remote plugin
2021-06-29 16:21:32 +02:00
d4a8e7a65f
Support short uuid for GET video/playlist
2021-06-29 14:56:35 +02:00
37a44fc915
Add ability to search playlists
2021-06-25 14:44:01 +02:00
8ebf2a5d5d
Refactor live manager
2021-06-17 08:47:03 +02:00
eae0365b5c
Fix missing transactions
2021-06-15 10:40:06 +02:00
79db409a41
More robust actor image lazy load
2021-06-14 16:14:45 +02:00
520bf885c5
Remove unnecessary log
2021-06-11 16:44:33 +02:00
31d5d916c3
Fix redundancy row grabbing
2021-06-11 16:31:41 +02:00
9f7657b6b6
Include transaction in video hooks
2021-06-11 16:22:54 +02:00
adddb12b04
Fix video with all files query
2021-06-11 16:05:32 +02:00
668f864f5a
Fix live files include
2021-06-11 16:02:26 +02:00
b8afe6f0cd
Fix video rights fetching
2021-06-11 15:53:52 +02:00
69322042c4
Cleanup video model scopes
2021-06-11 14:45:58 +02:00
7c3a6636fd
Move get* video methods to formatter
2021-06-11 14:36:07 +02:00
fd6a74a835
Refactor schedule update
2021-06-11 14:33:34 +02:00
20a206c3d1
Refactor include checks
2021-06-11 14:26:37 +02:00
71d4af1efc
Use raw SQL for most of video queries
2021-06-11 14:09:52 +02:00
3c79c2ce86
Optimize join build
2021-06-11 11:27:45 +02:00
17bb45388e
Optimize rows parsing
2021-06-11 11:15:44 +02:00
ca4b4b2e55
Fetch directly all video attributes for get API
2021-06-11 09:57:19 +02:00
1d43c3a613
Use separate queries for video files
2021-06-11 09:31:59 +02:00
d9bf974f5d
Use raw SQL for video get request
2021-06-10 15:26:18 +02:00
e5dbd5084e
Refactor video query builder
2021-06-10 09:22:58 +02:00
ff0ea0cd8e
Transaction is optional
2021-06-09 17:06:56 +02:00
57a0a9cde4
Optimize AP video captions update
2021-06-09 16:22:01 +02:00
28dfb44b14
Try to speed up AP update transaction
2021-06-08 17:41:48 +02:00
99cb53fd5a
Don't count deleted comment for replies
2021-06-08 13:48:58 +02:00
ea54cd04c1
Fix video upload with a capitalized ext
2021-06-08 10:01:50 +02:00
49af5ac8c2
Refactor AP playlists
2021-06-03 16:40:32 +02:00
304a84d59c
Refactor getOrCreateAPVideo
2021-06-02 16:57:53 +02:00
a7944e89bc
Add expiresOn index
2021-06-01 11:03:52 +02:00
9e2e51dc0b
Add tags to live logger
2021-05-28 13:05:59 +02:00
a1eda903a4
Support '/w/' and '/w/p/' for watch page
...
And use them as default in client
2021-05-28 11:38:08 +02:00
2539932e16
Instance homepage support ( #4007 )
...
* Prepare homepage parsers
* Add ability to update instance hompage
* Add ability to set homepage as landing page
* Add homepage preview in admin
* Dynamically update left menu for homepage
* Inject home content in homepage
* Add videos list and channel miniature custom markup
* Remove unused elements in markup service
2021-05-27 15:59:55 +02:00
5ec3cbdf22
Merge branch 'release/3.2.0' into develop
2021-05-26 15:38:44 +02:00
6c8a99d12a
Fix lint
2021-05-26 15:38:09 +02:00
8ee37c5f38
Merge branch 'release/3.2.0' into develop
2021-05-26 13:41:15 +02:00
f49b3231d0
Fix broken lives handling
2021-05-26 12:03:44 +02:00
53e7f82fe3
Fix redundancy max size
2021-05-26 09:44:16 +02:00
c4d125527a
Avoid error when file has no torrent file
2021-05-25 11:35:29 +02:00
fa263eeff7
Fix 4106: default boolean plugin setting for frontend
2021-05-25 11:35:29 +02:00
be89e66895
Avoid error when file has no torrent file
2021-05-25 11:34:43 +02:00
8bb6e70be3
Fix 4106: default boolean plugin setting for frontend
2021-05-25 11:34:22 +02:00
aea0b0e7cd
Inject server config in HTML
2021-05-14 13:37:23 +02:00
c158a5faab
Refactor a little bit controllers
2021-05-12 16:57:31 +02:00
16c016e8b1
Stricter models typing
2021-05-12 14:12:14 +02:00
428ccb8b7a
Reorganize plugin models
2021-05-11 13:33:11 +02:00
2b02c520e6
Cleanup shared models
2021-05-11 11:32:31 +02:00
7d9ba5c089
Cleanup models directory organization
2021-05-11 11:32:31 +02:00
e024fd6a74
Update channel updatedAt when uploading a video
2021-05-10 09:38:11 +02:00
a66c2e3252
Fix remote actor creation date
2021-05-07 09:00:09 +02:00
b31d72625d
Plugin user.getAuthUser is now async
...
So we can load the full user
2021-05-05 15:26:28 +02:00
1fd61899ea
Add ability to filter my videos by live
2021-05-03 16:42:15 +02:00
9172345484
Fix banner error in actor json
2021-04-13 09:40:20 +02:00
c24822a8fd
Don't save other attributes when set as refreshed
2021-04-12 13:44:58 +02:00
fe19f600da
add channel and playlist stats to server stats endpoint ( #3747 )
...
* add channel and playlist stats to nodeinfo
* add tests for active video channels stats
* fix tests for active channel stats
2021-04-12 11:19:07 +02:00
dc48fdbe68
Merge branch 'release/3.1.0' into develop
2021-04-08 15:16:01 +02:00
1afb3c474a
Fix outbox fetch with subtitled videos
2021-04-08 15:04:14 +02:00
84531547bc
Add size info in db for actor images
2021-04-08 13:38:04 +02:00
a0eeb45f14
Update data in DB when regenerate thumbnails
2021-04-08 13:38:04 +02:00
cdeddff142
Add ability to update the banner
2021-04-08 10:07:53 +02:00
213e30ef90
Add banner tests
2021-04-08 10:07:53 +02:00
2cb03dc1f4
Add banners support
2021-04-08 10:07:53 +02:00
f479685678
Agnostic actor image storage
2021-04-08 10:07:53 +02:00
93544419d2
Fix tests
2021-04-08 08:10:13 +02:00
8e286cdca4
add sort select to my videos, change default sort to -publishedAt
...
fixes #3779
2021-04-06 10:03:19 +02:00
f43db2f46e
Refactor auth flow
...
Reimplement some node-oauth2-server methods to remove hacky code needed by our external
login workflow
2021-03-24 18:18:41 +01:00
32a18cbf33
Add new plugin/peertube version notifs
2021-03-24 18:18:41 +01:00
42ec411bf8
Fix abuse tests
2021-03-09 16:10:52 +01:00
5a22296bdb
Fix broken hot algorithm
2021-03-03 13:52:58 +01:00
534556052e
Optimize views sort sql query
2021-03-03 11:24:16 +01:00
74d249bc13
Add ability to cleanup remote AP interactions
2021-03-03 10:08:08 +01:00
494df94036
Fix abuse crash on deleted video
2021-02-26 09:28:50 +01:00
031ea8efed
Use named chunk for embed on analyze
2021-02-25 15:01:29 +01:00
d61893f723
Async torrent creation
2021-02-25 15:01:07 +01:00
d7df188f23
Fix separate SQL query for video get
2021-02-25 11:18:11 +01:00
2a021e6cb6
Fix thread replies API response
2021-02-19 10:52:31 +01:00
9d6b9d10ef
Fix video comments display with deleted comments
2021-02-19 10:06:52 +01:00
8efc27bf14
Cleanup
2021-02-18 13:38:09 +01:00
f66db4d5c8
Fix feeds
2021-02-18 13:38:09 +01:00
b3d5cb92b1
Add ability to update torrents cache in client
2021-02-18 13:38:09 +01:00
d9a2a03196
Don't guess remote tracker URL
2021-02-18 13:38:09 +01:00
90a8bd305d
Dissociate video file names and video uuid
2021-02-18 13:38:09 +01:00
374b725df5
Optimize remote thumbnail processing
2021-02-16 10:36:44 +01:00
Jelle Besseling
Chocobozzz
kontrollanten
Chocobozzz
John Livingston
Rigel Kent