41fb13c330
esModuleInterop to true
2021-08-27 15:12:22 +02:00
0305db28c9
Add support for saving video files to object storage ( #4290 )
...
* Add support for saving video files to object storage
* Add support for custom url generation on s3 stored files
Uses two config keys to support url generation that doesn't directly go
to (compatible s3). Can be used to generate urls to any cache server or
CDN.
* Upload files to s3 concurrently and delete originals afterwards
* Only publish after move to object storage is complete
* Use base url instead of url template
* Fix mistyped config field
* Add rudenmentary way to download before transcode
* Implement Chocobozzz suggestions
https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478
The remarks in question:
Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names
Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket
Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET)
I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs)
https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer
I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs
* Import correct function
* Support multipart upload
* Remove import of node 15.0 module stream/promises
* Extend maximum upload job length
Using the same value as for redundancy downloading seems logical
* Use dynamic part size for really large uploads
Also adds very small part size for local testing
* Fix decreasePendingMove query
* Resolve various PR comments
* Move to object storage after optimize
* Make upload size configurable and increase default
* Prune webtorrent files that are stored in object storage
* Move files after transcoding jobs
* Fix federation
* Add video path manager
* Support move to external storage job in client
* Fix live object storage tests
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-17 08:26:20 +02:00
d26836cd95
Refactor notifier
2021-08-02 10:39:51 +02:00
764b1a14fc
Use random names for VOD HLS playlists
2021-07-26 11:29:31 +02:00
4d029ef8ec
Add ability for instances to follow any actor
2021-07-21 13:35:31 +02:00
c0e8b12e7f
Refactor requests
2021-07-20 15:27:18 +02:00
c171d85253
Fix webfinger tls only
2021-07-01 17:04:13 +02:00
06c2759338
Fix hooks definition
2021-07-01 16:47:14 +02:00
d4a8e7a65f
Support short uuid for GET video/playlist
2021-06-29 14:56:35 +02:00
37a44fc915
Add ability to search playlists
2021-06-25 14:44:01 +02:00
52fb1d97ba
Fix prune storage tests
2021-06-17 11:09:54 +02:00
8ebf2a5d5d
Refactor live manager
2021-06-17 08:47:03 +02:00
fd6584844b
Resolve object id from actors too
2021-06-16 10:01:31 +02:00
eae0365b5c
Fix missing transactions
2021-06-15 10:40:06 +02:00
e54bd458c1
Remove unnecessary await
2021-06-15 10:40:06 +02:00
7500d6c900
Use a global request timeout
2021-06-15 08:21:31 +02:00
a9fbc2aaa1
Handle short video links
2021-06-14 16:05:25 +02:00
367994c521
Improve captions AP updater
2021-06-14 13:35:52 +02:00
57a0a9cde4
Optimize AP video captions update
2021-06-09 16:22:01 +02:00
4ead40e776
Cache refresh actor promise
2021-06-09 13:34:40 +02:00
bbfdeb3921
Fix actor image update
2021-06-08 18:12:49 +02:00
28dfb44b14
Try to speed up AP update transaction
2021-06-08 17:41:48 +02:00
9de33c6bd9
Add logs for refresh functions
2021-06-08 16:22:23 +02:00
7ffa92d6e4
Log rate AP fetch error as info
...
It's not really a warning, we could have many invalid rates in the
federation
2021-06-08 13:57:08 +02:00
ea54cd04c1
Fix video upload with a capitalized ext
2021-06-08 10:01:50 +02:00
9cc4b9c61f
Fix process delete promise return
2021-06-07 13:20:08 +02:00
91f8f8db97
createThumbnail -> updateThumbnail
2021-06-04 15:45:44 +02:00
868fce62f8
fetch -> load
...
avoid confusion with AP fetch functions
2021-06-03 18:10:56 +02:00
10363c74c1
Move middleware utils in middlewares
...
helpers modules should not import models
2021-06-03 18:03:36 +02:00
5e08989ede
Add tags to AP rate logger
2021-06-03 17:12:38 +02:00
908e6ead78
Refactor AP video logger tags
2021-06-03 16:56:42 +02:00
136d7efde7
Refactor AP actors
2021-06-03 16:40:32 +02:00
49af5ac8c2
Refactor AP playlists
2021-06-03 16:40:32 +02:00
463206948d
Add tags to logs in AP videos
2021-06-02 16:57:53 +02:00
e872632091
Refactor sync attributes
2021-06-02 16:57:53 +02:00
304a84d59c
Refactor getOrCreateAPVideo
2021-06-02 16:57:53 +02:00
c56faf0d94
Move AP video channel creation
2021-06-02 16:57:53 +02:00
08a47c75f9
Refactor AP video create/update
2021-06-02 16:57:53 +02:00
69290ab37b
Refactor AP video update
2021-06-02 16:57:53 +02:00
d88a3ea5e3
Process redundancy stats in series
2021-06-01 11:03:52 +02:00
16c016e8b1
Stricter models typing
2021-05-12 14:12:14 +02:00
7d9ba5c089
Cleanup models directory organization
2021-05-11 11:32:31 +02:00
e024fd6a74
Update channel updatedAt when uploading a video
2021-05-10 09:38:11 +02:00
a66c2e3252
Fix remote actor creation date
2021-05-07 09:00:09 +02:00
c24822a8fd
Don't save other attributes when set as refreshed
2021-04-12 13:44:58 +02:00
78ec174cca
Don't retry failed transaction for refresh
...
It's just a refresh and could lead to some issues in our tests. If there
is a transaction conflict, it means that the same video is already
beeing updated so aborting should be fine
2021-04-09 16:29:38 +02:00
84531547bc
Add size info in db for actor images
2021-04-08 13:38:04 +02:00
a0eeb45f14
Update data in DB when regenerate thumbnails
2021-04-08 13:38:04 +02:00
213e30ef90
Add banner tests
2021-04-08 10:07:53 +02:00
2cb03dc1f4
Add banners support
2021-04-08 10:07:53 +02:00
f479685678
Agnostic actor image storage
2021-04-08 10:07:53 +02:00
266131e0ca
Don't skip all threads on AP fetcher error
2021-03-24 18:18:40 +01:00
b5c361089f
Fix 404 AP status codes
2021-03-24 18:18:40 +01:00
db4b15f21f
Use got instead of request
2021-03-24 18:18:40 +01:00
452b3bea08
Introduce tags to the logger
...
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
6c9c3b7b14
Fix emptying video tags
2021-03-03 11:03:30 +01:00
dcd75f786c
Rename request timeout
2021-03-03 10:10:55 +01:00
74d249bc13
Add ability to cleanup remote AP interactions
2021-03-03 10:08:08 +01:00
8795d6f254
Fix broken local actors
...
Some channels can't federate because they don't have public/private
keys, maybe because the generation failed for various reasons
2021-02-26 14:22:25 +01:00
92315d979c
More robust channel change federation
2021-02-26 11:50:18 +01:00
543442a3be
Add more AP stats to stats endpoint
...
It will help to understand if the federation correctly works or not
2021-02-26 10:28:11 +01:00
a77c7327a2
Fix process abuse transactions
2021-02-25 15:55:31 +01:00
4cbdcf4409
Fix sending accept to old follows
2021-02-19 13:55:00 +01:00
fae6e4da8f
Try to speed up server tests
2021-02-19 09:36:50 +01:00
8efc27bf14
Cleanup
2021-02-18 13:38:09 +01:00
d9a2a03196
Don't guess remote tracker URL
2021-02-18 13:38:09 +01:00
90a8bd305d
Dissociate video file names and video uuid
2021-02-18 13:38:09 +01:00
a35a22797c
Remove previous thumbnail if needed
2021-02-16 10:36:44 +01:00
6302d599cd
Generate a name for caption files
2021-02-16 10:36:44 +01:00
a8b1b40485
Generate a name for thumbnails
...
Allows aggressive caching
2021-02-16 10:36:44 +01:00
c15aff666d
Remove old code
...
We consider Dislike as a classic activity now
2021-02-12 14:53:07 +01:00
ba5a8d89bb
Update server dependencies
2021-02-03 09:45:08 +01:00
471251fa14
Fix error log
2021-01-20 09:50:22 +01:00
1ea7da819e
add ability to remove one's avatar for account and channels ( #3467 )
...
* add ability to remove one's avatar for account and channels
* add ability to remove one's avatar for account and channels
* only display avatar edition options after input change
2021-01-13 09:12:55 +01:00
9cfeb3cf98
Fix redundancy federation in some cases
2020-12-17 09:23:57 +01:00
99afa081bc
Add AP stats
2020-12-15 13:34:58 +01:00
a800dbf345
Live views update
2020-12-09 16:31:42 +01:00
0d8de2756f
Fix live tests
2020-12-09 14:45:47 +01:00
b49f22d8f9
Upgrade sequelize to v6
2020-12-09 11:41:22 +01:00
f2eb23cd87
emit more specific status codes on video upload ( #3423 )
...
- reduce http status codes list to potentially useful codes
- convert more codes to typed ones
- factorize html generator for error responses
2020-12-08 21:16:10 +01:00
e26dc0cdbd
Strict actor url comparison
2020-12-03 16:12:58 +01:00
5147a6d945
Try to fix non HTTPS remote accounts
2020-12-03 15:21:16 +01:00
bb4ba6d94c
Add permanent live support
2020-12-03 15:21:16 +01:00
55d5fbc583
Fix lint
2020-11-20 11:35:33 +01:00
de94ac86a2
Fix incorrect IDs in AP federation
2020-11-20 11:21:08 +01:00
10c8b0b743
Fix comments federation
2020-11-10 15:26:33 +01:00
403c69c5a3
Check threads resolve on non federated videos
2020-11-10 14:34:04 +01:00
c655c9ef6f
Update ffmpeg static version for tests
2020-11-09 15:33:04 +01:00
e4bf785617
Handle views for live videos
2020-11-09 15:33:04 +01:00
053aed43fb
Regenerate miniature on live save
2020-11-09 15:33:04 +01:00
68e70a745b
Add save replay live tests
2020-11-09 15:33:04 +01:00
af4ae64f6f
Begin live tests
2020-11-09 15:33:04 +01:00
a5cf76afa3
Add watch messages if live has not started
2020-11-09 15:33:04 +01:00
de6310b2fc
Handle live federation
2020-11-09 15:33:04 +01:00
1ef65f4c03
Refactor video creation
2020-11-09 15:33:04 +01:00
371906639e
Add ability to a video multiple times in a playlist
2020-08-19 11:30:21 +02:00
30bc55c88b
Refactor video extensions logic in server
2020-08-11 07:57:20 +02:00
bd45d503e5
Reorganize shared models
2020-08-07 08:28:14 +02:00
d95d155988
Use 3 tables to represent abuses
2020-07-10 14:02:41 +02:00
1ebddadd07
predefined report reasons & improved reporter UI ( #2842 )
...
- added `startAt` and `endAt` optional timestamps to help pin down reported sections of a video
- added predefined report reasons
- added video player with report modal
2020-06-22 13:00:39 +02:00
Chocobozzz
Jelle Besseling
Rigel Kent