0305db28c9
Add support for saving video files to object storage ( #4290 )
...
* Add support for saving video files to object storage
* Add support for custom url generation on s3 stored files
Uses two config keys to support url generation that doesn't directly go
to (compatible s3). Can be used to generate urls to any cache server or
CDN.
* Upload files to s3 concurrently and delete originals afterwards
* Only publish after move to object storage is complete
* Use base url instead of url template
* Fix mistyped config field
* Add rudenmentary way to download before transcode
* Implement Chocobozzz suggestions
https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478
The remarks in question:
Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names
Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket
Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET)
I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs)
https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer
I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs
* Import correct function
* Support multipart upload
* Remove import of node 15.0 module stream/promises
* Extend maximum upload job length
Using the same value as for redundancy downloading seems logical
* Use dynamic part size for really large uploads
Also adds very small part size for local testing
* Fix decreasePendingMove query
* Resolve various PR comments
* Move to object storage after optimize
* Make upload size configurable and increase default
* Prune webtorrent files that are stored in object storage
* Move files after transcoding jobs
* Fix federation
* Add video path manager
* Support move to external storage job in client
* Fix live object storage tests
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-17 08:26:20 +02:00
de34551da3
Typo
2021-08-11 14:43:03 +02:00
eb8bd9ab1a
Try to fix openapi script
2021-08-11 14:38:07 +02:00
679c12e69c
Improve target bitrate calculation
2021-08-06 14:13:26 +02:00
15a7eafb89
Refactor video links builders
2021-07-26 15:04:37 +02:00
764b1a14fc
Use random names for VOD HLS playlists
2021-07-26 11:29:31 +02:00
13e1337791
Speed up plugin transcoding tests
2021-07-22 08:55:14 +02:00
a24bd1ed41
Merge branch 'next' into develop
2021-07-21 15:51:30 +02:00
2b9f672b58
Improve plugin constant tests
2021-07-21 15:44:28 +02:00
c63830f154
Rename captions commands
2021-07-21 13:58:35 +02:00
254d3579f5
Use an object to represent a server
2021-07-20 15:27:18 +02:00
89d241a79c
Shorter server command names
2021-07-20 15:27:18 +02:00
d23dd9fbfc
Introduce videos command
2021-07-20 15:27:18 +02:00
9293139fde
Introduce sql command
2021-07-20 15:27:18 +02:00
12edc1495a
Introduce comments command
2021-07-20 15:27:18 +02:00
a2470c9f4b
Introduce captions command
2021-07-20 15:27:17 +02:00
329619b345
Introduce CLI command
2021-07-20 15:27:16 +02:00
13c749c121
Fix angular builds
2021-07-05 15:19:23 +02:00
ed76401ecd
Add ms to parse log dates
2021-06-29 15:22:12 +02:00
8cbc40b2fe
Move to sass module
2021-06-28 10:54:02 +02:00
8cc6120118
Upgrade server dependencies
2021-06-25 17:39:27 +02:00
29b7426c67
Speedup CI
2021-06-17 10:43:34 +02:00
8ebf2a5d5d
Refactor live manager
2021-06-17 08:47:03 +02:00
51f636ad0f
Display locale date time in parse log
2021-06-15 10:40:06 +02:00
3d470a530c
Faster ci using compiled ts files
2021-06-15 08:21:12 +02:00
71d4af1efc
Use raw SQL for most of video queries
2021-06-11 14:09:52 +02:00
27bc958674
Bidi support
2021-06-07 18:14:05 +02:00
5ed25fb76e
Remove annoying superfluous slash
2021-06-02 16:51:23 +02:00
4d7ce9218a
Merge branch 'release/3.2.0' into develop
2021-05-28 15:54:48 +02:00
86dc0b9cc9
Fix upgrade script
2021-05-28 15:54:18 +02:00
5351a0584f
Replace local menu icon
2021-05-28 14:56:42 +02:00
662351e626
Update embed webpack config
2021-05-27 16:00:13 +02:00
5b1a6d45b5
Merge branch 'release/3.2.0' into develop
2021-05-14 14:18:13 +02:00
cb4bab61c1
Fix logger warning level
2021-05-14 14:17:53 +02:00
7d9ba5c089
Cleanup models directory organization
2021-05-11 11:32:31 +02:00
c07902b908
Move transcoding files in their own directory
2021-05-11 11:32:31 +02:00
d15aebf511
Run misc tests in parallel
2021-05-07 15:42:23 +02:00
a0e69167ae
Remove scripty
...
It breaks our tests with the latest npm release
2021-05-05 10:51:32 +02:00
a45050e09e
Add player mode to player stats
2021-04-28 09:07:57 +02:00
6b4e74c2b7
Add log to prune storage script
2021-04-28 09:05:53 +02:00
4e11d8f3ca
Cleanup stats for nerds
2021-04-28 09:05:44 +02:00
3e0e8d4afd
Revert "Fix context menu when watching a playlist"
...
This reverts commit e8bb5b6b3a
2021-04-21 15:28:12 +02:00
e8bb5b6b3a
Fix context menu when watching a playlist
2021-04-21 15:18:22 +02:00
84531547bc
Add size info in db for actor images
2021-04-08 13:38:04 +02:00
a0eeb45f14
Update data in DB when regenerate thumbnails
2021-04-08 13:38:04 +02:00
f479685678
Agnostic actor image storage
2021-04-08 10:07:53 +02:00
47099aba46
Fix benchmark
2021-03-30 08:57:17 +02:00
6794d1005f
Add log to regenerate thumbnails
2021-03-29 17:25:11 +02:00
f076d49f37
script/upgrade: use db.name config value
2021-03-29 09:19:11 +02:00
c2bd7a6fcf
Add regenrate thumbnails scripts
2021-03-24 18:18:41 +01:00
452b3bea08
Introduce tags to the logger
...
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
c221d62a8b
Fix CI moderation tests
2021-03-09 15:45:37 +01:00
f45aa01a1b
Support sq
2021-03-09 11:34:46 +01:00
a5ea2aeaf0
use .load instead of .safeLoad in js-yaml
2021-03-04 12:08:02 +01:00
cd2c3dcdc4
Remove unnecessary transcoding job error
2021-03-03 10:23:05 +01:00
0ce8d34ea0
Add other routes to benchmark
2021-02-25 16:23:30 +01:00
5ac593b7f7
Update credits
2021-02-24 13:59:41 +01:00
e3234c3b4c
Fix credit script link
2021-02-24 13:56:04 +01:00
4abbeff575
Add benchmark script
2021-02-22 15:54:15 +01:00
94d721efdc
Fix stats tests
2021-02-19 16:29:00 +01:00
8efc27bf14
Cleanup
2021-02-18 13:38:09 +01:00
90a8bd305d
Dissociate video file names and video uuid
2021-02-18 13:38:09 +01:00
a35a22797c
Remove previous thumbnail if needed
2021-02-16 10:36:44 +01:00
a8b1b40485
Generate a name for thumbnails
...
Allows aggressive caching
2021-02-16 10:36:44 +01:00
9aeef9aafa
Check video UUID in args scripts
2021-02-11 16:15:11 +01:00
20eb3a5be0
Fix create transcoding job script
...
Depending on the file and the webtorrent/hls config
2021-02-11 15:40:54 +01:00
cdce5af7fc
Provide sourcemaps in release
2021-02-11 14:02:54 +01:00
d93e43164a
Update stats format
2021-02-11 08:41:12 +01:00
99bb59fa77
Add client build stats script
2021-02-10 16:16:46 +01:00
9129b7694d
Allow to specify transcoding and import jobs concurrency
2021-02-08 15:38:45 +01:00
ba5a8d89bb
Update server dependencies
2021-02-03 09:45:08 +01:00
29f148a613
Add ability to build client with source maps
2021-02-02 15:37:18 +01:00
cb5c2abc99
Improve parse log with sql
2021-02-01 15:04:45 +01:00
529b37527c
Use a profile manager for transcoding
2021-01-28 15:55:39 +01:00
1e743faafe
Disable sql prettifier by default
...
It adds too much lines, leading to difficulties when reading dev logs
2021-01-26 09:54:32 +01:00
24516aa26a
Refactor transcoding job handlers
2021-01-21 15:58:17 +01:00
1d22d2514f
Fix service worker
2021-01-18 11:16:37 +01:00
0984960345
add script printing command to generate a resolution for a given file
2021-01-13 09:13:31 +01:00
75dd1b641f
chore: add manifest in light build
2021-01-13 09:10:52 +01:00
dfdd158a75
client: register service worker
...
closes #296
2021-01-13 09:10:52 +01:00
0f4f2ab007
Use more threads for xz release compression
2021-01-07 09:04:10 +01:00
2b6c55528d
Clearer error in parse log
2020-12-22 10:15:06 +01:00
c7444546e7
Add gl support
2020-12-16 11:45:12 +01:00
113d4a3f73
Only link to crowdfunding pages in credits
2020-12-14 16:08:15 +01:00
5d221dcda4
Fix deprecated i18n option
2020-12-14 09:26:21 +01:00
59fd824cf3
Fix tests timeout
2020-12-14 09:11:27 +01:00
34caef7fc0
Add joblog at the end of ci
2020-12-14 09:11:27 +01:00
17ae86e912
Display job log in ci
2020-12-14 09:11:27 +01:00
e078371807
Parse log script can take files as args
2020-12-14 09:11:27 +01:00
0d9c2cc0a2
Test with 3 parallel tests
2020-12-14 09:11:27 +01:00
5f8327c507
Update i18n with new player keys
2020-12-04 16:37:21 +01:00
12394e7e4b
Update contributors
2020-12-02 16:14:56 +01:00
6b88559bab
Fix local e2e tests
2020-11-30 09:11:12 +01:00
ae48a71d57
Remove unused i18n build
2020-11-26 11:30:18 +01:00
c5ce27f864
Regen translations
2020-11-25 16:54:59 +01:00
2245e3df8c
Update scripts to update translations
2020-11-25 16:52:02 +01:00
5a547f69d5
Support encoding profiles
2020-11-25 10:07:51 +01:00
daf6e48010
Split ffmpeg utils with ffprobe utils
2020-11-25 10:07:51 +01:00
488a80fbc0
fix lint for credits generation
2020-11-22 14:17:32 +01:00
63703d8cee
move crowdfunding information to CREDITS.md
...
moving that information to properly group contributions and crediting
in preparation of the v3 crowdfunding end
2020-11-21 17:51:07 +01:00
de94ac86a2
Fix incorrect IDs in AP federation
2020-11-20 11:21:08 +01:00
93ab83893d
Fix npm run test
2020-11-10 15:04:08 +01:00
c1961762b3
Fix player translations
2020-11-09 16:29:58 +01:00
d0510fc1d3
Add missing player localized strings
2020-11-09 16:16:07 +01:00
c8f3cfeba7
Fix tests
2020-11-09 15:33:04 +01:00
af4ae64f6f
Begin live tests
2020-11-09 15:33:04 +01:00
c6c0fa6cd8
Live streaming implementation first step
2020-11-09 15:33:04 +01:00
ed78004086
Update angular to fix localize extraction bug
2020-08-25 10:48:35 +02:00
ee90f1bc19
Remove unused gitlab ci env variables
2020-08-24 18:54:37 +02:00
49a56c7203
Use update i18n command instead of generate
2020-08-24 09:35:27 +02:00
66357162f8
Migrate to $localize
...
* Remove i18n polyfill to translate things in components
* Reduce bundle sizes
* Improve runtime perf
* Reduce a lot the time to make a full client build
* Reduce client build complexity
* We don't need a service to translate things anymore (so we will be able to translate title pages etc)
Unfortunately we may loose some translations in the migration process.
I'll put a message on weblate to notify translators
2020-08-14 10:28:30 +02:00
8c36074799
Update build steps for localization
2020-08-14 09:34:07 +02:00
ef78fdbb2e
Update player and server translations
2020-08-14 09:32:20 +02:00
241c1a558c
Fix icon names
2020-08-07 14:55:55 +02:00
56674bb9f8
Handle unavailable videos in embed playlists
2020-08-07 08:58:29 +02:00
4572c3d0d9
Handle basic playlist in embed
2020-08-07 08:58:29 +02:00
5abc96fca2
Add logic to handle playlist in embed
2020-08-07 08:58:29 +02:00
583eb04b54
Upgrade to angular 10
2020-08-07 08:28:14 +02:00
a3b5e78af3
Analyze embed too with bundlewatch
2020-08-06 10:58:36 +02:00
5c145fe7ab
Fix embed build
2020-08-06 09:57:07 +02:00
fc3b14e413
Limit github actions on PR
2020-08-06 09:47:46 +02:00
d632a14729
Test bundlewatch
2020-08-06 09:42:33 +02:00
d573926e9b
Add migrations for abuse messages
2020-07-31 11:35:19 +02:00
f9e1ca7087
Execute external tests on schedule
2020-07-31 11:32:04 +02:00
f9d2deae3a
Correctly test ldap
2020-07-31 11:32:04 +02:00
12e8547f07
Test ldap
2020-07-31 11:32:04 +02:00
f4659d73fb
Don't run in parallel cli and plugin tests
2020-07-31 11:32:04 +02:00
f0af38e696
Fix lint
2020-07-31 11:32:04 +02:00
7abb5c5da1
Use parallel to run tests in parallel
2020-07-31 11:32:04 +02:00
b488ba1e26
Don't rely on youtube for tests
...
Use another import URL when possible, and disable import tests when we
want to do a youtube import test
2020-07-31 11:32:04 +02:00
345b4a22a8
Update my-account sub-menus icons ( #2977 )
...
* Update user sub-menus icons
* Update admin sub-menu icons
* Add solendp to credits
Co-authored-by: kimsible <kimsible@users.noreply.github.com>
2020-07-23 14:42:06 +02:00
d28d13581f
Fix embed p2p warning localization
2020-07-20 17:05:08 +02:00
d10538b47d
ar to ar-001 locale
2020-07-02 14:08:02 +02:00
bfbc596643
update scripts/generate-code-contributors.ts
2020-07-01 17:03:48 +02:00
07aea1a264
Use TS_NODE_FILES instead of --files
...
Because --files does not work
2020-06-18 17:14:02 +02:00
c24aac6bc7
Fix tests
2020-06-18 13:39:13 +02:00
50a04125da
Fix npm run build
2020-06-11 09:29:33 +02:00
e6dfa58689
space optimizations for `node_modules` and client stats removal
...
- replace sharp with lighter jimp alternative
- remove stats in builds
fixes #2807
2020-06-10 14:01:49 +02:00
0bd558a0f9
Add kab and vi locales
2020-06-04 14:45:19 +02:00
520745bfe8
Add ar-001 locale
2020-06-04 10:35:08 +02:00
b763f88dd0
Merge branch 'release/2.2.0' into develop
2020-06-02 09:31:30 +02:00
03fc19281b
Fix emails
2020-06-02 09:21:33 +02:00
2a39506c7d
Revert "Remove oc locale support"
...
This reverts commit 8323c435c1
2020-05-29 17:09:55 +02:00
ee68bbc481
Unblock ios e2e tests
2020-05-12 16:49:35 +02:00
80fdaf0645
Add moderation helpers to plugins
2020-05-07 15:07:01 +02:00
829b794a85
Add plugin ldap tests
2020-05-04 16:21:39 +02:00
e1c5503114
Support logout and add id and pass tests
2020-05-04 16:21:39 +02:00
8dc8a34ee8
Avoir some circular dependencies
2020-05-04 16:21:39 +02:00
1b05d82d86
Add SQL query support in plugins
2020-04-09 11:00:30 +02:00
8323c435c1
Remove oc locale support
...
It does not work since many releases, and nobody created an issue so we
don't consider it much use
It does not work because Angular does not know this locale: https://github.com/angular/angular/tree/master/packages/common/locales
2020-04-06 14:37:26 +02:00
b6e0e6a31b
Update credits
2020-02-28 14:01:13 +01:00
99ae9d3a9e
Fix npm run dev
2020-02-19 09:14:28 +01:00
302cb3f92e
Use bash to run build scripts too
2020-02-19 08:57:32 +01:00
0e4338fd1d
Use bash to run scripts
2020-02-19 08:56:24 +01:00
170cd639a7
Create template dirs for openapi clients and provide README templates
...
The definitions are now also in the template dirs because they will be ignored by the generator.
Generators only use overrides and don't do anything with new files.
2020-02-18 14:49:11 +01:00
1d17d86349
Merge branch 'open-api-clients' into develop
2020-02-17 09:03:49 +01:00
002df3813d
OpenAPI clients generation
2020-02-14 14:14:14 +01:00
5a208b004a
Extend build commands
2020-02-13 14:13:48 +01:00
2779f29a66
Keep our current build system for now
2020-02-12 14:09:55 +01:00
8cc56c7fdd
Only use basehref
...
Will allow us to use the localize option
2020-02-12 14:09:55 +01:00
30ff193a60
Update code contributors
2020-02-11 15:18:25 +01:00
0e9c3b2810
Fix client dev mode
2020-02-10 16:39:28 +01:00
361dcebc75
Support i18n build
2020-02-10 16:39:28 +01:00
0df21c799f
Add hu and th locales
2020-02-04 16:48:54 +01:00
a15871560f
Move to eslint
2020-02-03 08:31:02 +01:00
144c6559ed
Relax travis build
2020-01-28 11:37:09 +01:00
b764380ac2
Add ability to list redundancies
2020-01-28 11:35:26 +01:00
44d4ee4fcb
Fix e2e tests
2020-01-23 14:59:44 +01:00
36619ac8f5
Reduce dev commands RAM usage
2020-01-22 10:18:57 +01:00
6848b9f4b2
Use a filtered proxy without variables
2020-01-22 09:36:03 +01:00
c76130067c
Fix travis regex
2020-01-21 17:02:01 +01:00
be7ca0c6b9
Use a proxy for youtube-dl to avoid travis errors
2020-01-21 16:46:59 +01:00
a3705089fa
Remove husky
2020-01-21 14:28:28 +01:00
c4a1811ea8
Use lower instead of ilike to login users
2020-01-09 14:00:34 +01:00
41ee83423c
Rebuild credits
...
Had to update the script now we use weblate
2020-01-09 12:00:47 +01:00
9b7668f527
Fix npm run dev script
2020-01-07 16:53:55 +01:00
9650937374
(fix) npm run dev invocation
2019-12-18 20:41:44 +01:00
e43ed96ef4
Fix build script
2019-12-18 15:40:59 +01:00
ba430d7516
Lazy load static objects
2019-12-18 15:40:59 +01:00
9f0327aeb3
Fix build
2019-12-18 11:01:59 +01:00
03d641a0d7
Add embed api build
2019-12-18 10:14:25 +01:00
c026a2e673
Remove watch scripts
2019-12-18 10:14:24 +01:00
3f9c4955af
Speedup embed first paint
2019-12-18 10:14:22 +01:00
907b8f9347
Use a migration script to generate HLS torrent files
2019-12-12 10:56:34 +01:00
47f6cb31d7
Fix email to field
2019-11-29 13:36:40 +01:00
20146df217
Fix update host
2019-11-27 11:38:40 +01:00
dee6fe1e4f
Add ability to generate HLS in CLI
2019-11-25 10:59:47 +01:00
66fb2aa39b
Don't always replace actor avatar
2019-11-25 10:59:44 +01:00
d7a25329f9
Add ability to disable webtorrent
...
In favour of HLS
2019-11-25 10:59:43 +01:00
df0fd9043c
Try to improve server watch script
2019-11-18 10:34:23 +01:00
cd1d6c23da
Remove unused files
2019-11-13 10:04:16 +01:00
350131cbaf
Make build works
2019-11-13 10:04:16 +01:00
24b1c65abd
Remove html tags
2019-11-07 14:12:57 +01:00
ad912c3de6
Update server and player
2019-11-07 12:00:09 +01:00
3aea8eb2e7
Try new worflow with french
2019-11-07 09:45:14 +01:00
41eac41b05
Fix typings
2019-11-05 09:21:04 +01:00
ebd316d142
Fix duplicated in CREDITS
2019-11-05 08:56:27 +01:00
8ed11b89b8
Try to update version of nightly builds
2019-10-23 11:46:57 +02:00
a0e6d26759
Fix tsconfig with CLI tools
2019-10-23 10:53:20 +02:00
1cd4fe501d
Fix i18n generation script
2019-10-22 10:30:09 +02:00
Jelle Besseling
Chocobozzz
frankstrater
kontrollanten
Lukas Winkler
Rigel Kent
Kim
Average Dude
ICabaleiro