Commit Graph

671 Commits

Author SHA1 Message Date
Chocobozzz 3396e65345
Optimize broadcast job creation 2022-06-17 11:55:03 +02:00
Chocobozzz b9e49a45f5
Add simulate many viewers script
Helps us to improve views scalability on peertube
2022-06-17 10:48:37 +02:00
Chocobozzz 65ba19ace9
Fix nightly build 2022-06-14 08:27:33 +02:00
John Livingston 7f6ff46222 Implement 5036: more readable diff on configuration file when upgrading. 2022-06-13 15:59:22 +02:00
Chocobozzz 92148a7a39
Fix ng build options 2022-06-13 15:15:20 +02:00
Chocobozzz 6d9ed0b6cb
Merge branch 'release/4.2.0' into develop 2022-06-07 09:59:15 +02:00
Chocobozzz 462a833e4c
Fix release script 2022-06-07 09:58:59 +02:00
Chocobozzz b0185d7351
Add new player string to custom translations 2022-05-31 14:34:47 +02:00
Chocobozzz d437596f86
Wait before uploading release files 2022-05-23 08:40:21 +02:00
Chocobozzz b003d57518
Add missing player translation 2022-05-04 09:18:39 +02:00
Chocobozzz b211106695 Support video views/viewers stats in server
* Add "currentTime" and "event" body params to view endpoint
 * Merge watching and view endpoints
 * Introduce WatchAction AP activity
 * Add tables to store viewer information of local videos
 * Add endpoints to fetch video views/viewers stats of local videos
 * Refactor views/viewers handlers
 * Support "views" and "viewers" counters for both VOD and live videos
2022-04-15 09:49:35 +02:00
Chocobozzz 68a4b1e587
Fix benchmark tests 2022-03-24 13:42:12 +01:00
Chocobozzz 1808a1f8e4
Add video edition finished notification 2022-03-22 16:25:14 +01:00
Chocobozzz 4e5e072c1b
Fix benchmark script 2022-03-21 08:24:07 +01:00
Chocobozzz 6afc0d374c
Add watching and views endpoints to benchmark 2022-03-18 14:52:53 +01:00
Chocobozzz 57e4e1c1a9
Don't store remote rates of remote videos
In the future we'll stop to expose all available rates to improve users
privacy
2022-03-18 11:21:50 +01:00
Chocobozzz 2e3f7a5a6f
Also fill avatar width for channels 2022-03-18 08:16:44 +01:00
Chocobozzz db1ccd05d3
Fix CI 2022-03-17 10:50:26 +01:00
Chocobozzz cc45cc9ac1
Remove uneeded memoize 2022-03-08 11:39:56 +01:00
Chocobozzz 55f0a7a762
Update translations and support fa 2022-03-03 10:54:52 +01:00
Chocobozzz 7bde625050
Fill width of local avatars 2022-03-01 13:37:40 +01:00
Chocobozzz 95faf1eaff
Create another test suite for transcoding jobs 2022-02-28 16:02:08 +01:00
Chocobozzz 1087427616
Improve benchmark script CLI options 2022-02-28 15:19:44 +01:00
Chocobozzz c729caf6cc Add basic video editor support 2022-02-28 10:42:19 +01:00
kontrollanten d0800f7661
Implement avatar miniatures (#4639)
* client: remove unused file

* refactor(client/my-actor-avatar): size from input

Read size from component input instead of scss, to make it possible to
use smaller avatar images when implemented.

* implement avatar miniatures

close #4560

* fix(test): max file size

* fix(search-index): normalize res acc to avatarMini

* refactor avatars to an array

* client/search: resize channel avatar to 120

* refactor(client/videos): remove unused function

* client(actor-avatar): set default size

* fix tests and avatars full result

When findOne is used only an array containting one avatar is returned.

* update migration version and version notations

* server/search: harmonize normalizing

* Cleanup avatar miniature PR

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-02-28 08:34:43 +01:00
Chocobozzz d8aa370f3d
Only generate types for stable releases 2022-02-10 10:41:15 +01:00
Chocobozzz c4207f978e
Fast forward on HLS decode error 2022-02-02 11:26:18 +01:00
Chocobozzz 0f11ec8dd3
Prevent HLS transcoding after webtorrent transcoding 2022-02-01 11:27:38 +01:00
Chocobozzz 2dd0a8a8fd
Add fast forward/rewind on mobile 2022-01-12 15:07:21 +01:00
Chocobozzz cbe2f36d93
Fix audio transcoding with video only file 2022-01-06 17:55:37 +01:00
Chocobozzz f8360396ff
Stop using tsconfig register
Prefer to replace paths at compile time
2022-01-06 08:26:14 +01:00
Chocobozzz 9270bd3a7e
Migrate to --enable-source-maps option 2022-01-03 16:37:16 +01:00
Chocobozzz 21d68e6803
Add version to generate types packages 2021-12-28 10:27:06 +01:00
Chocobozzz 6627dbc957
Move types package in packages/ 2021-12-28 10:18:15 +01:00
Chocobozzz b969539c83
Fix types dist paths 2021-12-24 13:28:33 +01:00
Chocobozzz dbd9fb44dd
Don't stuck state when move transcoding job failed 2021-12-23 11:13:06 +01:00
Chocobozzz 709565486f
Ensure we don't move live to object storage 2021-12-17 14:55:12 +01:00
Chocobozzz bf54587a3e
shared/ typescript types dir server-commands 2021-12-17 09:29:23 +01:00
Chocobozzz b8a4afa4c2
Optimize server eslint 2021-12-16 17:06:52 +01:00
Chocobozzz 9576506763
Add P2P enabled/disabled in player card 2021-12-16 10:08:55 +01:00
Chocobozzz 9b3294a849 Fix some build scripts and lint 2021-12-16 10:08:43 +01:00
lutangar b8fa3e8c73 refactor(types): create dedicated folder for types package src
fix guide examples and add types package readme

refactor(tsconfig): move back base tsconfig to base directory
2021-12-16 10:08:43 +01:00
lutangar 8b03e2ce1a feat(types): create peertube-types package
Create dedicated Typescript "types" configuration file for each "projects".
Create a types folder which includes every useful ts definition:
- client
- server
- shared
Add tooling to create a proper package, extract dependencies, etc...
Add CI Github task.

refactor(types): publish types package in release script
2021-12-16 10:08:43 +01:00
lutangar 06aad80165 chore(refactor): remove shared folder dependencies to the server
Many files from the `shared` folder were importing files from the `server` folder.
When attempting to use Typescript project references to describe dependencies,
it highlighted a circular dependency beetween `shared` <-> `server`.

The Typescript project forbid such usages.
Using project references greatly improve performance by rebuilding only
the updated project and not all source files.
> see https://www.typescriptlang.org/docs/handbook/project-references.html
2021-12-16 10:08:43 +01:00
Chocobozzz 3cf68b869d
Ability for admins to set default upload values 2021-12-14 17:17:01 +01:00
Chocobozzz 66175ae8b4
Support nn and nb locales 2021-12-13 17:06:44 +01:00
Chocobozzz e2a1ccd174
Add meaning to date short names 2021-12-13 16:44:34 +01:00
Chocobozzz 42fea41a48
Update translations 2021-12-13 16:36:56 +01:00
Chocobozzz 90701ec1d8
Add hls to prune storage script 2021-12-10 10:28:46 +01:00
Chocobozzz 6ba93fa639
Fix torrent metadata update for hls 2021-12-09 11:23:02 +01:00
Chocobozzz 9b293cd6a2
Update torrent metadata on video update 2021-12-08 11:07:19 +01:00
kontrollanten 875c402bab scripts/test: make MacOS compatible
man xargs:
     -r      Compatibility with GNU xargs.  The GNU version of xargs runs the utility argument at least once, even if xargs input is empty, and it supports a -r option to inhibit this
             behavior.  The FreeBSD version of xargs does not run the utility argument on empty input, but it supports the -r option for command-line compatibility with GNU xargs, but the
             -r option does nothing in the FreeBSD version of xargs.
2021-12-08 08:38:22 +01:00
Chocobozzz d91be8acd7
Don't process live in migration script 2021-11-25 14:39:37 +01:00
Jocelyn Jaubert e7542293d6 Launch yarn before linking to new peertube version
If yarn fails because we don't have the correct nodejs version, we want to know
earlier, so that we can upgrade nodejs without having a non-working
half-installed peertube instance.
2021-11-22 08:52:54 +01:00
Chocobozzz 98846837eb
Stop killing peertube in ci script 2021-11-18 15:32:06 +01:00
Chocobozzz ad5db1044c
Add ability to run transcoding jobs 2021-11-18 15:20:57 +01:00
Chocobozzz 5638df0895
Update xliffmerge 2021-11-15 15:56:58 +01:00
Chocobozzz 8f2608e9a9
Update credits 2021-11-12 11:28:04 +01:00
kontrollanten e1ab52d7ec
Add migrate-to-object-storage script (#4481)
* add migrate-to-object-storage-script

closes #4467

* add migrate-to-unique-playlist-filenames script

* fix(migrate-to-unique-playlist-filenames): update master/segments256

run updateMasterHLSPlaylist and updateSha256VODSegments after
file rename.

* Improve move to object storage scripts

* PR remarks

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-11-09 11:05:35 +01:00
Chocobozzz 8aad7ae413
Fix transcoding job with resolution 2021-11-05 14:19:42 +01:00
Chocobozzz fd5fa94bbc
Remove unnecessary env 2021-11-02 10:30:50 +01:00
Chocobozzz efcadd3d14
Fix "Off" player string localization 2021-10-21 11:01:10 +02:00
Chocobozzz cf0c8ee588
Fix admin edition disabling feature 2021-10-14 11:35:43 +02:00
Chocobozzz 87c0f718dc
Fix benchmark test 2021-10-14 08:30:17 +02:00
Chocobozzz 624e42fdf9
Remove peertube watch and peertube repl
I don't have time to maintain these scripts that cause some install
issues/maintenance difficulties
2021-09-10 15:06:57 +02:00
Chocobozzz 6d210220be
Fix NSFW filter and add tests 2021-09-03 16:35:18 +02:00
Chocobozzz 2ede07153c
Fix hls redundancy pruning 2021-09-02 15:10:40 +02:00
Chocobozzz 12d6b873cd
Improve e2e workflow and add doc 2021-09-02 09:07:42 +02:00
Chocobozzz 3419e0e1fe
Migrate to webdriverio 2021-09-01 15:06:46 +02:00
Chocobozzz 736c64ca4b
Fix benchmark script 2021-08-30 14:22:33 +02:00
Chocobozzz 41fb13c330
esModuleInterop to true 2021-08-27 15:12:22 +02:00
Chocobozzz 50bb71bade
Update contributor list 2021-08-26 10:01:42 +02:00
Chocobozzz 1fd8803cd9
No need to install client dependencies for upgrade 2021-08-26 10:01:42 +02:00
Chocobozzz 33a53638c2
Embed stats only when needed 2021-08-18 10:32:40 +02:00
Chocobozzz 1f6125be8b
Optimize torrent URL update 2021-08-18 09:14:51 +02:00
Chocobozzz c186a67f90
Support short uuid for scripts 2021-08-17 11:06:10 +02:00
Chocobozzz 66a7fc9473
Remove optimize old videos script
It is not compatible with HLS and I don't have time to maintain it
2021-08-17 10:33:36 +02:00
Jelle Besseling 0305db28c9
Add support for saving video files to object storage (#4290)
* Add support for saving video files to object storage

* Add support for custom url generation on s3 stored files

Uses two config keys to support url generation that doesn't directly go
to (compatible s3). Can be used to generate urls to any cache server or
CDN.

* Upload files to s3 concurrently and delete originals afterwards

* Only publish after move to object storage is complete

* Use base url instead of url template

* Fix mistyped config field

* Add rudenmentary way to download before transcode

* Implement Chocobozzz suggestions

https://github.com/Chocobozzz/PeerTube/pull/4290#issuecomment-891670478

The remarks in question:
    Try to use objectStorage prefix instead of s3 prefix for your function/variables/config names
    Prefer to use a tree for the config: s3.streaming_playlists_bucket -> object_storage.streaming_playlists.bucket
    Use uppercase for config: S3.STREAMING_PLAYLISTS_BUCKETINFO.bucket -> OBJECT_STORAGE.STREAMING_PLAYLISTS.BUCKET (maybe BUCKET_NAME instead of BUCKET)
    I suggest to rename moveJobsRunning to pendingMovingJobs (or better, create a dedicated videoJobInfo table with a pendingMove & videoId columns so we could also use this table to track pending transcoding jobs)
    https://github.com/Chocobozzz/PeerTube/pull/4290/files#diff-3e26d41ca4bda1de8e1747af70ca2af642abcc1e9e0bfb94239ff2165acfbde5R19 uses a string instead of an integer
    I think we should store the origin object storage URL in fileUrl, without base_url injection. Instead, inject the base_url at "runtime" so admins can easily change this configuration without running a script to update DB URLs

* Import correct function

* Support multipart upload

* Remove import of node 15.0 module stream/promises

* Extend maximum upload job length

Using the same value as for redundancy downloading seems logical

* Use dynamic part size for really large uploads

Also adds very small part size for local testing

* Fix decreasePendingMove query

* Resolve various PR comments

* Move to object storage after optimize

* Make upload size configurable and increase default

* Prune webtorrent files that are stored in object storage

* Move files after transcoding jobs

* Fix federation

* Add video path manager

* Support move to external storage job in client

* Fix live object storage tests

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-08-17 08:26:20 +02:00
Chocobozzz de34551da3
Typo 2021-08-11 14:43:03 +02:00
Chocobozzz eb8bd9ab1a
Try to fix openapi script 2021-08-11 14:38:07 +02:00
Chocobozzz 679c12e69c
Improve target bitrate calculation 2021-08-06 14:13:26 +02:00
Chocobozzz 15a7eafb89
Refactor video links builders 2021-07-26 15:04:37 +02:00
Chocobozzz 764b1a14fc Use random names for VOD HLS playlists 2021-07-26 11:29:31 +02:00
Chocobozzz 13e1337791
Speed up plugin transcoding tests 2021-07-22 08:55:14 +02:00
Chocobozzz a24bd1ed41
Merge branch 'next' into develop 2021-07-21 15:51:30 +02:00
Chocobozzz 2b9f672b58
Improve plugin constant tests 2021-07-21 15:44:28 +02:00
Chocobozzz c63830f154
Rename captions commands 2021-07-21 13:58:35 +02:00
Chocobozzz 254d3579f5
Use an object to represent a server 2021-07-20 15:27:18 +02:00
Chocobozzz 89d241a79c
Shorter server command names 2021-07-20 15:27:18 +02:00
Chocobozzz d23dd9fbfc
Introduce videos command 2021-07-20 15:27:18 +02:00
Chocobozzz 9293139fde
Introduce sql command 2021-07-20 15:27:18 +02:00
Chocobozzz 12edc1495a
Introduce comments command 2021-07-20 15:27:18 +02:00
Chocobozzz a2470c9f4b
Introduce captions command 2021-07-20 15:27:17 +02:00
Chocobozzz 329619b345
Introduce CLI command 2021-07-20 15:27:16 +02:00
Chocobozzz 13c749c121
Fix angular builds 2021-07-05 15:19:23 +02:00
Chocobozzz ed76401ecd
Add ms to parse log dates 2021-06-29 15:22:12 +02:00
Chocobozzz 8cbc40b2fe
Move to sass module 2021-06-28 10:54:02 +02:00
Chocobozzz 8cc6120118
Upgrade server dependencies 2021-06-25 17:39:27 +02:00
Chocobozzz 29b7426c67
Speedup CI 2021-06-17 10:43:34 +02:00
Chocobozzz 8ebf2a5d5d
Refactor live manager 2021-06-17 08:47:03 +02:00
Chocobozzz 51f636ad0f
Display locale date time in parse log 2021-06-15 10:40:06 +02:00
Chocobozzz 3d470a530c Faster ci using compiled ts files 2021-06-15 08:21:12 +02:00
Chocobozzz 71d4af1efc
Use raw SQL for most of video queries 2021-06-11 14:09:52 +02:00
Chocobozzz 27bc958674
Bidi support 2021-06-07 18:14:05 +02:00
frankstrater 5ed25fb76e Remove annoying superfluous slash 2021-06-02 16:51:23 +02:00
Chocobozzz 4d7ce9218a
Merge branch 'release/3.2.0' into develop 2021-05-28 15:54:48 +02:00
Chocobozzz 86dc0b9cc9
Fix upgrade script 2021-05-28 15:54:18 +02:00
Chocobozzz 5351a0584f
Replace local menu icon 2021-05-28 14:56:42 +02:00
Chocobozzz 662351e626 Update embed webpack config 2021-05-27 16:00:13 +02:00
Chocobozzz 5b1a6d45b5
Merge branch 'release/3.2.0' into develop 2021-05-14 14:18:13 +02:00
Chocobozzz cb4bab61c1
Fix logger warning level 2021-05-14 14:17:53 +02:00
Chocobozzz 7d9ba5c089
Cleanup models directory organization 2021-05-11 11:32:31 +02:00
Chocobozzz c07902b908
Move transcoding files in their own directory 2021-05-11 11:32:31 +02:00
Chocobozzz d15aebf511 Run misc tests in parallel 2021-05-07 15:42:23 +02:00
Chocobozzz a0e69167ae
Remove scripty
It breaks our tests with the latest npm release
2021-05-05 10:51:32 +02:00
Chocobozzz a45050e09e
Add player mode to player stats 2021-04-28 09:07:57 +02:00
Chocobozzz 6b4e74c2b7
Add log to prune storage script 2021-04-28 09:05:53 +02:00
Chocobozzz 4e11d8f3ca Cleanup stats for nerds 2021-04-28 09:05:44 +02:00
Chocobozzz 3e0e8d4afd
Revert "Fix context menu when watching a playlist"
This reverts commit e8bb5b6b3a.

We'll refactor this error handler in hls.js v1 upgrade
2021-04-21 15:28:12 +02:00
Chocobozzz e8bb5b6b3a
Fix context menu when watching a playlist 2021-04-21 15:18:22 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz a0eeb45f14
Update data in DB when regenerate thumbnails 2021-04-08 13:38:04 +02:00
Chocobozzz f479685678 Agnostic actor image storage 2021-04-08 10:07:53 +02:00
Chocobozzz 47099aba46
Fix benchmark 2021-03-30 08:57:17 +02:00
Chocobozzz 6794d1005f
Add log to regenerate thumbnails 2021-03-29 17:25:11 +02:00
kontrollanten f076d49f37 script/upgrade: use db.name config value 2021-03-29 09:19:11 +02:00
Chocobozzz c2bd7a6fcf
Add regenrate thumbnails scripts 2021-03-24 18:18:41 +01:00
Chocobozzz 452b3bea08
Introduce tags to the logger
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
Chocobozzz c221d62a8b
Fix CI moderation tests 2021-03-09 15:45:37 +01:00
Chocobozzz f45aa01a1b
Support sq 2021-03-09 11:34:46 +01:00
Lukas Winkler a5ea2aeaf0 use .load instead of .safeLoad in js-yaml 2021-03-04 12:08:02 +01:00
Chocobozzz cd2c3dcdc4
Remove unnecessary transcoding job error 2021-03-03 10:23:05 +01:00
Chocobozzz 0ce8d34ea0
Add other routes to benchmark 2021-02-25 16:23:30 +01:00
Chocobozzz 5ac593b7f7
Update credits 2021-02-24 13:59:41 +01:00
Chocobozzz e3234c3b4c
Fix credit script link 2021-02-24 13:56:04 +01:00
Chocobozzz 4abbeff575
Add benchmark script 2021-02-22 15:54:15 +01:00
Chocobozzz 94d721efdc Fix stats tests 2021-02-19 16:29:00 +01:00
Chocobozzz 8efc27bf14 Cleanup 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz a35a22797c Remove previous thumbnail if needed 2021-02-16 10:36:44 +01:00
Chocobozzz a8b1b40485 Generate a name for thumbnails
Allows aggressive caching
2021-02-16 10:36:44 +01:00
Chocobozzz 9aeef9aafa
Check video UUID in args scripts 2021-02-11 16:15:11 +01:00
Chocobozzz 20eb3a5be0
Fix create transcoding job script
Depending on the file and the webtorrent/hls config
2021-02-11 15:40:54 +01:00
Chocobozzz cdce5af7fc
Provide sourcemaps in release 2021-02-11 14:02:54 +01:00
Chocobozzz d93e43164a
Update stats format 2021-02-11 08:41:12 +01:00
Chocobozzz 99bb59fa77
Add client build stats script 2021-02-10 16:16:46 +01:00
Chocobozzz 9129b7694d Allow to specify transcoding and import jobs concurrency 2021-02-08 15:38:45 +01:00
Chocobozzz ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Chocobozzz 29f148a613
Add ability to build client with source maps 2021-02-02 15:37:18 +01:00
Chocobozzz cb5c2abc99
Improve parse log with sql 2021-02-01 15:04:45 +01:00