Commit Graph

579 Commits

Author SHA1 Message Date
Chocobozzz 3a4992633e
Migrate server to ESM
Sorry for the very big commit that may lead to git log issues and merge
conflicts, but it's a major step forward:

 * Server can be faster at startup because imports() are async and we can
   easily lazy import big modules
 * Angular doesn't seem to support ES import (with .js extension), so we
   had to correctly organize peertube into a monorepo:
    * Use yarn workspace feature
    * Use typescript reference projects for dependencies
    * Shared projects have been moved into "packages", each one is now a
      node module (with a dedicated package.json/tsconfig.json)
    * server/tools have been moved into apps/ and is now a dedicated app
      bundled and published on NPM so users don't have to build peertube
      cli tools manually
    * server/tests have been moved into packages/ so we don't compile
      them every time we want to run the server
 * Use isolatedModule option:
   * Had to move from const enum to const
     (https://www.typescriptlang.org/docs/handbook/enums.html#objects-vs-enums)
   * Had to explictely specify "type" imports when used in decorators
 * Prefer tsx (that uses esbuild under the hood) instead of ts-node to
   load typescript files (tests with mocha or scripts):
     * To reduce test complexity as esbuild doesn't support decorator
       metadata, we only test server files that do not import server
       models
     * We still build tests files into js files for a faster CI
 * Remove unmaintained peertube CLI import script
 * Removed some barrels to speed up execution (less imports)
2023-08-11 15:02:33 +02:00
Chocobozzz 881958d179
Correctly download files from S3
Use pre signed URLs to inject content disposition header
2023-06-29 10:18:21 +02:00
Chocobozzz 9c5cc50133
Add script to generate storyboards 2023-06-29 10:16:57 +02:00
Chocobozzz b4516abc39
Bumped to version v5.2.0 2023-06-21 09:06:13 +02:00
Chocobozzz 9a64621975
Bumped to version v5.2.0-rc.1 2023-05-30 10:18:25 +02:00
Chocobozzz 51afea5411
Update client dependencies 2023-05-25 09:24:55 +02:00
Chocobozzz 83002a8234
Update server dependencies 2023-05-22 17:11:31 +02:00
Alecks Gates cb0eda5602
Add Podcast RSS feeds (#5487)
* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Add correct feed image to RSS channel

* Prefer HLS videos for podcast RSS

Remove video/stream titles, add optional height attribute to podcast RSS

* Prefix podcast RSS images with root server URL

* Add optional video query support to include captions

* Add transcripts & person images to podcast RSS feed

* Prefer webseed/webtorrent files over HLS fragmented mp4s

* Experimentally adding podcast fields to basic config page

* Add validation for new basic config fields

* Don't include "content" in podcast feed, use full description for "description"

* Initial test implementation of Podcast RSS

This is a pretty simple implementation to add support for The Podcast Namespace in RSS -- instead of affecting the existing RSS implementation, this adds a new UI option.

I attempted to retain compatibility with the rest of the RSS feed implementation as much as possible and have created a temporary fork of the "pfeed" library to support this effort.

* Update to pfeed-podcast 1.2.2

* Add correct feed image to RSS channel

* Prefer HLS videos for podcast RSS

Remove video/stream titles, add optional height attribute to podcast RSS

* Prefix podcast RSS images with root server URL

* Add optional video query support to include captions

* Add transcripts & person images to podcast RSS feed

* Prefer webseed/webtorrent files over HLS fragmented mp4s

* Experimentally adding podcast fields to basic config page

* Add validation for new basic config fields

* Don't include "content" in podcast feed, use full description for "description"

* Add medium/socialInteract to podcast RSS feeds. Use HTML for description

* Change base production image to bullseye, install prosody in image

* Add liveItem and trackers to Podcast RSS feeds

Remove height from alternateEnclosure, replaced with title.

* Clear Podcast RSS feed cache when live streams start/end

* Upgrade to Node 16

* Refactor clearCacheRoute to use ApiCache

* Remove unnecessary type hint

* Update dockerfile to node 16, install python-is-python2

* Use new file paths for captions/playlists

* Fix legacy videos in RSS after migration to object storage

* Improve method of identifying non-fragmented mp4s in podcast RSS feeds

* Don't include fragmented MP4s in podcast RSS feeds

* Add experimental support for podcast:categories on the podcast RSS item

* Fix undefined category when no videos exist

Allows for empty feeds to exist (important for feeds that might only go live)

* Add support for podcast:locked -- user has to opt in to show their email

* Use comma for podcast:categories delimiter

* Make cache clearing async

* Fix merge, temporarily test with pfeed-podcast

* Syntax changes

* Add EXT_MIMETYPE constants for captions

* Update & fix tests, fix enclosure mimetypes, remove admin email

* Add test for podacst:socialInteract

* Add filters hooks for podcast customTags

* Remove showdown, updated to pfeed-podcast 6.1.2

* Add 'action:api.live-video.state.updated' hook

* Avoid assigning undefined category to podcast feeds

* Remove nvmrc

* Remove comment

* Remove unused podcast config

* Remove more unused podcast config

* Fix MChannelAccountDefault type hint missed in merge

* Remove extra line

* Re-add newline in config

* Fix lint errors for isEmailPublic

* Fix thumbnails in podcast feeds

* Requested changes based on review

* Provide podcast rss 2.0 only on video channels

* Misc cleanup for a less messy PR

* Lint fixes

* Remove pfeed-podcast

* Add peertube version to new hooks

* Don't use query include, remove TODO

* Remove film medium hack

* Clear podcast rss cache before video/channel update hooks

* Clear podcast rss cache before video uploaded/deleted hooks

* Refactor podcast feed cache clearing

* Set correct person name from video channel

* Styling

* Fix tests

---------

Co-authored-by: Chocobozzz <me@florianbigard.com>
2023-05-22 16:00:05 +02:00
Chocobozzz 1772b383de Add peertube runner cli 2023-05-09 08:57:34 +02:00
Chocobozzz c50bb3c561 Remove unmaintained help script 2023-05-09 08:57:34 +02:00
Chocobozzz 2fe978744e Remove transcoding scripts
We don't have enough energy to maintain them
2023-05-09 08:57:34 +02:00
Chocobozzz ae0e4fdbd1
Bumped to version v5.1.0 2023-03-21 09:11:02 +01:00
Chocobozzz 1bc7aaf47f
Bumped to version v5.1.0-rc.1 2023-02-28 11:18:32 +01:00
Chocobozzz 2be1fecc99
Upgrade sequelize dependency 2023-02-27 09:53:02 +01:00
Chocobozzz d41f4a6dc6
Improve image test comparison 2023-02-24 15:10:13 +01:00
Chocobozzz 82e3ae1523
Fix CI 2023-02-17 09:21:11 +01:00
Chocobozzz 182082f52d
Update bullmq
Requires redis >= 6.2
2023-02-16 14:10:11 +01:00
Chocobozzz e65ef81cf5
Update server dependencies 2023-02-16 11:56:58 +01:00
Chocobozzz 57f255bebc
Fix types build
We use server tsc to build client types so it causes issues with client
typescript version
2023-01-09 15:01:17 +01:00
Chocobozzz bde404995e
Fix build 2023-01-09 11:44:08 +01:00
Chocobozzz 2cb9f8b9c7
Bumped to version v5.0.1 2023-01-05 10:26:56 +01:00
Chocobozzz ce502d09a9
Bumped to version v5.0.0 2022-12-13 08:32:14 +01:00
Chocobozzz 495c81400d
Bumped to version v5.0.0-rc.1 2022-11-29 08:44:30 +01:00
Chocobozzz f7b00376a8
Update socket io 2022-11-28 09:16:34 +01:00
Chocobozzz 77239b425a
Merge branch 'release/4.3.0' into develop 2022-11-17 08:58:33 +01:00
Chocobozzz b961118540
Bumped to version v4.3.1 2022-11-17 08:48:47 +01:00
Chocobozzz 077600e037
Upgrade server dependencies 2022-11-15 13:48:51 +01:00
kontrollanten 564b9b5597
refactor(server): redis > ioredis (#5371)
* refactor(server): redis > ioredis

* refactor(JobQueue): reuse redis connection builder

* fix(redisio)

* fix(redis): setValue

* feat(redis): showFriendlyErrorStack

* feat(redis): auto pipelining

308017a6b9/README.md (autopipelining)

* dont use autopipelining for bullmq

* ioredis events
2022-11-14 16:26:20 +01:00
Chocobozzz 9ab330b90d Use private ACL for private videos in s3 2022-10-24 14:48:24 +02:00
Chocobozzz 3545e72c68 Put private videos under a specific subdirectory 2022-10-24 14:48:24 +02:00
Chocobozzz 56f4783075
Support two factor authentication in backend 2022-10-07 10:51:16 +02:00
Chocobozzz 31ee48d579
Fix error when running setup:cli 2022-09-28 14:55:15 +02:00
Chocobozzz 4a9de87793
Bumped to version v4.3.0 2022-09-21 11:30:14 +02:00
Chocobozzz e1e823faaa
Bumped to version v4.3.0-rc.1 2022-09-12 10:48:46 +02:00
Chocobozzz 3d2e4f0371
Update server dependencies 2022-09-09 11:11:52 +02:00
Chocobozzz fd3c2e8705
Add playback metric endpoint sent to OTEL 2022-08-16 10:33:27 +02:00
Chocobozzz 5a921e7b74
Move to bullmq 2022-08-09 09:18:07 +02:00
Chocobozzz 7a9e420a02
Remove uneeded async 2022-08-02 14:41:44 +02:00
Chocobozzz 1bb4c9ab2e
Add ability to delete a specific video file 2022-08-01 14:55:10 +02:00
Chocobozzz ce6b3765a2
Add more tracing 2022-07-28 10:56:05 +02:00
Chocobozzz b4890f2149
Upgrade server dependencies 2022-07-27 14:56:08 +02:00
Chocobozzz ba2684cedd
Fix lint 2022-07-13 12:14:26 +02:00
Chocobozzz 5ed6bc77ca
Upgrade server dependencies 2022-07-13 09:58:34 +02:00
Chocobozzz 5d7cb63ede
Add compat with openssl 3 2022-07-12 12:08:12 +02:00
Chocobozzz 630d0a1bf5
Introduce experimental telemetry 2022-07-06 15:13:55 +02:00
Chocobozzz 031bbcd25d
Upgrade oauth2-server 2022-06-28 15:32:57 +02:00
Chocobozzz a2fb5fb8b1
Merge branch 'release/4.2.0' into develop 2022-06-28 14:37:30 +02:00
Chocobozzz 28368da087
Bumped to version v4.2.2 2022-06-28 14:31:07 +02:00
Chocobozzz c53853ca1b
Introduce worker threads to process remote images 2022-06-24 16:31:48 +02:00
Chocobozzz 35540f3806
Upgrade server dependencies 2022-06-21 11:10:58 +02:00
Chocobozzz 50341c8fe9
Upgrade critital server dependencies 2022-06-21 11:07:00 +02:00
Chocobozzz ae64a66ac5
Upgrade vulnerable dependencies 2022-06-21 10:59:46 +02:00
Chocobozzz 3471c631ad
Bumped to version v4.2.1 2022-06-21 09:14:42 +02:00
Chocobozzz 8b126121da
Bumped to version v4.2.0 2022-06-07 09:52:42 +02:00
Chocobozzz 2e94ddee41
Bumped to version v4.2.0-rc.1 2022-05-23 08:29:22 +02:00
Chocobozzz f4120aea7a
Fix video upload with some characters in filename 2022-05-17 14:54:54 +02:00
Chocobozzz f15f20a5be
Upgrade http signature
To support hs2019 implementation of other federated softwaare that use
rsa256 instead of rsa512
2022-05-10 11:49:21 +02:00
Chocobozzz ca3d5912e1
Add use proxy for s3 (#4973)
* Fix object storage to be accessible via proxy

* fix lint

* Use hpagent

* Fix lint

* Fix PR

Co-authored-by: noellabo <noel.yoshiba@gmail.com>
2022-05-09 11:49:25 +02:00
kontrollanten 156cdbac22
object-storage: @aws-sdk/lib-storage for multipart (#4903)
* object-storage: @aws-sdk/lib-storage for multipart

* gitignore: add .DS_Store

* test(object-storage): remove only

* test(object-storage/multipart): generate video

* fix lint issue

* test(obj-storage/video): ensure file size

* Styling

Co-authored-by: Chocobozzz <me@florianbigard.com>
2022-04-19 15:22:18 +02:00
Chocobozzz 77d0ae7b20
Fix 2 hours limit on uploads 2022-04-19 11:44:03 +02:00
Chocobozzz 322ef8bb9c
Upgrade server dependencies 2022-04-15 10:56:46 +02:00
Chocobozzz b211106695 Support video views/viewers stats in server
* Add "currentTime" and "event" body params to view endpoint
 * Merge watching and view endpoints
 * Introduce WatchAction AP activity
 * Add tables to store viewer information of local videos
 * Add endpoints to fetch video views/viewers stats of local videos
 * Refactor views/viewers handlers
 * Support "views" and "viewers" counters for both VOD and live videos
2022-04-15 09:49:35 +02:00
Chocobozzz 1575be6825
Upgrade server dependencies 2022-04-08 14:32:31 +02:00
Chocobozzz f4110e0a72
Update server dependencies 2022-03-21 14:20:47 +01:00
Chocobozzz 70f3012acd
Bumped to version v4.1.1 2022-03-08 10:41:59 +01:00
Chocobozzz 57a8d07ed4
Bumped to version v4.1.0 2022-02-23 09:31:11 +01:00
Chocobozzz adc61ac6a3
Bumped to version v4.1.0-rc.1 2022-02-10 09:57:09 +01:00
Chocobozzz 119418b9d9
Upgrade lru-cache
The bug has been fixed on their side
2022-02-10 09:05:44 +01:00
Chocobozzz 9d8ef212ff
Fix broken dep 2022-02-09 17:48:15 +01:00
Chocobozzz c9f27d9881
Update server dependencies 2022-02-09 13:51:24 +01:00
Chocobozzz 0e00f894c6
Fix xml feed validity 2022-02-09 11:54:18 +01:00
Chocobozzz 4393b2552c
Move to peertube feed fork 2022-02-07 09:38:24 +01:00
Chocobozzz ebe4b3df5c
Remove fixed workaround 2022-01-21 11:11:03 +01:00
Chocobozzz 2f0a0ae217
Upgrade uploadx dependency 2022-01-18 10:47:26 +01:00
Chocobozzz c0d2eac332
Update server dependencies 2022-01-14 14:15:23 +01:00
Chocobozzz f8360396ff
Stop using tsconfig register
Prefer to replace paths at compile time
2022-01-06 08:26:14 +01:00
Chocobozzz 9270bd3a7e
Migrate to --enable-source-maps option 2022-01-03 16:37:16 +01:00
Chocobozzz 81d02aac56
Fix types package dependencies 2021-12-28 15:10:55 +01:00
Chocobozzz 6627dbc957
Move types package in packages/ 2021-12-28 10:18:15 +01:00
Chocobozzz fafc7c3cbd
Update resolve tspaths and enabled CI for types 2021-12-28 08:37:05 +01:00
Chocobozzz e5d91a9b9c
Upgrade redis dep 2021-12-24 14:52:24 +01:00
Chocobozzz 2d28b0c21d
Upgrade fast xml parser dep 2021-12-24 13:55:47 +01:00
Chocobozzz b969539c83
Fix types dist paths 2021-12-24 13:28:33 +01:00
Chocobozzz 9b3294a849 Fix some build scripts and lint 2021-12-16 10:08:43 +01:00
lutangar b8fa3e8c73 refactor(types): create dedicated folder for types package src
fix guide examples and add types package readme

refactor(tsconfig): move back base tsconfig to base directory
2021-12-16 10:08:43 +01:00
lutangar 8b03e2ce1a feat(types): create peertube-types package
Create dedicated Typescript "types" configuration file for each "projects".
Create a types folder which includes every useful ts definition:
- client
- server
- shared
Add tooling to create a proper package, extract dependencies, etc...
Add CI Github task.

refactor(types): publish types package in release script
2021-12-16 10:08:43 +01:00
Chocobozzz fa66c9a601
Merge branch 'release/4.0.0' into develop 2021-12-13 10:22:05 +01:00
Chocobozzz 37a5d6318b
Bumped to version v4.0.0 2021-12-13 09:22:25 +01:00
Chocobozzz 2d4822e3af
Upgrade multer 2021-12-08 08:58:52 +01:00
Chocobozzz ca87d95bcb
Fix plugin upgrade
Correctly decache all plugin paths
2021-12-03 10:14:17 +01:00
Chocobozzz 7e587ea46d
Bumped to version v4.0.0-rc.1 2021-11-30 09:01:37 +01:00
Chocobozzz 7fc1012ffd
Update server depencencies 2021-11-10 15:30:44 +01:00
kontrollanten e1ab52d7ec
Add migrate-to-object-storage script (#4481)
* add migrate-to-object-storage-script

closes #4467

* add migrate-to-unique-playlist-filenames script

* fix(migrate-to-unique-playlist-filenames): update master/segments256

run updateMasterHLSPlaylist and updateSha256VODSegments after
file rename.

* Improve move to object storage scripts

* PR remarks

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-11-09 11:05:35 +01:00
Chocobozzz 605450a6f7
Fix dependency security alert 2021-10-26 16:04:09 +02:00
Chocobozzz 5842a85424
Remove unnecessary package
https://github.com/standard/standard/issues/1316
2021-10-26 15:08:58 +02:00
Chocobozzz d91a0341a7
Add missing dependency proxy-addr 2021-10-26 09:57:38 +02:00
Chocobozzz 62549e6c98
Rewrite youtube-dl import
Use python3 binary
Allows to use a custom youtube-dl release URL
Allows to use yt-dlp (youtube-dl fork)
Remove proxy config from configuration to use HTTP_PROXY and HTTPS_PROXY
env variables
2021-10-22 10:25:24 +02:00
Chocobozzz 05c0535df7
Update server dependencies 2021-10-12 14:31:59 +02:00
Chocobozzz d00e9c54f3
Merge branch 'next' into develop 2021-10-11 09:37:30 +02:00
Chocobozzz eecd7ac359
Bumped to version v3.4.1 2021-10-11 09:37:27 +02:00