Commit Graph

52 Commits

Author SHA1 Message Date
Chocobozzz 6def7d34e6
Fix migrations 2021-12-17 14:27:16 +01:00
Chocobozzz f304a1580b
Fix build 2021-12-17 13:58:07 +01:00
Chocobozzz bf54587a3e
shared/ typescript types dir server-commands 2021-12-17 09:29:23 +01:00
lutangar 06aad80165 chore(refactor): remove shared folder dependencies to the server
Many files from the `shared` folder were importing files from the `server` folder.
When attempting to use Typescript project references to describe dependencies,
it highlighted a circular dependency beetween `shared` <-> `server`.

The Typescript project forbid such usages.
Using project references greatly improve performance by rebuilding only
the updated project and not all source files.
> see https://www.typescriptlang.org/docs/handbook/project-references.html
2021-12-16 10:08:43 +01:00
mj-saunders e291096f78
Apply import interval only when reasonable (#4552)
* Apply import interval only when reasonable

When importing videos from another service, an interval can be applied
between each download.
It only really makes sense to apply this interval when the last
attempted download actually happened, and not when it was skipped.

* Fix boolean notation
2021-11-22 16:10:00 +01:00
Chocobozzz 62549e6c98
Rewrite youtube-dl import
Use python3 binary
Allows to use a custom youtube-dl release URL
Allows to use yt-dlp (youtube-dl fork)
Remove proxy config from configuration to use HTTP_PROXY and HTTPS_PROXY
env variables
2021-10-22 10:25:24 +02:00
Chocobozzz 41fb13c330
esModuleInterop to true 2021-08-27 15:12:22 +02:00
Chocobozzz 98ab5dc810
Remove useless async 2021-08-26 10:01:42 +02:00
Chocobozzz 254d3579f5
Use an object to represent a server 2021-07-20 15:27:18 +02:00
Chocobozzz 89d241a79c
Shorter server command names 2021-07-20 15:27:18 +02:00
Chocobozzz d23dd9fbfc
Introduce videos command 2021-07-20 15:27:18 +02:00
Chocobozzz d0a0fa429d
Adapt CLI to new commands 2021-07-20 15:27:18 +02:00
Chocobozzz 078f17e6d9
Fix CLI tools 2021-07-20 15:27:18 +02:00
Chocobozzz 12152aa09f
Update tools dependencies 2021-06-25 17:48:27 +02:00
Chocobozzz 1bcb03a100
Use a class for youtube-dl 2021-05-11 11:32:31 +02:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz d7764e2eb7
Fix import script when using the search index 2021-02-19 15:04:58 +01:00
Chocobozzz ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Florent 82f5527fc3
add option for video-import CLI to wait between two video imports (#3310)
* peertube-import-videos: Add --wait-interval option

* peertube-import-videos: various code style improvements

Co-authored-by: Florent F <florent.fayolle69@gmail.com>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
2021-01-25 15:42:44 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516)
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
2021-01-15 15:56:56 +01:00
Chocobozzz 5721fd8374
Fix channels import with a channel with playlists 2020-12-14 09:55:30 +01:00
Chocobozzz d8794cf855
More robust import script when using since/until 2020-12-14 09:55:30 +01:00
Chocobozzz de29e90c37
Fix CLI import script 2020-11-17 15:28:24 +01:00
Chocobozzz f940e81469
Catch error in import script 2020-04-14 09:41:20 +02:00
Chocobozzz bdd428a6d9
Update dependencies 2020-02-28 16:21:34 +01:00
Raphaël Droz 79ee77eab1 peertube-import-videos passes remaining options down to youtube-dl 2020-02-12 11:33:14 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz 005b51b977
Force node 10 usage 2020-01-28 14:03:12 +01:00
Chocobozzz 4449d2695b
Fix URL normalization in import script 2019-12-30 10:44:50 +01:00
Chocobozzz da69b88638
Auto add https:// in import script URLs 2019-12-30 10:40:09 +01:00
Florent F 71056d0008 Fix hours in peertube-import-videos
Youtube doesn't store the hour of the publish date so
set it to zero in order to compare it.
2019-11-04 17:38:03 +01:00
Chocobozzz bcd4cf0586
Fix remove end slash function 2019-11-04 16:34:58 +01:00
Chocobozzz 2aaa1a3fdc
Use typescript paths in cli scripts too 2019-10-21 17:13:22 +02:00
Frank de Lange bda3b70537 peertube-import-videos.ts: add --tmpdir, --first, --last and --verbose [level] parameters (#2045)
* peertube-import-videos.ts: add --tmpdir <tmpdir> parameter, used to designate working directory for downloading and converting imported videos

* peertube-import-videos.ts: add --first and --last parameters to limit processing of the returned playlist to the first/last N elements

* peertube-import-videos.ts: add --verbose [verbosity] parameter, set this from 0 (only errors are reported) to 4 (for trace debugging), default is 2 (info). When --verbose is used without the optional parameter the logging level is set to 3 (debug). At level 1 (warn) it will only report on successfully uploaded videos (and/or errors), use this when running peertube-import-videos in a cron job to mirror a channel.

* package.json: remove dependency on loglevel

cli.ts: add getLogger(loglevel), to be used in CLI tools, add --verbose to set log level

peertube-import-videos: use getLogger (from cli) instead of loglevel, add error_exit (log error and exit), move --verbose to cli.ts, etc.

* cli.ts: remove superfluous reference to default logging level

* peertube-import-videos: exit_error -> exitError
2019-08-26 11:35:28 +02:00
Florent F d0198ff99f Add since parameter to peertube-import-videos (#1991)
* Add since parameter to peertube-import-videos

* PR remarks + --until <date>
2019-08-01 10:21:55 +02:00
Chocobozzz 8d2be0ed7b WIP plugins: move plugin CLI in peertube script
Install/uninstall/list plugins remotely
2019-07-24 10:58:16 +02:00
Chocobozzz 1205823fec
Add ability to override CLI import attributes 2019-06-13 13:53:28 +02:00
Chocobozzz 1a12f66d63
Add more CLI tests 2019-06-13 11:09:38 +02:00
Chocobozzz 2b4dd7e26d
Fix optional privacy in upload endpoint 2019-04-25 13:55:28 +02:00
Chocobozzz e8a739e880
Fix video import CLI script
Password is an optional argument
2019-04-24 16:51:25 +02:00
Chocobozzz 94565d52bb
Shared utils -> extra-utils
Because they need dev dependencies
2019-04-24 16:25:52 +02:00
Chocobozzz 74dc3bca2b
Don't expose constants directly in initializers/ 2019-04-11 14:26:41 +02:00
Chocobozzz c74c9be934
Refractor published date on video import 2019-02-12 11:47:23 +01:00
Andrés Maldonado 84929846e7
Import original publication date (CLI)
* Import original publication date when importing a video from YouTube using the CLI
* Fix lint issues
2019-02-12 11:19:19 +01:00
Chocobozzz 88108880bb
Merge branch 'develop' into pr/1217 2019-02-11 11:52:34 +01:00
Chocobozzz bb8f7872f5
Fix peertube CLI documentation 2019-01-14 11:01:40 +01:00
HesioZ ab4dbe3657 Remove the eventual trailing '/' at the end of urls (see #1453) (#1480)
remove eventual trailing '/' at the end of urls in import script (see #1453)
2018-12-15 01:43:04 +01:00
buoyantair 9639bd1757 Move utils to /shared
Move utils used by /server/tools/* & /server/tests/**/* into
/shared folder.

Issue: #1336
2018-10-29 22:18:31 +05:30
BRAINS YUM fa27f07637 allow peertube-import-videos.ts CLI script to run concurrently (#1334)
allows running multiple imports at the same time, whereas previously a concurrent instance of the script deleted another processe's file.
2018-10-24 20:07:51 +02:00
Lucas Declercq 7f2cfe3a79 Rename downloadingEnabled property to downloadEnabled 2018-10-08 14:45:22 +02:00