Commit Graph

60 Commits

Author SHA1 Message Date
Chocobozzz 8f35e76928
Don't display "null%" progress 2025-02-20 10:52:53 +01:00
Chocobozzz 4682e2c85a
Update runner version to 0.1.0 2025-02-20 10:52:42 +01:00
Chocobozzz b9a53addc9
Add list-jobs runner command 2025-02-20 10:31:04 +01:00
Chocobozzz 0392af3bb8
Prevent exception when sending latest chunks 2025-02-20 08:46:35 +01:00
Chocobozzz ac0bf90758
Fix live ending when using remote runners 2025-02-11 14:51:00 +01:00
Chocobozzz 66023f9c98
Fix runner files watch 2025-02-11 11:38:17 +01:00
Chocobozzz 0130fb4664
Node 20 required for PeerTube tools too 2025-02-05 08:52:24 +01:00
Chocobozzz cfd8c3c40e
Update server dependencies
Breaking: rename parse-log script option `-nt` to `-n`
2025-02-03 10:19:11 +01:00
Chocobozzz 40ac4ce3da
Fix redundancy CLI 2025-01-31 14:04:22 +01:00
Chocobozzz 05733a859c
Update runner version 2025-01-14 10:31:27 +01:00
Chocobozzz 5b4c7fc20d
Add custom upload ability for runners 2025-01-09 10:29:16 +01:00
Chocobozzz 33a68f74dd
Implement graceful shutdown in runner 2025-01-09 10:29:16 +01:00
Chocobozzz c7a7e49dd0
CLI dep maintenance upgrade 2024-11-25 10:07:29 +01:00
Chocobozzz 29307ae414
Runner dep maintenance upgrade 2024-11-25 10:07:25 +01:00
Chocobozzz 6e2f86e1b3
Add more log when refusing to process another job 2024-11-07 16:50:57 +01:00
Chocobozzz 20dbdaaeea
Update peertube runner version 2024-11-07 10:07:17 +01:00
Chocobozzz 6b2dc0b444
Fix runner requesting jobs 2024-11-05 09:11:54 +01:00
Chocobozzz a91bd80087
Add ability to filter requested runner jobs 2024-10-31 10:22:35 +01:00
Chocobozzz 018a675ac2
Update peertube-runner version 2024-09-18 15:19:54 +02:00
Chocobozzz 75e7ded018
Merge branch 'release/6.2.0' into develop 2024-08-06 11:03:54 +02:00
Chocobozzz 6fc73f4972
Update runner version 2024-08-06 11:03:39 +02:00
Chocobozzz c5255e784c
Fix concurrency issues when sendin chunks 2024-08-06 10:57:19 +02:00
Chocobozzz 51b5b96286
Merge branch 'release/6.2.0' into develop 2024-08-06 08:37:30 +02:00
Chocobozzz 0ac25c0531
Update runner version 2024-08-06 08:37:12 +02:00
Chocobozzz 0a87497a11
Merge branch 'release/6.2.0' into develop 2024-08-05 16:34:58 +02:00
Chocobozzz e7b9311e92
Don't send m3u8 containing non existing chunks 2024-08-05 16:32:55 +02:00
Chocobozzz 816f346a60 Separate HLS audio and video streams
Allows:
  * The HLS player to propose an "Audio only" resolution
  * The live to output an "Audio only" resolution
  * The live to ingest and output an "Audio only" stream

 This feature is under a config for VOD videos and is enabled by default for lives

 In the future we can imagine:
  * To propose multiple audio streams for a specific video
  * To ingest an audio only VOD and just output an audio only "video"
    (the player would play the audio file and PeerTube would not
    generate additional resolutions)

This commit introduce a new way to download videos:
 * Add "/download/videos/generate/:videoId" endpoint where PeerTube can
   mux an audio only and a video only file to a mp4 container
 * The download client modal introduces a new default panel where the
   user can choose resolutions it wants to download
2024-08-05 08:31:59 +02:00
Chocobozzz e8ac84f1b3
Update runner version
Includes VAD filter for transcription in whisper-ctranslate2
2024-07-03 15:23:32 +02:00
Chocobozzz a7fc7c97a6
Update peertube runner version
Includes transcription
2024-07-01 11:30:13 +02:00
Chocobozzz b66963fe6f
Runner can choose job type 2024-06-28 08:44:59 +02:00
Chocobozzz 1bfb791e05
Integrate transcription in PeerTube 2024-06-28 08:44:58 +02:00
lutangar ef14cf4a5c
feat(transcription): groundwork
chore: fiddling around some more

chore: add ctranslate2 and timestamped

chore: add performance markers

chore: refactor test

chore: change worflow name

chore: ensure Python3

chore(duration): convert to chai/mocha syntahx

chore(transcription): add individual tests for others transcribers

chore(transcription): implement formats test of all implementations

Also compare result of other implementation to the reference implementation

chore(transcription): add more test case with other language and models size and local model

chore(test): wip ctranslate 2 adapat

chore(transcription): wip transcript file and benchmark

chore(test): clean a bit

chore(test): clean a bit

chore(test): refacto timestamed spec

chore(test): update workflow

chore(test): fix glob expansion with sh

chore(test): extract some hw info

chore(test): fix async tests

chore(benchmark): add model info

feat(transcription): allow use of a local mode in timestamped-whisper

feat(transcription): extract run and profiling info in own value object

feat(transcription): extract run concept in own class an run more bench

chore(transcription): somplify run object only a uuid is now needed and add more benchmark scenario

docs(transcription): creates own package readme

docs(transcription): add local model usage

docs(transcription): update README

fix(transcription): use fr video for better comparison

chore(transcription): make openai comparison passed

docs(timestamped): clea

chore(transcription): change transcribers transcribe method signature

Introduce whisper builtin model.

fix(transcription): activate language detection

Forbid transcript creation without a language.
Add `languageDetection` flag to an engine and some assertions.

Fix an issue in `whisper-ctranslate2` :
https://github.com/Softcatala/whisper-ctranslate2/pull/93

chore(transcription): use PeerTube time helpers instead of custom ones

Update existing time function to output an integer number of seconds and add a ms human-readable time formatter with hints of tests.

chore(transcription): use PeerTube UUID helpers

chore(transcription): enable CER evaluation

Thanks to this recent fix in Jiwer <3
https://github.com/jitsi/jiwer/issues/873

chore(jiwer): creates JiWer package

I'm not very happy with the TranscriptFileEvaluator constructor... suggestions ?

chore(JiWer): add usage in README

docs(jiwer): update JiWer readme

chore(transcription): use FunMOOC video in fixtures

chore(transcription): add proper english video fixture

chore(transcription): use os tmp directory where relevant

chore(transcription): fix jiwer cli test reference.txt

chore(transcription): move benchmark out of tests

chore(transcription): remove transcription workflow

docs(transcription): add benchmark info

fix(transcription): use ms precision in other transcribers

chore(transcription): simplify most of the tests

chore(transcription): remove slashes when building path with join

chore(transcription): make fromPath method async

chore(transcription): assert path to model is a directory for CTranslate2 transcriber

chore(transcription): ctranslate2 assertion

chore(transcription): ctranslate2 assertion

chore(transcription): add preinstall script for Python dependencies

chore(transcription): add download and unzip utils functions

chore(transcription): add download and unzip utils functions

chore(transcription): download & unzip models fixtures

chore(transcription): zip

chore(transcription): raise download file test timeout

chore(transcription): simplify download file test

chore(transcription): add transcriptions test to CI

chore(transcription): raise test preconditions timeout

chore(transcription): run preinstall scripts before running ci

chore(transcription): create dedicated tmp folder for transcriber tests

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): use short video for local model test

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): raise timeout some more

chore(transcription): setup verbosity based on NODE_ENV value
2024-06-28 08:43:40 +02:00
Chocobozzz 4f4d3adf73
Update apps dependencies 2024-06-21 14:49:17 +02:00
Chocobozzz 29329d6c45 Implement auto tag on comments and videos
* Comments and videos can be automatically tagged using core rules or
   watched word lists
 * These tags can be used to automatically filter videos and comments
 * Introduce a new video comment policy where comments must be approved
   first
 * Comments may have to be approved if the user auto block them using
   core rules or watched word lists
 * Implement FEP-5624 to federate reply control policies
2024-05-29 15:03:14 +02:00
Chocobozzz 3bfecf4890
Update runner version 2024-04-04 16:33:06 +02:00
Chocobozzz 93b09bf891
Fix stuck runner 2024-04-04 16:31:01 +02:00
Chocobozzz faabe996ba
Update runner version 2024-04-03 14:25:14 +02:00
Chocobozzz 17fb4fd6d0
Update runner version 2024-04-03 09:17:58 +02:00
Chocobozzz 0794fe2ac1
Fix runner ffmpeg logger 2024-04-03 09:17:45 +02:00
Chocobozzz 121efedde2
Update peertube-runner version 2024-03-29 15:04:44 +01:00
Chocobozzz 33607e3268
Add ping debug in peertube-runner 2024-03-29 15:04:03 +01:00
Chocobozzz 128748e6e4
Update peertube-runner version 2024-03-19 09:59:47 +01:00
Chocobozzz c09e27d77a
Optimize transcoding profile building 2024-03-19 09:53:59 +01:00
Chocobozzz 4e98d843da
Success on update "not in processing state" error
Or the job is never "ended"
2024-03-19 09:26:40 +01:00
Chocobozzz c727a34cb6
Prevent aborting another live session 2024-03-18 16:09:22 +01:00
Chocobozzz 1a8b20ba30
Less verbose on expected error 2024-03-18 11:28:43 +01:00
Chocobozzz 54a7183b11
Check available jobs on reconnection 2024-03-08 10:29:41 +01:00
Chocobozzz 473840e890
Update apps versions 2024-02-23 15:49:09 +01:00
Chocobozzz ed77d65699
Upgrade peertube runner dependencies 2024-02-23 15:26:08 +01:00
Chocobozzz 1e17dece73
Update peertube-cli dependencies 2024-02-23 15:23:53 +01:00