Commit Graph

103 Commits

Author SHA1 Message Date
Jerop Kipruto 4421ef126f
Refactor: Use telemetry_utils.js in local_telemetry.js (#1066)
## TLDR

Refactors `scripts/local_telemetry.js` to use shared helper functions and constants from `scripts/telemetry_utils.js`.

## Dive Deeper

This change centralizes common telemetry-related logic, including:
- Binary downloading and management (`ensureBinary`)
- Reading and writing JSON files
- Waiting for network ports
- Managing workspace telemetry settings (`manageTelemetrySettings`)
- Process cleanup and signal handling (`registerCleanup`)

By using the shared utilities, `local_telemetry.js` becomes more concise and focused on its specific task of setting up the local OpenTelemetry and Jaeger environment.

## Docs

https://github.com/google-gemini/gemini-cli/blob/main/docs/core/telemetry.md#local-telemetry-with-jaeger-ui-for-traces

## Issue

#750
2025-06-15 19:32:12 +00:00
Billy Biggs e717c51aa1
Avoid import.meta.dirname to be backwards compatible to Node.js 18+ (#1058) 2025-06-15 05:30:00 +00:00
Jerop Kipruto 53753f0455
Add telemetry command and refactor telemetry settings (#1060)
#750 

### Telemetry Settings
Refactors telemetry configuration to use a nested `telemetry` object in `settings.json`, for example:

```json
{
  "telemetry": {
    "enabled": true,
    "target": "gcp"
    "log-prompts": "true"
  },
  "sandbox": false
}
```

The above includes
- Centralized telemetry settings under a `telemetry` object in `settings.json`.
- CLI flags for the `gemini` command to override all telemetry sub-settings:
    - `--telemetry` / `--no-telemetry`
    - `--telemetry-target <local|gcp>`
    - `--telemetry-otlp-endpoint <URL>`
    - `--telemetry-log-prompts` / `--no-telemetry-log-prompts`
- Updates `packages/cli/src/config/config.ts` and `packages/core/src/config/config.ts` to read from the new settings structure and respect the new CLI flags.
- Modifies `scripts/handle-telemetry.js`, `scripts/local_telemetry.js`, and `scripts/telemetry_utils.js` to align with the new settings structure.
- Updates `docs/core/telemetry.md` to reflect the new settings structure, CLI flags, and order of precedence.
- Renames `logUserPromptsEnabled` to `logPrompts` for brevity.

### `npm run telemetry`

Add a new `npm run telemetry` command that uses `scripts/telemetry.js`, automates the entire process of setting up a local and GCP telemetry pipelines, including configuring the necessary settings in the `.gemini/settings.json` workspace file and installing required binaries (e.g. `otelcol-contrib`).

---
```shell
$ npm run telemetry -- --target=gcp

> gemini-cli@0.1.0 telemetry
> node scripts/telemetry.js --target=gcp

⚙️  Using command-line target: gcp
🚀 Running telemetry script for target: gcp.
 Starting Local Telemetry Exporter for Google Cloud 
⚙️  Enabled telemetry in workspace settings.
🔧 Set telemetry OTLP endpoint to http://localhost:4317.
🎯 Set telemetry target to gcp.
 Workspace settings updated.
 Using Google Cloud Project ID: foo-bar

🔑 Please ensure you are authenticated with Google Cloud:
  - Run `gcloud auth application-default login` OR ensure `GOOGLE_APPLICATION_CREDENTIALS` environment variable points to a valid service account key.
  - The account needs "Cloud Trace Agent", "Monitoring Metric Writer", and "Logs Writer" roles.
 otelcol-contrib already exists at /Users/jerop/github/gemini-cli/.gemini/otel/bin/otelcol-contrib
🧹 Cleaning up old processes and logs...
 Deleted old GCP collector log.
📄 Wrote OTEL collector config to /Users/jerop/github/gemini-cli/.gemini/otel/collector-gcp.yaml
🚀 Starting OTEL collector for GCP... Logs: /Users/jerop/github/gemini-cli/.gemini/otel/collector-gcp.log
 Waiting for OTEL collector to start (PID: 17013)...
 OTEL collector started successfully on port 4317.

 Local OTEL collector for GCP is running.

🚀 To send telemetry, run the Gemini CLI in a separate terminal window.

📄 Collector logs are being written to: /Users/jerop/github/gemini-cli/.gemini/otel/collector-gcp.log

📊 View your telemetry data in Google Cloud Console:
   - Logs: https://console.cloud.google.com/logs/query;query=logName%3D%22projects%2Ffoo-bar%2Flogs%2Fgemini_cli%22?project=foo-bar
   - Metrics: https://console.cloud.google.com/monitoring/metrics-explorer?project=foo-bar
   - Traces: https://console.cloud.google.com/traces/list?project=foo-bar

Press Ctrl+C to exit.
^C
👋 Shutting down...
⚙️  Disabled telemetry in workspace settings.
🔧 Cleared telemetry OTLP endpoint.
🎯 Cleared telemetry target.
 Workspace settings updated.
🛑 Stopping otelcol-contrib (PID: 17013)...
 otelcol-contrib stopped.
```
2025-06-15 00:47:32 -04:00
Jerop Kipruto 3bcb3c3666
docs: clarify telemetry script usage (#1034)
Updates the telemetry documentation and the GCP telemetry script to make it clearer that the Gemini CLI needs to be run in a separate terminal session after starting the collector script.

This addresses potential user confusion where they might expect telemetry data to appear without actively using the CLI.

#750
2025-06-14 14:49:21 +00:00
Jerop Kipruto 1452bb4ca4
Add GCP telemetry script (#1033)
Adds a script -  `scripts/telemetry_gcp.js` - to simplify setting up a local OpenTelemetry collector that forwards data to Google Cloud. This is a follow up to the script for local telemetry `scripts/local_telemetry.js` added in #1015.

This script automates downloading necessary binaries, configuring the collector, and updating workspace settings.

Also includes `scripts/telemetry_utils.js` with shared helper functions for telemetry scripts. Will refactor `scripts/local_t elemetry.js` in next steps to use this shared functionality.

Updates `docs/core/telemetry.md` to include:
- A new "Quick Start" section
- Detailed instructions for the new GCP automated script
- Reorganization of existing sections for clarity

#750 

---
```
 Starting Local Telemetry Exporter for Google Cloud 
⚙️  Enabled telemetry in workspace settings.
🔧 Set telemetry OTLP endpoint to http://localhost:4317.
 Workspace settings updated.
 Using Google Cloud Project ID: foo-bar

🔑 Please ensure you are authenticated with Google Cloud:
  - Run `gcloud auth application-default login` OR ensure `GOOGLE_APPLICATION_CREDENTIALS` environment variable points to a valid service account key.
  - The account needs "Cloud Trace Agent", "Monitoring Metric Writer", and "Logs Writer" roles.
 otelcol-contrib already exists at /Users/jerop/github/gemini-cli/.gemini/otel/bin/otelcol-contrib
🧹 Cleaning up old processes and logs...
 Deleted old GCP collector log.
📄 Wrote OTEL collector config to /Users/jerop/github/gemini-cli/.gemini/otel/collector-gcp.yaml
🚀 Starting OTEL collector for GCP... Logs: /Users/jerop/github/gemini-cli/.gemini/otel/collector-gcp.log
 Waiting for OTEL collector to start (PID: 65145)...
 OTEL collector started successfully on port 4317.

 Local OTEL collector for GCP is running.

📄 Collector logs are being written to: /Users/jerop/github/gemini-cli/.gemini/otel/collector-gcp.log

📊 View your telemetry data in Google Cloud Console:
   - Traces: https://console.cloud.google.com/traces/list?project=foo-bar
   - Metrics: https://console.cloud.google.com/monitoring/metrics-explorer?project=foo-bar
   - Logs: https://console.cloud.google.com/logs/query;query=logName%3D%22projects%2Ffoo-bar%2Flogs%2Fgemini_cli%22?project=foo-bar

Press Ctrl+C to exit.
^C
👋 Shutting down...
⚙️ Disabled telemetry in workspace settings.
🔧 Cleared telemetry OTLP endpoint.
 Workspace settings updated.
🛑 Stopping otelcol-contrib (PID: 65145)...
 otelcol-contrib stopped.
```
2025-06-13 23:28:18 -04:00
Keir Mierle 9954779739
Add a local telemetry launcher (#1015) 2025-06-13 18:08:03 -07:00
Olcan f75c48323c
fixes to proxy on macos: prevent curl from hanging during wait-for-proxy by adding ipv6 support and timeout (#947) 2025-06-11 11:31:38 -07:00
Zach Sais 5bab5a7378
fix version set for cli and slash commands (#892) 2025-06-11 08:25:26 -05:00
Tommaso Sciortino 97e08fc804
Remove warnings for old env var names. (#920) 2025-06-10 18:34:36 -07:00
Seth Troisi 6f4226fde1 Fix debug to work when not running under a sandbox 2025-06-10 13:33:18 -07:00
Olcan e38d2078cc
restricted networking for all sandboxing methods, new seatbelt profiles, updated docs, fixes to sandbox build, debugging through sandbox (#891) 2025-06-10 08:58:37 -07:00
Olcan 749c010a0d
strip json comments in newly created sandbox_command.js (consistent w/ bash version) (#886) 2025-06-10 00:33:42 -07:00
matt korwel 3b943c1582
Windows: Refactor Shell Scripts to Node.js for Cross-Platform Compatibility (#784) 2025-06-09 12:19:42 -07:00
Olcan a3d11e8fef
replace reference to "README" with "docs" to avoid confusion for folks who do not have access to README (#866) 2025-06-08 19:19:33 -07:00
matt korwel 584286cfd9
fix(deps): externalize tree-sitter (#840)
Submitting without approval to fix broken deployment on main. But also, we should lock this down.
2025-06-08 02:05:55 -07:00
Keith Ballinger 84678c6448
Makefile for convenience (#833) 2025-06-07 22:22:32 -07:00
cperry-goog 18d6a11c04
refactor: rename gemini-code to gemini-cli (#822) 2025-06-07 14:27:22 -07:00
cperry-goog 51cd5ffd91
fix(build): correct sandbox warning link (#819) 2025-06-07 12:42:32 -07:00
Bryan Morgan 28ff62e7b1
Added /mcp command support and cleaned up broken tests (#817) 2025-06-07 15:06:18 -04:00
Olcan d3e43437a0
use execSync (vs spawnSync) so launch fails if build_sandbox fails; tweaks in build_sandbox to fix some shellcheck warnings, and to simplify the logic slightly (#767) 2025-06-05 13:02:56 -07:00
Tolik Malibroda 4d4cf0f2f9
feat: Add multi-stage docker build support for custom sandbox.Dockerfile (#746) 2025-06-05 17:46:54 +02:00
Olcan 8365c8f954
prefer to load gemini-specific .env file from .gemini folder when it exists there (#697) 2025-06-02 14:16:48 -07:00
Taylor Mullen 190e6be800 Fix: Update sandbox build to use new core package name
- The `packages/core` tarball name changed from `gemini-code-server-*.tgz` to `gemini-code-core-*.tgz` after the `server` to `core` rename.
- This updates `scripts/build_sandbox.sh` and the root `Dockerfile` to use the new `gemini-code-core-*.tgz` naming, resolving the CI failure during the Docker build step of the publish process.
2025-05-30 22:41:54 -07:00
Tommaso Sciortino 21fba832d1
Rename server->core (#638) 2025-05-30 18:25:47 -07:00
Brandon Keiji 31a7affb74
feat(sandbox): use package config to dictate sandbox image name (#624) 2025-05-30 19:28:46 +00:00
Brandon Keiji 816cc08a8f
fix(sandbox): set default env var to 'gemini-cli-sandbox' (#620) 2025-05-30 18:02:27 +00:00
Olcan 094b9dc474
use npx json instead of jq (#617) 2025-05-30 01:12:36 -07:00
Brandon Keiji 7468d3cddf
fix(sandbox): add SHORT_SHA to image tag (#604) 2025-05-29 23:43:20 +00:00
Olcan 1c066548b4
allow comments in settings when parsed in scripts via jq (#603) 2025-05-29 16:25:16 -07:00
Brandon Keiji dc94a03f39
feat: publish root Dockerfile to our image registry (#599) 2025-05-29 14:01:44 -07:00
DeWitt Clinton 27a773d5b2
Display git commit info in the /about section. (#567)
This change detects the most recent git commit short hash and writes it to the `GIT_COMMIT_INFO` constant in `packages/cli/src/generated/git-commit.sh`, optionally appending the string "(local modifications)" if additional local changes after that commit are detected.

If set, this string is displayed in the `/about` dialog as well as passed into the `/bug` template.

Example:

```
> /about

╭───────────────────────────────────────────────────────────────────────────╮
│                                                                           │
│ About Gemini CLI                                                          │
│                                                                           │
│ CLI Version               development                                     │
│ Git Commit                43370ab (local modifications)                   │
│ Model                     gemini-2.5-pro-preview-05-06                    │
│ Sandbox                   sandbox-exec (minimal)                          │
│ OS                        darwin v23.11.0                                 │
│                                                                           │
╰───────────────────────────────────────────────────────────────────────────╯
```

Additionally, this change updates `.gitignore` to ignore the generated files, `scripts/clean.sh` to remove them, and adds a `npm run generate` stage for this and any other generators we need to write.
2025-05-28 00:04:26 -07:00
Olcan 9595e98db8
replace error with warning if sandbox build is triggered without enabling, improve README to reduce confusion (#570) 2025-05-27 10:55:07 -07:00
Taylor Mullen 8b8fa6c1ae Refactor: Convert copy_files.cjs to ES module syntax
- Converted scripts/copy_files.cjs to use ES module syntax (renaming to copy_files.js).
- This change aligns with the project's preference for ES modules over CommonJS for better modernity and future-proofing.
- Updated eslint.config.js to remove the .cjs override.
- Adjusted scripts/build_package.sh to call the new .js file.
2025-05-18 23:49:48 -07:00
Olcan 76cf5e9fc1
rename env vars GEMINI_CODE_{MODEL,SANDBOX,SANDBOX_IMAGE} (#411) 2025-05-17 17:28:44 -07:00
Taylor Mullen 8af970061e Fix: Use Node.js script for cross-platform file copying in build
- Replaces the rsync command in the build_package.sh script with a
  Node.js script (copy_files.cjs) to handle copying necessary files
  (e.g., .md, .json, .sb) during the build process.
- Addresses an issue where the build would fail on systems
  that do not have rsync installed (e.g., some Windows environments or
  minimal Linux distributions) by using a Node.js script, providing a
  cross-platform solution as Node.js is already a project dependency.
- Updates the ESLint configuration to correctly lint .cjs files as
  CommonJS modules.

Fixes https://github.com/google-gemini/gemini-cli/issues/387
2025-05-16 13:59:07 -07:00
Olcan 1fa40405ea
improve sandboxing status message, update README, remove dead code from a previous change to build_sandbox.sh (#346) 2025-05-14 11:23:06 -07:00
Olcan 4a0f5476c0
fall back to ~/.env if .env is not found in current directory or any ancestors (#338) 2025-05-13 15:36:34 -07:00
Olcan b35a3856a2
fix debugging with seatbelt, including in strict profile (#300) 2025-05-09 08:44:40 -07:00
Taylor Mullen 41b82ce796 Add bundling support.
- This can now be invoked with `npm run bundle`, it creates a `bundle/` folder that has:
  - gemini.js
  - sandbox-macos-minimal.sb
  - sandbox-macos-strict.sb
  - shell.json
  - shell.md

- This doesn't include any sort of automation for auto bundling pieces. It's just the root capability which we can weave into other locations.

Fixes https://b.corp.google.com/issues/411432723
2025-05-08 23:36:42 -07:00
Olcan d524309e3c
use seatbelt on macos, with two profiles: minimal (default) which only restricts writes, and strict, which is deny-by-default and only allows specific operations (#283) 2025-05-07 20:03:29 -07:00
cornmander 95ab38e8d6
Create simple script for setting up a dev environment. (#277) 2025-05-07 16:21:16 -04:00
Olcan 4649026312
make sandbox build quiet by default but allow VERBOSE=1 option. enable caching by default but allow disabling via BUILD_SANDBOX_FLAGS="--no-cache" (#278) 2025-05-07 11:00:48 -07:00
Brandon Keiji ed0b90644a
fix: build image with --no-cache (#275) 2025-05-07 08:18:04 -07:00
Brandon Keiji 739654bb25
fix(sandbox): consolidate dev and prod sandbox (#273) 2025-05-07 07:23:13 -07:00
Olcan ae96b8914e
fix sandbox prod build w/ custom Dockerfile (#255) 2025-05-03 10:07:51 -07:00
Olcan 69d1c644d9
custom sandboxing via sandbox.Dockerfile and sandbox.bashrc in project settings (#249) 2025-05-02 14:07:40 -07:00
Olcan f237082c37
pass PATH and PYTHONPATH into sandbox, let sandbox scripts recognize user settings for sandbox (#247) 2025-05-02 12:04:22 -07:00
Olcan a18eea8c23
remove start_sandbox.sh script (#238) 2025-05-01 09:16:33 -07:00
Olcan e85db8aa3c
drop the "RE" from REBUILD_SANDBOX (#218) 2025-04-29 14:45:11 -07:00
Olcan 825cecc089
SANDBOX_SET_UID_GID option for systems where this is necessary (should be only rootful docker on linux w/o userns-remap configured) (#210)
* SANDBOX_SET_UID_GID option for systems where this is necessary (should be only rootful docker on linux w/o userns-remap configured)

* Merge remote-tracking branch 'origin/main' into sandbox_uid_gid
2025-04-29 08:43:24 -07:00