Release and Packaging: Clean up (#3489)

This commit is contained in:
matt korwel 2025-07-07 16:36:51 -07:00 committed by GitHub
parent 4e84989d8f
commit a4097ae6f9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
16 changed files with 35 additions and 463 deletions

View File

@ -57,9 +57,14 @@ steps:
args:
- -c
- |
export GEMINI_SANDBOX_IMAGE_TAG=$$(cat /workspace/image_tag.txt)
echo "Using Docker image tag for publish: $$GEMINI_SANDBOX_IMAGE_TAG"
npm run publish:sandbox
set -e
IMAGE_TAG=$(cat /workspace/image_tag.txt)
BASE_IMAGE_URI=$(npm run -s config get sandboxImageUri)
IMAGE_URI_NO_TAG=${BASE_IMAGE_URI%:*}
FINAL_IMAGE_URI="${IMAGE_URI_NO_TAG}:${IMAGE_TAG}"
echo "Pushing sandbox image: ${FINAL_IMAGE_URI}"
$_CONTAINER_TOOL push "${FINAL_IMAGE_URI}"
env:
- 'GEMINI_SANDBOX=$_CONTAINER_TOOL'

View File

@ -284,7 +284,7 @@ Container-based sandboxing mounts the project directory (and system temp directo
#### Proxied Networking
All sandboxing methods, including MacOS Seatbelt using `*-proxied` profiles, support restricting outbound network traffic through a custom proxy server that can be specified as `GEMINI_SANDBOX_PROXY_COMMAND=<command>`, where `<command>` must start a proxy server that listens on `:::8877` for relevant requests. See `scripts/example-proxy.js` for a minimal proxy that only allows `HTTPS` connections to `example.com:443` (e.g. `curl https://example.com`) and declines all other requests. The proxy is started and stopped automatically alongside the sandbox.
All sandboxing methods, including MacOS Seatbelt using `*-proxied` profiles, support restricting outbound network traffic through a custom proxy server that can be specified as `GEMINI_SANDBOX_PROXY_COMMAND=<command>`, where `<command>` must start a proxy server that listens on `:::8877` for relevant requests. See `docs/examples/proxy-script.md` for a minimal proxy that only allows `HTTPS` connections to `example.com:443` (e.g. `curl https://example.com`) and declines all other requests. The proxy is started and stopped automatically alongside the sandbox.
## Manual Publish

View File

@ -8,8 +8,7 @@ help:
@echo "Usage:"
@echo " make install - Install npm dependencies"
@echo " make build - Build the entire project"
@echo " make build-sandbox - Build the sandbox container"
@echo " make build-all - Build the project and the sandbox"
@echo " make build-all - Build the entire project"
@echo " make test - Run the test suite"
@echo " make lint - Lint the code"
@echo " make format - Format the code"
@ -17,7 +16,7 @@ help:
@echo " make clean - Remove generated files"
@echo " make start - Start the Gemini CLI"
@echo " make debug - Start the Gemini CLI in debug mode"
@echo " make release - Publish a new release"
@echo ""
@echo " make run-npx - Run the CLI using npx (for testing the published package)"
@echo " make create-alias - Create a 'gemini' alias for your shell"
@ -27,8 +26,6 @@ install:
build:
npm run build
build-sandbox:
npm run build:sandbox
build-all:
npm run build:all
@ -54,8 +51,6 @@ start:
debug:
npm run debug
release:
npm run publish:release
run-npx:
npx https://github.com/google-gemini/gemini-cli

View File

@ -103,14 +103,12 @@ There are two distinct build processes used, depending on the distribution chann
**Docker sandbox image**
The Docker-based execution method is supported by the `gemini-cli-sandbox` container image. This image is published to a container registry and contains a pre-installed, global version of Gemini CLI. The `scripts/prepare-cli-packagejson.js` script dynamically injects the URI of this image into the CLI's `package.json` before publishing, so the CLI knows which image to pull when the `--sandbox` flag is used.
The Docker-based execution method is supported by the `gemini-cli-sandbox` container image. This image is published to a container registry and contains a pre-installed, global version of Gemini CLI.
## Release process
A unified script, `npm run publish:release`, orchestrates the release process. The script performs the following actions:
The release process is automated through GitHub Actions. The release workflow performs the following actions:
1. Build the NPM packages using `tsc`.
2. Update the CLI's `package.json` with the Docker image URI.
3. Build and tag the `gemini-cli-sandbox` Docker image.
4. Push the Docker image to the container registry.
5. Publish the NPM packages to the artifact registry.
2. Publish the NPM packages to the artifact registry.
3. Create GitHub releases with bundled assets.

View File

@ -1,3 +1,8 @@
# Example Proxy Script
The following is an example of a proxy script that can be used with the `GEMINI_SANDBOX_PROXY_COMMAND` environment variable. This script only allows `HTTPS` connections to `example.com:443` and declines all other requests.
```javascript
#!/usr/bin/env node
/**
@ -73,3 +78,4 @@ server.listen(PROXY_PORT, () => {
`[PROXY] Allowing HTTPS connections to domains: ${ALLOWED_DOMAINS.join(', ')}`,
);
});
```

View File

@ -183,8 +183,7 @@ This is the most critical stage where files are moved and transformed into their
`bundle` folder is created at the project root to house the final package contents.
1. The `package.json` is Transformed:
- What happens: The package.json from packages/cli/ is read, modified, and written into the root `bundle`/ directory. The
script scripts/prepare-cli-packagejson.js is responsible for this.
- What happens: The package.json from packages/cli/ is read, modified, and written into the root `bundle`/ directory.
- File movement: packages/cli/package.json -> (in-memory transformation) -> `bundle`/package.json
- Why: The final package.json must be different from the one used in development. Key changes include:
- Removing devDependencies.

View File

@ -17,44 +17,33 @@
"sandboxImageUri": "us-docker.pkg.dev/gemini-code-dev/gemini-cli/sandbox:0.1.9"
},
"scripts": {
"start": "node scripts/start.js",
"debug": "cross-env DEBUG=1 node --inspect-brk scripts/start.js",
"generate": "node scripts/generate-git-commit-info.js",
"build": "node scripts/build.js",
"build:sandbox": "node scripts/build_sandbox.js",
"build:all": "npm run build && npm run build:sandbox",
"clean": "node scripts/clean.js",
"prepare": "npm run bundle",
"build:packages": "npm run build --workspaces",
"build:sandbox": "node scripts/build_sandbox.js --skip-npm-install-build",
"bundle": "npm run generate && node esbuild.config.js && node scripts/copy_bundle_assets.js",
"test": "npm run test --workspaces",
"test:ci": "npm run test:ci --workspaces --if-present && npm run test:scripts",
"test:scripts": "vitest run --config ./scripts/tests/vitest.config.ts",
"test:e2e": "npm run test:integration:sandbox:none -- --verbose --keep-output",
"test:integration:all": "npm run test:integration:sandbox:none && npm run test:integration:sandbox:docker && npm run test:integration:sandbox:podman",
"test:integration:sandbox:none": "GEMINI_SANDBOX=false node integration-tests/run-tests.js",
"test:integration:sandbox:docker": "GEMINI_SANDBOX=docker node integration-tests/run-tests.js",
"test:integration:sandbox:podman": "GEMINI_SANDBOX=podman node integration-tests/run-tests.js",
"test:scripts": "vitest run --config ./scripts/tests/vitest.config.ts",
"start": "node scripts/start.js",
"debug": "cross-env DEBUG=1 node --inspect-brk scripts/start.js",
"lint:fix": "eslint . --fix && eslint integration-tests --fix",
"lint": "eslint . --ext .ts,.tsx && eslint integration-tests",
"lint:fix": "eslint . --fix && eslint integration-tests --fix",
"lint:ci": "eslint . --ext .ts,.tsx --max-warnings 0 && eslint integration-tests --max-warnings 0",
"typecheck": "npm run typecheck --workspaces --if-present",
"format": "prettier --write .",
"typecheck": "npm run typecheck --workspaces --if-present",
"preflight": "npm run clean && npm ci && npm run format && npm run lint:ci && npm run build && npm run typecheck && npm run test:ci",
"auth:npm": "npx google-artifactregistry-auth",
"auth:docker": "gcloud auth configure-docker us-west1-docker.pkg.dev",
"auth": "npm run auth:npm && npm run auth:docker",
"prerelease:dev": "npm run prerelease:version --workspaces && npm run prerelease:deps --workspaces",
"bundle": "npm run generate && node esbuild.config.js && node scripts/copy_bundle_assets.js",
"build:packages": "npm run build --workspaces",
"build:sandbox:fast": "node scripts/build_sandbox.js --skip-npm-install-build",
"prepare": "npm run bundle",
"prepare:package": "node scripts/prepare-package.js",
"publish:sandbox": "node scripts/publish-sandbox.js",
"publish:npm": "npm publish --workspaces ${NPM_PUBLISH_TAG:+--tag=$NPM_PUBLISH_TAG} ${NPM_DRY_RUN:+--dry-run}",
"publish:release": "npm run prepare:package && npm run build:packages && npm run build:sandbox:fast && npm run publish:sandbox && npm run publish:npm",
"prepublishOnly": "node scripts/check-versions.js && node scripts/prepublish.js",
"release:version": "node scripts/version.js",
"tag:release:nightly": "node scripts/tag-release.js",
"check:versions": "node scripts/check-versions.js",
"publish:actions-release": "npm run prepare:package && npm run build:packages && npm run publish:npm"
"telemetry": "node scripts/telemetry.js",
"clean": "node scripts/clean.js"
},
"bin": {
"gemini": "bundle/gemini.js"

View File

@ -24,7 +24,7 @@
;; deny all outbound network traffic EXCEPT through proxy on localhost:8877
;; set `GEMINI_SANDBOX_PROXY_COMMAND=<command>` to run proxy alongside sandbox
;; proxy must listen on :::8877 (see scripts/example-proxy.js)
;; proxy must listen on :::8877 (see docs/examples/proxy-script.md)
(deny network-outbound)
(allow network-outbound (remote tcp "localhost:8877"))

View File

@ -88,5 +88,5 @@
;; allow outbound network traffic through proxy on localhost:8877
;; set `GEMINI_SANDBOX_PROXY_COMMAND=<command>` to run proxy alongside sandbox
;; proxy must listen on :::8877 (see scripts/example-proxy.js)
;; proxy must listen on :::8877 (see docs/examples/proxy-script.md)
(allow network-outbound (remote tcp "localhost:8877"))

View File

@ -1,65 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import { readFileSync } from 'fs';
import path from 'path';
function readPackageJson(dir) {
const p = path.join(dir, 'package.json');
return JSON.parse(readFileSync(p, 'utf-8'));
}
const root = readPackageJson('.');
const cli = readPackageJson('packages/cli');
const core = readPackageJson('packages/core');
const errors = [];
console.log('Checking version consistency...');
// 1. Check that all package versions are the same.
if (root.version !== cli.version || root.version !== core.version) {
errors.push(
`Version mismatch: root (${root.version}), cli (${cli.version}), core (${core.version})`,
);
} else {
console.log(`- All packages are at version ${root.version}.`);
}
// 2. Check that the cli's dependency on core matches the core version.
const coreDepVersion = cli.dependencies['@google/gemini-cli-core'];
const expectedCoreVersion = `^${core.version}`;
if (
coreDepVersion !== expectedCoreVersion &&
coreDepVersion !== 'file:../core'
) {
errors.push(
`CLI dependency on core is wrong: expected ${expectedCoreVersion} or "file:../core", got ${coreDepVersion}`,
);
} else {
console.log(`- CLI dependency on core (${coreDepVersion}) is correct.`);
}
// 3. Check that the sandbox image tag matches the root version.
const imageUri = root.config.sandboxImageUri;
const imageTag = imageUri.split(':').pop();
if (imageTag !== root.version) {
errors.push(
`Sandbox image tag mismatch: expected ${root.version}, got ${imageTag}`,
);
} else {
console.log(`- Sandbox image tag (${imageTag}) is correct.`);
}
if (errors.length > 0) {
console.error('\nVersion consistency checks failed:');
for (const error of errors) {
console.error(`- ${error}`);
}
process.exit(1);
}
console.log('\nAll version checks passed!');

View File

@ -1,11 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
// esbuild-banner.js
import { createRequire } from 'module';
const require = createRequire(import.meta.url);
globalThis.__filename = require('url').fileURLToPath(import.meta.url);
globalThis.__dirname = require('path').dirname(globalThis.__filename);

View File

@ -1,82 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
// ES module equivalent of __dirname
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const cliPackageJsonPath = path.resolve(
__dirname,
'../packages/cli/package.json',
);
const cliPackageJson = JSON.parse(fs.readFileSync(cliPackageJsonPath, 'utf8'));
// Get version from root package.json (accessible via env var in npm scripts)
const version = process.env.npm_package_version;
// Get Docker registry and image name directly from PUBLISH_ environment variables.
// These are expected to be set by the CI/build environment.
const containerImageRegistry = process.env.SANDBOX_IMAGE_REGISTRY;
const containerImageName = process.env.SANDBOX_IMAGE_NAME;
if (!version || !containerImageRegistry || !containerImageName) {
console.error(
'Error: Missing required environment variables. Need: ' +
'npm_package_version, SANDBOX_IMAGE_REGISTRY, and SANDBOX_IMAGE_NAME.',
);
console.error(
'These should be passed from the CI environment (e.g., Cloud Build substitutions) ' +
'to the npm publish:release script.',
);
process.exit(1);
}
const containerImageUri = `${containerImageRegistry}/${containerImageName}:${version}`;
// Add or update fields in cliPackageJson.config to store this information
if (!cliPackageJson.config) {
cliPackageJson.config = {};
}
cliPackageJson.config.sandboxImageUri = containerImageUri;
fs.writeFileSync(
cliPackageJsonPath,
JSON.stringify(cliPackageJson, null, 2) + '\n',
);
console.log(
`Updated ${path.relative(process.cwd(), cliPackageJsonPath)} with Docker image details:`,
);
console.log(` URI: ${containerImageUri}`);
console.log(` Registry: ${containerImageRegistry}`);
console.log(` Image Name: ${containerImageName}`);
// Copy README.md to packages/cli
const rootReadmePath = path.resolve(__dirname, '../README.md');
const cliReadmePath = path.resolve(__dirname, '../packages/cli/README.md');
try {
fs.copyFileSync(rootReadmePath, cliReadmePath);
console.log('Copied root README.md to packages/cli/');
} catch (err) {
console.error('Error copying README.md:', err);
process.exit(1);
}
// Copy README.md to packages/cli
const rootLicensePath = path.resolve(__dirname, '../LICENSE');
const cliLicensePath = path.resolve(__dirname, '../packages/cli/LICENSE');
try {
fs.copyFileSync(rootLicensePath, cliLicensePath);
console.log('Copied root LICENSE to packages/cli/');
} catch (err) {
console.error('Error copying LICENSE:', err);
process.exit(1);
}

View File

@ -1,50 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
import fs from 'fs';
import path from 'path';
const packageJsonPath = path.resolve(process.cwd(), 'package.json');
const readmePath = path.resolve(process.cwd(), 'README.md');
const licensePath = path.resolve(process.cwd(), 'LICENSE');
const errors = [];
// 1. Check for package.json and the 'repository' field
// Required for publishing through wombat-dressing-room
if (!fs.existsSync(packageJsonPath)) {
errors.push(`Error: package.json not found in ${process.cwd()}`);
} else {
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, 'utf8'));
if (
!packageJson.repository ||
typeof packageJson.repository !== 'object' ||
packageJson.repository.type !== 'git' ||
!packageJson.repository.url.includes('google-gemini/gemini-cli')
) {
errors.push(
`Error: The "repository" field in ${packageJsonPath} must be an object pointing to the "google-gemini/gemini-cli" git repository.`,
);
}
}
// 2. Check for README.md
if (!fs.existsSync(readmePath)) {
errors.push(`Error: README.md not found in ${process.cwd()}`);
}
// 3. Check for LICENSE
if (!fs.existsSync(licensePath)) {
errors.push(`Error: LICENSE file not found in ${process.cwd()}`);
}
if (errors.length > 0) {
console.error('Pre-publish checks failed:');
errors.forEach((error) => console.error(`- ${error}`));
process.exit(1);
}
console.log('Pre-publish checks passed.');

View File

@ -1,47 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { execSync } from 'child_process';
const {
npm_package_config_sandboxImageUri,
DOCKER_DRY_RUN,
GEMINI_SANDBOX_IMAGE_TAG,
} = process.env;
if (!npm_package_config_sandboxImageUri) {
console.error(
'Error: npm_package_config_sandboxImageUri environment variable is not set (should be run via npm).',
);
process.exit(1);
}
let imageUri = npm_package_config_sandboxImageUri;
if (GEMINI_SANDBOX_IMAGE_TAG) {
const [baseUri] = imageUri.split(':');
imageUri = `${baseUri}:${GEMINI_SANDBOX_IMAGE_TAG}`;
}
if (DOCKER_DRY_RUN) {
console.log(`DRY RUN: Would execute: docker push "${imageUri}"`);
} else {
console.log(`Executing: docker push "${imageUri}"`);
execSync(`docker push "${imageUri}"`, { stdio: 'inherit' });
}

View File

@ -1,123 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { execSync, spawn } from 'child_process';
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
try {
execSync('node scripts/sandbox_command.js -q');
} catch {
console.error('ERROR: sandboxing disabled. See docs to enable sandboxing.');
process.exit(1);
}
const argv = yargs(hideBin(process.argv)).option('i', {
alias: 'interactive',
type: 'boolean',
default: false,
}).argv;
if (argv.i && !process.stdin.isTTY) {
console.error(
'ERROR: interactive mode (-i) requested without a terminal attached',
);
process.exit(1);
}
const image = 'gemini-cli-sandbox';
const sandboxCommand = execSync('node scripts/sandbox_command.js')
.toString()
.trim();
const sandboxes = execSync(
`${sandboxCommand} ps --filter "ancestor=${image}" --format "{{.Names}}"`,
)
.toString()
.trim()
.split('\n')
.filter(Boolean);
let sandboxName;
const firstArg = argv._[0];
if (firstArg) {
if (firstArg.startsWith(image) || /^\d+$/.test(firstArg)) {
sandboxName = firstArg.startsWith(image)
? firstArg
: `${image}-${firstArg}`;
argv._.shift();
}
}
if (!sandboxName) {
if (sandboxes.length === 0) {
console.error(
'No sandboxes found. Are you running gemini-cli with sandboxing enabled?',
);
process.exit(1);
}
if (sandboxes.length > 1) {
console.error('Multiple sandboxes found:');
sandboxes.forEach((s) => console.error(` ${s}`));
console.error(
'Sandbox name or index (0,1,...) must be specified as first argument',
);
process.exit(1);
}
sandboxName = sandboxes[0];
}
if (!sandboxes.includes(sandboxName)) {
console.error(`unknown sandbox ${sandboxName}`);
console.error('known sandboxes:');
sandboxes.forEach((s) => console.error(` ${s}`));
process.exit(1);
}
const execArgs = [];
let commandToRun = [];
// Determine interactive flags.
// If a command is provided, only be interactive if -i is passed.
// If no command is provided, always be interactive.
if (argv._.length > 0) {
if (argv.i) {
execArgs.push('-it');
}
} else {
execArgs.push('-it');
}
// Determine the command to run inside the container.
if (argv._.length > 0) {
// Join all positional arguments into a single command string.
const userCommand = argv._.join(' ');
// The container is Linux, so we use bash -l -c to execute the command string.
// This is cross-platform because it's what the container runs, not the host.
commandToRun = ['bash', '-l', '-c', userCommand];
} else {
// No command provided, so we start an interactive bash login shell.
commandToRun = ['bash', '-l'];
}
const spawnArgs = ['exec', ...execArgs, sandboxName, ...commandToRun];
// Use spawn to avoid shell injection issues and handle arguments correctly.
spawn(sandboxCommand, spawnArgs, { stdio: 'inherit' });

View File

@ -1,42 +0,0 @@
/**
* @license
* Copyright 2025 Google LLC
* SPDX-License-Identifier: Apache-2.0
*/
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
import { execSync } from 'child_process';
try {
execSync('command -v npm', { stdio: 'ignore' });
} catch {
console.log('npm not found. Installing npm via nvm...');
try {
execSync(
'curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash',
{ stdio: 'inherit' },
);
const nvmsh = `\\. "$HOME/.nvm/nvm.sh"`;
execSync(`${nvmsh} && nvm install 22`, { stdio: 'inherit' });
execSync(`${nvmsh} && node -v`, { stdio: 'inherit' });
execSync(`${nvmsh} && nvm current`, { stdio: 'inherit' });
execSync(`${nvmsh} && npm -v`, { stdio: 'inherit' });
} catch {
console.error('Failed to install nvm or node.');
process.exit(1);
}
}
console.log('Development environment setup complete.');