Genai Toolbox
by ckchzh
Connect AI agents to databases via MCP with schema-aware query support. Use when querying DBs from agents, configuring connections, or benchmarking.
安装
claude skill add --url github.com/openclaw/skills/tree/main/skills/ckchzh/genai-toolbox文档
Genai Toolbox
Genai Toolbox v2.0.0 — an AI toolkit for managing generative AI database workflows from the command line. Log configurations, benchmarks, prompts, evaluations, fine-tuning runs, cost tracking, and optimization notes. Each entry is timestamped and persisted locally. Works entirely offline — your data never leaves your machine.
Inspired by googleapis/genai-toolbox (13,412+ GitHub stars).
Why Genai Toolbox?
- Works entirely offline — your data never leaves your machine
- Simple command-line interface with no GUI dependency
- Export to JSON, CSV, or plain text at any time for sharing or archival
- Automatic activity history logging across all commands
- Each domain command doubles as both a logger and a viewer
Commands
Domain Commands
Each domain command works in two modes: log mode (with arguments) saves a timestamped entry, view mode (no arguments) shows the 20 most recent entries.
| Command | Description |
|---|---|
genai-toolbox configure <input> | Log a configuration note such as database connection strings, MCP server settings, or schema definitions. Track which configurations were active during each experiment or deployment. |
genai-toolbox benchmark <input> | Log a benchmark result or performance observation. Record query latency, throughput, p99 response times, and row-scan efficiency across different database backends. |
genai-toolbox compare <input> | Log a comparison note between models, tools, or database configurations. Useful for side-by-side evaluations like GPT-4 vs Claude for SQL generation accuracy. |
genai-toolbox prompt <input> | Log a prompt template or prompt engineering note. Track iterations on database query generation prompts, schema descriptions, and few-shot examples for SQL synthesis. |
genai-toolbox evaluate <input> | Log an evaluation result or quality metric. Record query accuracy, semantic correctness scores, and human review outcomes for AI-generated database operations. |
genai-toolbox fine-tune <input> | Log a fine-tuning run or hyperparameter note. Track training on domain-specific SQL patterns, schema-aware models, and the resulting improvements in query generation. |
genai-toolbox analyze <input> | Log an analysis observation or insight. Record failure patterns, query plan analysis, common error modes, and data quality issues found across AI-database interactions. |
genai-toolbox cost <input> | Log cost tracking data including API costs, database compute charges, and token consumption. Essential for monitoring expenses across multiple projects and cloud providers. |
genai-toolbox usage <input> | Log usage metrics or consumption data. Track query volumes, token counts, connection pool utilization, and rate limit encounters across AI-database workflows. |
genai-toolbox optimize <input> | Log optimization attempts or performance improvements. Record query plan changes, index additions, caching strategies, and their measured impact on performance. |
genai-toolbox test <input> | Log test results or test case notes. Record integration test outcomes, edge case coverage for SQL generation, and regression test results across schema changes. |
genai-toolbox report <input> | Log a report entry or summary finding. Capture weekly performance summaries, migration reports, or executive-level findings from AI-database integration projects. |
Utility Commands
| Command | Description |
|---|---|
genai-toolbox stats | Show summary statistics across all log files, including entry counts per category and total data size on disk. |
genai-toolbox export <fmt> | Export all data to a file in the specified format. Supported formats: json, csv, txt. Output is saved to the data directory. |
genai-toolbox search <term> | Search all log entries for a term using case-insensitive matching. Results are grouped by log category for easy scanning. |
genai-toolbox recent | Show the 20 most recent entries from the unified activity log, giving a quick overview of recent work across all commands. |
genai-toolbox status | Health check showing version, data directory path, total entry count, disk usage, and last activity timestamp. |
genai-toolbox help | Show the built-in help message listing all available commands and usage information. |
genai-toolbox version | Print the current version (v2.0.0). |
Data Storage
All data is stored locally at ~/.local/share/genai-toolbox/. Each domain command writes to its own log file (e.g., configure.log, benchmark.log). A unified history.log tracks all actions across commands. Use export to back up your data at any time.
Requirements
- Bash (4.0+)
- No external dependencies — pure shell script
- No network access required
When to Use
- Tracking GenAI agent-to-database connection configurations and MCP server setups across environments
- Logging benchmark results for query latency, throughput, and AI model accuracy on SQL generation tasks
- Comparing different AI models or prompt strategies for database query generation accuracy
- Managing prompt templates and few-shot examples for schema-aware SQL synthesis workflows
- Tracking API costs, token usage, and compute expenses across multiple GenAI database integration projects
Examples
bash
# Log a database connection configuration
genai-toolbox configure "PostgreSQL connection via MCP, schema=public, pool_size=10, ssl=required"
# Record a benchmark result
genai-toolbox benchmark "Query latency: avg 120ms, p99 350ms on 1M rows, index=btree"
# Compare two approaches
genai-toolbox compare "GPT-4 vs Claude for SQL generation: Claude +8% accuracy, GPT-4 2x faster"
# Log a prompt template
genai-toolbox prompt "v3: Generate SELECT query for {table} filtering by {condition}, include schema context"
# Track costs
genai-toolbox cost "March total: $42.50 across 150k queries, avg $0.00028/query"
# View all statistics
genai-toolbox stats
# Export everything as JSON
genai-toolbox export json
# Search across all logs
genai-toolbox search "PostgreSQL"
# Check recent activity
genai-toolbox recent
# Health check
genai-toolbox status
Powered by BytesAgain | bytesagain.com | hello@bytesagain.com