CompletionKit

Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.

com.completionkit/evals · Repository · Website · version 1.0.0 · 3 stars · listed from registry

Install

Hosted endpoint, so there is nothing to install. The probe connects straight to it:

https://completionkit.com/mcp

Needs credentials: Authorization. The probe used placeholder values, so an amber needs credentials result below means the server asked for the real ones, not that it is broken. Maintainers: how to go green.

Results by platform

Linux

installs and connects2026-08-15 · probed over remote-http

3 recorded runs, oldest first.

tools/list returned 54 tools.

prompts_list prompts_get prompts_create prompts_update prompts_delete prompts_publish prompts_suggest_improvement runs_list runs_get runs_create runs_update runs_delete runs_generate runs_regrade runs_rerun runs_retry_failures responses_list responses_get datasets_list datasets_get datasets_create datasets_update datasets_delete datasets_create_from_url metrics_list metrics_get metrics_create metrics_update metrics_delete metrics_suggest_variants metric_groups_list metric_groups_get metric_groups_create metric_groups_update metric_groups_delete metric_versions_list metric_versions_publish metric_versions_dismiss provider_credentials_list provider_credentials_get provider_credentials_create provider_credentials_update provider_credentials_delete tags_list tags_get tags_create tags_update tags_delete agreements_list agreements_create

macOS

installs and connects2026-08-15 · probed over remote-http

3 recorded runs, oldest first.

tools/list returned 54 tools.

Windows

installs and connects2026-08-15 · probed over remote-http

3 recorded runs, oldest first.

tools/list returned 54 tools.

Badge

Paste this into the project README to show the current result:

[![does it install](https://img.shields.io/endpoint?url=https://doesitinstall.com/badge/com.completionkit__evals.json)](https://doesitinstall.com/s/com.completionkit__evals.html)