> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getcargo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Monitoring a tool

> Inspect runs, pull metrics, re-queue failures, and set alerts from the CLI — or the UI

Every run a tool produces is queryable from the CLI, so you can monitor health, compute error rates, and re-queue failures from a terminal or CI job. The same data backs the **Records** and **Metrics** views in the UI.

## Inspect runs from the CLI

```bash theme={null}
cargo-ai orchestration tool list                       # → workflowUuid

# List recent runs, optionally filtered by status
cargo-ai orchestration run list \
  --workflow-uuid <uuid> \
  --statuses error \
  --created-after 2025-01-12

# Drill into one run — inputs, node-by-node outputs, timings, errors
cargo-ai orchestration run get <run-uuid>
```

Run statuses are `running`, `success`, `error`, and `cancelled`.

## Metrics

Pull per-node execution counts and credit usage over a date range:

```bash theme={null}
cargo-ai orchestration run get-metrics \
  --workflow-uuid <uuid> \
  --created-after 2025-01-12 \
  --created-before 2025-01-19
```

```json theme={null}
{
  "runMetrics": [
    {
      "nodeUuid": "enrich-node-uuid",
      "totalExecutionsCount": 1000,
      "successExecutionsCount": 950,
      "errorExecutionsCount": 48,
      "cancelledExecutionsCount": 2,
      "creditsUsedCount": 450
    }
  ]
}
```

Error rate per node = `errorExecutionsCount / totalExecutionsCount` (here `48 / 1000 = 4.8%`).

## Re-queue failed runs

Download the failed runs, extract their record IDs, and re-run only those:

```bash theme={null}
# 1. Download failures
cargo-ai orchestration run download \
  --workflow-uuid <uuid> \
  --statuses error \
  --created-after 2025-01-12 > failed_runs.json

# 2. Re-run just the failed records
RECORD_IDS=$(jq '[.[].recordId]' failed_runs.json)
cargo-ai orchestration batch create \
  --workflow-uuid <uuid> \
  --data "{\"kind\":\"recordIds\",\"recordIds\":$RECORD_IDS}"
```

<Tip>
  Before re-queuing, check whether failures are transient (rate limits) or
  permanent (invalid data). Fix permanent issues first — retrying won't help.
</Tip>

## From the UI

The tool editor's **Records** view lists the same runs with status filters (by batch, version, status, date, node outcome); click a run for the node-by-node breakdown. The **Metrics** view charts success rate, latency (p50/p95/p99), volume, and error distribution, grouped by batch, version, or date. Failed runs can be retried individually or in bulk from Records.

<iframe className="w-full aspect-video rounded-xl" src="https://www.youtube.com/embed/TmVNoHot68Q" title="Retrying a failed run" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen />

## Alerts

Send alerts to your own systems with a webhook. Cargo POSTs a payload like this on failures:

```json theme={null}
{
  "event": "tool.execution.failed",
  "tool_id": "tool_abc123",
  "tool_name": "Company Enrichment",
  "execution_id": "exec_xyz789",
  "error": "Rate limit exceeded",
  "timestamp": "2024-01-15T10:30:00Z"
}
```

Slack and email notifications (failure thresholds, success-rate drops, unusual volume) are configured under **Settings → Notifications**.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Tool execution times out">
    Break complex tools into smaller ones, add timeouts to slow external calls,
    reduce batch sizes, or use async execution for long-running operations.
  </Accordion>

  <Accordion title="Rate limit errors from integrations">
    Only connector nodes (`kind: "connector"`) are rate-limited. Add delays
    between calls, distribute load, reduce concurrency, or check the
    integration's documented limits.
  </Accordion>

  <Accordion title="Data not found in enrichment">
    Verify the input format (e.g. domain without `https://`), confirm the record
    exists in the provider's database, and add fallback providers or validation
    before the enrichment step.
  </Accordion>

  <Accordion title="CRM write failures">
    Confirm the record exists first, check field permissions and required
    fields, validate data types against CRM field types, and watch for
    duplicate-detection rules.
  </Accordion>
</AccordionGroup>

## Best practices

* **Set alerts before production** — don't wait for users to report issues.
* **Review metrics regularly** — spot trends before they become incidents.
* **Fix root causes** — investigate repeated failures instead of blindly retrying.
* **Version changes** — correlate performance shifts with deploys.