Files
cursor/plans/HTTP-c8f2b444.plan.md
ray zhou 2dd9f17da9 ok
2026-06-29 14:51:55 +08:00

7.0 KiB
Raw Blame History


todos:

  • id: "redis-keys" content: "RedisKeyManagerService 增加 duration buckets/sum/count 三个 key 常量" status: pending
  • id: "prometheus-service" content: "PrometheusService 实现 recordHttpRequestDuration + render histogram 输出" status: pending
  • id: "middleware-timing" content: "PrometheusMiddleware 计时并在 finally 记录 duration" status: pending
  • id: "grafana-panels" content: "app_http_services_dashboard.json 增加 P50/P95/P99/Avg 与 route 耗时面板" status: pending
  • id: "verify" content: "curl /metrics 验证 + 跑 completion-report 门禁" status: pending isProject: false

slot_wallet HTTP 响应时长监控

现状

当前链路:

sequenceDiagram
    participant Client
    participant Middleware as PrometheusMiddleware
    participant Handler
    participant Redis
    participant Metrics as MetricsController

    Client->>Middleware: HTTP Request
    Middleware->>Handler: handler()
    Handler-->>Middleware: Response
    Middleware->>Redis: hIncrBy requests_total
    Middleware-->>Client: Response
    Note over Metrics,Redis: Prometheus scrape
    Metrics->>Redis: hGetAll
    Metrics-->>Client: text/plain metrics

结论:可以加响应时长,且与现有架构兼容;推荐新增标准 Prometheus Histogram而不是改现有 counter。


目标指标

新增 metricapp_http_request_duration_secondshistogram

子指标 含义
_bucket{le="..."} 各耗时桶累计次数
_sum 累计耗时(秒)
_count 观测次数

Labels 与现有请求 counter 保持一致:service, method, route, status,便于在 Grafana 与请求量 join。

Histogram bucketsWebman API 常用区间):

0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, +Inf


代码改动

1. Redis Key — slot_wallet/app/service/RedisKeyManagerService.php

新增 3 个常量(与现有 PROMETHEUS_* 命名一致):

  • PROMETHEUS_HTTP_REQUEST_DURATION_BUCKETSprometheus:http:duration:buckets
  • PROMETHEUS_HTTP_REQUEST_DURATION_SUMprometheus:http:duration:sum
  • PROMETHEUS_HTTP_REQUEST_DURATION_COUNTprometheus:http:duration:count

2. 记录耗时 — slot_wallet/app/service/PrometheusService.php

新增 recordHttpRequestDuration(string $method, string $route, int $status, float $durationSeconds)

  • field 仍用 json_encode 存 labelsbucket 额外带 le
  • _bucket / _countRedis::hIncrBy(..., 1)
  • _sumRedis::hIncrByFloat(..., $durationSeconds)(避免整数毫秒精度损失)
  • 对每个 le bucket$durationSeconds <= le 则递增;最后递增 +Inf bucket
  • 抽取私有方法 buildHttpRequestLabels(),供 recordHttpRequest 与 duration 复用,避免重复 JSON 拼装

扩展 render(),在现有 counter 之后输出:

# HELP app_http_request_duration_seconds HTTP request latency in seconds.
# TYPE app_http_request_duration_seconds histogram
app_http_request_duration_seconds_bucket{...,le="0.1"} N
...
app_http_request_duration_seconds_sum{...} X.XXX
app_http_request_duration_seconds_count{...} N

3. 中间件计时 — slot_wallet/app/middleware/PrometheusMiddleware.php

process() 开头 $startedAt = microtime(true)finally 中:

$durationSeconds = microtime(true) - $startedAt;
PrometheusService::recordHttpRequestDuration($method, $route, $status, $durationSeconds);

优化:将 resolveRoute() 结果缓存到局部变量,避免 finally 里重复解析;recordHttpRequestrecordHttpRequestDuration 共用同一份 method/route/status

耗时口径:中间件包裹的完整 handler 执行时间(含业务逻辑,不含 Prometheus 写 Redis 本身;写在 finally 末尾,影响极小)。


Grafana 仪表盘扩展

更新 slot_wallet/doc/grafana/app_http_services_dashboard.json在现有请求量面板下方新增一行「Latency」区域version 递增为 3

面板 类型 PromQL 示例
P50 Latency stat histogram_quantile(0.50, sum by (le) (rate(app_http_request_duration_seconds_bucket{service="$service"}[$__rate_interval])))
P95 Latency stat 同上 0.95
P99 Latency stat 同上 0.99
Avg Latency stat sum(rate(..._sum...)) / sum(rate(..._count...))
Latency Trend (P50/P95/P99) timeseries 三条 quantile 曲线unit=s
Top Slow Routes (P95) bargauge topk(10, histogram_quantile(0.95, sum by (route, le) (rate(..._bucket{service="$service"}[$__range]))))
Route Latency Details table 合并 route 的 P95、Avg、Requests与现有 Route Request Details 面板并列或扩列)

所有新面板 unit 设为 sthreshold 可按业务再调。


数据流(改后)

sequenceDiagram
    participant Middleware as PrometheusMiddleware
    participant Redis
    participant Grafana

    Middleware->>Middleware: startedAt = microtime(true)
    Middleware->>Middleware: handler()
    Middleware->>Redis: incr requests_total
    Middleware->>Redis: incr duration buckets/sum/count
    Note over Grafana: rate + histogram_quantile
    Grafana->>Grafana: P50/P95/P99/Avg

验证步骤

  1. 本地发若干 HTTP 请求到 slot_wallet含快/慢接口)。
  2. curl /metrics,确认出现 app_http_request_duration_seconds_bucket/_sum/_count
  3. Prometheus scrape 后,在 Grafana 导入更新后的 dashboard JSON选择 service=slot_wallet,确认 P50/P95 有数据。
  4. slot-backend-completion-report 门禁(php -l + verify 脚本)。

注意事项

  • 仅改 slot_wallet(按你的选择);slot_console / slot_pwa 结构相同,后续可 copy 同一套改动。
  • Redis Hash 会随 route × status × bucket 增长;与现有 app_http_requests_total 同一量级,可接受。
  • Histogram 在 Prometheus 侧用 rate() + histogram_quantile() 算分位Grafana 查询写法与标准 Prometheus 一致。
  • 无需改 Prometheus scrape 配置metric 名新增,非替换)。