7.0 KiB
todos:
- id: "redis-keys" content: "RedisKeyManagerService 增加 duration buckets/sum/count 三个 key 常量" status: pending
- id: "prometheus-service" content: "PrometheusService 实现 recordHttpRequestDuration + render histogram 输出" status: pending
- id: "middleware-timing" content: "PrometheusMiddleware 计时并在 finally 记录 duration" status: pending
- id: "grafana-panels" content: "app_http_services_dashboard.json 增加 P50/P95/P99/Avg 与 route 耗时面板" status: pending
- id: "verify" content: "curl /metrics 验证 + 跑 completion-report 门禁" status: pending isProject: false
slot_wallet HTTP 响应时长监控
现状
当前链路:
sequenceDiagram
participant Client
participant Middleware as PrometheusMiddleware
participant Handler
participant Redis
participant Metrics as MetricsController
Client->>Middleware: HTTP Request
Middleware->>Handler: handler()
Handler-->>Middleware: Response
Middleware->>Redis: hIncrBy requests_total
Middleware-->>Client: Response
Note over Metrics,Redis: Prometheus scrape
Metrics->>Redis: hGetAll
Metrics-->>Client: text/plain metrics
- 中间件:
slot_wallet/app/middleware/PrometheusMiddleware.php在finally中调用recordHttpRequest(),只记请求数。 - 指标服务:
slot_wallet/app/service/PrometheusService.php用 Redis Hash 存 counter,经slot_wallet/app/controller/MetricsController.php的/metrics输出。 - Grafana:
slot_wallet/doc/grafana/app_http_services_dashboard.json仅查询app_http_requests_total/app_business_responses_total。
结论:可以加响应时长,且与现有架构兼容;推荐新增标准 Prometheus Histogram,而不是改现有 counter。
目标指标
新增 metric:app_http_request_duration_seconds(histogram)
| 子指标 | 含义 |
|---|---|
_bucket{le="..."} |
各耗时桶累计次数 |
_sum |
累计耗时(秒) |
_count |
观测次数 |
Labels 与现有请求 counter 保持一致:service, method, route, status,便于在 Grafana 与请求量 join。
Histogram buckets(秒)(Webman API 常用区间):
0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, +Inf
代码改动
1. Redis Key — slot_wallet/app/service/RedisKeyManagerService.php
新增 3 个常量(与现有 PROMETHEUS_* 命名一致):
PROMETHEUS_HTTP_REQUEST_DURATION_BUCKETS→prometheus:http:duration:bucketsPROMETHEUS_HTTP_REQUEST_DURATION_SUM→prometheus:http:duration:sumPROMETHEUS_HTTP_REQUEST_DURATION_COUNT→prometheus:http:duration:count
2. 记录耗时 — slot_wallet/app/service/PrometheusService.php
新增 recordHttpRequestDuration(string $method, string $route, int $status, float $durationSeconds):
- field 仍用
json_encode存 labels(bucket 额外带le) _bucket/_count:Redis::hIncrBy(..., 1)_sum:Redis::hIncrByFloat(..., $durationSeconds)(避免整数毫秒精度损失)- 对每个
lebucket,若$durationSeconds <= le则递增;最后递增+Infbucket - 抽取私有方法
buildHttpRequestLabels(),供recordHttpRequest与 duration 复用,避免重复 JSON 拼装
扩展 render(),在现有 counter 之后输出:
# HELP app_http_request_duration_seconds HTTP request latency in seconds.
# TYPE app_http_request_duration_seconds histogram
app_http_request_duration_seconds_bucket{...,le="0.1"} N
...
app_http_request_duration_seconds_sum{...} X.XXX
app_http_request_duration_seconds_count{...} N
3. 中间件计时 — slot_wallet/app/middleware/PrometheusMiddleware.php
在 process() 开头 $startedAt = microtime(true),finally 中:
$durationSeconds = microtime(true) - $startedAt;
PrometheusService::recordHttpRequestDuration($method, $route, $status, $durationSeconds);
优化:将 resolveRoute() 结果缓存到局部变量,避免 finally 里重复解析;recordHttpRequest 与 recordHttpRequestDuration 共用同一份 method/route/status。
耗时口径:中间件包裹的完整 handler 执行时间(含业务逻辑,不含 Prometheus 写 Redis 本身;写在 finally 末尾,影响极小)。
Grafana 仪表盘扩展
更新 slot_wallet/doc/grafana/app_http_services_dashboard.json,在现有请求量面板下方新增一行「Latency」区域(version 递增为 3):
| 面板 | 类型 | PromQL 示例 |
|---|---|---|
| P50 Latency | stat | histogram_quantile(0.50, sum by (le) (rate(app_http_request_duration_seconds_bucket{service="$service"}[$__rate_interval]))) |
| P95 Latency | stat | 同上 0.95 |
| P99 Latency | stat | 同上 0.99 |
| Avg Latency | stat | sum(rate(..._sum...)) / sum(rate(..._count...)) |
| Latency Trend (P50/P95/P99) | timeseries | 三条 quantile 曲线,unit=s |
| Top Slow Routes (P95) | bargauge | topk(10, histogram_quantile(0.95, sum by (route, le) (rate(..._bucket{service="$service"}[$__range])))) |
| Route Latency Details | table | 合并 route 的 P95、Avg、Requests(与现有 Route Request Details 面板并列或扩列) |
所有新面板 unit 设为 s(秒),threshold 可按业务再调。
数据流(改后)
sequenceDiagram
participant Middleware as PrometheusMiddleware
participant Redis
participant Grafana
Middleware->>Middleware: startedAt = microtime(true)
Middleware->>Middleware: handler()
Middleware->>Redis: incr requests_total
Middleware->>Redis: incr duration buckets/sum/count
Note over Grafana: rate + histogram_quantile
Grafana->>Grafana: P50/P95/P99/Avg
验证步骤
- 本地发若干 HTTP 请求到 slot_wallet(含快/慢接口)。
curl /metrics,确认出现app_http_request_duration_seconds_bucket/_sum/_count。- Prometheus scrape 后,在 Grafana 导入更新后的 dashboard JSON,选择
service=slot_wallet,确认 P50/P95 有数据。 - 跑
slot-backend-completion-report门禁(php -l+ verify 脚本)。
注意事项
- 仅改 slot_wallet(按你的选择);
slot_console/slot_pwa结构相同,后续可 copy 同一套改动。 - Redis Hash 会随
route × status × bucket增长;与现有app_http_requests_total同一量级,可接受。 - Histogram 在 Prometheus 侧用
rate()+histogram_quantile()算分位;Grafana 查询写法与标准 Prometheus 一致。 - 无需改 Prometheus scrape 配置(metric 名新增,非替换)。