GitLab

erio authored Apr 16, 2026 and 陈曦 committed Apr 17, 2026

refactor: extract ReadUpstreamResponseBody to deduplicate upstream response read + too-large error handling

Consolidates 9 call sites of resolveUpstreamResponseReadLimit + readUpstreamResponseBodyLimited + ErrUpstreamResponseBodyTooLarge error handling into a single ReadUpstreamResponseBody function with TooLargeWriter callback for API-format-specific error responses (Anthropic, OpenAI, countTokens).

Elysia authored Apr 16, 2026 and 陈曦 committed Apr 17, 2026

Each retry in the SetOutboxWatermark loop now gets its own 5s context.
Previously a shared context could already be expired when the second or
third attempt ran, making the retries pointless.
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Elysia authored Apr 16, 2026 and 陈曦 committed Apr 17, 2026

Fixes #1691

- pollOutbox() reused a 10s context for SetOutboxWatermark after event
  processing could take much longer, causing "outbox watermark write
  failed: context deadline exceeded". The watermark never advanced so
  the same 200 events were reprocessed every poll cycle, spiking CPU.
  Now uses an independent 5s context with up to 3 retries (200ms apart).

- When multiple Codex accounts sharing the same 21-22 groups are all
  rate-limited in quick succession, each account_changed event triggered
  redundant bucket rebuild attempts for the same groups. Introduce
  batchSeenKey{groupID, platform} and thread a seen map through the
  handler chain; rebuildBucketsForPlatform skips (group, platform) pairs
  already rebuilt within the same poll batch (~80% fewer rebuild calls in
  the 5-accounts-same-groups scenario).
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

- UserBreakdownItem: add AccountCost field + SQL aggregation
- UserBreakdownSubTable: add orange account cost column
- Admin usage table: add account_cost column (after cost, default visible)
- Column settings: add account_cost toggle option

- Fix mock for GetModelStatsWithFilters: add account_cost column
- Add assertion: GetStatsWithFilters always returns TotalAccountCost
- New test: GetModelStatsAccountCostColumn verifies scan of AccountCost
- New test: GetGroupStatsAccountCostColumn verifies scan of AccountCost
- New test: GetStatsWithFiltersAlwaysReturnsAccountCost (no AccountID filter)
- Integration test: add TotalAccountCost/TodayAccountCost assertions
- Fix gofmt alignment in usage_log_types.go

- Add account_cost column to dashboard aggregation tables (migration 107)
- DashboardStats: add TotalAccountCost/TodayAccountCost fields
- ModelStat/GroupStat: add AccountCost field with SQL aggregation
- GetStatsWithFilters: always return TotalAccountCost (remove accountID filter)
- Dashboard Token cards: show user(green)/cost(orange)/standard(gray)
- Usage stats card: show account cost and standard below main value
- Model/Group distribution tables: add orange cost column

Product name (e.g. "快代码科技工作室 100 元") should show the user's
original recharge amount (limitAmount), not the fee-inclusive pay amount.
The gateway receives payAmount separately for actual charging.

- Fix gofmt alignment in setting_handler.go, settings.go, payment_config_service.go
- Add payment_balance_recharge_multiplier and payment_recharge_fee_rate
  to API contract test expected JSON

Backend:
- Validate recharge_fee_rate: 0 ≤ rate ≤ 100, max 2 decimal places

Frontend settings:
- Add % suffix icon to fee rate input
- Enforce max=100, min=0, step=0.01 with 2 decimal precision

Backend:
- Use cfg.RechargeFeeRate in order creation instead of hardcoded 0
- Remove dead getFeeRate stub method
- All amounts computed server-side: order_amount, pay_amount, fee_rate

Frontend - PaymentView:
- Read recharge_fee_rate from checkout-info API (not per-method)
- Show fee breakdown only when fee_rate > 0
- Show credited amount only when multiplier ≠ 1

Frontend - Order display (user + admin):
- Fix fee_rate * 100 bug (fee_rate is already a percentage)
- OrderTable: show pay_amount as primary, fee/credited as sub-lines
- AdminOrderDetail: full breakdown (base/fee/paid/credited)
- AdminRefundDialog: label "到账金额" for clarity
- PaymentResultView: show pay_amount with fee info

Types + i18n:
- Add recharge_fee_rate to CheckoutInfoResponse
- Add fee_rate to CreateOrderResult
- Add translations: creditedAmount, fee, baseAmount, includedInPayAmount

- Add recharge_fee_rate system setting (percentage fee on top of recharge amount)
- Full backend chain: config constant, PaymentConfig struct, update validation,
  read/write persistence, DTO, handler GET/PUT responses
- Frontend: settings input with preview, i18n (zh/en), API types
- Fix provider card toggle layout: labels above switches to save width
- Fix Chinese translation: "EasyPay" → "易支付" in provider description

- Add balance_recharge_multiplier system setting (e.g. 1.2 = charge 100 get 120)
- Separate order_amount (credited balance) from pay_amount (actual payment)
- Refund calculates gateway amount proportionally from pay_amount
- Frontend shows both amounts in order details, payment status, refund dialog
- Admin settings UI for configuring recharge multiplier

- Refactor OpenAI Messages() routing: pre-compute dispatch model using
  resolveOpenAIMessagesDispatchMappedModel + NormalizeOpenAICompatRequestedModel
  instead of try-fail-retry pattern with gin context passing
- Remove openai_messages_fallback_model context anti-pattern
- Use effectiveMappedModel directly for forward default mapped model
- Add 3 subscription group tests covering all branch paths:
  _Blocked (no active subscription → SUBSCRIPTION_REQUIRED),
  _RequiresRepo (nil repo → SUBSCRIPTION_REPOSITORY_UNAVAILABLE),
  _AllowsActiveSubscription (valid subscription → success)

wire_gen.go was calling NewSchedulerCache(redisClient) but wire.go had
been updated to register ProvideSchedulerCache(redisClient, config),
which reads SnapshotMGetChunkSize and SnapshotWriteChunkSize from config.
Without this fix, those config values were silently ignored.

- Fix flaky WebSocket passthrough test: allow StatusNormalClosure after
  client close instead of requiring NoError (race condition fix)
- Fix ratelimit 401 test: use PlatformOpenAI instead of PlatformGemini
  for OAuth token cache invalidation scenario (more accurate)
- Add usageLoadQueue: Anthropic OAuth/setup-token accounts sharing the
  same proxy exit are serialized with 1-2s jitter to prevent upstream 429
- AccountUsageCell: add module-level usage cache (5min TTL), unmounted
  safety guard, and integrate enqueueUsageRequest for throttled fetching

Backend:
- gateway_handler: pass subject.UserID instead of int64(0) for user-level routing
- setting_handler: add missing BalanceLowNotifyRechargeURL to UpdateSettings response
- openai_gateway_service: use applyAccountStatsCost for account stats pricing integration
- embed_on: add local file override (data/public/) for embedded frontend assets

Frontend:
- useTableSelection: add batchUpdate method for batch operations
- AccountsView: virtual scrolling params, Set-based isSelected, swipe virtualization
- ProxiesView: add batchUpdate to selection and swipe-select
- BulkEditAccountModal: fix submit handler to prevent event object as argument
- SettingsView: move payload construction outside try block
- i18n: add general translation keys (saved, deleted, view, validation, allowUserRefund)
- api/client: reorder error fields for consistency
- stores/payment: clarify pollOrderStatus JSDoc

1. PrepareRefund: block refund on provider instance lookup failure
   instead of silently skipping permission check (medium severity)

2. UpdateProviderInstance: allow enabling refund_enabled and
   allow_user_refund in the same request by checking req.RefundEnabled
   value before falling back to DB read

3. ExecuteRefund: only revoke subscription on ErrAdjustWouldExpire,
   abort on other errors (DB failure, not found) instead of
   unconditionally revoking

- gateway_handler: pass ParsedRequest to RecordUsage + set in gin.Context
- channel_handler: add FeaturesConfig to CRUD (WebSearch channel toggle)
- channel_repo: features_config JSONB persistence (Create/Get/Update/List)
- security_headers: add Stripe CSP domains (script-src + frame-src)

Bug fixes:
- Detached context for GetAccountConcurrencyBatch (prevent all-zero on request cancel)
- Filter soft-deleted users in GetByGroupID
- Stripe CSP policy (allow Stripe.js in script-src and frame-src)
- WebSearch API key validation on save
- RECHARGING status in payment result success check
- Windows test fixes (logger Sync deadlock, config path escaping)

Feature enhancements:
- Webhook multi-instance dispatch (extractOutTradeNo + GetWebhookProvider)
- EasyPay mobile H5 payment (device param + PayURL2)
- SSE error propagation in WebSearch emulation
- AccountStatsCost DTO field for admin usage logs
- Plans sort by sort_order instead of created_at
- UsageMapHook for streaming response usage data
- apicompat Instructions field passthrough
- EffectiveLoadFactor for ops concurrency/metrics
- Usage billing RETURNING balance for notify system
- BulkUpdate mixed channel warning with details
- println to slog migration in auth cache
- Wire ProviderSet cleanup
- CI cache-dependency-path optimization

Frontend:
- Refund eligibility check per provider (canRequestRefund)
- Plan sort_order editing
- Dead code cleanup (simulate_claude_max, client_affinity)
- GroupsView platform switch guard
- channels features_config API type
- UsageView account_stats_cost export

allow_user_refund:
- Add allow_user_refund field to PaymentProviderInstance ent schema
- Migration 103: ALTER TABLE payment_provider_instances ADD COLUMN
- Cascade logic: disabling refund_enabled auto-disables allow_user_refund
- User refund validation: check provider instance allows user refund
- Admin refund validation: check provider instance allows admin refund
- Subscription refund: deduct days on refund, rollback on failure
- New endpoint: GET /payment/orders/refund-eligible-providers
- Frontend: ToggleSwitch in ProviderCard/Dialog, cascade in SettingsView

Wildcard matching:
- Change findPricingForModel from "longest prefix wins" to "config order
  priority (first match wins)", aligning with channel service behavior

- Fix errcheck: brave.go resp.Body.Close, manager_test.go Encode
- Fix gofmt: payment_config_service.go
- Fix unused: use shouldFallbackGeminiModel (with modelName param) in handler

- Fix errcheck: handle Write/Encode return values in brave_test.go
- Fix errcheck: defer resp.Body.Close() with _ assignment in tavily.go
- Fix gofmt: payment.go, channel.go, payment_config_providers.go
- Fix unused: remove dead decodeURLValue in easypay.go
- Restore shouldFallbackGeminiModel function (deleted during cherry-pick)
- Add missing balanceNotifyService param to NewGatewayService in test
- Fix platform default test expectation (empty stays empty)
- Fix wildcard pricing test (longest prefix wins, not config order)
- Fix subscription group test (SUBSCRIPTION_REPOSITORY_UNAVAILABLE)

Checkout page aggregates Stripe sub-types (card,link,alipay,wxpay) under
"stripe", but SelectInstance matched against supported_types literally,
which doesn't contain "stripe". Now matches by provider_key for Stripe.

- Add missing interface methods to test stubs (RemoveGroupFromUserAllowedGroups,
  GetNotifyCodeUserRate, IncrNotifyCodeUserRate, UpdateGroupIDByUserAndGroup)
- Fix NewUserService call signatures (add 4th param)
- Fix GetAccountCount return signature (3 values)
- Update api_contract_test.go snapshots for balance_notify fields
- Restore resolveOpenAIMessagesDispatchMappedModel function
- Reset VERSION to upstream 0.1.112

- Restore function deleted during cherry-pick conflict resolution
- Reset VERSION to upstream 0.1.112

- Restore gateway_cache.go to upstream (no lua embeds)
- Restore payment_order.go to upstream (use out_trade_no lookup)
- Restore payment_fulfillment.go to upstream (same reason)
- Add FeaturesConfig field and IsWebSearchEmulationEnabled to Channel
- Add applyAccountStatsCost wrapper function
- Add SettingKeyWebSearchEmulationConfig constant
- Add WebSearchEmulationEnabled to SystemSettings
- Add notify code rate limiting methods to EmailCache interface
- Remove AllowUserRefund references (ent schema not present)
- Fix duplicate import in payment_handler.go
- Fix wire_gen.go argument mismatches

shaw authored Apr 13, 2026 and 陈曦 committed Apr 14, 2026

Legacy instances created the settings table via ent auto-migration,
which emits Go-level defaults only. Migration 005 uses CREATE TABLE
IF NOT EXISTS, so the missing SQL DEFAULT was never backfilled. This
caused 098's raw INSERT to fail with a NOT NULL violation on
updated_at. The new migration is idempotent and safe for fresh
installs (no-op) and historical instances (backfills the default).

shuanbao0 authored Apr 11, 2026 and 陈曦 committed Apr 14, 2026

在前一个 commit 的 isResponsesShape 短路路径基础上,补充对 Cursor 云端
带过来的、Codex 上游统一不支持的顶层 Responses API 参数的剥离:

  - prompt_cache_retention
  - safety_identifier
  - metadata
  - stream_options

根因补充:这条 raw-body 透传路径为了保留 Cursor 的 input 数组整体结构,
不再经过 ChatCompletionsRequest 的反序列化过滤,所以这些 Go 结构体里
没有对应字段的参数会被原样发到上游,上游返回:
    Unsupported parameter: <field>
常规 Chat Completions 转换路径天然通过 ChatCompletionsRequest 丢弃未知字段,
不受影响;此处仅在 isResponsesShape 分支内用 sjson.DeleteBytes 显式过滤,
作用域最小。剥离列表与 openai_gateway_service.go:2034 的
unsupportedFields 语义对齐。

另外在 applyCodexOAuthTransform 的 OAuth 兜底 strip 列表里同步追加
prompt_cache_retention,作为对该函数所有其他 OAuth 调用点的 defense
in depth(当前只有 Cursor 路径的短路已在前面剥过,但保留这一层更稳)。

测试:
- TestCursorMixedShape_StripsUnsupportedFields — 验证所有 4 个字段都被剥
- TestApplyCodexOAuthTransform_StripsPromptCacheRetention — OAuth 兜底路径
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

shuanbao0 authored Apr 11, 2026 and 陈曦 committed Apr 14, 2026

Cursor 云端 (User-Agent: Go-http-client/2.0) 发往 /v1/chat/completions 的
body 使用 Responses API 格式:
    {"model":"gpt-5.4","input":[{"role":"system","content":"..."}],"stream":true}

原代码用 ChatCompletionsRequest 反序列化,该结构体没有 Input 字段,
Cursor 的 input 数组被静默丢弃,ChatCompletionsToResponses 转换后产出
input: null,Codex 上游以 "Invalid type for 'input': expected a string,
but got an object" 拒绝请求(上游 typeof null === 'object')。

修复:在 ForwardAsChatCompletions 里用 gjson 检测 body shape,当 input
存在且 messages 缺失时,跳过 Chat→Responses 转换,用 sjson 仅改写 model
字段后原样透传 body。billing 所需的 ServiceTier 和 Reasoning.Effort 通过
gjson 从 raw body 提取,下游 codex OAuth transform 路径保持不变。

测试:新增 openai_cursor_warmup_pipeline_test.go,覆盖 5 个 shape 检测
用例(正向/标准请求不误伤/两字段共存/空 body/JSON 回读)。
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

erio authored Apr 13, 2026 and 陈曦 committed Apr 14, 2026

fix(payment): fix Alipay/Wxpay direct provider type mapping and enable cross-provider load balancing

Two issues fixed:

1. Alipay.SupportedTypes() returned ["alipay_direct"] and Wxpay returned
   ["wxpay_direct"], but the frontend sends payment_type="alipay"/"wxpay".
   The registry lookup failed with "payment method (alipay) is not
   configured". Fix: return the base types ["alipay"]/["wxpay"].

2. When multiple providers support the same payment type (e.g. EasyPay
   and Alipay direct both handle "alipay"), only the last-registered
   provider's instances were reachable — the registry mapped one type to
   one provider key, and SelectInstance queried by that single key.

   Fix: bypass the registry in invokeProvider and let SelectInstance
   query across all providers when providerKey is empty. The selected
   instance's own ProviderKey (now included in InstanceSelection) is
   used to create the correct provider, enabling true cross-provider
   load balancing.

Closes #1592

sakurawztlt authored Apr 13, 2026 and 陈曦 committed Apr 14, 2026

b2e379cf 引入的 BufferedResponseAccumulator 已修复了 chat_completions
非流式路径和 responses OAuth 非流式路径，但遗漏了 Anthropic /v1/messages
非流式路径 (handleAnthropicBufferedStreamingResponse)。

当客户端请求 stream=false 且模型开启思考时，上游 response.completed
终态事件的 output 字段可能为空，实际 message 内容通过
response.output_text.delta 增量事件下发。旧代码只读终态事件的 Response，
导致客户端收到的 content 字段为空 ([{"type":"text"}])。

本 commit 将 b2e379cf 的相同修复模式镜像到 Anthropic 路径：在 SSE 扫描
过程中用 BufferedResponseAccumulator 累积 delta 内容，终态 output 为空
时通过 SupplementResponseOutput 补充重建。

同时修复 handleAnthropicBufferedStreamingResponse 遗漏 response.done
事件类型的问题，与 chat completions 路径保持一致，避免上游发送
response.done 时 handler 认不出终态事件、最终返回 502 的潜在问题。

BufferedResponseAccumulator 已在 chatcompletions_responses_test.go 有
完整单元测试覆盖（TextOnly/ToolCalls/Reasoning/Mixed/SupplementEmpty/
NoSupplementWhenOutputExists/EmptyDeltas/IgnoresNonFunctionCallItems），
本次复用相同累加器无需新增测试。

Billing (25 tests):
- CalculateCostUnified: nil resolver fallback, token/per_request/image modes
- GetModelPricingWithChannel: nil/partial/full channel overrides
- resolveAccountStatsCost: four-level priority chain integration tests

WebSearch (18 tests):
- PopulateWebSearchUsage: nil input, manager states, QuotaLimit nil/*int64
- ResetWebSearchUsage: nil manager error
- Manager.ResetUsage: nil Redis
- shouldEmulateWebSearch: full decision chain (8 scenarios)

Notify (36 tests):
- ParseNotifyEmails/MarshalNotifyEmails: old/new format, roundtrip
- crossedDownward: boundary values, threshold semantics
- checkQuotaDimCrossings: mixed dimensions, disabled/zero skip