Skip to content

Gate Simple Earn Pipeline

SyncGateSimpleEarnWorkflow снимает snapshot Gate Simple Earn из confirmed web API market/list и пишет snapshots в Timescale через SQL ingest.

Компоненты

  • SyncGateSimpleEarnWorkflow — вызывает Gate Simple Earn overview API, нормализует asset rows и пишет snapshots в Timescale через SQL ingest.
  • internal/worker/models/gate_simple_earn.go
  • internal/worker/activities/sync_gate_simple_earn.go
  • internal/worker/workflows/sync_gate_simple_earn.go
  • internal/db/migrations/00020_gate_simple_earn_snapshot_schema.sql
  • internal/db/migrations/00021_gate_simple_earn_aggregates_policies.sql

API Endpoint

  • GET https://www.gate.com/apiw/v2/uni-loan/earn/market/list

Confirmed request params for overview:

  • available=false Observed meaning: do not restrict the list to only currently available products.
  • limit=1000 by default Page size for one API request.
  • have_balance=2 Observed meaning: do not filter the list by current user balance.
  • have_award=0 Observed meaning: do not restrict the list to award-specific products.
  • is_subscribed=0 Observed meaning: do not restrict the list to products already subscribed by the current user.
  • sort_business=1 Observed meaning: use Gate's default business sorting for the market list.
  • search_type=0 Observed meaning: use the default search mode for search_coin.
  • page=<n> Page number for pagination.
  • optional search_coin=<SYMBOL> Optional asset filter, normalized to uppercase by the activity.

Discovery notes:

  • plain curl https://www.gate.com/simple-earn returned Access Denied from this environment;
  • browser network inspection confirmed the usable source is GET /apiw/v2/uni-loan/earn/market/list;
  • direct HTTP requests from the worker environment became unreliable because Gate started requiring browser-generated session state;
  • the current pipeline bootstraps a browser session on https://www.gate.com/en-us/simple-earn and then calls market/list through Playwright BrowserContext.Request();
  • these parameter meanings are based on parameter names and observed response behavior, not on official Gate API documentation.

Mapping

  • project = asset row from data.list
  • subpool = item from fixed_list or fixable_list
  • project_coin <= top-level asset
  • project_name <= top-level name fallback to asset symbol
  • reward_coin <= bonus_asset when present, otherwise the staked asset itself
  • project_status aggregates subpool statuses
  • subpool_status:
  • future start_time => Upcoming
  • past end_time => Ended
  • sale_status in (1,2) without ended timing => Active
  • sale_status_text:
  • 1 => Subscribe
  • 2 => Sold Out

Detail route is not required for current mapping. The confirmed overview payload already contains embedded fixed_list and fixable_list.

Persistence

  • ingest_gate_simple_earn_snapshot($1::jsonb)
  • internal path /apiw/v2/uni-loan/earn/market/list
  • source marker playwright

Tables:

  • fct_gate_simple_earn_snapshot
  • fct_gate_simple_earn_subpool_snapshot
  • fct_gate_simple_earn_response_snapshot
  • fct_gate_simple_earn_quarantine
  • fct_gate_simple_earn_5min
  • fct_gate_simple_earn_daily

Manual Run

docker compose exec -T temporal-admin-tools temporal workflow start \
  --address temporal:7233 --namespace default --task-queue default \
  --type SyncGateSimpleEarnWorkflow \
  --workflow-id sync-gate-simple-earn-manual-$(date +%s) \
  --input '{}'

Verification

  • Workflow result counters: projects_seen, projects_upserted, subpools_seen, subpools_upserted, quarantine_rows
  • SQL:
  • SELECT COUNT(*) FROM fct_gate_simple_earn_snapshot;
  • SELECT COUNT(*) FROM fct_gate_simple_earn_subpool_snapshot;
  • SELECT reason, COUNT(*) FROM fct_gate_simple_earn_quarantine GROUP BY reason ORDER BY COUNT(*) DESC;