Skip to content

System Overview

Актуальный scope

Проект содержит восемь рабочих направлений:

  • синхронизация бирж из CoinGecko в exchanges;
  • синхронизация каталога сетей в networks_catalog и связанных таблицах;
  • синхронизация монет/сетей с 8 бирж в exchange_assets* и conservative global mapping;
  • snapshot Gate Launchpool в fct_gate_launchpool_* (Timescale hypertables + continuous aggregates);
  • scraping BingX Launchpool через Playwright;
  • snapshot BingX Launchpool в fct_bingx_launchpool_* (Timescale hypertables + continuous aggregates).
  • scraping Bitget Launchpool через Playwright;
  • snapshot Bitget Launchpool в fct_bitget_launchpool_* (Timescale hypertables + continuous aggregates).
  • scraping BingX XPool через Playwright;
  • snapshot BingX XPool в fct_bingx_xpool_* (Timescale hypertables + continuous aggregates).

Компоненты

  • cmd/worker — Temporal worker
  • internal/worker/workflows/sync_exchanges.go
  • internal/worker/workflows/sync_networks_catalog.go
  • internal/worker/workflows/sync_exchange_coins.go
  • internal/worker/workflows/sync_exchange_coins_child.go
  • internal/worker/workflows/sync_gate_launchpool.go
  • internal/worker/workflows/sync_gate_launchpool_schedule.go
  • internal/worker/workflows/scrape_bingx_launchpool.go
  • internal/worker/workflows/sync_bingx_launchpool.go
  • internal/worker/workflows/scrape_bitget_launchpool.go
  • internal/worker/workflows/sync_bitget_launchpool.go
  • internal/worker/workflows/scrape_bingx_xpool.go
  • internal/worker/workflows/sync_bingx_xpool.go
  • internal/worker/activities/sync_exchanges.go
  • internal/worker/activities/sync_networks_catalog.go
  • internal/worker/activities/sync_exchange_coins.go
  • internal/worker/activities/sync_gate_launchpool.go
  • internal/worker/activities/sync_gate_launchpool_schedule.go
  • internal/worker/activities/scrape_bingx_launchpool.go
  • internal/worker/activities/sync_bingx_launchpool.go
  • internal/worker/activities/scrape_bitget_launchpool.go
  • internal/worker/activities/sync_bitget_launchpool.go
  • internal/worker/activities/scrape_bingx_xpool.go
  • internal/worker/activities/sync_bingx_xpool.go
  • internal/db/migrations/00001_baseline.sql
  • internal/db/migrations/00002_network_catalog.sql
  • internal/db/migrations/00003_exchange_coins_catalog.sql
  • internal/db/migrations/00005_gate_launchpool_snapshot_schema.sql
  • internal/db/migrations/00006_gate_launchpool_aggregates_policies.sql
  • internal/db/migrations/00007_bingx_launchpool_snapshot_schema.sql
  • internal/db/migrations/00008_bingx_launchpool_aggregates_policies.sql
  • internal/db/migrations/00009_bingx_xpool_snapshot_schema.sql
  • internal/db/migrations/00010_bingx_xpool_aggregates_policies.sql
  • internal/db/migrations/00013_bitget_launchpool_snapshot_schema.sql
  • internal/db/migrations/00014_bitget_launchpool_aggregates_policies.sql

Поток данных

flowchart LR
  CG[CoinGecko API] --> EX[SyncExchangesWorkflow]
  EX --> DB[(PostgreSQL exchanges)]
  CGP[CoinGecko /asset_platforms] --> NC[SyncNetworksCatalogWorkflow]
  CH[chainid.network] --> NC
  LL[DefiLlama /v2/chains] --> NC
  NC --> NDB[(PostgreSQL networks_catalog)]
  EXS[Bybit/Binance/Gate/Bitget/BingX/KuCoin/OKX/MEXC APIs] --> EC[SyncExchangeCoinsWorkflow]
  EC --> CDB[(PostgreSQL exchange_assets/global_assets)]
  GLP[Gate Launchpool API project-list] --> GLPW[SyncGateLaunchpoolWorkflow]
  GLPW --> TS[(Timescale fct_gate_launchpool_* + cagg)]
  BX[BingX Launchpool UI/API] --> XP[ScrapeBingXLaunchpoolWorkflow]
  XP --> BLPW[SyncBingXLaunchpoolWorkflow]
  BLPW --> BTS[(Timescale fct_bingx_launchpool_* + cagg)]
  BTP[Bitget Launchpool UI/API] --> BTPW[ScrapeBitgetLaunchpoolWorkflow]
  BTPW --> BTLPW[SyncBitgetLaunchpoolWorkflow]
  BTLPW --> BTTS[(Timescale fct_bitget_launchpool_* + cagg)]
  BXX[BingX XPool UI/API] --> XPW[ScrapeBingXXPoolWorkflow]
  XPW --> BXPW[SyncBingXXPoolWorkflow]
  BXPW --> BXTS[(Timescale fct_bingx_xpool_* + cagg)]

DB стратегия

После squashed baseline поддерживается только fresh-install через reset окружений.