OScope: every text file of commit 9d9eae4d711c8575b6fbb4d882cc53b3af7dc798, part 1 of 3. Code: https://git.oscope.trade/oscope · git clone https://git.oscope.trade/oscope.git · manifest: https://git.oscope.trade/oscope/manifest.json Each file starts with "===== FILE: =====" and a line with its sha256, size and line count. Nothing is left out or changed. ===== FILE: README.md ===== sha256: 2a63c60ece6f44fe9e6299f5e8dfc4974b3e5a51d356a6798696b3c37137e28d · bytes: 11986 · lines: 249
OScope # OScope **The open source memecoin terminal for Solana.** Live Pulse, a trade firehose, charts, holders, wallet trackers and paper trading.
It runs on your own computer. No account, no wallet, no keys, no fees. [Website](https://oscope.trade)  ·  [X](https://x.com/OScopetrade)  ·  [Start with your AI](#start-it-with-your-ai)  ·  [Safety](SAFETY.md)  ·  [Verify with AI](https://git.oscope.trade/oscope/ai)  ·  [For AI agents](AGENTS.md) ![License](https://img.shields.io/badge/license-MIT-4d8dff?style=flat-square) ![Node](https://img.shields.io/badge/node-18.17%2B-4d8dff?style=flat-square) ![Dependencies](https://img.shields.io/badge/dependencies-2-4d8dff?style=flat-square) ![Private keys](https://img.shields.io/badge/private%20keys-never-2fe39b?style=flat-square) ![Fees](https://img.shields.io/badge/fees-0%25-2fe39b?style=flat-square) OScope trailer Trailer with sound: latest release
--- ## Start it with your AI You do not need to code. Paste the prompt below into your AI. It checks that the code is safe, installs everything and starts OScope. A couple of minutes later trades are flowing on your screen. - **Best:** an AI that can run commands on your computer, like Claude Code, Codex, Cursor, Windsurf or Gemini CLI. It does every step itself. - **Also works:** a chat app like Claude, ChatGPT or Gemini. It gives you one command at a time and you paste back what you see. ```text I want to run OScope, an open source memecoin terminal for Solana, on my computer. The code is here: https://git.oscope.trade/oscope STEP 1. SAFETY CHECK. Do this before you install or run anything. Get the code (git clone https://git.oscope.trade/oscope.git) or read it in the browser. Do not run it yet. If you cannot run commands, read https://git.oscope.trade/oscope/ai.txt: every text file of the project in one page. Read every file in src/, tools/ and public/js/, and also package.json, package-lock.json and .npmrc. Then tell me in plain words: 1. Does it ever ask for, read, store or send a private key, a seed phrase or a wallet file? Can it sign or send a transaction? 2. Every internet address it connects to, and what for. 3. Does it read or write any file outside its own folder? 4. Does anything run at install time (install scripts in package.json, in its dependencies, in .npmrc)? How many dependencies does it have and what are they? 5. Does it only listen on my own computer (127.0.0.1) unless I change that? 6. Is anything hidden: scrambled code, encoded text, code downloaded at run time, eval? Compare what you found with SAFETY.md in the same repository. That file is the project's own list of what the code does and does not do. If the code does anything SAFETY.md does not say, or anything looks wrong or unclear, STOP and tell me. Do not go on until I say so. STEP 2. INSTALL AND START. Only if step 1 was clean. 1. Run `node -v`. OScope needs Node.js 18.17 or newer (20 or newer is best). If Node.js is missing or too old, help me install the current LTS version from https://nodejs.org the normal way for my system. 2. In the oscope folder run `npm ci`. It installs exactly what package-lock.json pins: ws, lightweight-charts and fancy-canvas (a helper lightweight-charts needs). Nothing else. 3. Run `npm run doctor`. It checks everything OScope needs. Fix whatever it reports. 4. Run `npm start` and leave it running. It prints the address to open, normally http://localhost:3000, and every 15 seconds a line with how many trades came in. 5. Tell me the address. Tell me that I should see new coins appear on the Pulse page and trades flowing on the Firehose page within a few seconds. If you cannot run commands on my computer, give me one step at a time, tell me exactly what to type and wait for me to paste the result. If a step fails, read AGENTS.md in the repository (it has a troubleshooting list), fix it, and tell me what you changed. ``` ### We want you to feel safe Never run code from the internet because a README says it is fine. That is how wallets get drained. So the prompt makes your AI **read the code first and tell you what it does, before anything is installed**, and stop if anything looks off. What it will find: | Check | Result | |---|---| | **No keys, ever** | OScope never asks for a private key, a seed phrase or a wallet file, and cannot sign or send a transaction. Trading is paper trading: practice money that lives in your browser. | | **Your computer only** | The server listens on `127.0.0.1`. Nobody on the internet, and no other website open in your browser, can reach it. | | **Public data only** | It reads a public trade stream, public SOL price pages and the pictures of the coins you look at. It sends nothing about you anywhere. No tracking. | | **Nothing runs at install** | Two libraries and one helper, exact versions, install scripts switched off. No build step: the code you read is the code that runs. | The full list, with the file behind every point so your AI can check it line by line: **[SAFETY.md](SAFETY.md)**. --- ## Inside Pulse: New Pairs, Final Stretch and Migrated, live Coin page: chart, live trades, holders and the paper trade panel | Feature | What it does | |---|---| | **Pulse** | **New Pairs**, **Final Stretch** and **Migrated**, live. Market cap, volume, trades, holders, top 10, dev, sniper and bundle holdings, socials and curve progress on every row. Filters per column. A column holds still while your mouse is on it. | | **Firehose** | Every pump.fun and PumpSwap trade the moment it lands, with trades per second, buy share and volume. Filter by size, side and venue. | | **Coin page** | Candles from 1 second to 4 hours, market cap or price, USD or SOL. Live trades, holders, top traders with PnL, the dev's other coins. Marks for the dev, your tracked wallets and your own trades. | | **Paper trading** | Quick buy from Pulse, buy and sell on the coin page, live PnL in Portfolio. Fills come from the pool's reserves and fee at the moment you click, so size moves the price like it does on chain. | | **Trackers** | Name any wallet and see its trades land, on every page. | --- ## Data ### Instant: the PumpApi firehose OScope starts with **no database, no Helius, no API key and no sign-up**. It reads the free public stream of [PumpApi](https://pumpapi.io): one message per pump.fun and PumpSwap transaction, with the pool's reserves and every trader's balance after it. Every number in OScope is built from that: - price and market cap from the pool reserves - holders from each trader's balance after their trade - dev = the wallet that launched the coin, bundle = buys in the launch block, snipers = buys in the next two blocks - charts from the trades You see everything **from the moment you start**. A coin launched after that is complete: every trade, every holder. A coin launched before shows its live price at once and builds its chart and holders from then on; where a number is not known yet OScope shows `–` instead of guessing. What it has seen is saved in `data/`, so a restart keeps it. ### The full picture A coin's past (its chart since launch, every holder of a week-old coin) needs history, and a live stream has none. Three ways to get it: | | What you do | What you get | Cost | |---|---|---|---| | **1. Start now with PumpApi** | Nothing. This is how OScope starts. Leave it running. | Everything from the moment you started. | Free | | **2. Helius** | An API key from [helius.dev](https://www.helius.dev); OScope asks it for a coin's past trades and full holder list when you open the coin. | The whole history of any coin, on demand. | Free plan for light use | | **3. Your own database + backfill** | Record the stream into a database, load past days once, start OScope from it. | Everything, for every coin, and it is yours. | Disk space and setup | 2 and 3 are **not built in yet**. History only has to be handed to the same function the live stream uses, and [AGENTS.md](AGENTS.md) describes exactly how. Ask your AI: *"Add a Helius backfill to OScope. Read AGENTS.md, section Adding history."* --- ## The firehose for your own tools PumpApi allows only a couple of connections per internet address. OScope opens one and **passes it on**, so your own bots and scripts read the same stream without opening another: ``` ws://localhost:3000/firehose every message ws://localhost:3000/firehose?pools=pump,pump-amm only pump.fun and PumpSwap ``` ```js import WebSocket from 'ws'; const ws = new WebSocket('ws://localhost:3000/firehose?pools=pump,pump-amm'); ws.on('message', m => { const t = JSON.parse(m); if (t.action === 'buy') console.log(t.symbol, t.quoteAmount, 'SOL'); }); ``` | Command | What it does | |---|---| | `npm run tape` | Trades scroll by in your terminal. `npm run tape -- 1` shows only trades of 1 SOL or more. | | `npm run firehose` | The firehose alone, without the page, on `ws://127.0.0.1:3100`. | | `npm run doctor` | Checks Node, the libraries, the port, the stream and the data folder, and says how to fix what is wrong. | --- ## Install by hand [Node.js](https://nodejs.org) 18.17 or newer (20+ recommended). ```sh git clone https://git.oscope.trade/oscope.git cd oscope npm ci npm start ``` Open **http://localhost:3000**. Something off? `npm run doctor`.
Settings
OScope needs no settings. To change one, copy `.env.example` to `.env`. | Setting | Default | What it does | |---|---|---| | `PORT` | `3000` | Port of the page. If it is taken and you set nothing, OScope takes the next free one and prints it. | | `HOST` | `127.0.0.1` | Only this computer. `0.0.0.0` opens it to your network (no password: trusted networks only). | | `FEED_URL` | `wss://stream.pumpapi.io/` | The live stream. Point it at a firehose you run to share one connection. | | `FIREHOSE` | `on` | `off` stops passing the stream on at `/firehose`. | | `IPFS_GATEWAYS` | three public gateways | Where coin pictures kept on IPFS are fetched from. | | `DATA_DIR` | `data` | Where OScope keeps what it saw between runs. | | `STRETCH_AT` | `60` | Curve progress (%) at which a coin moves to Final Stretch. | | `NEW_MAX_AGE_MIN` | `60` | Minutes a coin stays in New Pairs. | | `LOG_EVERY` | `15` | Seconds between the trade-count lines in the terminal. `0` turns them off. | Everything you set in the page (paper wallet, tracked wallets, filters, hidden coins) stays in your browser. Settings → Export saves it to a file; Import brings it back.
How it is built
``` src/ server.mjs web server, the page's websocket, the firehose feed.mjs stream connection and SOL price store.mjs coins, trades, candles, holders, the Pulse columns meta.mjs coin pictures and socials, every address checked first config.mjs settings tools/ doctor.mjs npm run doctor tape.mjs npm run tape firehose.mjs npm run firehose public/ index.html, css/, js/ the page: plain JavaScript modules, no build step ``` It uses about 400–500 MB of memory: full detail for the coins you can see or open, a light copy of the rest.
--- ## Credits Charts by [TradingView Lightweight Charts](https://github.com/tradingview/lightweight-charts) (Apache 2.0). Fonts: [Geist](https://vercel.com/font) (SIL Open Font License). Live data: [PumpApi](https://pumpapi.io). ## Not financial advice OScope shows public market data. Memecoins are extremely risky and most go to zero. ## License [MIT](LICENSE) ===== FILE: SAFETY.md ===== sha256: c6743a59b2a7f593a8db1db1934fcc7a29d8fe5e211ff3ec79802185a706f3c7 · bytes: 5735 · lines: 101 # Safety This file is the project's own list of everything OScope does and does not do on your computer. It is written to be **checked against the code**, by you or by your AI. Each point names the file it lives in. If the code ever does something this file does not say, that is a bug: please open an issue. ## What OScope never does - It never asks for, reads, stores or sends a **private key, seed phrase or wallet file**. There is no place to enter one. - It never **signs or sends a transaction**. It has no code that can. - It has **no accounts, no logins, no tracking, no analytics** and no "phone home". - It never **downloads or runs other code**. No auto-update, no `eval`, no scrambled or encoded code. - It never reads or writes files **outside its own folder**. (It reads `.env` if you made one, and writes `data/`.) Trading in OScope is **paper trading**: a practice wallet that exists only in your browser (`public/js/paper.js`). No real SOL and no real coins are involved. ## What it connects to All outgoing connections are made by the server, in two files. The server only *reads* from them; it sends no information about you. (Like any web request, the other side sees your IP address.) | Where | What for | Code | |---|---|---| | `wss://stream.pumpapi.io/` | The live trade stream. OScope listens; it sends nothing. You can change the address with `FEED_URL`. | `src/feed.mjs` `connectFeed` | | `api.coinbase.com`, `api.kraken.com`, `api.binance.com` | The SOL price in dollars, every 20 seconds. The first that answers is used. | `src/feed.mjs` `solPrice` | | The metadata and picture links of the coins you look at | Each coin's picture, description and social links. These addresses are chosen by whoever launched the coin. | `src/meta.mjs` | | `ipfs.filebase.io`, `4everland.io`, `pump.mypinata.cloud` | Public IPFS gateways. Most coins keep their picture on IPFS; a link of that kind is asked for on these. You can change the list with `IPFS_GATEWAYS`. | `src/meta.mjs` `candidates`, `src/config.mjs` | Because coin creators choose those addresses, every one is checked before it is fetched (`src/meta.mjs` `safeUrl` and `get`): - only `http` and `https`; - never an address on your own computer or your local network (the name is looked up first; private, loopback and link-local addresses are refused); - every redirect is checked the same way, three at most; - size limits (256 KB for metadata, 6 MB for a picture) and an 8 second time limit. `npm run doctor` and `npm run tape` also ask `api.coinbase.com` for the SOL price (`tools/`). **Your browser** talks only to your own OScope server. The page's content policy (`csp` in `src/server.mjs`) allows scripts, styles, fonts, pictures and connections from that server only: coin pictures reach the page through the server, never straight from the internet. Links to pump.fun, Solscan and X open only when you click them, in a new tab. ## What it listens on - `127.0.0.1:3000` by default: **only your own computer**. If port 3000 is taken, the next free port. - It answers only requests addressed to `localhost` / `127.0.0.1` (`hostOk` in `src/server.mjs`), so a website that points its own name at your computer gets nothing. - Two websockets: `/ws` for the page and `/firehose` for your own tools. A web page from any other site cannot open either (the Origin check in the `upgrade` handler in `src/server.mjs`). - It accepts only `GET` and `HEAD`. Nothing can be uploaded or changed through it. - `HOST=0.0.0.0` opens it to other devices on your network. Only do that on a network you trust: it has no password. - `npm run firehose` listens on `127.0.0.1:3100` and refuses every web page (`tools/firehose.mjs`). ## What it stores - `data/coins.json` and `data/devs.json`: coins, trades and holders it saw. Public chain data, nothing about you. - In your browser's local storage: your paper wallet, tracked wallet names, filters, hidden coins and settings. It stays in your browser. Settings → Export gives you a copy. ## What gets installed - Two libraries and one helper, exact versions, with checksums in `package-lock.json`: [`ws`](https://github.com/websockets/ws) (websockets), [`lightweight-charts`](https://github.com/tradingview/lightweight-charts) (TradingView's open source chart library) and `fancy-canvas` (a helper `lightweight-charts` needs). - `.npmrc` sets `ignore-scripts=true`: no package can run code while it installs. - `package.json` has no install scripts. - No build step. The files in `src/` and `public/` are what runs. The only minified file is the chart library's own release inside `node_modules/lightweight-charts/dist/`, as published on npm. ## Content from strangers Coin names, descriptions, links and pictures are written by whoever launched the coin. OScope treats them as untrusted: - names and descriptions are put on the page as plain text, never as HTML (`h()` in `public/js/util.js`); - links must be `http` or `https`, and open in a new tab with no access to the OScope page (`safeHref` in `public/js/util.js`, `cleanLink` in `src/meta.mjs`); - pictures are served with a policy that stops them from running scripts (the `/pic/` handler in `src/server.mjs`). ## Check it yourself ```sh # every outgoing connection in the code grep -rn "fetch(\|new WebSocket(" src tools public/js # anything that touches files grep -rn "node:fs\|node:child_process" src tools # nothing runs at install cat .npmrc grep -n "\"scripts\"" -A 6 package.json # keys, signing: there should be no match at all grep -rniE "private.?key|secret.?key|seed phrase|mnemonic|signTransaction|sendTransaction|keypair" src tools public/js ``` ## Found a problem? Tell us on X: https://x.com/OScopetrade ===== FILE: AGENTS.md ===== sha256: 7650af048b8e6721d76fd4c49f9e1668c385037f97bd024dacf6b53187ec130f · bytes: 9182 · lines: 146 # For AI assistants You are helping someone run OScope, an open source memecoin terminal for Solana, on their own computer. They may not know how to code. Use plain words with them, do one step at a time, and say what you did. ## Safety check first Before installing or running anything, read the code and report to the user. [SAFETY.md](SAFETY.md) lists what the code does and does not do, with the file for each point. Check the code against it: - no private key, seed phrase or wallet file is ever asked for, read, stored or sent; nothing signs or sends a transaction; - outgoing connections: only `src/feed.mjs` (the trade stream, SOL price) and `src/meta.mjs` (coin metadata and pictures, each address checked first); `tools/` asks for the SOL price and talks to the local server; - files: `src/config.mjs` reads `.env`; `src/store.mjs` reads and writes `data/`; `src/server.mjs` serves `public/` and one file of `node_modules/lightweight-charts/dist/`; - install: `.npmrc` has `ignore-scripts=true`, `package.json` has no install scripts, `package-lock.json` pins `ws`, `lightweight-charts` and `fancy-canvas`; - the server listens on `127.0.0.1` unless `HOST` says otherwise. If the code does anything SAFETY.md does not say, stop and tell the user. ## Install and run ```sh node -v # 18.17 or newer; 20 or newer is best git clone https://git.oscope.trade/oscope.git cd oscope npm ci # exactly what package-lock.json pins npm run doctor # checks Node, the libraries, the port, the stream, the data folder npm start # leave it running ``` `npm start` prints: ``` OScope is running Open http://127.0.0.1:3000 Firehose ws://127.0.0.1:3000/firehose Data wss://stream.pumpapi.io/ ``` then `feed: connected …`, and every 15 seconds a line like `live: 2,910 trades in the last 15 s (194/s) · 3 launches and 0 migrations seen · 2,513 coins in memory`. It works when: - that `live:` line shows trades (well above zero), and - `http://localhost:3000/api/status` answers with `"feed":{"connected":true…}` and a growing `stats.trades`. Tell the user to open the printed address. New Pairs fills within seconds. Final Stretch fills as curve coins trade. Migrated fills as coins migrate, which can take some minutes on a fresh start. The Firehose page shows every trade. No settings are needed. On Windows, set a setting by putting it in a `.env` file (copy `.env.example`) instead of typing `NAME=value` before the command. ## Troubleshooting | What you see | Why | What to do | |---|---|---| | `node: command not found`, or a version under 18.17 | Node.js missing or old | Install the current LTS from https://nodejs.org (or with the system's package manager, nvm, fnm, winget). | | `npm ci` fails | No `package-lock.json` in the folder, or no internet | Run it inside the `oscope` folder. Check the connection. | | `port 3000 is taken, trying 3001` | Another program uses the port | Nothing: OScope takes the next free port and prints the address. To choose one, set `PORT`. | | `feed: closed (…) after 0 s … only a few connections per IP address` | The stream allows only a couple of connections per internet address | Close other programs that read the same stream (another OScope, a bot). Or run one firehose (`npm run firehose`) and point everything at it with `FEED_URL=ws://127.0.0.1:3100`. | | `feed:` errors like `ENOTFOUND`, `ETIMEDOUT`, `certificate` | No internet, a firewall, a VPN or a proxy blocks websockets | It needs outgoing `wss://` on port 443. Try without the VPN. | | Dollar values show `–` | No SOL price source answered yet | It retries every 20 seconds by itself. Check that `api.coinbase.com` is reachable. | | The page says `Stream down` or `Stream quiet` | The server lost the stream | Read the terminal: the `feed:` lines say why. It reconnects by itself. | | A coin page says `Not seen yet` | The coin has not traded since OScope started | It appears at its next trade. See "Adding history". | | Holders, top 10, dev show `–` | The coin launched before OScope started, or its tokens were moved by plain transfers the stream does not carry | Expected with the live stream alone. See "Adding history". | | `403 OScope answers only on localhost` | The page was opened under another name while `HOST` is `127.0.0.1` | Open `http://localhost:`. To reach it from another device set `HOST=0.0.0.0`. | `npm run doctor` checks most of this and prints a fix for each problem. ## How the code is laid out - `src/server.mjs`: the web server. Serves `public/`, answers `/api/*`, runs the page's websocket `/ws` and the firehose `/firehose`. Only `GET` and `HEAD`. - `src/feed.mjs`: `connectFeed(url, onFrame)` keeps one websocket to the stream open and calls `onFrame(frame, rawText)` for every message. `solPrice()` keeps the SOL price. - `src/store.mjs`: `Store`. `ingest(frame)` is the only way data gets in. It keeps coins, trades, candles (one second and one minute), holders and the three Pulse columns, and saves a snapshot to `data/` every minute and on exit. - `src/meta.mjs`: fetches coin metadata and pictures, with the address checks. - `public/js/`: the page. Plain ES modules, no framework, no build step. One file per view (`pulse.js`, `live.js`, `coin.js`, `portfolio.js`, `trackerview.js`), plus `paper.js` (the paper wallet), `net.js` (the websocket). ## The frame format One frame is one transaction. `Store.ingest` uses these fields (amounts are whole SOL and whole tokens, not lamports): ```js { pool: 'pump' | 'pump-amm', // bonding curve | PumpSwap. Other pools are ignored. action: 'create' | 'buy' | 'sell' | 'migrate', mint, name, symbol, uri, // uri = the coin's metadata link quoteMint, // must be SOL (So11111111111111111111111111111111111111112) txSigner, // the wallet that sent the transaction (for create: the dev) breakdown: [{ action, trader, tokenAmount, quoteAmount }], // optional; without it the four fields above/below are used tokenAmount, quoteAmount, price, // SOL per token after the transaction marketCapQuote, // market cap in SOL after the transaction tokensInPool, quoteInPool, // real reserves after the transaction vTokensInBondingCurve, vQuoteInBondingCurve, // pump: virtual reserves virtualQuoteInPool, poolCreatedBy, // pump-amm: extra virtual SOL; 'pump' = the pool made by the migration supply, poolFeeRate, decimals, mayhemMode, postBalances: { [wallet]: { [mint]: tokensHeldAfter } }, block, timestamp, signature, priorityFee // slot, milliseconds } ``` ## Adding history The live stream starts at the moment OScope starts. A coin that launched earlier has no chart, trades or holders from before that. To add the past, **build frames in the format above and pass them to `store.ingest()` oldest first, before the live stream is connected** (in `src/server.mjs`, between `store.load()` and `connectFeed(...)`). Nothing else in OScope has to change: the columns, charts, holders and the page all read the store. Two ways people do it: **Helius (or any Solana RPC that keeps history).** Good for filling one coin when the user opens it. - Past trades: list the signatures for the coin's bonding curve or pool account (`getSignaturesForAddress`), load the transactions (`getTransaction` with `maxSupportedTransactionVersion`, or Helius' parsed transactions), and turn each pump.fun / PumpSwap trade into a frame. Reserves and the trader's balance after the trade are in the transaction's post balances. - Current holders: Helius' token-accounts API or `getTokenLargestAccounts` gives balances per wallet. Write them into the coin's `holders` map (`wallet -> { bal, … }`) instead of making frames. - Keep the API key in `.env` (for example `HELIUS_API_KEY`), read it in `src/config.mjs`, never send it to the page and never commit it. Stay inside the plan's rate limits: fetch on demand, cache the result in `data/`. **A local database and a backfill.** Good for running OScope all the time or for many users. - Record: read `ws://localhost:3000/firehose` (or the stream itself) and store every message, as rows in a database or as one file per hour. - Backfill: PumpApi publishes past hours of the same stream as downloadable archives (see their documentation). An hour is about half a gigabyte compressed, so load only the hours you need. - Start: at boot, read the stored frames for the window you want, oldest first, into `store.ingest()`, then connect the live stream. Raise `TRADE_CAP`, `MAX_COINS` and `IDLE_DROP_MIN` if you load a long window: everything is held in memory. ## Rules for changing this code - Never add code that takes a private key or signs a transaction unless the user asks for exactly that and understands it. - Keep the server on `127.0.0.1` by default. - Coin names, descriptions and links are untrusted: put them on the page as text (`h()` in `public/js/util.js`), never as HTML. - Every new outgoing connection goes into SAFETY.md. - No build step, no new dependency without a reason. ===== FILE: package.json ===== sha256: 5f445c2f61752dfb3894cab385039c3baaee5828b0067436d666e8da3e993b9c · bytes: 630 · lines: 25 { "name": "oscope", "version": "0.1.0", "description": "Open source memecoin terminal for Solana: live pulse, charts, holders, wallet tracking and paper trading.", "license": "MIT", "author": "OScope contributors", "type": "module", "engines": { "node": ">=18.17" }, "scripts": { "start": "node src/server.mjs", "tape": "node tools/tape.mjs", "firehose": "node tools/firehose.mjs", "doctor": "node tools/doctor.mjs" }, "dependencies": { "lightweight-charts": "5.2.1", "ws": "8.22.0" }, "repository": { "type": "git", "url": "https://git.oscope.trade/oscope.git" } } ===== FILE: package-lock.json ===== sha256: b57642727d96f6a889bee8feeec9c9ce6aaed68979a2c99b2980489e54a737b0 · bytes: 1623 · lines: 56 { "name": "oscope", "version": "0.1.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "oscope", "version": "0.1.0", "license": "MIT", "dependencies": { "lightweight-charts": "5.2.1", "ws": "8.22.0" }, "engines": { "node": ">=18.17" } }, "node_modules/fancy-canvas": { "version": "2.1.0", "resolved": "https://registry.npmjs.org/fancy-canvas/-/fancy-canvas-2.1.0.tgz", "integrity": "sha512-nifxXJ95JNLFR2NgRV4/MxVP45G9909wJTEKz5fg/TZS20JJZA6hfgRVh/bC9bwl2zBtBNcYPjiBE4njQHVBwQ==", "license": "MIT" }, "node_modules/lightweight-charts": { "version": "5.2.1", "resolved": "https://registry.npmjs.org/lightweight-charts/-/lightweight-charts-5.2.1.tgz", "integrity": "sha512-IVwoK1RLFiLPubaKIjNbtjWLnpPMqiABSrTay6whmNa8L1+19292VtHJ+BWyPUuLCwF0tcQlhEWd1CLB2a1nsQ==", "license": "Apache-2.0", "dependencies": { "fancy-canvas": "2.1.0" } }, "node_modules/ws": { "version": "8.22.0", "resolved": "https://registry.npmjs.org/ws/-/ws-8.22.0.tgz", "integrity": "sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg==", "license": "MIT", "engines": { "node": ">=10.0.0" }, "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "peerDependenciesMeta": { "bufferutil": { "optional": true }, "utf-8-validate": { "optional": true } } } } } ===== FILE: .npmrc ===== sha256: f7d367cb01a037434a5a1d32328abe5cfdff2eb7efad417bf879ea2c49ff1314 · bytes: 136 · lines: 4 # Never run install scripts of packages (OScope needs none). npm start / npm run still work. ignore-scripts=true audit=false fund=false ===== FILE: src/config.mjs ===== sha256: 5eb44a44a5ab64846170f9881b5df46eff8dfdcc6f9b6566f57bcbb64866589c · bytes: 2571 · lines: 45 // Settings come from the environment, or from a .env file next to package.json (see .env.example). // Every setting has a default: OScope runs with no .env at all. import fs from 'node:fs'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; export const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); // .env: KEY=value lines; a value already set in the environment wins try { for (const line of fs.readFileSync(path.join(ROOT, '.env'), 'utf8').split(/\r?\n/)) { const m = line.match(/^\s*([A-Z_][A-Z0-9_]*)\s*=\s*(.*?)\s*$/i); if (m && process.env[m[1]] == null) process.env[m[1]] = m[2].replace(/^(['"])(.*)\1$/, '$2'); } } catch {} const env = process.env; const num = (v, d) => (v != null && v !== '' && Number.isFinite(+v) ? +v : d); export const config = { port: num(env.PORT, 3000), portFixed: env.PORT != null && env.PORT !== '', // a port you chose is never swapped for a free one host: env.HOST || '127.0.0.1', // Live pump.fun + PumpSwap trade stream (JSON, one message per transaction). Free, no key, no sign-up. feedUrl: env.FEED_URL || 'wss://stream.pumpapi.io/', // OScope shares its one stream connection at ws://localhost:/firehose for your own tools. "off" turns it off. firehose: (env.FIREHOSE || 'on').toLowerCase() !== 'off', dataDir: path.resolve(ROOT, env.DATA_DIR || 'data'), // Public IPFS gateways asked for coin pictures and metadata, in this order (comma separated, each ending in /ipfs/). ipfsGateways: (env.IPFS_GATEWAYS || 'https://ipfs.filebase.io/ipfs/,https://4everland.io/ipfs/,https://pump.mypinata.cloud/ipfs/').split(',').map(s => s.trim()).filter(s => /^https:\/\/[^\s]+\/ipfs\/$/.test(s)), // A curve coin moves from New Pairs to Final Stretch at this bonding-curve progress (percent). stretchAt: num(env.STRETCH_AT, 60), // New Pairs keeps coins younger than this (minutes). newMaxAgeMin: num(env.NEW_MAX_AGE_MIN, 60), // Coins kept per Pulse column (the page filters further). columnCap: num(env.COLUMN_CAP, 80), // Trades kept in memory per coin that is on screen, open or young (the coin page's trade list reads these). tradeCap: num(env.TRADE_CAP, 1000), // Quiet coins are dropped from memory after this many minutes without a trade. idleDropMin: num(env.IDLE_DROP_MIN, 120), // Coins kept in memory at most; the ones traded least recently go first. maxCoins: num(env.MAX_COINS, 6000), // A line in the terminal every N seconds saying how many trades came in (0 = off). logEvery: num(env.LOG_EVERY, 15), }; ===== FILE: src/feed.mjs ===== sha256: a8ba8ede2edd8f7e23929c246668e295bc6c8f47286b3a696122783e75a42ac4 · bytes: 3137 · lines: 65 // The live trade stream (one websocket, reopened when it drops) and the SOL price. import WebSocket from 'ws'; export function connectFeed(url, onFrame, log = console.log) { let ws = null, wait = 1000, alive = 0, timer = null, closed = false; const state = { url, connected: false, since: 0, reconnects: 0 }; const open = () => { if (closed) return; ws = new WebSocket(url, { perMessageDeflate: true, handshakeTimeout: 15000 }); ws.on('open', () => { state.connected = true; state.since = Date.now(); if (!state.reconnects) log(`feed: connected to ${url}`); }); ws.on('message', buf => { alive = Date.now(); const raw = buf.toString(); let f; try { f = JSON.parse(raw); } catch { return; } // onFrame gets each transaction and its exact text (the firehose passes the text on untouched) if (Array.isArray(f)) for (const x of f) onFrame(x, null); else onFrame(f, raw); }); ws.on('close', (code, reason) => { const lived = state.connected ? Date.now() - state.since : 0; // a connection that lasted a while starts the wait over; one that drops at once backs off if (lived > 15000) wait = 1000; const why = `${code}${reason && reason.length ? ' ' + reason.toString().slice(0, 120) : ''}`; if (lived > 15000 || state.reconnects < 3 || state.reconnects % 20 === 0) { log(`feed: closed (${why}) after ${Math.round(lived / 1000)} s, retrying in ${Math.round(wait / 1000)} s${lived < 2000 && state.reconnects > 1 ? '. The stream may allow only a few connections per IP address: close other apps reading it.' : ''}`); } state.connected = false; state.reconnects++; setTimeout(open, wait); wait = Math.min(wait * 2, 30000); }); ws.on('error', err => { log(`feed: ${err.message}`); try { ws.terminate(); } catch {} }); }; open(); // a stream that goes quiet for 30 s is reopened timer = setInterval(() => { if (state.connected && Date.now() - alive > 30000) { log('feed: quiet for 30 s, reopening'); try { ws.terminate(); } catch {} } }, 10000); state.close = () => { closed = true; clearInterval(timer); try { ws.close(); } catch {} }; return state; } // SOL in dollars, from public exchange prices. export function solPrice(log = console.log) { const st = { usd: 0, at: 0 }; const sources = [ async () => +(await (await fetch('https://api.coinbase.com/v2/prices/SOL-USD/spot', { signal: AbortSignal.timeout(6000) })).json()).data.amount, async () => +(await (await fetch('https://api.kraken.com/0/public/Ticker?pair=SOLUSD', { signal: AbortSignal.timeout(6000) })).json()).result.SOLUSD.c[0], async () => +(await (await fetch('https://api.binance.com/api/v3/ticker/price?symbol=SOLUSDT', { signal: AbortSignal.timeout(6000) })).json()).price, ]; const tick = async () => { for (const s of sources) { try { const v = await s(); if (v > 1) { st.usd = v; st.at = Date.now(); return; } } catch {} } if (!st.usd) log('sol price: no source answered, retrying'); }; tick(); setInterval(tick, 20000).unref(); return st; } ===== FILE: src/meta.mjs ===== sha256: 05e36cc7c917937ee3af1f2772da62be16567495f35bc04b0c3a464d05e0716f · bytes: 7725 · lines: 183 // Coin metadata (picture, description, socials) from each coin's metadata link, and a small picture cache. // // Metadata links are chosen by whoever launched the coin, so every fetch is checked first: only http(s), and // never an address on this machine or a private network (a link like http://127.0.0.1/... must not reach // services running next to OScope). import dns from 'node:dns/promises'; import net from 'node:net'; import { config } from './config.mjs'; const MAX_JSON = 256 * 1024; const MAX_IMG = 6 * 1024 * 1024; const IMG_CACHE_BYTES = 64 * 1024 * 1024; function privateIp(ip) { if (net.isIPv4(ip)) { const [a, b] = ip.split('.').map(Number); return a === 10 || a === 127 || a === 0 || (a === 169 && b === 254) || (a === 172 && b >= 16 && b <= 31) || (a === 192 && b === 168) || (a === 100 && b >= 64 && b <= 127) || a >= 224; } const v = ip.toLowerCase(); if (v.startsWith('::ffff:')) return privateIp(v.slice(7)); return v === '::1' || v === '::' || v.startsWith('fc') || v.startsWith('fd') || v.startsWith('fe80'); } async function safeUrl(raw) { let u; try { u = new URL(raw); } catch { return null; } if (u.protocol !== 'https:' && u.protocol !== 'http:') return null; if (u.username || u.password) return null; const host = u.hostname.replace(/^\[|\]$/g, ''); if (host === 'localhost' || host.endsWith('.localhost') || host.endsWith('.local') || host.endsWith('.internal')) return null; try { const addrs = net.isIP(host) ? [{ address: host }] : await dns.lookup(host, { all: true }); if (!addrs.length || addrs.some(a => privateIp(a.address))) return null; } catch { return null; } return u; } async function get(raw, maxBytes, accept, hops = 0) { const u = await safeUrl(raw); if (!u) throw new Error('blocked url'); const res = await fetch(u, { redirect: 'manual', signal: AbortSignal.timeout(8000), headers: { accept, 'user-agent': 'OScope' } }); if (res.status >= 300 && res.status < 400 && res.headers.get('location') && hops < 3) { return get(new URL(res.headers.get('location'), u).href, maxBytes, accept, hops + 1); } if (!res.ok) throw new Error('http ' + res.status); const len = +res.headers.get('content-length') || 0; if (len > maxBytes) throw new Error('too big'); const chunks = []; let size = 0; for await (const ch of res.body) { size += ch.length; if (size > maxBytes) throw new Error('too big'); chunks.push(ch); } return { type: (res.headers.get('content-type') || '').split(';')[0].trim().toLowerCase(), body: Buffer.concat(chunks) }; } // Most coins keep their metadata and picture on IPFS and name one gateway in the link. Gateways come and go // (several big ones no longer answer programs at all), so a link with /ipfs/ is asked for on the gateways // in IPFS_GATEWAYS. The one that answered last is asked first. const DEAD_FOR_PROGRAMS = /^https?:\/\/(ipfs\.io|dweb\.link|w3s\.link|nftstorage\.link|gateway\.pinata\.cloud|cloudflare-ipfs\.com)\//i; const gateways = [...config.ipfsGateways]; function candidates(raw) { const m = String(raw).match(/^(?:https?:\/\/[^/]+\/ipfs\/|ipfs:\/\/(?:ipfs\/)?)([A-Za-z0-9]{20,}(?:[/?].*)?)$/); if (!m) return [raw]; const own = DEAD_FOR_PROGRAMS.test(raw) || raw.startsWith('ipfs:') ? [] : [raw]; return [...own, ...gateways.filter(g => !raw.startsWith(g)).map(g => g + m[1])]; } async function getAny(raw, maxBytes, accept) { let last; for (const url of candidates(raw)) { try { const r = await get(url, maxBytes, accept); const i = gateways.findIndex(g => url.startsWith(g)); if (i > 0) gateways.unshift(...gateways.splice(i, 1)); return r; } catch (e) { last = e; } } throw last || new Error('no address'); } const cleanLink = v => { if (typeof v !== 'string') return ''; v = v.trim(); if (!v) return ''; if (!/^https?:\/\//i.test(v)) { if (/^@?[A-Za-z0-9_]{1,15}$/.test(v)) return 'https://x.com/' + v.replace(/^@/, ''); if (/^[\w.-]+\.[a-z]{2,}(\/\S*)?$/i.test(v)) v = 'https://' + v; else return ''; } try { const u = new URL(v); return u.protocol === 'https:' || u.protocol === 'http:' ? u.href.slice(0, 300) : ''; } catch { return ''; } }; export class Meta { constructor(store) { this.store = store; this.queue = []; this.active = 0; this.limit = 12; this.imgs = new Map(); // mint -> { type, body } this.imgBytes = 0; this.imgWait = new Map(); this.imgBad = new Map(); // mint -> when its picture last failed: not asked for again for 5 minutes } want(c) { if (c.metaState) return; c.metaState = 1; this.queue.push(c); this.pump(); } // a page is looking at this coin: fetch it before the rest urgent(c) { if (c.metaState === 1) { const i = this.queue.indexOf(c); if (i > 0) { this.queue.splice(i, 1); this.queue.unshift(c); } } else if (!c.metaState) { c.metaState = 1; this.queue.unshift(c); this.pump(); } } pump() { while (this.active < this.limit && this.queue.length) { // newest first: a coin launched a second ago matters more than one from the backlog const c = this.queue.length > 400 ? this.queue.pop() : this.queue.shift(); this.active++; this.load(c).finally(() => { this.active--; this.pump(); }); } if (this.queue.length > 4000) this.queue.splice(0, this.queue.length - 4000); } async load(c) { if (!c.uri) { c.metaState = 3; return; } try { const r = await getAny(c.uri, MAX_JSON, 'application/json,*/*'); if (r.type.startsWith('image/')) { c.image = c.uri; } else { const j = JSON.parse(r.body.toString('utf8')); const img = typeof j.image === 'string' ? j.image : ''; c.image = img; c.description = typeof j.description === 'string' ? j.description.slice(0, 600) : ''; const ext = j.extensions || {}; c.twitter = cleanLink(j.twitter || ext.twitter || ''); c.website = cleanLink(j.website || ext.website || ''); c.telegram = cleanLink(j.telegram || ext.telegram || ''); if (!c.name && typeof j.name === 'string') c.name = j.name.slice(0, 60); if (!c.symbol && typeof j.symbol === 'string') c.symbol = j.symbol.slice(0, 20); } c.metaState = 2; } catch { // a link is often not reachable yet in the first seconds after a launch: try again a few times c.metaTries = (c.metaTries || 0) + 1; c.metaState = 3; const wait = [3000, 15000, 60000][c.metaTries - 1]; if (wait) setTimeout(() => { if (c.metaState === 3) { c.metaState = 0; this.want(c); } }, wait).unref(); } c.ver++; } async image(mint) { const hit = this.imgs.get(mint); if (hit) { this.imgs.delete(mint); this.imgs.set(mint, hit); return hit; } if (this.imgWait.has(mint)) return this.imgWait.get(mint); const c = this.store.coins.get(mint); if (!c || !c.image) return null; const bad = this.imgBad.get(mint); if (bad && Date.now() - bad < 300000) return null; const p = getAny(c.image, MAX_IMG, 'image/*').then(r => { if (!r.type.startsWith('image/')) { this.imgBad.set(mint, Date.now()); return null; } this.imgBad.delete(mint); const v = { type: r.type, body: r.body }; this.imgs.set(mint, v); this.imgBytes += r.body.length; for (const [k, old] of this.imgs) { if (this.imgBytes <= IMG_CACHE_BYTES) break; this.imgs.delete(k); this.imgBytes -= old.body.length; } return v; }).catch(() => { this.imgBad.set(mint, Date.now()); if (this.imgBad.size > 5000) this.imgBad.clear(); return null; }).finally(() => this.imgWait.delete(mint)); this.imgWait.set(mint, p); return p; } } ===== FILE: src/server.mjs ===== sha256: bbd0dd94f4ec70cdfd8d53515108ced30247cc5e5fa0e36559b20e2e3623189d · bytes: 18779 · lines: 382 // OScope server: reads the live trade stream, keeps the coins in memory, serves the terminal page and talks to it // over one websocket. It also passes the raw stream on at /firehose, so your own tools can share its one connection. import http from 'node:http'; import fs from 'node:fs'; import path from 'node:path'; import zlib from 'node:zlib'; import { WebSocketServer } from 'ws'; import { config, ROOT } from './config.mjs'; import { Store } from './store.mjs'; import { Meta } from './meta.mjs'; import { connectFeed, solPrice } from './feed.mjs'; const PUBLIC = path.join(ROOT, 'public'); const log = (...a) => console.log(new Date().toISOString().slice(11, 19), ...a); const store = new Store(); const meta = new Meta(store); store.wantMeta = c => meta.want(c); const loaded = store.load(); if (loaded) log(`loaded ${loaded} coins from ${config.dataDir}`); // pictures that were being fetched or had failed when OScope stopped are fetched again when their coin is looked at for (const c of store.coins.values()) if (c.metaState === 1 || c.metaState === 3) { c.metaState = 0; c.metaTries = 0; } const sol = solPrice(log); // ---------- firehose: the raw stream, passed on to local tools ---------- // ws://localhost:/firehose every message // ws://localhost:/firehose?pools=pump,pump-amm only those pools const fire = new Set(); function relay(f, raw) { let text = raw; for (const cl of fire) { if (cl.pools && !cl.pools.has(f.pool)) continue; if (cl.ws.readyState !== 1) continue; if (cl.ws.bufferedAmount > 8e6) { cl.ws.terminate(); continue; } // a reader that cannot keep up is cut, never the stream cl.ws.send(text ??= JSON.stringify(f)); } } const feed = connectFeed(config.feedUrl, (f, raw) => { store.ingest(f); if (fire.size) relay(f, raw); }, log); // ---------- websocket clients ---------- const clients = new Set(); const coinSubs = new Map(); // mint -> Set(client) const walletSubs = new Map(); // wallet -> Set(client) const MINT_RE = /^[1-9A-HJ-NP-Za-km-z]{32,44}$/; function send(cl, obj) { if (cl.ws.readyState !== 1) return; if (cl.ws.bufferedAmount > 16e6) { cl.ws.terminate(); return; } cl.ws.send(JSON.stringify(obj)); } function unsubCoin(cl) { if (!cl.coin) return; const s = coinSubs.get(cl.coin); if (s) { s.delete(cl); if (!s.size) { coinSubs.delete(cl.coin); store.pinned.delete(cl.coin); } } cl.coin = null; } function setWallets(cl, list) { for (const w of cl.wallets) { const s = walletSubs.get(w); if (s) { s.delete(cl); if (!s.size) walletSubs.delete(w); } } cl.wallets = new Set(list.filter(w => typeof w === 'string' && MINT_RE.test(w)).slice(0, 1000)); for (const w of cl.wallets) { if (!walletSubs.has(w)) walletSubs.set(w, new Set()); walletSubs.get(w).add(cl); } } let tape = []; let tapeOn = 0; store.onTrade = (c, tr) => { if (tapeOn) { if (!c.metaState) meta.want(c); tape.push([tr[0], c.mint, c.symbol, c.image ? `/pic/${c.mint}` : '', tr[1], tr[2], tr[5], tr[6], c.pool, tr[7], c.name]); if (tape.length > 3000) tape = tape.slice(-1500); } const subs = coinSubs.get(c.mint); if (subs) for (const cl of subs) cl.trades.push(tr); const ws = walletSubs.get(tr[6]); if (ws) { const item = { w: tr[6], m: c.mint, n: c.name, s: c.symbol, img: c.image ? `/pic/${c.mint}` : '', buy: tr[1], sol: tr[2], tok: tr[3], mc: tr[5], ts: tr[0], sig: tr[7] }; for (const cl of ws) cl.wt.push(item); } }; const wss = new WebSocketServer({ noServer: true, maxPayload: 256 * 1024, perMessageDeflate: { threshold: 1024 } }); wss.on('connection', ws => { const cl = { ws, pulse: false, tape: false, sent: new Map(), cols: '', coin: null, coinVer: 0, trades: [], watch: new Set(), wallets: new Set(), wt: [] }; clients.add(cl); send(cl, { t: 'hello', sol: sol.usd, feed: feed.connected, stretchAt: config.stretchAt }); ws.on('message', buf => { let msg; try { msg = JSON.parse(buf.toString()); } catch { return; } if (!msg || typeof msg !== 'object') return; if (msg.t === 'pulse') { cl.pulse = !!msg.on; cl.sent.clear(); cl.cols = ''; } else if (msg.t === 'tape') { if (!!msg.on !== cl.tape) { cl.tape = !!msg.on; tapeOn += cl.tape ? 1 : -1; } } else if (msg.t === 'coin') { unsubCoin(cl); if (typeof msg.mint === 'string' && MINT_RE.test(msg.mint)) { cl.coin = msg.mint; cl.coinVer = 0; cl.trades = []; if (!coinSubs.has(msg.mint)) coinSubs.set(msg.mint, new Set()); coinSubs.get(msg.mint).add(cl); store.pinned.add(msg.mint); const c = store.coins.get(msg.mint); if (c) { c.viewed = Date.now(); meta.urgent(c); } } } else if (msg.t === 'watch' && Array.isArray(msg.mints)) { cl.watch = new Set(msg.mints.filter(m => typeof m === 'string' && MINT_RE.test(m)).slice(0, 300)); } else if (msg.t === 'track' && Array.isArray(msg.wallets)) setWallets(cl, msg.wallets); }); ws.on('close', () => { unsubCoin(cl); setWallets(cl, []); if (cl.tape) tapeOn--; clients.delete(cl); }); }); // Pulse: every 400 ms each page gets the column order (when it changed) and the rows that changed since its last send. setInterval(() => { const cols = store.placeColumns(); const ids = { new: cols.new.map(c => c.mint), stretch: cols.stretch.map(c => c.mint), migrated: cols.migrated.map(c => c.mint) }; const idsKey = JSON.stringify(ids); const all = [...cols.new, ...cols.stretch, ...cols.migrated]; for (const c of all) if (!c.metaState || (c.metaState === 1 && !c.urged)) { c.urged = true; meta.urgent(c); } for (const cl of clients) { if (!cl.pulse || cl.ws.bufferedAmount > 4e6) continue; const rows = []; const present = new Set(); for (const c of all) { present.add(c.mint); if (cl.sent.get(c.mint) !== c.ver) { rows.push(store.row(c)); cl.sent.set(c.mint, c.ver); } } for (const m of cl.sent.keys()) if (!present.has(m)) cl.sent.delete(m); if (!rows.length && cl.cols === idsKey) continue; const out = { t: 'p', rows }; if (cl.cols !== idsKey) { out.cols = ids; cl.cols = idsKey; } send(cl, out); } }, 400); // Coin page: live trades every 250 ms, the coin's numbers when they change. Tracked wallets' trades. setInterval(() => { for (const cl of clients) { if (cl.coin) { const c = store.coins.get(cl.coin); if (c && cl.coinVer !== c.ver) { cl.coinVer = c.ver; send(cl, { t: 'c', d: store.detail(c) }); } if (cl.trades.length) { send(cl, { t: 'trs', m: cl.coin, list: cl.trades.slice(-200) }); cl.trades = []; } } if (cl.wt.length) { send(cl, { t: 'wt', list: cl.wt.slice(-100) }); cl.wt = []; } } if (tape.length) { const list = tape.slice(-400); tape = []; for (const cl of clients) if (cl.tape && cl.ws.bufferedAmount < 4e6) send(cl, { t: 'tape', list, n: store.stats.trades }); } }, 250); // Prices of the coins a page holds (paper positions), once a second. setInterval(() => { for (const cl of clients) { if (!cl.watch.size) continue; const p = {}; for (const m of cl.watch) { const c = store.coins.get(m); if (c && c.price) p[m] = [c.price, c.mc, c.pool, c.rq, c.rt, c.fee, c.name, c.symbol, c.image ? `/pic/${c.mint}` : '']; } send(cl, { t: 'px', p }); } }, 1000); setInterval(() => { const st = { t: 'st', sol: sol.usd, feed: feed.connected, lag: store.stats.lastFrame ? Date.now() - store.stats.lastFrame : null, coins: store.coins.size, trades: store.stats.trades, fire: fire.size }; for (const cl of clients) send(cl, st); }, 3000); setInterval(() => { // coins open on a page or held in a paper wallet keep their full detail const pinned = new Set(coinSubs.keys()); for (const cl of clients) for (const m of cl.watch) pinned.add(m); store.pinned = pinned; for (const m of coinSubs.keys()) { const c = store.coins.get(m); if (c) c.viewed = Date.now(); } const n = store.prune(); if (n > 2000) log(`dropped ${n} quiet coins, ${store.coins.size} kept`); }, 30000); setInterval(() => { try { store.save(); } catch (e) { log('save failed:', e.message); } }, 60000).unref(); // ---------- http ---------- const TYPES = { '.html': 'text/html; charset=utf-8', '.js': 'text/javascript; charset=utf-8', '.mjs': 'text/javascript; charset=utf-8', '.css': 'text/css; charset=utf-8', '.svg': 'image/svg+xml', '.png': 'image/png', '.ico': 'image/x-icon', '.woff2': 'font/woff2', '.json': 'application/json', '.txt': 'text/plain; charset=utf-8', '.webmanifest': 'application/manifest+json' }; // The page may load and connect to this server only: no outside scripts, pictures, fonts or connections. const csp = host => `default-src 'self'; img-src 'self' data: blob:; connect-src 'self' ws://${host} wss://${host}; style-src 'self' 'unsafe-inline'; font-src 'self'; script-src 'self'; object-src 'none'; base-uri 'none'; frame-ancestors 'self'`; const VENDOR = { '/vendor/lightweight-charts.mjs': path.join(ROOT, 'node_modules/lightweight-charts/dist/lightweight-charts.standalone.production.mjs') }; const fileCache = new Map(); function staticFile(file) { const st = fs.statSync(file); const hit = fileCache.get(file); if (hit && hit.mtime === st.mtimeMs) return hit; const body = fs.readFileSync(file); const type = TYPES[path.extname(file)] || 'application/octet-stream'; const text = /^(text|application\/(json|manifest))|svg/.test(type); const v = { mtime: st.mtimeMs, body, type, etag: `"${st.size.toString(36)}-${Math.floor(st.mtimeMs).toString(36)}"`, br: text ? zlib.brotliCompressSync(body) : null, gz: text ? zlib.gzipSync(body) : null }; fileCache.set(file, v); return v; } function sendStatic(req, res, file, cache) { let f; try { f = staticFile(file); } catch { return notFound(res); } const h = { 'content-type': f.type, etag: f.etag, 'cache-control': cache, 'x-content-type-options': 'nosniff', vary: 'accept-encoding' }; if (f.type.startsWith('text/html')) { const host = String(req.headers.host || '').replace(/[^a-zA-Z0-9.:\-\[\]]/g, '') || 'localhost'; h['content-security-policy'] = csp(host); h['referrer-policy'] = 'no-referrer'; } if (req.headers['if-none-match'] === f.etag) { res.writeHead(304, h); return res.end(); } const ae = req.headers['accept-encoding'] || ''; let body = f.body; if (f.br && /\bbr\b/.test(ae)) { body = f.br; h['content-encoding'] = 'br'; } else if (f.gz && /\bgzip\b/.test(ae)) { body = f.gz; h['content-encoding'] = 'gzip'; } res.writeHead(200, h); res.end(req.method === 'HEAD' ? undefined : body); } function json(res, obj, status = 200) { const body = JSON.stringify(obj); res.writeHead(status, { 'content-type': 'application/json', 'cache-control': 'no-store', 'x-content-type-options': 'nosniff' }); res.end(body); } function notFound(res) { res.writeHead(404, { 'content-type': 'text/plain' }); res.end('not found'); } const SPA = /^\/(|live|coin\/[1-9A-HJ-NP-Za-km-z]{32,44}|portfolio|trackers|wallet\/[1-9A-HJ-NP-Za-km-z]{32,44}|settings)$/; async function api(req, res, url) { const parts = url.pathname.split('/').filter(Boolean); // api, what, id const what = parts[1], id = parts[2]; if (what === 'status') { return json(res, { feed: { connected: feed.connected, reconnects: feed.reconnects, url: config.feedUrl }, firehose: config.firehose ? { path: '/firehose', readers: fire.size } : null, sol: sol.usd, coins: store.coins.size, clients: clients.size, stats: store.stats, uptime: Math.round(process.uptime()), memoryMB: Math.round(process.memoryUsage().rss / 1048576), held: { ...store.memory(), picturesMB: Math.round(meta.imgBytes / 1048576) } }); } if (what === 'search') { const q = (url.searchParams.get('q') || '').trim().slice(0, 60); if (!q) return json(res, []); if (MINT_RE.test(q) && store.coins.has(q)) return json(res, [store.row(store.coins.get(q))]); const lq = q.toLowerCase(), out = []; for (const c of store.coins.values()) { if (c.origin !== 'pump') continue; const s = c.symbol.toLowerCase(), n = c.name.toLowerCase(); const score = s === lq ? 3 : s.startsWith(lq) ? 2 : n.includes(lq) ? 1 : c.mint.startsWith(q) ? 1 : 0; if (score) out.push([score, c]); } out.sort((a, b) => b[0] - a[0] || b[1].mc - a[1].mc); const top = out.slice(0, 20).map(([, c]) => c); for (const c of top) if (!c.metaState) meta.urgent(c); return json(res, top.map(c => store.row(c))); } if (what === 'dev' && id && MINT_RE.test(id)) { const list = (store.devs.get(id) || []).map(e => { const c = store.coins.get(e.m); return { m: e.m, n: c?.name || e.n, s: c?.symbol || e.s, t: e.t, mig: c?.migratedAt || e.mig, mc: c?.mc || 0, ath: Math.max(c?.ath || 0, e.ath || 0), img: c?.image ? `/pic/${e.m}` : '', live: !!c }; }).sort((a, b) => b.t - a.t); return json(res, list); } if (what === 'wallet' && id && MINT_RE.test(id)) { const out = []; for (const c of store.coins.values()) { const h = c.holders.get(id); if (!h) continue; out.push({ m: c.mint, n: c.name, s: c.symbol, img: c.image ? `/pic/${c.mint}` : '', bal: h.bal, bs: h.bs, ss: h.ss, n_: h.n, last: h.last, mc: c.mc, pnl: h.ss - h.bs + h.bal * c.price, value: h.bal * c.price }); } out.sort((a, b) => b.last - a.last); return json(res, out.slice(0, 300)); } if (!id || !MINT_RE.test(id)) return notFound(res); const c = store.coins.get(id); if (!c) return json(res, { error: 'not seen', mint: id }, 404); c.viewed = Date.now(); if (what === 'coin') { meta.urgent(c); return json(res, store.detail(c)); } if (what === 'candles') { const tf = Math.max(1, Math.min(86400, parseInt(url.searchParams.get('tf') || '60', 10) || 60)); return json(res, store.candles(c, tf)); } if (what === 'trades') { const limit = Math.max(1, Math.min(2000, parseInt(url.searchParams.get('limit') || '200', 10) || 200)); return json(res, c.trades.slice(-limit).reverse()); } if (what === 'holders') return json(res, store.holdersOf(c, 100)); return notFound(res); } // When OScope listens only on this computer, it answers only requests addressed to this computer. A website // that points its own name at 127.0.0.1 (DNS rebinding) gets nothing. const LOCAL_ONLY = ['127.0.0.1', 'localhost', '::1'].includes(config.host); const LOCAL_NAMES = new Set(['localhost', '127.0.0.1', '[::1]']); function hostOk(req) { if (!LOCAL_ONLY) return true; const name = String(req.headers.host || '').replace(/:\d+$/, '').toLowerCase(); return LOCAL_NAMES.has(name) || name.endsWith('.localhost'); } const server = http.createServer(async (req, res) => { try { if (!hostOk(req)) { res.writeHead(403, { 'content-type': 'text/plain' }); return res.end('OScope answers only on localhost'); } const url = new URL(req.url, 'http://x'); const p = decodeURIComponent(url.pathname); if (req.method !== 'GET' && req.method !== 'HEAD') { res.writeHead(405); return res.end(); } if (p.startsWith('/api/')) return await api(req, res, url); if (p.startsWith('/pic/')) { const mint = p.slice(5); if (!MINT_RE.test(mint)) return notFound(res); const img = await meta.image(mint); if (!img) return notFound(res); res.writeHead(200, { 'content-type': img.type, 'cache-control': 'public, max-age=86400', 'x-content-type-options': 'nosniff', 'content-security-policy': "default-src 'none'; style-src 'unsafe-inline'; sandbox" }); return res.end(img.body); } if (VENDOR[p]) return sendStatic(req, res, VENDOR[p], 'public, max-age=86400'); if (SPA.test(p)) return sendStatic(req, res, path.join(PUBLIC, 'index.html'), 'no-cache'); const file = path.join(PUBLIC, path.normalize(p)); if (!file.startsWith(PUBLIC + path.sep)) return notFound(res); return sendStatic(req, res, file, 'no-cache'); } catch (e) { if (!res.headersSent) { res.writeHead(500, { 'content-type': 'text/plain' }); res.end('error'); } log('http error:', e.message); } }); const fireWss = new WebSocketServer({ noServer: true, maxPayload: 4096, perMessageDeflate: false }); fireWss.on('connection', (ws, req) => { const pools = new URL(req.url, 'http://x').searchParams.get('pools'); const cl = { ws, pools: pools ? new Set(pools.split(',').map(p => p.trim()).filter(Boolean)) : null }; fire.add(cl); ws.on('close', () => fire.delete(cl)); ws.on('message', () => {}); // readers only listen }); server.on('upgrade', (req, sock, head) => { const p = req.url.split('?')[0]; if (!hostOk(req)) return sock.destroy(); if (p !== '/ws' && !(p === '/firehose' && config.firehose)) return sock.destroy(); // A web page may open these sockets only if this server served it: other websites you visit cannot read them. // Tools (no Origin header) may. const origin = req.headers.origin; if (origin) { try { if (new URL(origin).host !== req.headers.host) return sock.destroy(); } catch { return sock.destroy(); } } const target = p === '/ws' ? wss : fireWss; target.handleUpgrade(req, sock, head, ws => target.emit('connection', ws, req)); }); // ---------- start ---------- let port = config.port; function listen() { server.listen(port, config.host); } server.on('error', err => { if (err.code === 'EADDRINUSE' && !config.portFixed && port < config.port + 20) { log(`port ${port} is taken, trying ${port + 1}`); port++; return setTimeout(listen, 50); } if (err.code === 'EADDRINUSE') { console.error(`\nPort ${port} is already in use. Pick another one: PORT=${port + 1} npm start\n`); process.exit(1); } console.error(err); process.exit(1); }); server.on('listening', () => { const host = config.host === '0.0.0.0' || config.host === '::' ? 'localhost' : config.host; const url = `http://${host}:${port}`; console.log(`\n OScope is running\n\n Open ${url}\n Firehose ${config.firehose ? url.replace('http', 'ws') + '/firehose' : 'off'}\n Data ${config.feedUrl}\n`); }); listen(); // a line every LOG_EVERY seconds, so you can see the trades coming in let lastTrades = 0; if (config.logEvery > 0) { setInterval(() => { const n = store.stats.trades - lastTrades; lastTrades = store.stats.trades; if (!feed.connected) return log('waiting for the stream…'); log(`live: ${n.toLocaleString('en-US')} trades in the last ${config.logEvery} s (${(n / config.logEvery).toFixed(0)}/s) · ${store.stats.creates} launches and ${store.stats.migrations} migrations seen · ${store.coins.size.toLocaleString('en-US')} coins in memory · ${Math.round(process.memoryUsage().rss / 1048576)} MB`); }, config.logEvery * 1000).unref(); } const stop = () => { try { store.save(); log('saved'); } catch {} process.exit(0); }; process.on('SIGINT', stop); process.on('SIGTERM', stop); ===== FILE: src/store.mjs ===== sha256: b93d06441ddafb200303675c5d97d9854f0e52b24d06374e7c3b65617090c65f · bytes: 18523 · lines: 415 // Everything the terminal knows about coins, built from the live trade stream. // // One frame = one transaction on the pump.fun bonding curve ("pump") or on PumpSwap ("pump-amm"). // Frames carry the pool reserves after the transaction, so price, market cap and curve progress are exact; // they also carry each trader's token balance after the transaction, so holder balances are exact for every // wallet that traded while OScope was running. // // All amounts are in SOL. The page turns them into dollars with the live SOL price. import fs from 'node:fs'; import path from 'node:path'; import { config } from './config.mjs'; const WSOL = 'So11111111111111111111111111111111111111112'; const CURVE_TOKENS = 793_100_000; // tokens a fresh pump.fun curve can sell before it migrates const isSol = q => q === WSOL || q === 'sol' || q == null; const now = () => Date.now(); // Memory: full detail only for coins somebody may look at (in a Pulse column, open on a page, held in the paper // wallet, or launched in the last ten minutes). Every other coin keeps a light copy: live numbers, its last trades, the // wallets that still hold it and two hours of minute candles. Without this, coins traded around the clock by bots // pile up until the process uses gigabytes. const LIGHT = { trades: 30, holders: 100, s1: 60, m1: 120 }; const YOUNG_MS = 10 * 60e3, VIEWED_MS = 30 * 60e3; const clamp = (v, a, b) => Math.min(b, Math.max(a, v)); function newCoin(f) { return { mint: f.mint, name: f.name || '', symbol: f.symbol || '', uri: f.uri || '', image: '', description: '', twitter: '', website: '', telegram: '', metaState: 0, created: 0, createdBlock: null, dev: '', origin: f.pool === 'pump' || f.poolCreatedBy === 'pump' ? 'pump' : 'other', pool: f.pool, poolId: f.poolId || '', supply: +f.supply || 1e9, decimals: f.decimals ?? 6, tokenProgram: f.tokenProgram || '', mayhem: !!f.mayhemMode, price: 0, mc: 0, ath: 0, liq: 0, rq: 0, rt: 0, inPool: 0, fee: +f.poolFeeRate || 0.0125, progress: 0, seenCurve: f.pool === 'pump', complete: false, migratedAt: 0, stretchAt: 0, buys: 0, sells: 0, volB: 0, volS: 0, fees: 0, firstSeen: now(), last: 0, holders: new Map(), // wallet -> { bal, bs, ss, bt, st, n, first, last } hGone: 0, // wallets still holding that were dropped from a light copy (counted in holders) viewed: 0, // when a page last asked for this coin snipers: new Set(), bundle: new Set(), trades: [], // [ts, isBuy, sol, tokens, priceSol, mcSol, wallet, signature] m1: [], // minute candles of market cap in SOL: [t, o, h, l, c, volSol] s1: [], // one-second candles, the same, for the last hour of trading seconds ver: 1, }; } export class Store { constructor() { this.coins = new Map(); this.devs = new Map(); // dev wallet -> [{ m, n, s, t, mig, ath }] this.onTrade = null; // (coin, trade) => void this.onCoin = null; // (coin) => void, a coin was created or migrated this.wantMeta = null; // (coin) => void. Pictures and socials are fetched only for coins somebody looks at. this.stats = { frames: 0, trades: 0, creates: 0, migrations: 0, lastFrame: 0, started: now() }; this.columns = { new: [], stretch: [], migrated: [] }; this.pinned = new Set(); // mints a page is looking at: never dropped } ingest(f) { this.stats.frames++; this.stats.lastFrame = now(); if (!f || !f.mint || (f.pool !== 'pump' && f.pool !== 'pump-amm')) return; const a = f.action; if (a === 'buy' || a === 'sell') this.trade(f); else if (a === 'create') this.create(f); else if (a === 'migrate' || a === 'complete' || (a === 'createPool' && f.poolCreatedBy === 'pump')) this.migrate(f); } coinOf(f) { let c = this.coins.get(f.mint); if (!c) { c = newCoin(f); this.coins.set(f.mint, c); } else { if (!c.name && f.name) c.name = f.name; if (!c.symbol && f.symbol) c.symbol = f.symbol; if (!c.uri && f.uri) c.uri = f.uri; if (f.poolCreatedBy === 'pump') c.origin = 'pump'; } return c; } applyPool(c, f, ts) { if (f.pool === 'pump') { c.seenCurve = true; c.rq = +f.vQuoteInBondingCurve || c.rq; c.rt = +f.vTokensInBondingCurve || c.rt; if (f.tokensInPool != null) { c.progress = clamp((CURVE_TOKENS - f.tokensInPool) / CURVE_TOKENS * 100, 0, 100); if (f.tokensInPool <= 0.5) c.complete = true; } } else { // PumpSwap: price = (SOL in pool + virtual SOL) / tokens in pool if (c.pool === 'pump' && c.seenCurve && !c.migratedAt && c.origin === 'pump' && f.poolCreatedBy === 'pump') this.markMigrated(c, ts); c.rq = (+f.quoteInPool || 0) + (+f.virtualQuoteInPool || 0); c.rt = +f.tokensInPool || c.rt; c.progress = 100; } c.pool = f.pool; if (f.poolId) c.poolId = f.poolId; if (f.supply) c.supply = +f.supply; if (f.poolFeeRate != null) c.fee = +f.poolFeeRate; if (f.price > 0) c.price = +f.price; if (f.marketCapQuote > 0) c.mc = +f.marketCapQuote; c.liq = (+f.quoteInPool || 0) + (+f.tokensInPool || 0) * c.price; if (f.tokensInPool != null) c.inPool = +f.tokensInPool; if (c.mc > c.ath) c.ath = c.mc; if (c.pool === 'pump' && c.progress >= config.stretchAt && !c.stretchAt) c.stretchAt = ts; } create(f) { if (!isSol(f.quoteMint)) return; const ts = +f.timestamp || now(); const c = this.coinOf(f); c.created = ts; c.createdBlock = f.block ?? null; c.dev = f.txSigner || f.creatorFeeAddress || ''; c.origin = 'pump'; c.mayhem = !!f.mayhemMode; this.applyPool(c, f, ts); this.stats.creates++; this.devAdd(c); if (this.wantMeta) this.wantMeta(c); // a new coin goes straight to New Pairs: get its picture now // a launch with a dev buy carries the buy in the same frame this.fills(c, f, ts); c.ver++; if (this.onCoin) this.onCoin(c, 'create'); } migrate(f) { const c = this.coins.get(f.mint); if (!c) return; const ts = +f.timestamp || now(); if (f.action === 'complete') { c.complete = true; c.ver++; return; } if (f.pool === 'pump-amm') this.applyPool(c, f, ts); if (!c.migratedAt) this.markMigrated(c, ts); } markMigrated(c, ts) { c.migratedAt = ts; c.complete = true; this.stats.migrations++; const d = c.dev && this.devs.get(c.dev); if (d) { const e = d.find(x => x.m === c.mint); if (e) e.mig = ts; } c.ver++; if (this.onCoin) this.onCoin(c, 'migrate'); } trade(f) { if (!isSol(f.quoteMint)) return; // anyone can open a side pool for a pump coin; only the pool made by the migration counts if (f.pool === 'pump-amm' && f.poolCreatedBy && f.poolCreatedBy !== 'pump') return; const ts = +f.timestamp || now(); const c = this.coinOf(f); this.applyPool(c, f, ts); this.fills(c, f, ts); c.ver++; } fills(c, f, ts) { const parts = Array.isArray(f.breakdown) && f.breakdown.length ? f.breakdown : [{ action: f.action, trader: f.txSigner, tokenAmount: f.tokenAmount, quoteAmount: f.quoteAmount }]; let any = false; for (const p of parts) { if (p.action !== 'buy' && p.action !== 'sell') continue; const buy = p.action === 'buy'; const sol = +p.quoteAmount || 0, tok = +p.tokenAmount || 0, w = p.trader || f.txSigner; if (!w || (!sol && !tok)) continue; let h = c.holders.get(w); if (!h) { h = { bal: 0, bs: 0, ss: 0, bt: 0, st: 0, n: 0, first: ts, last: ts }; c.holders.set(w, h); } if (buy) { h.bs += sol; h.bt += tok; c.buys++; c.volB += sol; } else { h.ss += sol; h.st += tok; c.sells++; c.volS += sol; } h.n++; h.last = ts; const post = f.postBalances?.[w]?.[c.mint]; h.bal = post != null && Number.isFinite(+post) ? +post : Math.max(0, h.bal + (buy ? tok : -tok)); if (buy && c.createdBlock != null && w !== c.dev && f.block != null) { const d = f.block - c.createdBlock; if (d === 0) c.bundle.add(w); else if (d > 0 && d <= 2) c.snipers.add(w); } c.fees += sol * c.fee; const tr = [ts, buy ? 1 : 0, sol, tok, c.price, c.mc, w, f.signature || '']; c.trades.push(tr); if (c.trades.length > config.tradeCap + 200) c.trades.splice(0, c.trades.length - config.tradeCap); this.candle(c, ts, c.mc, sol); this.stats.trades++; any = true; if (this.onTrade) this.onTrade(c, tr); } if (any) { c.fees += +f.priorityFee || 0; c.last = ts; } } candle(c, ts, v, sol) { if (!(v > 0)) return; this.bucket(c.s1, Math.floor(ts / 1000), v, sol, 3600); this.bucket(c.m1, Math.floor(ts / 60000) * 60, v, sol, 4320); } bucket(m, t, v, sol, cap) { const last = m[m.length - 1]; if (last && last[0] === t) { if (v > last[2]) last[2] = v; if (v < last[3]) last[3] = v; last[4] = v; last[5] += sol; } else if (!last || t > last[0]) { const o = last ? last[4] : v; m.push([t, o, Math.max(o, v), Math.min(o, v), v, sol]); if (m.length > cap + 100) m.splice(0, m.length - cap); } } devAdd(c) { if (!c.dev) return; let d = this.devs.get(c.dev); if (!d) { d = []; this.devs.set(c.dev, d); } if (!d.some(x => x.m === c.mint)) { d.push({ m: c.mint, n: c.name, s: c.symbol, t: c.created, mig: 0, ath: c.ath }); if (d.length > 100) d.splice(0, d.length - 100); } } // ---------- what the page reads ---------- holderStats(c) { let top = [], n = 0, devBal = 0, snBal = 0, buBal = 0, sum = 0; for (const [w, h] of c.holders) { if (h.bal <= 0.5) continue; n++; sum += h.bal; if (w === c.dev) devBal = h.bal; if (c.snipers.has(w)) snBal += h.bal; if (c.bundle.has(w)) buBal += h.bal; if (top.length < 10) { top.push(h.bal); if (top.length === 10) top.sort((a, b) => a - b); } else if (h.bal > top[0]) { top[0] = h.bal; top.sort((a, b) => a - b); } } const s = c.supply || 1e9, pct = v => +Math.min(100, v / s * 100).toFixed(2); // Tokens sent by plain transfer never show in the trade stream, so a sender still looks like a holder. // When the wallets add up to more than the tokens outside the pool, the split is not known. if (sum > (s - c.inPool) * 1.03 + 1) return { holders: n + c.hGone, top10: null, dev: null, snipers: null, bundle: null, devSold: false }; return { holders: n + c.hGone, top10: pct(top.reduce((a, b) => a + b, 0)), dev: pct(devBal), snipers: pct(snBal), bundle: pct(buBal), devSold: !!c.dev && c.holders.has(c.dev) && devBal <= 0.5 }; } row(c) { if (c._rowVer === c.ver && c._row) return c._row; const hs = this.holderStats(c); const d = c.dev ? this.devs.get(c.dev) : null; c._row = { m: c.mint, n: c.name, s: c.symbol, img: c.image ? `/pic/${c.mint}` : '', tw: c.twitter, web: c.website, tg: c.telegram, t: c.created, mig: c.migratedAt, last: c.last, mc: c.mc, ath: c.ath, liq: c.liq, vol: c.volB + c.volS, buys: c.buys, sells: c.sells, fees: c.fees, p: c.price, rq: c.rq, rt: c.rt, fee: c.fee, pr: +c.progress.toFixed(1), pool: c.pool, may: c.mayhem ? 1 : 0, done: c.complete ? 1 : 0, h: hs.holders, top10: hs.top10, dh: hs.dev, sn: hs.snipers, bu: hs.bundle, ds: hs.devSold ? 1 : 0, dev: c.dev, dc: d ? d.length : 0, dm: d ? d.filter(x => x.mig).length : 0, v: c.ver, }; c._rowVer = c.ver; return c._row; } detail(c) { return { ...this.row(c), price: c.price, supply: c.supply, rq: c.rq, rt: c.rt, fee: c.fee, poolId: c.poolId, desc: c.description, uri: c.uri, firstSeen: c.firstSeen, decimals: c.decimals, tp: c.tokenProgram, seen: !!c.created }; } holdersOf(c, limit = 100) { const list = []; for (const [w, h] of c.holders) list.push([w, h]); const tag = w => (w === c.dev ? 'dev' : c.bundle.has(w) ? 'bundle' : c.snipers.has(w) ? 'sniper' : ''); const holders = list.filter(([, h]) => h.bal > 0.5).sort((a, b) => b[1].bal - a[1].bal).slice(0, limit) .map(([w, h]) => ({ w, bal: h.bal, pct: h.bal / c.supply * 100, bs: h.bs, ss: h.ss, n: h.n, first: h.first, tag: tag(w) })); const traders = list.map(([w, h]) => [w, h, h.ss - h.bs + h.bal * c.price]).sort((a, b) => b[2] - a[2]).slice(0, limit) .map(([w, h, pnl]) => ({ w, bal: h.bal, pct: h.bal / c.supply * 100, bs: h.bs, ss: h.ss, bt: h.bt, st: h.st, n: h.n, pnl, tag: tag(w) })); return { holders, traders, total: list.filter(([, h]) => h.bal > 0.5).length }; } candles(c, tf) { // tf in seconds: under a minute from the one-second candles (the last hour), a minute and up from minute candles const src = tf < 60 ? c.s1 : c.m1; const step = tf; const out = []; for (const [t0, o, h, l, cl, v] of src) { const t = Math.floor(t0 / step) * step; const last = out[out.length - 1]; if (last && last[0] === t) { if (h > last[2]) last[2] = h; if (l < last[3]) last[3] = l; last[4] = cl; last[5] += v; } else out.push([t, o, h, l, cl, v]); } return out; } // ---------- Pulse columns ---------- placeColumns() { const t = now(), maxAge = config.newMaxAgeMin * 60e3, cap = config.columnCap; const nw = [], st = [], mg = []; for (const c of this.coins.values()) { if (c.origin !== 'pump') continue; if (c.migratedAt) { if (t - c.migratedAt < 24 * 3600e3) mg.push(c); continue; } if (c.pool !== 'pump') continue; if (c.progress >= config.stretchAt || c.complete) { if (t - c.last < 15 * 60e3) st.push(c); } else if (c.progress < config.stretchAt - 5 && c.stretchAt) c.stretchAt = 0; if (c.created && t - c.created < maxAge && c.progress < config.stretchAt && !c.complete) nw.push(c); } nw.sort((a, b) => b.created - a.created); st.sort((a, b) => b.progress - a.progress); mg.sort((a, b) => b.migratedAt - a.migratedAt); this.columns = { new: nw.slice(0, cap), stretch: st.slice(0, cap), migrated: mg.slice(0, cap) }; return this.columns; } // ---------- memory ---------- // a light copy of a coin nobody is looking at (see LIGHT) lighten(c) { if (c.trades.length > LIGHT.trades) c.trades.splice(0, c.trades.length - LIGHT.trades); if (c.s1.length > LIGHT.s1) c.s1.splice(0, c.s1.length - LIGHT.s1); if (c.m1.length > LIGHT.m1) c.m1.splice(0, c.m1.length - LIGHT.m1); if (c.holders.size > LIGHT.holders) { for (const [w, h] of c.holders) if (h.bal <= 0.5 && w !== c.dev) c.holders.delete(w); if (c.holders.size > LIGHT.holders) { const small = [...c.holders].filter(([w]) => w !== c.dev).sort((a, b) => a[1].bal - b[1].bal); for (const [w] of small.slice(0, c.holders.size - LIGHT.holders)) { c.holders.delete(w); c.hGone++; } } } c.light = true; } // what the memory holds, for /api/status memory() { let trades = 0, holders = 0, candles = 0, light = 0; for (const c of this.coins.values()) { trades += c.trades.length; holders += c.holders.size; candles += c.s1.length + c.m1.length; if (c.light) light++; } return { coins: this.coins.size, light, trades, holders, candles }; } prune() { const t = now(), idle = config.idleDropMin * 60e3; const keep = new Set(); for (const col of Object.values(this.columns)) for (const c of col) keep.add(c.mint); for (const m of this.pinned) keep.add(m); for (const c of this.coins.values()) { if (keep.has(c.mint) || t - c.viewed < VIEWED_MS || (c.created && t - c.created < YOUNG_MS)) c.light = false; else this.lighten(c); } let dropped = 0; for (const [m, c] of this.coins) { if (keep.has(m)) continue; const lastAct = Math.max(c.last, c.created, c.migratedAt, c.firstSeen); if (t - lastAct > idle) { this.coins.delete(m); dropped++; } } if (this.coins.size > config.maxCoins) { const list = [...this.coins.values()].filter(c => !keep.has(c.mint) && c.light).sort((a, b) => a.last - b.last); for (const c of list.slice(0, this.coins.size - config.maxCoins)) { this.coins.delete(c.mint); dropped++; } } // a busy coin's holder map: drop wallets that sold out long ago for (const c of this.coins.values()) { if (c.holders.size > 4000) { for (const [w, h] of c.holders) if (h.bal <= 0.5 && t - h.last > 10 * 60e3 && w !== c.dev) c.holders.delete(w); } } // dev index: keep 20,000 devs if (this.devs.size > 20000) { const list = [...this.devs.entries()].sort((a, b) => a[1].at(-1).t - b[1].at(-1).t); for (const [d] of list.slice(0, this.devs.size - 20000)) this.devs.delete(d); } return dropped; } // ---------- saved between runs ---------- save() { const dir = config.dataDir; fs.mkdirSync(dir, { recursive: true }); const t = now(), keep = []; for (const c of this.coins.values()) { if (c.origin !== 'pump') continue; // what the Pulse columns need after a restart, and what pages have open const fresh = (c.created && t - c.created < config.newMaxAgeMin * 60e3) || (c.migratedAt && t - c.migratedAt < 24 * 3600e3) || this.pinned.has(c.mint) || (c.progress >= config.stretchAt && t - c.last < 15 * 60e3); if (!fresh) continue; const holders = [...c.holders].sort((a, b) => b[1].bal - a[1].bal).slice(0, 200); keep.push({ ...c, holders, snipers: [...c.snipers], bundle: [...c.bundle], trades: c.trades.slice(-200), s1: c.s1.slice(-600), m1: c.m1.slice(-1440), _row: undefined, _rowVer: undefined }); } const write = (name, data) => { const p = path.join(dir, name); fs.writeFileSync(p + '.tmp', JSON.stringify(data)); fs.renameSync(p + '.tmp', p); }; write('coins.json', { saved: t, coins: keep }); write('devs.json', [...this.devs]); return keep.length; } load() { const dir = config.dataDir; try { const d = JSON.parse(fs.readFileSync(path.join(dir, 'devs.json'), 'utf8')); this.devs = new Map(d); } catch {} try { const { coins } = JSON.parse(fs.readFileSync(path.join(dir, 'coins.json'), 'utf8')); for (const o of coins) { const c = { ...newCoin(o), ...o, holders: new Map(o.holders), snipers: new Set(o.snipers), bundle: new Set(o.bundle), ver: 1 }; if (!c.s1.length) for (const tr of c.trades) this.bucket(c.s1, Math.floor(tr[0] / 1000), tr[5], tr[2], 3600); this.coins.set(c.mint, c); } return coins.length; } catch { return 0; } } } ===== FILE: tools/doctor.mjs ===== sha256: ea013ea0c59384c664836e8f218f335098596455315798f0d772371f360d3691 · bytes: 3865 · lines: 65 // npm run doctor: checks everything OScope needs and says how to fix what is missing. import fs from 'node:fs'; import path from 'node:path'; import net from 'node:net'; import { config, ROOT } from '../src/config.mjs'; const ok = s => console.log(` \x1b[32m✓\x1b[0m ${s}`); const bad = (s, fix) => { console.log(` \x1b[31m✗\x1b[0m ${s}${fix ? `\n fix: ${fix}` : ''}`); problems++; }; const note = s => console.log(` \x1b[2m· ${s}\x1b[0m`); let problems = 0; console.log('\nOScope doctor\n'); // 1. Node.js const [maj, min] = process.versions.node.split('.').map(Number); if (maj > 18 || (maj === 18 && min >= 17)) ok(`Node.js ${process.versions.node}`); else bad(`Node.js ${process.versions.node} is too old`, 'install Node.js 20 or newer (LTS) from https://nodejs.org'); // 2. dependencies for (const p of ['ws', 'lightweight-charts']) { if (fs.existsSync(path.join(ROOT, 'node_modules', p, 'package.json'))) ok(`${p} installed`); else bad(`${p} is not installed`, 'run: npm ci'); } // 3. the port: free, or OScope already running there const status = await fetch(`http://127.0.0.1:${config.port}/api/status`, { signal: AbortSignal.timeout(2000) }).then(r => r.json()).catch(() => null); let localFire = false; if (status && status.feed) { ok(`OScope is already running on http://localhost:${config.port} (stream ${status.feed.connected ? 'live' : 'not connected'}, ${status.stats?.trades ?? 0} trades so far)`); localFire = !!status.firehose && status.feed.connected; } else { const free = await new Promise(res => { const s = net.createServer().once('error', () => res(false)).once('listening', () => s.close(() => res(true))).listen(config.port, '127.0.0.1'); }); if (free) ok(`port ${config.port} is free`); else note(`port ${config.port} is used by another program: OScope will take the next free port and print its address`); } // 4. the stream { const { default: WebSocket } = await import('ws'); const url = localFire ? `ws://127.0.0.1:${config.port}/firehose` : config.feedUrl; const r = await new Promise(res => { let n = 0, opened = false; const ws = new WebSocket(url, { handshakeTimeout: 8000 }); const done = why => { try { ws.terminate(); } catch {} res({ n, opened, why }); }; ws.on('open', () => { opened = true; setTimeout(() => done('time'), 5000); }); ws.on('message', () => n++); ws.on('close', code => done('closed ' + code)); ws.on('error', e => done(e.message)); }); if (r.n > 0) ok(`stream works: ${r.n} messages in 5 s from ${url}`); else if (r.opened) bad(`the stream opened but sent nothing (${r.why})`, 'the stream allows only a few connections per internet address: close other programs reading it, or point FEED_URL at a firehose you already run'); else bad(`cannot reach the stream at ${url} (${r.why})`, 'check your internet connection, VPN or firewall (it needs outgoing websockets on port 443)'); } // 5. SOL price const px = await fetch('https://api.coinbase.com/v2/prices/SOL-USD/spot', { signal: AbortSignal.timeout(6000) }).then(r => r.json()).then(j => +j.data.amount).catch(() => 0); if (px > 0) ok(`SOL price: $${px.toFixed(2)}`); else note('SOL price source did not answer: OScope tries two more, dollar values wait until one answers'); // 6. data folder try { fs.mkdirSync(config.dataDir, { recursive: true }); fs.accessSync(config.dataDir, fs.constants.W_OK); ok(`data folder ${path.relative(ROOT, config.dataDir) || config.dataDir} is writable`); } catch { bad(`cannot write ${config.dataDir}`, 'set DATA_DIR in .env to a folder you can write to'); } console.log(problems ? `\n${problems} problem(s). Fix them and run npm run doctor again.\n` : status ? `\nAll good. OScope is running: open http://localhost:${config.port}\n` : `\nAll good. Run: npm start (then open http://localhost:${config.port})\n`); process.exit(problems ? 1 : 0); ===== FILE: tools/firehose.mjs ===== sha256: 4cd35a01cebfe50f8f99557a3181361b563ddd0d60758ee24bdf50a16ffbabd4 · bytes: 1999 · lines: 39 // The firehose on its own: one connection to the stream, passed on to as many local readers as you like. // OScope itself already does this at ws://localhost:3000/firehose; use this when you want the stream without the page. // // npm run firehose -> ws://127.0.0.1:3100 // readers may add ?pools=pump,pump-amm to get only those pools // // Settings: FEED_URL (the stream), FIREHOSE_PORT (3100), FIREHOSE_HOST (127.0.0.1). // Point OScope at it with FEED_URL=ws://127.0.0.1:3100 to run several things on one stream connection. import { WebSocketServer } from 'ws'; import { config } from '../src/config.mjs'; import { connectFeed } from '../src/feed.mjs'; const port = Number(process.env.FIREHOSE_PORT) || 3100; const host = process.env.FIREHOSE_HOST || '127.0.0.1'; const readers = new Set(); let n = 0; const wss = new WebSocketServer({ port, host, maxPayload: 4096, perMessageDeflate: false }); wss.on('connection', (ws, req) => { if (req.headers.origin) return ws.close(1008, 'tools only'); // web pages you visit cannot read it const pools = new URL(req.url, 'http://x').searchParams.get('pools'); const r = { ws, pools: pools ? new Set(pools.split(',')) : null }; readers.add(r); ws.on('close', () => readers.delete(r)); }); wss.on('listening', () => console.log(`\n Firehose on ws://${host}:${port}\n Stream ${config.feedUrl}\n`)); wss.on('error', e => { console.error(e.code === 'EADDRINUSE' ? `Port ${port} is in use: FIREHOSE_PORT=${port + 1} npm run firehose` : e.message); process.exit(1); }); connectFeed(config.feedUrl, (f, raw) => { n++; let text = raw; for (const r of readers) { if (r.pools && !r.pools.has(f.pool)) continue; if (r.ws.bufferedAmount > 8e6) { r.ws.terminate(); continue; } r.ws.send(text ??= JSON.stringify(f)); } }); setInterval(() => { console.log(`${new Date().toTimeString().slice(0, 8)} ${n.toLocaleString('en-US')} messages in the last 30 s, ${readers.size} reader(s)`); n = 0; }, 30000).unref();