1
0
mirror of https://github.com/misskey-dev/misskey.git synced 2026-07-09 16:54:54 +02:00

Compare commits

..

59 Commits

Author SHA1 Message Date
syuilo
d5609abf25 wip 2026-07-09 22:02:37 +09:00
syuilo
cf3b206ef4 wip 2026-07-09 21:24:33 +09:00
syuilo
951e22242d wip 2026-07-09 21:05:14 +09:00
syuilo
cc356cf542 wip 2026-07-09 20:32:20 +09:00
syuilo
f0b3a7241d wip 2026-07-09 18:48:28 +09:00
syuilo
e7bc3821d1 Update MkImageGallery.item.vue 2026-07-09 18:25:08 +09:00
syuilo
d2647e066e Update MkImageGallery.item.vue 2026-07-09 17:53:09 +09:00
syuilo
b7ca70bd1e wip 2026-07-09 17:42:34 +09:00
syuilo
564ed21b74 Update MkImageGallery.item.vue 2026-07-09 17:31:09 +09:00
syuilo
5c7f547342 wip 2026-07-09 17:03:20 +09:00
syuilo
75e97da41e wip 2026-07-09 16:13:16 +09:00
かっこかり
059fbaa92b fix(frontend/test): follow-up of #17682 of 2 (#17685)
* fix(frontend/test): follow-up of #17682 of 2

* fix
2026-07-08 22:37:19 +09:00
かっこかり
76ab562810 fix: follow-up of #17682 [ci skip] 2026-07-08 21:19:35 +09:00
かっこかり
9472e72678 enhance(frontend/test): frontend e2eをPlaywrightに移行 (#17682)
* enhance(frontend/test): frontend e2eをVitest Browser Modeに移行

* fix

* fix unit tests

* fix

* fix

* update playwright

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* perf

* Revert "perf"

This reverts commit e60e965c3c.

* fix
2026-07-08 20:58:02 +09:00
かっこかり
9440423a73 refactor(frontend): フロントエンドのfluent-emojisをemoji-assetsから取ってくるように (#17662)
refactor(frontend): フロントエンドのfluent-emojis使用箇所を `@misskey-dev/emoji-assets` のものに変更
2026-07-08 20:11:13 +09:00
かっこかり
b2867eb0d7 fix(frontend): ディレクティブのイベントリスナーがマウント解除後も残る問題を修正 (#17683)
* fix(frontend): ツールチップがマウント解除後もイベントリスナーが残る問題を修正

* more fix

* fix

* fix/refactor

* Update Changelog

* update comment

* refactor

* fix
2026-07-08 19:45:43 +09:00
syuilo
5590f8bad9 chore(backend): COOP/COEPヘッダーを送信するテスト用引数 2026-07-08 13:15:03 +09:00
おさむのひと
c15f93e48d fix: Sentry 使用環境下にて、外部送信リクエストへ sentry-trace / baggage ヘッダが既定で付与されないように (#17677)
* fix: Sentry 使用環境下にて、外部送信リクエストへ `sentry-trace` / `baggage` ヘッダーが既定で付与されないように

* fix
2026-07-08 09:46:44 +09:00
github-actions[bot]
03d224acd0 Bump version to 2026.7.0-alpha.2 2026-07-06 10:03:45 +00:00
かっこかり
c24ce9b55a deps: update dependencies (#17669)
* deps: update dependencies

* deps: update dependencies (major)

* migrate js-yaml

* migrate js-yaml

* run pnpm dedupe

* fix

* chore: reorganize dependencies

* perf: lazy-load id generators

* Revert "perf: lazy-load id generators"

This reverts commit 041f7c4206.
2026-07-06 18:55:06 +09:00
おさむのひと
7243bd34b9 fix: Sentry 使用環境下にて、Misskey が発行した SQL クエリが span に含まれない問題を修正 (#17671)
* fix: pgをexternalにしてSentryのspanが動くようにする

* fix changelog

* add comment
2026-07-06 18:54:04 +09:00
おさむのひと
d4dd02c744 feat: Sentryバックエンドの自動計装を個別で無効化出来る仕組みを追加する (#17673)
* feat: Sentryバックエンドの自動計装を個別で無効化出来る仕組みを追加する

* fix comment
2026-07-06 18:52:57 +09:00
syuilo
3817f00803 Revert "fix(dev): fix typo of report-api-diff.yml"
This reverts commit d1cac26cfb.
2026-07-05 14:11:05 +09:00
github-actions[bot]
f066c1c2ef Bump version to 2026.7.0-alpha.1 2026-07-05 05:06:33 +00:00
かっこかり
1d41cb3d29 perf(frontend): UNIX時計ウィジェットでrequestAnimationFrameを使用するように (#17665) 2026-07-05 13:08:44 +09:00
かっこかり
1858d56918 fix(frontend): ワードミュートの判定が誤っている問題を修正 (#17668) 2026-07-05 13:07:17 +09:00
かっこかり
97082efd99 Update CHANGELOG.md [ci skip] 2026-07-04 20:46:03 +09:00
Johann150
187610d516 fix(frontend): correctly detect isMe in mention (#17664) 2026-07-04 20:43:42 +09:00
かっこかり
00b2cb2076 fix(backend/test): streaming.ts e2e の検証不備を修正 (#17663)
fix(backend/test): streaming.ts e2e の検証不備を修正 (tiramiss-community/endolphin#113)

Co-authored-by: samunohito <46447427+samunohito@users.noreply.github.com>
2026-07-04 18:53:10 +09:00
おさむのひと
426f6748e5 fix: frontend-builderのutils.tsが型エラーを起こしていたのを修正 (#17649) 2026-07-04 17:44:42 +09:00
かっこかり
c423bf92a8 Update CHANGELOG.md 2026-07-04 16:47:31 +09:00
かっこかり
f59bf35615 feat: 条件に一致したURLプレビューのサムネイルを隠すことができるように (#17635)
* URLプレビューのサムネイルを隠す機能を追加 (MisskeyIO#214)

* Update Changelog

* fix

* Update Changelog

* fix

* fix: もうセンシティブと判定されている場合はそれをそのまま通すように

* fix: unnecessary import

* fix lint

---------

Co-authored-by: CyberRex <hspwinx86@gmail.com>
2026-07-04 16:26:22 +09:00
おさむのひと
0f056c4955 refactor: Sentry呼び出し個所の抽象化 (#17661) 2026-07-04 16:06:24 +09:00
かっこかり
eb2c7ff6c6 fix(backend/test): follow-up of #17654 (waitForを他のe2eテストにも展開) (#17659)
test(backend): block.ts等のfanout timeline反映待ちレースを修正する (tiramiss-community/endolphin#112)

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>
2026-07-04 11:21:23 +09:00
おさむのひと
9f614517c0 fix: tsconfig.jsonにskipLibCheck: trueを追加する (#17651)
* fix: tsconfig.jsonにskipLibCheck: trueを追加する

* skipLibCheckで抜けたチェックを自前でやる
2026-07-03 19:22:23 +09:00
github-actions[bot]
5432984af8 Bump version to 2026.7.0-alpha.0 2026-07-03 10:15:24 +00:00
かっこかり
c29a3d902b refactor(frontend): MkNote/MkNoteDetailedのロジックを統合 (#17636)
* refactor(frontend): MkNote/MkNoteDetailedのロジックを統合

* refactor

* fix

* fix: 差分を解消

* fix lint

* fix types

---------

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2026-07-03 19:14:48 +09:00
かっこかり
721b1b06a0 fix(backend/test): backend e2e test の beforeAll でdispose前にschema dropが走るのを修正 (#17653)
fix(backend): setup.e2e.ts の beforeAll でdispose前にschema dropが走る順序バグを修正 (Phase 6) (tiramiss-community/endolphin#52)

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>
2026-07-03 15:57:39 +09:00
かっこかり
97e54a1ee8 fix(frontend): APIキーを指定するいくつかのフォームでautocompleteが発動する問題を修正 (#17655) 2026-07-03 14:58:58 +09:00
かっこかり
96d6a09ebc fix(backend/test): e2e testの固定待ち時間をポーリングに変更 (#17654)
* test(backend): timelines.ts の固定250ms待ちをポーリング化する (tiramiss-community/endolphin#47)

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>

* test(backend): move.ts のflaky/低速sleepをポーリング化する (tiramiss-community/endolphin#49)

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>

---------

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>
2026-07-03 10:46:04 +09:00
かっこかり
1eedf04d9a fix(frontend): デバイスサイズをスマートフォンに固定している場合はページヘッダーのアイコンを常に表示するように (#17590)
* fix(frontend): デバイスサイズをスマートフォンに固定している場合はページヘッダーのアイコンを常に表示するように

* Update Changelog

* Update Changelog

* fix

---------

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2026-07-02 22:36:57 +09:00
ihoronir
2d67380f2a fix(backend): 同名の新規ハッシュタグが複数同時に作成されたとき、DBで発生した競合を適切にハンドリング (#17640)
* Hashtag Timeline のテストを復活させる

* HashtagService.ts を修正

* fix indent

* refactor

* Update CHANGELOG.md

* fix review
2026-07-02 09:48:26 +09:00
おさむのひと
174fb434cc fix: frontend-embedで見つけた型エラーの修正 (#17650) 2026-07-02 09:24:50 +09:00
greymoth
7e29f04287 fix(chat): don't send message while the IME is composing (#17646)
Co-authored-by: greymoth <246701683+greymoth-jp@users.noreply.github.com>
2026-07-01 10:06:16 +09:00
renovate[bot]
bf88122140 chore(deps): update [github actions] update dependencies [ci skip] (#17645)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-30 13:11:23 +09:00
かっこかり
4daa1ffe05 chore(deps): 未使用依存パッケージを削除 (#17641)
chore(deps): 未使用依存パッケージを削除 (tiramiss-community/endolphin#20)

Co-authored-by: おさむのひと <46447427+samunohito@users.noreply.github.com>
2026-06-29 23:08:09 +09:00
syuilo
62f8589c05 Update CONTRIBUTING.md 2026-06-28 17:52:08 +09:00
かっこかり
6193c35f9f fix: stats API の型に reactionCount が定義されていない問題を修正 (#17634)
* fix: stats API の型に reactionCount が定義されていない問題を修正

* Update Changelog
2026-06-28 01:49:00 +09:00
かっこかり
1220f05903 Update CHANGELOG.md 2026-06-28 01:02:03 +09:00
かっこかり
7544ebf7a3 Update CHANGELOG.md 2026-06-28 00:54:52 +09:00
かっこかり
ffe65caf10 Update CHANGELOG.md 2026-06-28 00:25:42 +09:00
かっこかり
4f993cef1b enhance: Set default Node.js version to v26 (#17623)
* enhance: Set default Node.js version to v26

* fix

* Update Changelog

* fix types

* drop node v22

* update changelog

* fix test

* update

* fix test

* fix test

* Revert "drop node v22"

This reverts commit bb4e011628.

* fix changelog

* attempt to fix test

* attempt to fix test

* fix: update re2 that supports node 26

* attempt to fix test

* attempt to fix test

* run pnpm dedupe

* restore 2fa e2e

* refactor

* attempt to fix test

* attempt to fix test

* run pnpm dedupe

* attempt to fix test
2026-06-28 00:06:49 +09:00
かっこかり
ba3fb4aa14 Merge branch 'master' into develop 2026-06-27 23:40:06 +09:00
かっこかり
0137b1c406 fix: update compatible node version on master (#17632)
Update package.json
2026-06-27 23:39:29 +09:00
かっこかり
797dec7d0e Update CHANGELOG.md 2026-06-27 23:36:35 +09:00
github-actions[bot]
42ff280163 Bump version to 2026.6.1-alpha.1 2026-06-27 13:08:35 +00:00
かっこかり
554339aaa1 deps: update dependencies (#17631)
* deps: update deps

* Update Changelog

* update dependencies (major)

* fix: broken lockfile on changelog-checker

* update vite

* update minimum working node version to 22.22.0

* update minimum working node version to 22.22.2

* fix types

* refactor: avoid using default import

* Revert "refactor: avoid using default import"

This reverts commit 5784df2ff0.

---------

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2026-06-27 22:04:08 +09:00
syuilo
529c4d4d0e chore(dev): tweak heap snapshot table 2026-06-27 20:21:17 +09:00
syuilo
cec23e5756 enhance(dev): improve heap snapshot analysis (#17628)
* Update measure-memory.mts

* Update utility.mts

* Update backend-memory-report.mts

* tweak
2026-06-27 18:59:46 +09:00
262 changed files with 6486 additions and 8465 deletions

View File

@@ -35,7 +35,7 @@ Misskey リポジトリの `.claude/` 構成を 7 カテゴリで採点し、改
| 2 | Context Efficiency | frontmatter description の冗長度、SKILL.md の長さ分布、重複情報、CLAUDE.md の肥大化 |
| 3 | Quality Gates | Stop / PreToolUse / PostToolUse hook の整備、`/quality-gate` 等の完了前ゲートの有無、自動 lint/typecheck |
| 4 | Memory Persistence | `.claude/skills/*/SKILL.md``references/` の同期状態を評価。プロジェクト側 `.claude/memory/` は未採用方針 (auto-memory はユーザーホーム側で自動運用) のため、ここを採点起点にせず既定 5/10 から開始する |
| 5 | Eval Coverage | `working-on-backend` / `working-on-frontend` の testing リファレンス (backend-testing.md / frontend-testing.md) の網羅、Misskey 固有の e2e/fed/Storybook/Cypress 適用ガイド |
| 5 | Eval Coverage | `working-on-backend` / `working-on-frontend` の testing リファレンス (backend-testing.md / frontend-testing.md) の網羅、Misskey 固有の e2e/fed/Storybook/Playwright 適用ガイド |
| 6 | Security Guardrails | SPDX 規約適用、migration 不変性ルール、ja-JP.yml 限定編集ルール、secrets 検出 |
| 7 | Cost Efficiency | enabledPlugins の重複・過剰、context-budget の整備、MCP 過剰登録なし |

View File

@@ -21,7 +21,7 @@ note: 元 ECC 版は言語自動判定 + format/lint/type のジェネリック
`/quality-gate [scope]`
完了前の **軽量** 品質チェック。重い E2E / 連合テスト (test:e2e / test:fed / Cypress) は CI 側で実行されるため、本コマンドには含めない。
完了前の **軽量** 品質チェック。重い E2E / 連合テスト (test:e2e / test:fed / Playwright) は CI 側で実行されるため、本コマンドには含めない。
## Scope
@@ -120,4 +120,4 @@ Frontend ut: PASS (87/87)
- ジェネリックな言語自動判定を排除し、Misskey 固定 pipeline に。
- formatter フェーズなし (Misskey は ESLint --fix のみ採用)。
- e2e / federation / Cypress は重いため除外し CI 側に委譲。
- e2e / federation / Playwright は重いため除外し CI 側に委譲。

View File

@@ -26,7 +26,7 @@ cp .github/misskey/test.yml .config/test.yml
補足:
- ルートの `pnpm start:test` (Cypress 用にテストサーバーを起動するコマンド) を使う経路では実行時に `ncp` で自動コピーされる ([package.json](../../../../../package.json))。それ以外で backend テストを直接走らせる時は上記の手動コピーが必要
- ルートの `pnpm start:test` (Playwright 用にテストサーバーを起動するコマンド) を使う経路では実行時に `ncp` で自動コピーされる ([package.json](../../../../../package.json))。それ以外で backend テストを直接走らせる時は上記の手動コピーが必要
- すでに `.config/test.yml` があれば各テストスクリプトの内部 `compile-config` で十分なので、追加で `pnpm --filter backend compile-config` を叩く必要はない
- `pnpm start:test` は backend e2e テスト (`pnpm --filter backend test:e2e`) の前提ではない (ポート競合の元になるため使わないこと)

View File

@@ -27,7 +27,7 @@ SKILL.md 本体は references への索引だけ。具体的な手順や規約
- SCSS Modules / `--MI_THEME-*` `--MI-*` CSS 変数 / グローバル utility class (`_button` 等) → [references/knowledge/scss-modules.md](references/knowledge/scss-modules.md)
- `os.alert` / `os.confirm` / `os.popup` 等 UI ヘルパー (ブラウザ標準 `alert()` 直呼びは禁止) → [references/knowledge/os-api.md](references/knowledge/os-api.md)
- `*.stories.impl.ts` 併設規則 + 複数 story / argTypes / layout / action パターン → [references/knowledge/storybook.md](references/knowledge/storybook.md)
- frontend Vitest / Cypress E2E の書き方と前提 → [references/knowledge/frontend-testing.md](references/knowledge/frontend-testing.md)
- frontend Vitest / Playwright E2E の書き方と前提 → [references/knowledge/frontend-testing.md](references/knowledge/frontend-testing.md)
## 必ず最後に通る場所

View File

@@ -1,4 +1,4 @@
# Frontend テスト (Vitest / Cypress)
# Frontend テスト (Vitest / Playwright)
Misskey frontend のテスト構成。
@@ -13,11 +13,11 @@ pnpm --filter frontend test-and-coverage # カバレッジ付き
- 主な配置: `packages/frontend/test/*.test.ts` (例: `i18n.test.ts`, `theme.test.ts`, `is-birthday.test.ts`)
- ビルドツール周りなど対象コードと隣接させた方が分かりやすいテストは、コードと同じディレクトリに `*.test.ts` として置く (例: [packages/frontend/lib/rollup-plugin-unwind-css-module-class-name.test.ts](../../../../../packages/frontend/lib/rollup-plugin-unwind-css-module-class-name.test.ts))
- 共有コンポーネント (`MkX.vue`) のユニットテストは現状少なく、`*.spec.ts` / `__tests__/` 形式は採用していない (Storybook + Cypress でカバー)
- 共有コンポーネント (`MkX.vue`) のユニットテストは現状少なく、`*.spec.ts` / `__tests__/` 形式は採用していない (Storybook + Playwright でカバー)
## Cypress E2E
## Playwright E2E
Cypress**起動済みのテストサーバー** に対して走るため、unit より前提が多い。[.github/workflows/test-frontend.yml](../../../../../.github/workflows/test-frontend.yml) の `e2e` ジョブと同じ手順をローカルで踏む:
Playwright**起動済みのテストサーバー** に対して走るため、unit より前提が多い。[.github/workflows/test-frontend.yml](../../../../../.github/workflows/test-frontend.yml) の `e2e` ジョブと同じ手順をローカルで踏む:
```bash
# 1. テスト用 DB / Redis を起動 (テスト用ポート。開発用の compose.local-db.yml ではない)
@@ -29,15 +29,14 @@ cp .github/misskey/test.yml .config/test.yml
# 3. 全体ビルド
pnpm build
# 4. テストサーバー起動 + Cypress 実行 (いずれもルートから)
pnpm e2e # 内部で pnpm start:test を起動し http://localhost:61812 を待って Cypress run
pnpm cy:open # 対話的に開く (サーバーは別途 pnpm start:test で起動しておく)
# 4. テストサーバー起動 + Playwright 実行 (いずれもルートから)
pnpm e2e # 内部で pnpm start:test を起動し http://localhost:61812 を待って Playwright を実行
```
- 設定: ルート [cypress.config.ts](../../../../../cypress.config.ts)
- テスト本体は [cypress/](../../../../../cypress/) 配下
- 設定: ルート [packages/frontend/playwright.config.ts](../../../../../packages/frontend/playwright.config.ts) で、`packages/frontend/test/e2e/` 配下の `*.spec.ts` を対象にする
- テスト本体は [packages/frontend/test/e2e/](../../../../../packages/frontend/test/e2e/) 配下に配置する。
新規 frontend 機能の E2E は Cypress に書くのが基本。ただし対象は主要 UI フロー (login / post / drive etc) に限定し、細かい単位テストは Vitest または Storybook で代替する慣習。
新規 frontend 機能の E2E は Playwright に書くのが基本。ただし対象は主要 UI フロー (login / post / drive etc) に限定し、細かい単位テストは Vitest または Storybook で代替する慣習。
## Storybook (視覚確認 + Chromatic 視覚回帰)
@@ -55,6 +54,6 @@ pnpm --filter frontend build-storybook # 静的ビルド
frontend のテスト種別で DB / Redis の要否が違う:
- **Vitest (unit)** — DB 不要。ロジック / コンポーネント単体のテストで backend に繋がない (CI の `vitest` ジョブにも `services:` は無い)
- **Cypress (E2E)** — テストサーバー (`pnpm start:test`) 経由で backend に繋ぐため DB / Redis が必要。**テスト用ポートの [packages/backend/test/compose.yml](../../../../../packages/backend/test/compose.yml)** を使う (上記 Cypress E2E の手順を参照)
- **Playwright (E2E)** — テストサーバー (`pnpm start:test`) 経由で backend に繋ぐため DB / Redis が必要。**テスト用ポートの [packages/backend/test/compose.yml](../../../../../packages/backend/test/compose.yml)** を使う (上記 Playwright E2E の手順を参照)
開発用の `compose.local-db.yml` (db `5432` / redis `6379`) は **テストには使わない**。テスト用の `packages/backend/test/compose.yml` (`54312` / `56312`) とはポートが異なり、混同すると接続できない。

View File

@@ -303,8 +303,31 @@ id: 'aidx'
#sentryForBackend:
# enableNodeProfiling: true
# # Specify Sentry integration names to disable individual auto-instrumentation.
# # The names are integration .name values, not factory function names.
# # To check enabled names, set `options.debug: true` and see
# # "Integration installed: <name>" in the Sentry logs.
# #
# # As of 2026-07-07 / @sentry/node 10.62.0, useful names for Misskey include:
# # Postgres ... DB queries (when pg is externalized)
# # Redis ... ioredis commands
# # Fastify ... inbound HTTP routes
# # Http ... inbound/outbound HTTP; disabling this can also affect request isolation
# # NodeFetch ... fetch/undici requests
# disabledIntegrations: ['Postgres']
# options:
# dsn: 'https://examplePublicKey@o0.ingest.sentry.io/0'
# # By default, Misskey prevents Sentry trace headers (`sentry-trace` and
# # `baggage`) from being sent to remote ActivityPub/Webhook/etc. hosts.
# # To intentionally propagate distributed traces to trusted internal services,
# # list only those internal URL patterns here.
# # Avoid broad patterns that match remote servers unless you intentionally
# # want to restore Sentry's legacy behavior of propagating traces to all
# # outbound HTTP requests.
# #tracePropagationTargets: []
# # Internal allowlist example:
# #tracePropagationTargets:
# # - 'internal-service.example'
#sentryForFrontend:
# vueIntegration:

View File

@@ -5,7 +5,7 @@
"workspaceFolder": "/workspace",
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "22.15.0"
"version": "22.23.1"
},
"ghcr.io/devcontainers-extra/features/pnpm:2": {
"version": "10.10.0"

View File

@@ -12,4 +12,4 @@ pnpm install --frozen-lockfile
cp .devcontainer/devcontainer.yml .config/default.yml
pnpm build
pnpm migrate
pnpm exec cypress install
pnpm --filter frontend exec playwright install --with-deps chromium

2
.github/labeler.yml vendored
View File

@@ -16,7 +16,7 @@
'packages/frontend:test':
- any:
- changed-files:
- any-glob-to-any-file: ['cypress/**/*']
- any-glob-to-any-file: ['packages/frontend/test/**/*']
'packages/sw':
- any:

View File

@@ -1 +1 @@
22.15.0
22.22.2

View File

@@ -54,7 +54,7 @@ const metrics = [
function formatMemoryMb(valueKiB: number | null | undefined) {
if (valueKiB == null) return '-';
return `${util.formatNumber(valueKiB / 1024)} MB`;
return `${util.formatNumber(valueKiB / 1000)} MB`;
}
function getMemoryValue(report: MemoryReport, phase: typeof memoryReportPhases[number]['key'], metric: typeof metrics[number]) {
@@ -79,8 +79,8 @@ function renderMainTableForPhase(base: MemoryReport, head: MemoryReport, phase:
'| --- | ---: | ---: | ---: | ---: | ---: | ---: |',
];
function formatDeltaMemory(diffKiB: number) {
return util.formatColoredDelta(formatMemoryMb(Math.abs(diffKiB)), diffKiB);
function formatDeltaMemory(deltaKiB: number) {
return util.formatColoredDelta(deltaKiB, v => formatMemoryMb(v), 100); // 0.1 MB threshold
}
for (const metric of metrics) {
@@ -91,7 +91,7 @@ function renderMainTableForPhase(base: MemoryReport, head: MemoryReport, phase:
const headSpread = getSampleSpread(head, phase, metric);
const summary = util.pairedDeltaSummary(base.samples, head.samples, (sample) => getMemoryValueFromSample(sample, phase, metric));
const percent = summary.median * 100 / baseValue;
const deltaMedian = summary == null ? '-' : `${formatDeltaMemory(summary.median)}<br>${util.formatDeltaPercent(percent).replaceAll('\\%', '\\\\%')}`;
const deltaMedian = summary == null ? '-' : `${formatDeltaMemory(summary.median)}<br>${util.formatDeltaPercent(percent, 0.1).replaceAll('\\%', '\\\\%')}`;
lines.push(`| **${metric}** | ${formatMemoryMb(baseValue)} <br> ± ${formatMemoryMb(baseSpread)} | ${formatMemoryMb(headValue)} <br> ± ${formatMemoryMb(headSpread)} | ${deltaMedian} | ${summary?.mad == null ? '-' : formatMemoryMb(summary.mad)} | ${summary == null ? '-' : formatDeltaMemory(summary.min)} | ${summary == null ? '-' : formatDeltaMemory(summary.max)} |`);
}
@@ -183,7 +183,7 @@ function renderJsFootprintMetricTable(base: RuntimeLoadedJsFootprintReport, head
const headValue = getJsFootprintValue(head, 'afterRequest', key);
if (baseValue == null || headValue == null) continue;
lines.push(`| **${title}** | ${formatter(baseValue)} | ${formatter(headValue)} | ${util.formatColoredDelta(formatter(headValue - baseValue), headValue - baseValue)} | ${util.calcAndFormatDeltaPercent(baseValue, headValue).replaceAll('\\%', '\\\\%')} |`);
lines.push(`| **${title}** | ${formatter(baseValue)} | ${formatter(headValue)} | ${util.formatColoredDelta(headValue - baseValue, v => formatter(v))} | ${util.calcAndFormatDeltaPercent(baseValue, headValue).replaceAll('\\%', '\\\\%')} |`);
}
return lines.join('\n');
@@ -278,7 +278,7 @@ function renderLargestPackageIncreases(base: RuntimeLoadedJsFootprintReport, hea
];
for (const packageSummary of increases) {
lines.push(`| ${packageDisplayName(packageSummary)} | ${util.formatBytes(packageSummary.baseSourceBytes)} | ${util.formatBytes(packageSummary.sourceBytes)} | ${util.formatColoredDelta(util.formatBytes(packageSummary.sourceBytes - packageSummary.baseSourceBytes), packageSummary.sourceBytes - packageSummary.baseSourceBytes)} | ${util.formatColoredDelta(util.formatNumber(packageSummary.modules - packageSummary.baseModules), packageSummary.modules - packageSummary.baseModules)} |`);
lines.push(`| ${packageDisplayName(packageSummary)} | ${util.formatBytes(packageSummary.baseSourceBytes)} | ${util.formatBytes(packageSummary.sourceBytes)} | ${util.formatColoredDelta(packageSummary.sourceBytes - packageSummary.baseSourceBytes, v => util.formatBytes(v))} | ${util.formatColoredDelta(packageSummary.modules - packageSummary.baseModules, v => util.formatNumber(v))} |`);
}
return lines.join('\n');
@@ -406,15 +406,15 @@ function isBeyondSampleNoise(base: MemoryReport, head: MemoryReport, phase: type
const headValue = getMemoryValue(head, phase, metric);
if (baseValue == null || headValue == null) return false;
const diff = headValue - baseValue;
if (diff <= 0) return false;
const delta = headValue - baseValue;
if (delta <= 0) return false;
const baseSpread = getSampleSpread(base, phase, metric);
const headSpread = getSampleSpread(head, phase, metric);
if (baseSpread == null || headSpread == null) return true;
const combinedSpread = Math.hypot(baseSpread, headSpread);
return diff > combinedSpread * 3;
return delta > combinedSpread * 3;
}
const warningMetric = getWarningMetric(base, head);

View File

@@ -1,271 +0,0 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import * as util from './utility.mts';
import * as heapSnapshotUtil from './heap-snapshot-util.mts';
import type { HeapSnapshotData } from './heap-snapshot-util.mts';
export type BrowserMeasurement = {
label: string;
timestamp: string;
url: string;
scenario: string;
durationMs: number;
network: {
requestCount: number;
finishedRequestCount: number;
failedRequestCount: number;
cachedRequestCount: number;
serviceWorkerRequestCount: number;
totalEncodedBytes: number;
totalDecodedBodyBytes: number;
sameOriginEncodedBytes: number;
thirdPartyEncodedBytes: number;
byResourceType: Record<string, {
requests: number;
encodedBytes: number;
decodedBodyBytes: number;
}>;
largestRequests: {
url: string;
method: string;
resourceType: string;
status?: number;
encodedBytes: number;
decodedBodyBytes: number;
}[];
failedRequests: {
url: string;
method: string;
resourceType: string;
errorText?: string;
status?: number;
}[];
};
performance: {
cdpMetrics: Record<string, number>;
runtimeHeap?: {
usedSize: number;
totalSize: number;
};
webVitals: {
firstPaintMs?: number;
firstContentfulPaintMs?: number;
domContentLoadedEventEndMs?: number;
loadEventEndMs?: number;
longTaskCount: number;
longTaskDurationMs: number;
maxLongTaskDurationMs: number;
resourceEntryCount: number;
domElements: number;
};
};
heapSnapshot: HeapSnapshotData;
};
function escapeCell(value: string) {
return String(value).replaceAll('|', '\\|').replaceAll('\n', '<br>');
}
function truncate(value: string, maxLength = 140) {
if (value.length <= maxLength) return value;
return `${value.slice(0, maxLength - 3)}...`;
}
function formatMs(value: number | null | undefined) {
if (value == null || !Number.isFinite(value)) return '-';
if (value >= 1_000) return `${util.formatNumber(value / 1_000)} s`;
return `${util.formatNumber(value)} ms`;
}
function formatSecondsAsMs(value: number | null | undefined) {
if (value == null || !Number.isFinite(value)) return '-';
return formatMs(value * 1_000);
}
function formatDelta(value: number, formatter: (value: number) => string) {
if (value === 0) return formatter(0);
return util.formatColoredDelta(formatter(Math.abs(value)), value);
}
function metricRow(label: string, baseValue: number | null | undefined, headValue: number | null | undefined, formatter: (value: number) => string) {
if (baseValue == null || headValue == null || !Number.isFinite(baseValue) || !Number.isFinite(headValue)) return null;
const delta = headValue - baseValue;
return `| **${label}** | ${formatter(baseValue)} | ${formatter(headValue)} | ${formatDelta(delta, formatter)} | ${util.calcAndFormatDeltaPercent(baseValue, headValue).replaceAll('\\%', '\\\\%')} |`;
}
function resourceTypeBytes(report: BrowserMeasurement, resourceTypes: string[]) {
return resourceTypes.reduce((sum, resourceType) => sum + (report.network.byResourceType[resourceType]?.encodedBytes ?? 0), 0);
}
function getMetric(report: BrowserMeasurement, key: string) {
return report.performance.cdpMetrics[key];
}
function renderSummaryTable(base: BrowserMeasurement, head: BrowserMeasurement) {
const rows = [
metricRow('Scenario duration', base.durationMs, head.durationMs, formatMs),
metricRow('Requests', base.network.requestCount, head.network.requestCount, util.formatNumber),
metricRow('Failed requests', base.network.failedRequestCount, head.network.failedRequestCount, util.formatNumber),
metricRow('Encoded network', base.network.totalEncodedBytes, head.network.totalEncodedBytes, util.formatBytes),
metricRow('Decoded body', base.network.totalDecodedBodyBytes, head.network.totalDecodedBodyBytes, util.formatBytes),
metricRow('Same-origin encoded', base.network.sameOriginEncodedBytes, head.network.sameOriginEncodedBytes, util.formatBytes),
metricRow('Third-party encoded', base.network.thirdPartyEncodedBytes, head.network.thirdPartyEncodedBytes, util.formatBytes),
metricRow('Script encoded', resourceTypeBytes(base, ['Script']), resourceTypeBytes(head, ['Script']), util.formatBytes),
metricRow('Stylesheet encoded', resourceTypeBytes(base, ['Stylesheet']), resourceTypeBytes(head, ['Stylesheet']), util.formatBytes),
metricRow('Fetch/XHR encoded', resourceTypeBytes(base, ['Fetch', 'XHR']), resourceTypeBytes(head, ['Fetch', 'XHR']), util.formatBytes),
metricRow('Image encoded', resourceTypeBytes(base, ['Image']), resourceTypeBytes(head, ['Image']), util.formatBytes),
metricRow('Font encoded', resourceTypeBytes(base, ['Font']), resourceTypeBytes(head, ['Font']), util.formatBytes),
metricRow('First contentful paint', base.performance.webVitals.firstContentfulPaintMs, head.performance.webVitals.firstContentfulPaintMs, formatMs),
metricRow('Load event end', base.performance.webVitals.loadEventEndMs, head.performance.webVitals.loadEventEndMs, formatMs),
metricRow('Long tasks', base.performance.webVitals.longTaskCount, head.performance.webVitals.longTaskCount, util.formatNumber),
metricRow('Long task duration', base.performance.webVitals.longTaskDurationMs, head.performance.webVitals.longTaskDurationMs, formatMs),
metricRow('Max long task', base.performance.webVitals.maxLongTaskDurationMs, head.performance.webVitals.maxLongTaskDurationMs, formatMs),
metricRow('JS heap used', base.performance.runtimeHeap?.usedSize ?? getMetric(base, 'JSHeapUsedSize'), head.performance.runtimeHeap?.usedSize ?? getMetric(head, 'JSHeapUsedSize'), util.formatBytes),
metricRow('JS heap total', base.performance.runtimeHeap?.totalSize ?? getMetric(base, 'JSHeapTotalSize'), head.performance.runtimeHeap?.totalSize ?? getMetric(head, 'JSHeapTotalSize'), util.formatBytes),
metricRow('V8 heap snapshot total', base.heapSnapshot.categories.total, head.heapSnapshot.categories.total, util.formatBytes),
metricRow('DOM elements', base.performance.webVitals.domElements, head.performance.webVitals.domElements, util.formatNumber),
metricRow('CDP nodes', getMetric(base, 'Nodes'), getMetric(head, 'Nodes'), util.formatNumber),
metricRow('JS event listeners', getMetric(base, 'JSEventListeners'), getMetric(head, 'JSEventListeners'), util.formatNumber),
metricRow('Layout count', getMetric(base, 'LayoutCount'), getMetric(head, 'LayoutCount'), util.formatNumber),
metricRow('Recalc style count', getMetric(base, 'RecalcStyleCount'), getMetric(head, 'RecalcStyleCount'), util.formatNumber),
metricRow('Script duration', getMetric(base, 'ScriptDuration'), getMetric(head, 'ScriptDuration'), formatSecondsAsMs),
metricRow('Task duration', getMetric(base, 'TaskDuration'), getMetric(head, 'TaskDuration'), formatSecondsAsMs),
].filter(row => row != null);
return [
'| Metric | Base | Head | Δ | Δ (%) |',
'| --- | ---: | ---: | ---: | ---: |',
...rows,
].join('\n');
}
function renderResourceTypeTable(base: BrowserMeasurement, head: BrowserMeasurement) {
const preferredOrder = ['Document', 'Script', 'Stylesheet', 'Fetch', 'XHR', 'Image', 'Font', 'Media', 'WebSocket', 'EventSource', 'Other'];
const keys = [...new Set([
...preferredOrder,
...Object.keys(base.network.byResourceType),
...Object.keys(head.network.byResourceType),
])].filter(key => base.network.byResourceType[key] != null || head.network.byResourceType[key] != null);
const lines = [
'| Type | Base reqs | Head reqs | Δ reqs | Base encoded | Head encoded | Δ encoded |',
'| --- | ---: | ---: | ---: | ---: | ---: | ---: |',
];
for (const key of keys) {
const baseRow = base.network.byResourceType[key] ?? { requests: 0, encodedBytes: 0 };
const headRow = head.network.byResourceType[key] ?? { requests: 0, encodedBytes: 0 };
lines.push(`| **${key}** | ${util.formatNumber(baseRow.requests)} | ${util.formatNumber(headRow.requests)} | ${formatDelta(headRow.requests - baseRow.requests, util.formatNumber)} | ${util.formatBytes(baseRow.encodedBytes)} | ${util.formatBytes(headRow.encodedBytes)} | ${formatDelta(headRow.encodedBytes - baseRow.encodedBytes, util.formatBytes)} |`);
}
return lines.join('\n');
}
function renderHeapSnapshotTable(base: BrowserMeasurement, head: BrowserMeasurement) {
const lines = [
'| Category | Base | Head | Δ | Δ (%) | Base nodes | Head nodes |',
'| --- | ---: | ---: | ---: | ---: | ---: | ---: |',
];
for (const category of Object.keys(heapSnapshotUtil.heapSnapshotCategory) as (keyof typeof heapSnapshotUtil.heapSnapshotCategory)[]) {
const baseValue = base.heapSnapshot.categories[category];
const headValue = head.heapSnapshot.categories[category];
const baseNodeCount = base.heapSnapshot.nodeCounts[category];
const headNodeCount = head.heapSnapshot.nodeCounts[category];
const categoryInfo = heapSnapshotUtil.heapSnapshotCategory[category];
const metricText = `$\\color{${categoryInfo.color}}{\\rule{8pt}{8pt}}$ **${categoryInfo.label}**`;
lines.push(`| ${metricText} | ${util.formatBytes(baseValue)} | ${util.formatBytes(headValue)} | ${formatDelta(headValue - baseValue, util.formatBytes)} | ${util.calcAndFormatDeltaPercent(baseValue, headValue).replaceAll('\\%', '\\\\%')} | ${util.formatNumber(baseNodeCount)} | ${util.formatNumber(headNodeCount)} |`);
}
return lines.join('\n');
}
function renderLargestRequests(report: BrowserMeasurement, title: string) {
if (report.network.largestRequests.length === 0) return null;
const lines = [
`<details><summary>${title}</summary>`,
'',
'| Resource | Type | Status | Encoded | Decoded |',
'| --- | --- | ---: | ---: | ---: |',
];
for (const request of report.network.largestRequests.slice(0, 10)) {
lines.push(`| \`${escapeCell(truncate(request.url))}\` | ${escapeCell(request.resourceType)} | ${request.status ?? '-'} | ${util.formatBytes(request.encodedBytes)} | ${util.formatBytes(request.decodedBodyBytes)} |`);
}
lines.push('', '</details>');
return lines.join('\n');
}
function renderFailedRequests(report: BrowserMeasurement, title: string) {
if (report.network.failedRequests.length === 0) return null;
const lines = [
`<details><summary>${title}</summary>`,
'',
'| Resource | Type | Status | Error |',
'| --- | --- | ---: | --- |',
];
for (const request of report.network.failedRequests.slice(0, 20)) {
lines.push(`| \`${escapeCell(truncate(request.url))}\` | ${escapeCell(request.resourceType)} | ${request.status ?? '-'} | ${escapeCell(request.errorText ?? '')} |`);
}
lines.push('', '</details>');
return lines.join('\n');
}
function renderHeadHeapSankey(head: BrowserMeasurement) {
return heapSnapshotUtil.renderHeapSnapshotSankey({
summary: head.heapSnapshot,
samples: [{
round: 1,
data: head.heapSnapshot,
}],
}, 'Head browser');
}
export function renderFrontendBrowserReport(base: BrowserMeasurement, head: BrowserMeasurement, options: {
headHeapSnapshotUrl?: string;
} = {}) {
const headHeapSnapshotUrl = options.headHeapSnapshotUrl;
const lines = [
'## Frontend Browser Metrics',
'',
renderSummaryTable(base, head),
'',
'_Measured once per side with a fresh headless Chrome profile, browser cache disabled, service workers bypassed, and a forced V8 GC before the heap snapshot. Scenario: sign up, dismiss the initial account setup dialog, create the first timeline note, then wait until that note is visible._',
'',
'<details>',
'<summary>Requests by resource type</summary>',
'',
renderResourceTypeTable(base, head),
'',
'</details>',
'',
'<details>',
'<summary>V8 heap snapshot statistics</summary>',
'',
renderHeapSnapshotTable(base, head),
'',
...(headHeapSnapshotUrl != null && headHeapSnapshotUrl !== '' ? [`[Download head heap snapshot](${headHeapSnapshotUrl})`, ''] : []),
'</details>',
'',
];
for (const section of [
renderHeadHeapSankey(head),
renderLargestRequests(head, 'Largest head requests'),
renderFailedRequests(base, 'Failed base requests'),
renderFailedRequests(head, 'Failed head requests'),
]) {
if (section == null) continue;
lines.push(section, '');
}
return lines.join('\n').trimEnd() + '\n';
}

View File

@@ -6,7 +6,6 @@
import { promises as fs } from 'node:fs';
import path from 'node:path';
import * as util from './utility.mts';
import { renderFrontendBrowserReport, type BrowserMeasurement } from './frontend-browser-report.mts';
const marker = '<!-- misskey-frontend-js-size -->';
@@ -300,13 +299,13 @@ function renderVisualizerSummaryTable(before: ReturnType<typeof collectVisualize
`<tr><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>`,
`<tr>`,
`<th><b>Δ</b></th>`,
...summary.map((key) => `<td>${util.calcAndFormatDeltaNumber(before.summary[key], after.summary[key])}</td>`),
...metrics.map((key) => `<td>${util.calcAndFormatDeltaBytes(before.metrics[key], after.metrics[key])}</td>`),
...summary.map((key) => `<td>${util.calcAndFormatDeltaNumber(before.summary[key], after.summary[key], 0)}</td>`),
...metrics.map((key) => `<td>${util.calcAndFormatDeltaBytes(before.metrics[key], after.metrics[key], 1000)}</td>`),
`</tr>`,
`<tr>`,
`<th><b>Δ (%)</b></th>`,
...summary.map((key) => `<td>${util.calcAndFormatDeltaPercent(before.summary[key], after.summary[key])}</td>`),
...metrics.map((key) => `<td>${util.calcAndFormatDeltaPercent(before.metrics[key], after.metrics[key])}</td>`),
...summary.map((key) => `<td>${util.calcAndFormatDeltaPercent(before.summary[key], after.summary[key], 0.1)}</td>`),
...metrics.map((key) => `<td>${util.calcAndFormatDeltaPercent(before.metrics[key], after.metrics[key], 0.1)}</td>`),
`</tr>`,
`</tbody>`,
`</table>`,
@@ -358,16 +357,16 @@ function chunkMarkdownTable(rows: ReturnType<typeof getChunkComparisonRows>, tot
'| --- | ---: | ---: | ---: | ---: |',
];
if (total != null) {
lines.push(`| (total) | ${util.formatBytes(total.beforeSize)} | ${util.formatBytes(total.afterSize)} | ${util.calcAndFormatDeltaBytes(total.beforeSize, total.afterSize)} | ${util.calcAndFormatDeltaPercent(total.beforeSize, total.afterSize).replaceAll('\\%', '\\\\%')} |`);
lines.push(`| (total) | ${util.formatBytes(total.beforeSize)} | ${util.formatBytes(total.afterSize)} | ${util.calcAndFormatDeltaBytes(total.beforeSize, total.afterSize, 1000)} | ${util.calcAndFormatDeltaPercent(total.beforeSize, total.afterSize, 0.1).replaceAll('\\%', '\\\\%')} |`);
lines.push('| | | | | |');
}
for (const row of rows) {
if (row.changeType === 'added') {
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${util.formatBytes(row.beforeSize)} | ${util.formatBytes(row.afterSize)} | ${util.calcAndFormatDeltaBytes(row.beforeSize, row.afterSize)} | $\\color{orange}{\\text{(+)}}$ |`);
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${util.formatBytes(row.beforeSize)} | ${util.formatBytes(row.afterSize)} | ${util.calcAndFormatDeltaBytes(row.beforeSize, row.afterSize, 1000)} | $\\color{orange}{\\text{( + )}}$ |`);
} else if (row.changeType === 'removed') {
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${util.formatBytes(row.beforeSize)} | ${util.formatBytes(row.afterSize)} | ${util.calcAndFormatDeltaBytes(row.beforeSize, row.afterSize)} | $\\color{green}{\\text{(-)}}$ |`);
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${util.formatBytes(row.beforeSize)} | ${util.formatBytes(row.afterSize)} | ${util.calcAndFormatDeltaBytes(row.beforeSize, row.afterSize, 1000)} | $\\color{green}{\\text{( - )}}$ |`);
} else {
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${util.formatBytes(row.beforeSize)} | ${util.formatBytes(row.afterSize)} | ${util.calcAndFormatDeltaBytes(row.beforeSize, row.afterSize)} | ${util.calcAndFormatDeltaPercent(row.beforeSize, row.afterSize).replaceAll('\\%', '\\\\%')} |`);
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${util.formatBytes(row.beforeSize)} | ${util.formatBytes(row.afterSize)} | ${util.calcAndFormatDeltaBytes(row.beforeSize, row.afterSize, 1000)} | ${util.calcAndFormatDeltaPercent(row.beforeSize, row.afterSize, 0.1).replaceAll('\\%', '\\\\%')} |`);
}
}
return lines.join('\n');
@@ -478,17 +477,11 @@ function renderFrontendBundleReport(before: ReturnType<typeof collectVisualizerR
}
const args = process.argv.slice(2);
const [beforeDir, afterDir, beforeStatsFile, afterStatsFile, beforeBrowserMetricsFile, afterBrowserMetricsFile, outFile] = args;
if (beforeDir == null || afterDir == null || beforeStatsFile == null || afterStatsFile == null || beforeBrowserMetricsFile == null || afterBrowserMetricsFile == null || outFile == null) {
throw new Error('Usage: node frontend-js-size.mts <before-dir> <after-dir> <before-stats.json> <after-stats.json> <before-browser.json> <after-browser.json> <output.md>');
}
const [beforeDir, afterDir, beforeStatsFile, afterStatsFile, outFile] = args;
const before = await collectReport(beforeDir);
const after = await collectReport(afterDir);
const beforeStats = JSON.parse(await fs.readFile(beforeStatsFile, 'utf8')) as VisualizerReport;
const afterStats = JSON.parse(await fs.readFile(afterStatsFile, 'utf8')) as VisualizerReport;
const beforeBrowserMetrics = JSON.parse(await fs.readFile(beforeBrowserMetricsFile, 'utf8')) as BrowserMeasurement;
const afterBrowserMetrics = JSON.parse(await fs.readFile(afterBrowserMetricsFile, 'utf8')) as BrowserMeasurement;
const beforeVisualizerReport = collectVisualizerReport(beforeStats);
const afterVisualizerReport = collectVisualizerReport(afterStats);
const visualizerArtifactLink = `[Open treemap HTML](${process.env.FRONTEND_BUNDLE_REPORT_ARTIFACT_URL})`;
@@ -504,10 +497,6 @@ const body = [
'',
renderFrontendBundleReport(beforeVisualizerReport, afterVisualizerReport),
'',
renderFrontendBrowserReport(beforeBrowserMetrics, afterBrowserMetrics, {
headHeapSnapshotUrl: process.env.FRONTEND_BROWSER_HEAD_HEAP_SNAPSHOT_ARTIFACT_URL,
}),
'',
visualizerArtifactLink,
].join('\n');

View File

@@ -63,20 +63,20 @@ export function renderHeapSnapshotTable(base: HeapSnapshotReport, head: HeapSnap
const percent = summary.median * 100 / baseValue;
if (category === 'total') {
const deltaMedian = `${util.formatDeltaBytes(summary.median)}<br>${util.formatDeltaPercent(percent).replaceAll('\\%', '\\\\%')}`;
const deltaMedian = `${util.formatDeltaBytes(summary.median, 100000)}<br>${util.formatDeltaPercent(percent, 0.1).replaceAll('\\%', '\\\\%')}`;
const baseText = `${util.formatBytes(baseValue)} <br> ± ${util.formatBytes(baseSpread)}`;
const headText = `${util.formatBytes(headValue)} <br> ± ${util.formatBytes(headSpread)}`;
const metricText = `$\\color{${heapSnapshotCategory[category].color}}{\\rule{8pt}{8pt}}$ **${heapSnapshotCategory[category].label}**`;
lines.push(`| ${metricText} | ${baseText} | ${headText} | ${deltaMedian} | ${util.formatBytes(summary.mad)} | ${util.formatDeltaBytes(summary.min)} | ${util.formatDeltaBytes(summary.max)} |`);
lines.push(`| ${metricText} | ${baseText} | ${headText} | ${deltaMedian} | ${util.formatBytes(summary.mad)} | ${util.formatDeltaBytes(summary.min, 100000)} | ${util.formatDeltaBytes(summary.max, 100000)} |`);
lines.push('| | | | | | | |');
} else {
const deltaMedian = util.formatDeltaBytes(summary.median);
const deltaMedian = util.formatDeltaBytes(summary.median, 100000);
const baseText = util.formatBytes(baseValue);
const headText = util.formatBytes(headValue);
const basePercent = util.formatPercent((baseValue * 100) / baseTotal);
const headPercent = util.formatPercent((headValue * 100) / headTotal);
const metricText = `<details><summary>$\\color{${heapSnapshotCategory[category].color}}{\\rule{8pt}{8pt}}$ **${heapSnapshotCategory[category].label}**</summary>${basePercent}${headPercent}</details>`;
lines.push(`| ${metricText} | ${baseText} | ${headText} | ${deltaMedian} | ${util.formatBytes(summary.mad)} | ${util.formatDeltaBytes(summary.min)} | ${util.formatDeltaBytes(summary.max)} |`);
lines.push(`| ${metricText} | ${baseText} | ${headText} | ${deltaMedian} | ${util.formatBytes(summary.mad)} | ${util.formatDeltaBytes(summary.min, 100000)} | ${util.formatDeltaBytes(summary.max, 100000)} |`);
}
}

View File

@@ -1,841 +0,0 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { spawn, spawnSync, type ChildProcessWithoutNullStreams } from 'node:child_process';
import { mkdtemp, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import * as util from './utility.mts';
import { heapSnapshotCategory, type HeapSnapshotData } from './heap-snapshot-util.mts';
const [baseDirArg, headDirArg, baseOutputArg, headOutputArg, headHeapSnapshotOutputArg] = process.argv.slice(2);
const baseUrl = process.env.FRONTEND_BROWSER_METRICS_URL ?? 'http://127.0.0.1:61812';
const serverReadyTimeoutMs = util.readIntegerEnv('FRONTEND_BROWSER_METRICS_SERVER_READY_TIMEOUT_MS', 120_000, 1);
const scenarioTimeoutMs = util.readIntegerEnv('FRONTEND_BROWSER_METRICS_SCENARIO_TIMEOUT_MS', 90_000, 1);
const settleMs = util.readIntegerEnv('FRONTEND_BROWSER_METRICS_SETTLE_MS', 1_000, 0);
const heapSnapshotBreakdownTopN = util.readIntegerEnv('FRONTEND_BROWSER_HEAP_SNAPSHOT_BREAKDOWN_TOP_N', 8, 1);
type ChromeHandle = {
process: ChildProcessWithoutNullStreams;
port: number;
userDataDir: string;
};
type CdpResponse<T = any> = {
id?: number;
method?: string;
params?: any;
result?: T;
error?: {
code: number;
message: string;
};
};
type NetworkRequest = {
requestId: string;
url: string;
method: string;
resourceType: string;
startedAt: number;
status?: number;
mimeType?: string;
encodedDataLength: number;
decodedBodyLength: number;
fromDiskCache: boolean;
fromServiceWorker: boolean;
finished: boolean;
failed: boolean;
errorText?: string;
};
type NetworkSummary = {
requestCount: number;
finishedRequestCount: number;
failedRequestCount: number;
cachedRequestCount: number;
serviceWorkerRequestCount: number;
totalEncodedBytes: number;
totalDecodedBodyBytes: number;
sameOriginEncodedBytes: number;
thirdPartyEncodedBytes: number;
byResourceType: Record<string, {
requests: number;
encodedBytes: number;
decodedBodyBytes: number;
}>;
largestRequests: {
url: string;
method: string;
resourceType: string;
status?: number;
encodedBytes: number;
decodedBodyBytes: number;
}[];
failedRequests: {
url: string;
method: string;
resourceType: string;
errorText?: string;
status?: number;
}[];
};
type BrowserMeasurement = {
label: string;
timestamp: string;
url: string;
scenario: string;
durationMs: number;
network: NetworkSummary;
performance: {
cdpMetrics: Record<string, number>;
runtimeHeap?: {
usedSize: number;
totalSize: number;
};
webVitals: {
firstPaintMs?: number;
firstContentfulPaintMs?: number;
domContentLoadedEventEndMs?: number;
loadEventEndMs?: number;
longTaskCount: number;
longTaskDurationMs: number;
maxLongTaskDurationMs: number;
resourceEntryCount: number;
domElements: number;
};
};
heapSnapshot: HeapSnapshotData;
};
class CdpClient {
private nextId = 1;
private callbacks = new Map<number, {
resolve: (value: any) => void;
reject: (error: Error) => void;
}>();
private eventHandlers = new Map<string, Set<(params: any) => void>>();
private ws: WebSocket;
private constructor(ws: WebSocket) {
this.ws = ws;
ws.addEventListener('message', event => {
const message = JSON.parse(String(event.data)) as CdpResponse;
if (message.id != null) {
const callback = this.callbacks.get(message.id);
if (callback == null) return;
this.callbacks.delete(message.id);
if (message.error != null) {
callback.reject(new Error(`${message.error.message} (${message.error.code})`));
} else {
callback.resolve(message.result);
}
return;
}
if (message.method != null) {
for (const handler of this.eventHandlers.get(message.method) ?? []) {
handler(message.params);
}
}
});
ws.addEventListener('close', () => {
for (const callback of this.callbacks.values()) {
callback.reject(new Error('CDP websocket closed'));
}
this.callbacks.clear();
});
}
static async connect(wsUrl: string) {
const ws = new WebSocket(wsUrl);
await new Promise<void>((resolvePromise, reject) => {
ws.addEventListener('open', () => resolvePromise(), { once: true });
ws.addEventListener('error', () => reject(new Error(`Failed to connect to ${wsUrl}`)), { once: true });
});
return new CdpClient(ws);
}
on(method: string, handler: (params: any) => void) {
const handlers = this.eventHandlers.get(method) ?? new Set();
handlers.add(handler);
this.eventHandlers.set(method, handlers);
}
send<T = any>(method: string, params: Record<string, unknown> = {}): Promise<T> {
const id = this.nextId++;
this.ws.send(JSON.stringify({ id, method, params }));
return new Promise<T>((resolvePromise, reject) => {
this.callbacks.set(id, {
resolve: resolvePromise,
reject,
});
});
}
close() {
this.ws.close();
}
}
function sleep(ms: number) {
return new Promise(resolvePromise => setTimeout(resolvePromise, ms));
}
async function fetchJson<T>(url: string, options?: RequestInit) {
const response = await fetch(url, options);
if (!response.ok) {
throw new Error(`${url} returned ${response.status}: ${await response.text()}`);
}
return await response.json() as T;
}
function findChrome() {
const envChrome = process.env.CHROME_BIN ?? process.env.GOOGLE_CHROME_BIN;
if (envChrome != null && envChrome !== '') return envChrome;
const candidates = process.platform === 'win32'
? [
'chrome.exe',
'msedge.exe',
]
: [
'google-chrome',
'google-chrome-stable',
'chromium',
'chromium-browser',
];
for (const candidate of candidates) {
const result = spawnSync(candidate, ['--version'], {
stdio: 'ignore',
shell: process.platform === 'win32',
});
if (result.status === 0) return candidate;
}
throw new Error('Could not find Chrome or Chromium. Set CHROME_BIN to the browser executable.');
}
async function launchChrome(label: string): Promise<ChromeHandle> {
const chrome = findChrome();
const port = label === 'base' ? 9222 : 9223;
const userDataDir = await mkdtemp(join(tmpdir(), `misskey-browser-metrics-${label}-`));
const child = spawn(chrome, [
'--headless=new',
'--disable-gpu',
'--disable-dev-shm-usage',
'--disable-background-networking',
'--disable-default-apps',
'--disable-extensions',
'--disable-sync',
'--metrics-recording-only',
'--no-first-run',
'--no-default-browser-check',
'--no-sandbox',
`--remote-debugging-port=${port}`,
`--user-data-dir=${userDataDir}`,
'about:blank',
], {
stdio: ['ignore', 'pipe', 'pipe'],
});
child.stdout.on('data', data => process.stderr.write(`[chrome:${label}] ${data}`));
child.stderr.on('data', data => process.stderr.write(`[chrome:${label}] ${data}`));
const startedAt = Date.now();
while (Date.now() - startedAt < 30_000) {
if (child.exitCode != null) throw new Error(`Chrome exited early with code ${child.exitCode}`);
try {
await fetchJson(`http://127.0.0.1:${port}/json/version`);
return {
process: child,
port,
userDataDir,
};
} catch {
await sleep(250);
}
}
throw new Error('Timed out waiting for Chrome DevTools Protocol');
}
async function closeChrome(handle: ChromeHandle) {
handle.process.kill();
await new Promise<void>(resolvePromise => {
if (handle.process.exitCode != null) {
resolvePromise();
return;
}
handle.process.once('exit', () => resolvePromise());
setTimeout(() => {
handle.process.kill('SIGKILL');
resolvePromise();
}, 5_000).unref();
});
await rm(handle.userDataDir, { recursive: true, force: true });
}
async function connectPage(port: number) {
const page = await fetchJson<{ webSocketDebuggerUrl: string }>(
`http://127.0.0.1:${port}/json/new?${encodeURIComponent('about:blank')}`,
{ method: 'PUT' },
).catch(async () => await fetchJson<{ webSocketDebuggerUrl: string }>(
`http://127.0.0.1:${port}/json/new?${encodeURIComponent('about:blank')}`,
));
return await CdpClient.connect(page.webSocketDebuggerUrl);
}
function startServer(label: string, repoDir: string) {
process.stderr.write(`[${label}] Starting Misskey test server\n`);
const child = spawn(util.commandName('pnpm'), ['start:test'], {
cwd: repoDir,
env: process.env,
stdio: ['ignore', 'pipe', 'pipe'],
detached: process.platform !== 'win32',
});
child.stdout.on('data', data => process.stderr.write(`[server:${label}] ${data}`));
child.stderr.on('data', data => process.stderr.write(`[server:${label}] ${data}`));
return child;
}
async function stopServer(child: ChildProcessWithoutNullStreams) {
if (child.exitCode != null) return;
if (process.platform === 'win32') {
spawnSync('taskkill', ['/pid', String(child.pid), '/t', '/f'], { stdio: 'ignore' });
} else if (child.pid != null) {
try {
process.kill(-child.pid, 'SIGTERM');
} catch {
child.kill('SIGTERM');
}
}
await new Promise<void>(resolvePromise => {
if (child.exitCode != null) {
resolvePromise();
return;
}
child.once('exit', () => resolvePromise());
setTimeout(() => {
if (child.pid != null) {
try {
if (process.platform === 'win32') {
spawnSync('taskkill', ['/pid', String(child.pid), '/t', '/f'], { stdio: 'ignore' });
} else {
process.kill(-child.pid, 'SIGKILL');
}
} catch {
child.kill('SIGKILL');
}
}
resolvePromise();
}, 10_000).unref();
});
}
async function waitForServer(child: ChildProcessWithoutNullStreams) {
const startedAt = Date.now();
while (Date.now() - startedAt < serverReadyTimeoutMs) {
if (child.exitCode != null) throw new Error(`Misskey server exited early with code ${child.exitCode}`);
try {
const response = await fetch(`${baseUrl}/`, { redirect: 'manual' });
if (response.status < 500) return;
} catch {
// retry
}
await sleep(1_000);
}
throw new Error(`Timed out waiting for ${baseUrl}`);
}
async function api(endpoint: string, body: Record<string, unknown>) {
const response = await fetch(`${baseUrl}/api/${endpoint}`, {
method: 'POST',
headers: {
'content-type': 'application/json',
},
body: JSON.stringify(body),
});
if (!response.ok) {
throw new Error(`/api/${endpoint} returned ${response.status}: ${await response.text()}`);
}
if (response.status === 204) return null;
return await response.json();
}
async function prepareInstance() {
await api('reset-db', {});
await api('admin/accounts/create', {
username: 'admin',
password: 'admin1234',
setupPassword: 'example_password_please_change_this_or_you_will_get_hacked',
});
}
function installNetworkTracker(cdp: CdpClient): NetworkRequest[] {
const requests = new Map<string, NetworkRequest>();
const requestRows: NetworkRequest[] = [];
cdp.on('Network.requestWillBeSent', params => {
if (params.request?.url == null) return;
const row: NetworkRequest = {
requestId: params.requestId,
url: params.request.url,
method: params.request.method ?? 'GET',
resourceType: params.type ?? 'Other',
startedAt: params.timestamp ?? 0,
encodedDataLength: 0,
decodedBodyLength: 0,
fromDiskCache: false,
fromServiceWorker: false,
finished: false,
failed: false,
};
requests.set(params.requestId, row);
requestRows.push(row);
});
cdp.on('Network.responseReceived', params => {
const row = requests.get(params.requestId);
if (row == null) return;
row.status = params.response?.status;
row.mimeType = params.response?.mimeType;
row.fromDiskCache = params.response?.fromDiskCache === true;
row.fromServiceWorker = params.response?.fromServiceWorker === true;
});
cdp.on('Network.dataReceived', params => {
const row = requests.get(params.requestId);
if (row == null) return;
row.decodedBodyLength += params.dataLength ?? 0;
row.encodedDataLength += params.encodedDataLength ?? 0;
});
cdp.on('Network.loadingFinished', params => {
const row = requests.get(params.requestId);
if (row == null) return;
row.finished = true;
row.encodedDataLength = Math.max(row.encodedDataLength, params.encodedDataLength ?? 0);
});
cdp.on('Network.loadingFailed', params => {
const row = requests.get(params.requestId);
if (row == null) return;
row.failed = true;
row.finished = true;
row.errorText = params.errorText;
});
return requestRows;
}
function isMeasurableRequest(row: NetworkRequest) {
return !row.url.startsWith('data:') && !row.url.startsWith('blob:') && !row.url.startsWith('devtools:');
}
function summarizeNetwork(requestRows: NetworkRequest[]): NetworkSummary {
const origin = new URL(baseUrl).origin;
const rows = requestRows.filter(isMeasurableRequest);
const byResourceType = {} as NetworkSummary['byResourceType'];
for (const row of rows) {
const summary = byResourceType[row.resourceType] ?? {
requests: 0,
encodedBytes: 0,
decodedBodyBytes: 0,
};
summary.requests += 1;
summary.encodedBytes += row.encodedDataLength;
summary.decodedBodyBytes += row.decodedBodyLength;
byResourceType[row.resourceType] = summary;
}
function isSameOrigin(url: string) {
try {
return new URL(url).origin === origin;
} catch {
return false;
}
}
return {
requestCount: rows.length,
finishedRequestCount: rows.filter(row => row.finished).length,
failedRequestCount: rows.filter(row => row.failed).length,
cachedRequestCount: rows.filter(row => row.fromDiskCache).length,
serviceWorkerRequestCount: rows.filter(row => row.fromServiceWorker).length,
totalEncodedBytes: rows.reduce((sum, row) => sum + row.encodedDataLength, 0),
totalDecodedBodyBytes: rows.reduce((sum, row) => sum + row.decodedBodyLength, 0),
sameOriginEncodedBytes: rows
.filter(row => isSameOrigin(row.url))
.reduce((sum, row) => sum + row.encodedDataLength, 0),
thirdPartyEncodedBytes: rows
.filter(row => !isSameOrigin(row.url))
.reduce((sum, row) => sum + row.encodedDataLength, 0),
byResourceType,
largestRequests: rows
.toSorted((a, b) => b.encodedDataLength - a.encodedDataLength)
.slice(0, 15)
.map(row => ({
url: row.url,
method: row.method,
resourceType: row.resourceType,
status: row.status,
encodedBytes: row.encodedDataLength,
decodedBodyBytes: row.decodedBodyLength,
})),
failedRequests: rows
.filter(row => row.failed)
.map(row => ({
url: row.url,
method: row.method,
resourceType: row.resourceType,
errorText: row.errorText,
status: row.status,
})),
};
}
async function evaluate<T>(cdp: CdpClient, expression: string, timeoutMs = 30_000): Promise<T> {
const result = await cdp.send<{
result: { value: T };
exceptionDetails?: unknown;
}>('Runtime.evaluate', {
expression,
awaitPromise: true,
returnByValue: true,
timeout: timeoutMs,
});
if (result.exceptionDetails != null) {
throw new Error(`Runtime.evaluate failed: ${JSON.stringify(result.exceptionDetails)}`);
}
return result.result.value;
}
function selectorReadyExpression(selector: string, options: { visible?: boolean; enabled?: boolean } = {}) {
return `(() => {
const el = document.querySelector(${JSON.stringify(selector)});
if (el == null) return false;
const style = window.getComputedStyle(el);
const rect = el.getBoundingClientRect();
if (${options.visible === true ? 'true' : 'false'} && (style.visibility === 'hidden' || style.display === 'none' || rect.width === 0 || rect.height === 0)) return false;
if (${options.enabled === true ? 'true' : 'false'} && (el.disabled || el.getAttribute('aria-disabled') === 'true')) return false;
return true;
})()`;
}
async function waitForSelector(cdp: CdpClient, selector: string, options: { timeoutMs?: number; visible?: boolean; enabled?: boolean } = {}) {
const startedAt = Date.now();
const timeoutMs = options.timeoutMs ?? scenarioTimeoutMs;
while (Date.now() - startedAt < timeoutMs) {
const ready = await evaluate<boolean>(cdp, selectorReadyExpression(selector, options), 5_000);
if (ready) return true;
await sleep(250);
}
return false;
}
async function waitForAnySelector(cdp: CdpClient, selectors: string[], options: { timeoutMs?: number; visible?: boolean; enabled?: boolean } = {}) {
const startedAt = Date.now();
const timeoutMs = options.timeoutMs ?? scenarioTimeoutMs;
while (Date.now() - startedAt < timeoutMs) {
for (const selector of selectors) {
const ready = await evaluate<boolean>(cdp, selectorReadyExpression(selector, options), 5_000);
if (ready) return selector;
}
await sleep(250);
}
return null;
}
async function click(cdp: CdpClient, selector: string) {
const found = await waitForSelector(cdp, selector, { visible: true, enabled: true });
if (!found) throw new Error(`Selector was not clickable: ${selector}`);
await evaluate<void>(cdp, `(() => {
const el = document.querySelector(${JSON.stringify(selector)});
if (el == null) throw new Error('Element not found');
el.scrollIntoView({ block: 'center', inline: 'center' });
el.click();
})()`);
}
async function maybeClick(cdp: CdpClient, selector: string, timeoutMs = 3_000) {
if (await waitForSelector(cdp, selector, { visible: true, enabled: true, timeoutMs })) {
await click(cdp, selector);
return true;
}
return false;
}
async function setValue(cdp: CdpClient, selector: string, value: string) {
const found = await waitForSelector(cdp, selector, { visible: true, enabled: true });
if (!found) throw new Error(`Selector was not editable: ${selector}`);
await evaluate<void>(cdp, `(() => {
const el = document.querySelector(${JSON.stringify(selector)});
if (el == null) throw new Error('Element not found');
el.scrollIntoView({ block: 'center', inline: 'center' });
el.focus();
const proto = Object.getPrototypeOf(el);
const descriptor = Object.getOwnPropertyDescriptor(proto, 'value');
if (descriptor?.set != null) {
descriptor.set.call(el, ${JSON.stringify(value)});
} else {
el.value = ${JSON.stringify(value)};
}
el.dispatchEvent(new InputEvent('input', { bubbles: true, inputType: 'insertText', data: ${JSON.stringify(value)} }));
el.dispatchEvent(new Event('change', { bubbles: true }));
})()`);
}
async function waitForText(cdp: CdpClient, text: string, timeoutMs = scenarioTimeoutMs) {
const startedAt = Date.now();
while (Date.now() - startedAt < timeoutMs) {
const found = await evaluate<boolean>(cdp, `document.body?.innerText?.includes(${JSON.stringify(text)}) === true`, 5_000);
if (found) return true;
await sleep(250);
}
return false;
}
async function runSignupAndPostScenario(cdp: CdpClient) {
const noteText = `Frontend browser metrics ${Date.now()}`;
await cdp.send('Page.navigate', { url: `${baseUrl}/` });
const initialSelector = await waitForAnySelector(cdp, ['[data-cy-signup]', '[data-cy-open-post-form]'], { visible: true, timeoutMs: scenarioTimeoutMs });
if (initialSelector == null) throw new Error('Timed out waiting for the signup or timeline entry point');
if (await waitForSelector(cdp, '[data-cy-signup]', { visible: true, enabled: true, timeoutMs: 5_000 })) {
await click(cdp, '[data-cy-signup]');
if (await waitForSelector(cdp, '[data-cy-signup-rules-continue]', { visible: true, timeoutMs: 5_000 })) {
await click(cdp, '[data-cy-signup-rules-notes-agree] [data-cy-switch-toggle]');
await maybeClick(cdp, '[data-cy-modal-dialog-ok]', 5_000);
await click(cdp, '[data-cy-signup-rules-continue]');
}
await setValue(cdp, '[data-cy-signup-username] input', 'alice');
await setValue(cdp, '[data-cy-signup-password] input', 'alice1234');
await setValue(cdp, '[data-cy-signup-password-retype] input', 'alice1234');
if (await waitForSelector(cdp, '[data-cy-signup-invitation-code] input', { visible: true, enabled: true, timeoutMs: 2_000 })) {
await setValue(cdp, '[data-cy-signup-invitation-code] input', 'test-invitation-code');
}
await click(cdp, '[data-cy-signup-submit]');
}
const firstReadySelector = await waitForAnySelector(cdp, [
'[data-cy-user-setup] [data-cy-modal-window-close]',
'[data-cy-open-post-form]',
], { visible: true, enabled: true, timeoutMs: scenarioTimeoutMs });
if (firstReadySelector == null) throw new Error('Timed out waiting for signed-in home timeline');
if (firstReadySelector === '[data-cy-user-setup] [data-cy-modal-window-close]') {
await click(cdp, '[data-cy-user-setup] [data-cy-modal-window-close]');
await maybeClick(cdp, '[data-cy-modal-dialog-ok]', 5_000);
}
await click(cdp, '[data-cy-open-post-form]');
await setValue(cdp, '[data-cy-post-form-text]', noteText);
await click(cdp, '[data-cy-open-post-form-submit]');
if (!await waitForText(cdp, noteText, scenarioTimeoutMs)) {
throw new Error('The first timeline note did not appear');
}
await sleep(settleMs);
}
async function collectPerformance(cdp: CdpClient): Promise<BrowserMeasurement['performance']> {
const cdpMetricsResult = await cdp.send<{ metrics: { name: string; value: number }[] }>('Performance.getMetrics');
const cdpMetrics = Object.fromEntries(cdpMetricsResult.metrics.map(metric => [metric.name, metric.value]));
const runtimeHeap = await cdp.send<{ usedSize: number; totalSize: number }>('Runtime.getHeapUsage').catch(() => undefined);
const webVitals = await evaluate<BrowserMeasurement['performance']['webVitals']>(cdp, `(() => {
const navigation = performance.getEntriesByType('navigation')[0];
const paintEntries = Object.fromEntries(performance.getEntriesByType('paint').map(entry => [entry.name, entry.startTime]));
const longTasks = performance.getEntriesByType('longtask');
const resourceEntries = performance.getEntriesByType('resource');
return {
firstPaintMs: paintEntries['first-paint'],
firstContentfulPaintMs: paintEntries['first-contentful-paint'],
domContentLoadedEventEndMs: navigation?.domContentLoadedEventEnd,
loadEventEndMs: navigation?.loadEventEnd,
longTaskCount: longTasks.length,
longTaskDurationMs: longTasks.reduce((sum, entry) => sum + entry.duration, 0),
maxLongTaskDurationMs: longTasks.reduce((max, entry) => Math.max(max, entry.duration), 0),
resourceEntryCount: resourceEntries.length,
domElements: document.getElementsByTagName('*').length,
};
})()`);
return {
cdpMetrics,
runtimeHeap,
webVitals,
};
}
function emptyHeapSnapshotData(): HeapSnapshotData {
const categories = {} as HeapSnapshotData['categories'];
const nodeCounts = {} as HeapSnapshotData['nodeCounts'];
for (const category of Object.keys(heapSnapshotCategory) as (keyof typeof heapSnapshotCategory)[]) {
categories[category] = 0;
nodeCounts[category] = 0;
}
return {
categories,
nodeCounts,
breakdowns: {} as HeapSnapshotData['breakdowns'],
};
}
function categorizeHeapNode(type: string, name: string): keyof typeof heapSnapshotCategory {
if (/^(ArrayBuffer|SharedArrayBuffer|DataView|(?:Big)?(?:Int|Uint|Float)(?:8|16|32|64)?(?:Clamped)?Array)$/u.test(name)) return 'typedArrays';
if (type === 'code' || type === 'closure') return 'code';
if (type === 'string' || type === 'concatenated string' || type === 'sliced string' || type === 'symbol') return 'strings';
if (type === 'array') return 'jsArrays';
if (type === 'hidden' || type === 'synthetic' || type === 'object shape') return 'systemObjects';
if (type === 'native') return 'otherNonJsObjects';
if (type === 'object' || type === 'regexp' || type === 'number' || type === 'bigint') return 'otherJsObjects';
return 'otherNonJsObjects';
}
function collapseBreakdown(entries: Map<string, number>) {
const sorted = [...entries]
.filter(([, value]) => value > 0)
.toSorted((a, b) => b[1] - a[1]);
const topEntries = sorted.slice(0, heapSnapshotBreakdownTopN);
const otherValue = sorted
.slice(heapSnapshotBreakdownTopN)
.reduce((sum, [, value]) => sum + value, 0);
const collapsed = Object.fromEntries(topEntries);
if (otherValue > 0) collapsed.Other = otherValue;
return collapsed;
}
function summarizeHeapSnapshot(snapshot: any): HeapSnapshotData {
const result = emptyHeapSnapshotData();
const nodeFields = snapshot.snapshot.meta.node_fields as string[];
const nodeTypes = snapshot.snapshot.meta.node_types as unknown[][];
const nodes = snapshot.nodes as number[];
const strings = snapshot.strings as string[];
const fieldCount = nodeFields.length;
const typeOffset = nodeFields.indexOf('type');
const nameOffset = nodeFields.indexOf('name');
const selfSizeOffset = nodeFields.indexOf('self_size');
const typeNames = nodeTypes[typeOffset] as string[];
const breakdownMaps = {} as Record<keyof typeof heapSnapshotCategory, Map<string, number>>;
for (const category of Object.keys(heapSnapshotCategory) as (keyof typeof heapSnapshotCategory)[]) {
if (category !== 'total') breakdownMaps[category] = new Map();
}
for (let offset = 0; offset < nodes.length; offset += fieldCount) {
const type = typeNames[nodes[offset + typeOffset]] ?? 'unknown';
const name = strings[nodes[offset + nameOffset]] ?? '';
const selfSize = nodes[offset + selfSizeOffset] ?? 0;
const category = categorizeHeapNode(type, name);
result.categories.total += selfSize;
result.nodeCounts.total += 1;
result.categories[category] += selfSize;
result.nodeCounts[category] += 1;
const label = `${type}: ${name || '(anonymous)'}`;
breakdownMaps[category].set(label, (breakdownMaps[category].get(label) ?? 0) + selfSize);
}
result.breakdowns = {} as HeapSnapshotData['breakdowns'];
for (const [category, entries] of Object.entries(breakdownMaps) as [keyof typeof heapSnapshotCategory, Map<string, number>][]) {
const collapsed = collapseBreakdown(entries);
if (Object.keys(collapsed).length > 0) {
result.breakdowns[category] = collapsed;
}
}
return result;
}
async function takeHeapSnapshot(cdp: CdpClient, savePath?: string) {
const chunks: string[] = [];
cdp.on('HeapProfiler.addHeapSnapshotChunk', params => {
chunks.push(params.chunk);
});
await cdp.send('HeapProfiler.enable');
await cdp.send('HeapProfiler.collectGarbage');
await cdp.send('HeapProfiler.takeHeapSnapshot', { reportProgress: false });
const content = chunks.join('');
if (savePath != null) {
await writeFile(savePath, content);
}
return summarizeHeapSnapshot(JSON.parse(content));
}
async function measureRepo(label: 'base' | 'head', repoDir: string, outputPath: string, heapSnapshotSavePath?: string) {
let server: ChildProcessWithoutNullStreams | null = null;
let chrome: ChromeHandle | null = null;
let cdp: CdpClient | null = null;
try {
server = startServer(label, repoDir);
await waitForServer(server);
await prepareInstance();
chrome = await launchChrome(label);
cdp = await connectPage(chrome.port);
const networkRequests = installNetworkTracker(cdp);
await cdp.send('Network.enable');
await cdp.send('Network.setCacheDisabled', { cacheDisabled: true });
await cdp.send('Network.setBypassServiceWorker', { bypass: true });
await cdp.send('Page.enable');
await cdp.send('Runtime.enable');
await cdp.send('Performance.enable');
const startedAt = Date.now();
await runSignupAndPostScenario(cdp);
const durationMs = Date.now() - startedAt;
const performance = await collectPerformance(cdp);
const heapSnapshot = await takeHeapSnapshot(cdp, heapSnapshotSavePath);
const measurement: BrowserMeasurement = {
label,
timestamp: new Date().toISOString(),
url: baseUrl,
scenario: 'fresh browser signup, first timeline note, after the note becomes visible',
durationMs,
network: summarizeNetwork(networkRequests),
performance,
heapSnapshot,
};
await writeFile(outputPath, JSON.stringify(measurement, null, '\t'));
process.stderr.write(`[${label}] Wrote browser measurement to ${outputPath}\n`);
} finally {
cdp?.close();
if (chrome != null) await closeChrome(chrome);
if (server != null) await stopServer(server);
}
}
async function main() {
if (baseDirArg == null || headDirArg == null || baseOutputArg == null || headOutputArg == null) {
throw new Error('Usage: node measure-frontend-browser-comparison.mts <base-dir> <head-dir> <base-output.json> <head-output.json> [head-heap-snapshot.heapsnapshot]');
}
await measureRepo('base', resolve(baseDirArg), resolve(baseOutputArg));
await measureRepo('head', resolve(headDirArg), resolve(headOutputArg), headHeapSnapshotOutputArg == null ? undefined : resolve(headHeapSnapshotOutputArg));
}
await main();

View File

@@ -94,11 +94,12 @@ export function escapeLatex(text: string) {
.replaceAll('%', '\\%');
}
export function formatColoredDelta(text: string, delta: number) {
if (delta === 0) return text;
const color = delta > 0 ? 'orange' : 'green';
export function formatColoredDelta(delta: number, text: (value: number) => string, colorThreshold = 0) {
if (delta === 0) return text(0);
const sign = delta > 0 ? '+' : '-';
return `$\\color{${color}}{\\text{${sign}${escapeLatex(text)}}}$`;
if (Math.abs(delta) < colorThreshold) return `$\\text{${sign}${escapeLatex(text(Math.abs(delta)))}}$`;
const color = delta > 0 ? 'orange' : 'green';
return `$\\color{${color}}{\\text{${sign}${escapeLatex(text(Math.abs(delta)))}}}$`;
}
const numberFormatter = new Intl.NumberFormat('en-US', {
@@ -114,8 +115,8 @@ export function formatBytes(value: number) {
const units = ['B', 'KB', 'MB', 'GB'];
let unitIndex = 0;
let size = value;
while (size >= 1024 && unitIndex < units.length - 1) {
size /= 1024;
while (size >= 1000 && unitIndex < units.length - 1) {
size /= 1000;
unitIndex += 1;
}
@@ -123,35 +124,34 @@ export function formatBytes(value: number) {
return `${numberFormatter.format(Number(size.toFixed(maximumFractionDigits)))} ${units[unitIndex]}`;
}
export function calcAndFormatDeltaNumber(before: number, after: number) {
export function calcAndFormatDeltaNumber(before: number, after: number, colorThreshold = 0) {
if (before == null || after == null) return '-';
const delta = after - before;
return formatColoredDelta(formatNumber(Math.abs(delta)), delta);
return formatColoredDelta(delta, v => formatNumber(v), colorThreshold);
}
export function formatDeltaBytes(deltaBytes: number) {
return formatColoredDelta(formatBytes(Math.abs(deltaBytes)), deltaBytes);
export function formatDeltaBytes(deltaBytes: number, colorThreshold = 0) {
return formatColoredDelta(deltaBytes, v => formatBytes(v), colorThreshold);
}
export function calcAndFormatDeltaBytes(before: number, after: number) {
export function calcAndFormatDeltaBytes(before: number, after: number, colorThreshold = 0) {
if (before == null || after == null) return '-';
const delta = after - before;
return formatDeltaBytes(delta);
return formatDeltaBytes(delta, colorThreshold);
}
export function formatPercent(value: number) {
return `${formatNumber(value)}%`;
}
export function formatDeltaPercent(deltaPercent: number) {
if (deltaPercent === 0) return '0%';
return formatColoredDelta(formatPercent(Math.abs(deltaPercent)), deltaPercent);
export function formatDeltaPercent(deltaPercent: number, colorThreshold = 0) {
return formatColoredDelta(deltaPercent, v => formatPercent(v), colorThreshold);
}
export function calcAndFormatDeltaPercent(before: number, after: number) {
export function calcAndFormatDeltaPercent(before: number, after: number, colorThreshold = 0) {
if (before == null || before === 0 || after == null || after === 0) return '-';
const delta = after - before;
return formatDeltaPercent(delta / before * 100);
return formatDeltaPercent(delta / before * 100, colorThreshold);
}
export function commandName(command: string) {

View File

@@ -16,10 +16,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Setup Node.js
uses: actions/setup-node@v6.4.0

View File

@@ -12,7 +12,7 @@ jobs:
steps:
- name: Checkout head
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Setup Node.js
uses: actions/setup-node@v6.4.0
with:

View File

@@ -18,7 +18,7 @@ jobs:
if: ${{ github.event.pull_request.mergeable == null || github.event.pull_request.mergeable == true }}
steps:
- name: checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
with:
submodules: true
persist-credentials: false
@@ -66,7 +66,7 @@ jobs:
if: ${{ github.event.pull_request.mergeable == null || github.event.pull_request.mergeable == true }}
steps:
- name: checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
with:
submodules: true
persist-credentials: false

View File

@@ -20,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Check version
run: |
if [ "$(jq -r '.version' package.json)" != "$(jq -r '.version' packages/misskey-js/package.json)" ]; then

View File

@@ -12,7 +12,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Check
run: |
counter=0
@@ -44,7 +44,6 @@ jobs:
}
directories=(
"cypress/e2e"
"packages/backend/migration"
"packages/backend/src"
"packages/backend/test"

View File

@@ -10,7 +10,7 @@ jobs:
check_copyright_year:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
- run: |
if [ "$(grep Copyright COPYING | sed -e 's/.*2014-\([0-9]*\) .*/\1/g')" -ne "$(date +%Y)" ]; then
echo "Please change copyright year!"

View File

@@ -27,7 +27,7 @@ jobs:
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Check out the repo
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to Docker Hub

View File

@@ -32,7 +32,7 @@ jobs:
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Check out the repo
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Docker meta

View File

@@ -17,7 +17,7 @@ jobs:
DOCKLE_VERSION: 0.4.15
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
- name: Download and install dockle v${{ env.DOCKLE_VERSION }}
run: |

View File

@@ -27,9 +27,6 @@ on:
- .node-version
- .github/scripts/utility.mts
- .github/scripts/frontend-js-size.mts
- .github/scripts/frontend-browser-report.mts
- .github/scripts/heap-snapshot-util.mts
- .github/scripts/measure-frontend-browser-comparison.mts
- .github/workflows/frontend-bundle-report.yml
- .github/workflows/frontend-bundle-report-comment.yml
@@ -184,6 +181,8 @@ jobs:
const path = require('node:path');
const jsSizeMarker = '<!-- misskey-frontend-js-size -->';
const visualizerMarker = '<!-- misskey-frontend-bundle-visualizer -->';
const reportMarkers = [jsSizeMarker, visualizerMarker];
const reportDir = path.join(process.env.RUNNER_TEMP, 'frontend-bundle-report');
const jsSizeReportPath = path.join(reportDir, 'frontend-js-size-report.md');
const prNumberPath = path.join(reportDir, 'pr-number.txt');
@@ -292,7 +291,7 @@ jobs:
per_page: 100,
});
const previousReports = comments.filter((comment) =>
comment.user?.type === 'Bot' && comment.body?.includes(jsSizeMarker));
comment.user?.type === 'Bot' && reportMarkers.some((reportMarker) => comment.body?.includes(reportMarker)));
if (previousReports.length > 0) {
const [previous, ...duplicates] = previousReports;

View File

@@ -22,14 +22,10 @@ on:
- .node-version
- .github/scripts/utility.mts
- .github/scripts/frontend-js-size.mts
- .github/scripts/frontend-browser-report.mts
- .github/scripts/heap-snapshot-util.mts
- .github/scripts/measure-frontend-browser-comparison.mts
- .github/workflows/frontend-bundle-report.yml
- .github/workflows/frontend-bundle-report-comment.yml
permissions:
actions: read
contents: read
pull-requests: read
@@ -40,16 +36,12 @@ concurrency:
jobs:
report:
name: Build frontend bundle report
needs:
- browser-report
runs-on: ubuntu-latest
outputs:
supported: ${{ steps.check-base-visualizer.outputs.supported }}
env:
FRONTEND_JS_SIZE_LOCALE: ja-JP
steps:
- name: Checkout base
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
with:
repository: ${{ github.event.pull_request.base.repo.full_name }}
ref: ${{ github.event.pull_request.base.sha }}
@@ -57,7 +49,7 @@ jobs:
submodules: true
- name: Checkout pull request
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha }}
@@ -77,7 +69,7 @@ jobs:
- name: Setup pnpm
if: steps.check-base-visualizer.outputs.supported == 'true'
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
with:
package_json_file: after/package.json
@@ -143,13 +135,6 @@ jobs:
archive: false
retention-days: 7
- name: Download browser metrics
if: steps.check-base-visualizer.outputs.supported == 'true'
uses: actions/download-artifact@v8
with:
name: frontend-browser-report-part
path: ${{ runner.temp }}/frontend-browser-report
- name: Generate report markdown
if: steps.check-base-visualizer.outputs.supported == 'true'
shell: bash
@@ -158,13 +143,9 @@ jobs:
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
FRONTEND_BUNDLE_REPORT_ARTIFACT_URL: ${{ steps.upload-bundle-visualizer.outputs.artifact-url }}
FRONTEND_BROWSER_HEAD_HEAP_SNAPSHOT_ARTIFACT_URL: ${{ needs.browser-report.outputs.head-heap-snapshot-url }}
run: |
REPORT_DIR="$RUNNER_TEMP/frontend-bundle-report"
BROWSER_REPORT_DIR="$RUNNER_TEMP/frontend-browser-report"
test -s "$BROWSER_REPORT_DIR/before-browser.json"
test -s "$BROWSER_REPORT_DIR/after-browser.json"
node after/.github/scripts/frontend-js-size.mts before after "$REPORT_DIR/before-stats.json" "$REPORT_DIR/after-stats.json" "$BROWSER_REPORT_DIR/before-browser.json" "$BROWSER_REPORT_DIR/after-browser.json" "$REPORT_DIR/frontend-js-size-report.md"
node after/.github/scripts/frontend-js-size.mts before after "$REPORT_DIR/before-stats.json" "$REPORT_DIR/after-stats.json" "$REPORT_DIR/frontend-js-size-report.md"
printf '%s\n' "$PR_NUMBER" > "$REPORT_DIR/pr-number.txt"
printf '%s\n' "$BASE_SHA" > "$REPORT_DIR/base-sha.txt"
printf '%s\n' "$HEAD_SHA" > "$REPORT_DIR/head-sha.txt"
@@ -187,114 +168,3 @@ jobs:
path: ${{ runner.temp }}/frontend-bundle-report/
if-no-files-found: error
retention-days: 7
browser-report:
name: Measure frontend browser metrics
runs-on: ubuntu-latest
timeout-minutes: 75
outputs:
head-heap-snapshot-url: ${{ steps.upload-browser-head-heap-snapshot.outputs.artifact-url }}
services:
postgres:
image: postgres:18
ports:
- 54312:5432
env:
POSTGRES_DB: test-misskey
POSTGRES_HOST_AUTH_METHOD: trust
redis:
image: redis:8
ports:
- 56312:6379
steps:
- name: Checkout base
uses: actions/checkout@v6.0.2
with:
repository: ${{ github.event.pull_request.base.repo.full_name }}
ref: ${{ github.event.pull_request.base.sha }}
path: before
submodules: true
- name: Checkout pull request
uses: actions/checkout@v6.0.2
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha }}
path: after
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
with:
package_json_file: after/package.json
- name: Setup Node.js
uses: actions/setup-node@v6.4.0
with:
node-version-file: after/.node-version
cache: pnpm
cache-dependency-path: |
before/pnpm-lock.yaml
after/pnpm-lock.yaml
- name: Install dependencies for base
working-directory: before
run: pnpm i --frozen-lockfile
- name: Configure base
working-directory: before
run: cp .github/misskey/test.yml .config
- name: Build base
working-directory: before
run: pnpm build
- name: Install dependencies for pull request
working-directory: after
run: pnpm i --frozen-lockfile
- name: Configure pull request
working-directory: after
run: cp .github/misskey/test.yml .config
- name: Build pull request
working-directory: after
run: pnpm build
- name: Measure frontend browser metrics
shell: bash
env:
FRONTEND_BROWSER_METRICS_SCENARIO_TIMEOUT_MS: 120000
FRONTEND_BROWSER_METRICS_SETTLE_MS: 1000
run: |
REPORT_DIR="$RUNNER_TEMP/frontend-browser-report"
mkdir -p "$REPORT_DIR"
node after/.github/scripts/measure-frontend-browser-comparison.mts before after "$REPORT_DIR/before-browser.json" "$REPORT_DIR/after-browser.json" "$REPORT_DIR/head-heap-snapshot.heapsnapshot"
- name: Upload browser head heap snapshot
id: upload-browser-head-heap-snapshot
uses: actions/upload-artifact@v7
with:
name: frontend-browser-head-heap-snapshot
path: ${{ runner.temp }}/frontend-browser-report/head-heap-snapshot.heapsnapshot
if-no-files-found: error
retention-days: 7
- name: Check browser metrics
shell: bash
run: |
REPORT_DIR="$RUNNER_TEMP/frontend-browser-report"
test -s "$REPORT_DIR/before-browser.json"
test -s "$REPORT_DIR/after-browser.json"
- name: Upload browser report part
uses: actions/upload-artifact@v7
with:
name: frontend-browser-report-part
path: |
${{ runner.temp }}/frontend-browser-report/before-browser.json
${{ runner.temp }}/frontend-browser-report/after-browser.json
if-no-files-found: error
retention-days: 7

View File

@@ -25,12 +25,12 @@ jobs:
ref: refs/pull/${{ github.event.number }}/merge
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
ref: ${{ matrix.ref }}
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:

View File

@@ -39,19 +39,19 @@ jobs:
steps:
- name: Checkout base
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
with:
ref: ${{ github.base_ref }}
path: base
submodules: true
- name: Checkout head
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
with:
ref: refs/pull/${{ github.event.number }}/merge
path: head
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
with:
package_json_file: head/package.json
- name: Use Node.js

View File

@@ -17,7 +17,9 @@ on:
- packages/misskey-bubble-game/**
- packages/misskey-reversi/**
- packages/shared/eslint.config.js
- scripts/check-dts*.mjs
- .github/workflows/lint.yml
- package.json
pull_request:
paths:
- packages/backend/**
@@ -31,17 +33,19 @@ on:
- packages/misskey-bubble-game/**
- packages/misskey-reversi/**
- packages/shared/eslint.config.js
- scripts/check-dts*.mjs
- .github/workflows/lint.yml
- package.json
jobs:
pnpm_install:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
fetch-depth: 0
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- uses: actions/setup-node@v6.4.0
with:
node-version-file: '.node-version'
@@ -69,12 +73,12 @@ jobs:
eslint-cache-version: v1
eslint-cache-path: ${{ github.workspace }}/node_modules/.cache/eslint-${{ matrix.workspace }}
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
fetch-depth: 0
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- uses: actions/setup-node@v6.4.0
with:
node-version-file: '.node-version'
@@ -100,6 +104,25 @@ jobs:
- sw
- misskey-js
steps:
- uses: actions/checkout@v6.0.3
with:
fetch-depth: 0
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.9
- uses: actions/setup-node@v6.4.0
with:
node-version-file: '.node-version'
cache: 'pnpm'
- run: pnpm i --frozen-lockfile
- run: pnpm --filter "${{ matrix.workspace }}^..." run build
- run: pnpm --filter ${{ matrix.workspace }} run typecheck
check-dts:
needs: [pnpm_install]
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v6.0.2
with:
fetch-depth: 0
@@ -111,5 +134,5 @@ jobs:
node-version-file: '.node-version'
cache: 'pnpm'
- run: pnpm i --frozen-lockfile
- run: pnpm --filter "${{ matrix.workspace }}^..." run build
- run: pnpm --filter ${{ matrix.workspace }} run typecheck
- run: node --test scripts/check-dts.test.mjs
- run: pnpm check-dts

View File

@@ -16,12 +16,12 @@ jobs:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
fetch-depth: 0
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- uses: actions/setup-node@v6.4.0
with:
node-version-file: ".node-version"

View File

@@ -16,11 +16,11 @@ jobs:
id-token: write
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:

View File

@@ -65,7 +65,7 @@ jobs:
echo '```diff' >> ./output.md
cat ./api.json.diff >> ./output.md
echo '```' >> ./output.md
echo '</details>' >> ./output.md
echo '</details>' >> .output.md
fi
echo "$FOOTER" >> ./output.md

View File

@@ -17,7 +17,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Download artifacts
uses: actions/download-artifact@v8

View File

@@ -22,12 +22,12 @@ jobs:
NODE_OPTIONS: "--max_old_space_size=7168"
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
if: github.event_name != 'pull_request_target'
with:
fetch-depth: 0
submodules: true
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
if: github.event_name == 'pull_request_target'
with:
fetch-depth: 0
@@ -37,7 +37,7 @@ jobs:
if: github.event_name == 'pull_request_target'
run: git checkout "$(git rev-list --parents -n1 HEAD | cut -d" " -f3)"
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:

View File

@@ -43,7 +43,7 @@ jobs:
ports:
- 56312:6379
meilisearch:
image: getmeili/meilisearch:v1.42.1
image: getmeili/meilisearch:v1.48.1
ports:
- 57712:7700
env:
@@ -51,11 +51,11 @@ jobs:
MEILI_ENV: development
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Install FFmpeg
run: |
sudo apt install -y ffmpeg
@@ -103,11 +103,11 @@ jobs:
- 56312:6379
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:
@@ -147,11 +147,11 @@ jobs:
POSTGRES_HOST_AUTH_METHOD: trust
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Get current date
id: current-date
run: echo "today=$(date +'%Y-%m-%d')" >> $GITHUB_OUTPUT

View File

@@ -30,7 +30,7 @@ jobs:
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Install FFmpeg
run: |
sudo apt install -y ffmpeg

View File

@@ -28,11 +28,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:
@@ -57,11 +57,6 @@ jobs:
name: E2E tests (frontend)
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
browser: [chrome]
services:
postgres:
image: postgres:18
@@ -76,17 +71,11 @@ jobs:
- 56312:6379
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
# https://github.com/cypress-io/cypress-docker-images/issues/150
#- name: Install mplayer for FireFox
# run: sudo apt install mplayer -y
# if: ${{ matrix.browser == 'firefox' }}
#- uses: browser-actions/setup-firefox@latest
# if: ${{ matrix.browser == 'firefox' }}
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:
@@ -97,29 +86,14 @@ jobs:
run: cp .github/misskey/test.yml .config
- name: Build
run: pnpm build
# https://github.com/cypress-io/cypress/issues/4351#issuecomment-559489091
- name: ALSA Env
run: echo -e 'pcm.!default {\n type hw\n card 0\n}\n\nctl.!default {\n type hw\n card 0\n}' > ~/.asoundrc
# XXX: This tries reinstalling Cypress if the binary is not cached
# Remove this when the cache issue is fixed
- name: Cypress install
run: pnpm exec cypress install
- name: Cypress run
uses: cypress-io/github-action@v7.1.9
- name: Playwright install
working-directory: packages/frontend
run: pnpm exec playwright install --with-deps chromium
- name: Test
run: pnpm start-server-and-test start:test http://localhost:61812 "pnpm --filter frontend test:e2e"
timeout-minutes: 15
with:
install: false
start: pnpm start:test
wait-on: 'http://localhost:61812'
headed: true
browser: ${{ matrix.browser }}
- uses: actions/upload-artifact@v7
if: failure()
with:
name: ${{ matrix.browser }}-cypress-screenshots
path: cypress/screenshots
- uses: actions/upload-artifact@v7
if: always()
with:
name: ${{ matrix.browser }}-cypress-videos
path: cypress/videos
name: playwright-e2e-artifacts
path: packages/frontend/test/e2e/artifacts

View File

@@ -22,10 +22,10 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
uses: actions/checkout@v6.0.3
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Setup Node.js
uses: actions/setup-node@v6.4.0

View File

@@ -16,11 +16,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:

View File

@@ -17,11 +17,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6.0.2
- uses: actions/checkout@v6.0.3
with:
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
uses: pnpm/action-setup@v6.0.9
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:

7
.gitignore vendored
View File

@@ -23,9 +23,8 @@ packages/sw/.yarn/cache
# pnpm
.pnpm-store
# Cypress
cypress/screenshots
cypress/videos
# Playwright
packages/frontend/test/e2e/artifacts
# Coverage
coverage
@@ -35,7 +34,7 @@ coverage
!/.config/example.yml
!/.config/docker_example.yml
!/.config/docker_example.env
!/.config/cypress-devcontainer.yml
!/.config/playwright-devcontainer.yml
docker-compose.yml
./compose.yml
.devcontainer/compose.yml

View File

@@ -1 +1 @@
22.18.0
26.4.0

View File

@@ -1,21 +1,40 @@
## 2026.6.1
## 2026.7.0
### Note
**今回のリリースではMisskeyの各種動作要件が変更されます。必ずアップグレード前にお使いの環境をご確認ください。**
- センシティブメディアの判定 (NSFW検出) が、本体に内蔵された nsfwjs による推論から、外部サービス [sensitive-detector](https://github.com/misskey-dev/sensitive-detector) への HTTP 呼び出し方式に変更されました。
- これに伴い、本体から `nsfwjs` / `@tensorflow/tfjs` / `@tensorflow/tfjs-node` および同梱の NSFW 判定モデルが削除され、インストール要件 (ネイティブ ML スタック) が緩和されました。
- **センシティブ判定機能を利用しているサーバーは対応が必要です。** 別途 [sensitive-detector](https://github.com/misskey-dev/sensitive-detector) サービスを立ち上げ、コントロールパネルの「モデレーション > センシティブなメディアの検出」で接続先 URL を設定してください。接続先が未設定の場合、センシティブ判定は行われません (すべて非センシティブ扱い)。
- 画像の正規化・動画フレームの抽出・しきい値判定・集約は引き続き本体側で行われ、外部サービスには正規化済み画像の推論のみを委譲します。
- Node.js v24, v26 をサポートしました。**Node.js v22 でも動作しますが、今後のリリースで v22 のサポートを終了する予定**ですので、Node.js のアップデートをご検討ください。
- Node.js のセキュリティアップデートに伴い、最低動作バージョンを 22.22.2 / 24.17.0 / 26.4.0 に引き上げました。
- Docker Image は Node.js 26.4.0-trixie に更新されています。
- バックエンドで画像処理に用いているライブラリ sharp のシステム要件の変更により、**SSE4.2 命令セットをサポートしていない x86_64 CPU では Misskey が正しく動作しなくなります**。仮想マシンに Misskey をデプロイしている場合や、古いハードウェアをお使いの場合は、アップデート前にお使いの環境をご確認ください。なお、ARM64 など x86_64 ではない環境においてはこの変更による影響はありません。
### General
- Feat: コントロールパネルから二要素認証を解除できるように
- Feat: 条件に一致したURLプレビューのサムネイルを隠すことができるように
(Based on https://github.com/MisskeyIO/misskey/pull/214)
### Client
- 2025.4.0 以前の設定情報の移行処理が削除されました
- 2025.4.0 から直接 2026.6.0 以上にアップデートする場合は設定が移行されませんので注意してください。移行したい場合は一度 2026.5.1 を経由してください。
- Fix: デバイスタイプをスマートフォンに固定している状態で画面幅が広いとき、画面左上のアイコンが表示されない問題を修正
- Fix: チャットでIMEの変換を確定するEnterでメッセージが送信されてしまうことがある問題を修正
- Fix: 自分へのメンションに対する色分けで、判定が大文字/小文字を区別していた問題を修正
- Fix: いくつかのイベントリスナーが正しく解除されない問題を修正(メモリ使用量の改善)
### Server
- Enhance: センシティブメディアの判定を外部サービス ([sensitive-detector](https://github.com/misskey-dev/sensitive-detector)) に分離し、`nsfwjs` / `@tensorflow/tfjs(-node)` の同梱と NSFW 判定モデルを廃止 (#16804)
- Enhance: Sentry バックエンドの自動計装を `sentryForBackend.disabledIntegrations` で個別に無効化できるように
- Enhance: Node.js 22.23.0以降、24.17.0以降、26.4.0以降をサポートするように
- Enhance: Docker Image の Node.js を 26.4.0 に、Debian を trixie (v13) に更新
- Fix: `/stats` API のレスポンス型が正しくない問題を修正
- Fix: ハッシュタグに関連するデータを更新する際のエラーハンドリングを修正
- Fix: Sentry 使用環境下にて、Misskey が発行した SQL クエリが span に含まれない問題を修正
- Fix: Sentry 使用環境下にて、外部送信リクエストへ `sentry-trace` / `baggage` ヘッダーが既定で付与されないように
## 2026.6.0

View File

@@ -600,6 +600,90 @@ TypeScriptでjsonをimportすると、tscでコンパイルするときにその
コンポーネント自身がmarginを設定するのは問題の元となることはよく知られている
marginはそのコンポーネントを使う側が設定する
### 命名規則
本来それが略称であっても、通常それでひとつのワードとして用いられるものは、略称として扱わない。
#### 例: IP address
Good: `ipAddress` / `IpAddress`
Bad: `IPAddress`
#### 例: User ID
Good: `userId` / `UserId`
Bad: `userID` / `UserID`
#### 例: XMLなHTTPのRequest
Good: `xmlHttpRequest` / `XmlHttpRequest`
Bad: `XMLHttpRequest` / `XMLHTTPRequest`
### 関数化の基準
汎用性が低く(例えばそれを関数化したとしてもその呼び出しが元の場所一か所しか存在しない)、内容も短い処理(例えば10行以下)は、かえって読みにくくなるため、関数化しない。
また、関数化する場合でも、呼び出しがある特定のスコープに限られる場合は、そのスコープ内に閉じ込めた方が分かりやすく簡潔になる場合がある(ただし本来その処理に不要であっても、構造上親のスコープにある関係のない変数や引数にもアクセスできるようになるため、必ずしもそうすれば設計上綺麗になるというわけでもない。状況に応じて判断すべし)。
Bad:
``` ts
function withBrankets(x) {
return `(${x})`;
}
function formatPercent(x) {
return `${x}%`;
}
function formatValue(x) {
return withBrankets(formatPercent(x));
}
function showData(a, b) {
console.log(formatValue(a));
console.log(formatValue(b));
}
```
Good:
``` ts
function formatValue(x) {
return `(${x}%)`;
}
function showData(a, b) {
console.log(formatValue(a));
console.log(formatValue(b));
}
```
or
``` ts
function showData(a, b) {
function formatValue(x) {
return `(${x}%)`;
}
console.log(formatValue(a));
console.log(formatValue(b));
}
```
or
``` ts
function showData(a, b) {
console.log(`(${a}%)`);
console.log(`(${b}%)`);
}
```
## その他
### HTMLのクラス名で follow という単語は使わない
広告ブロッカーで誤ってブロックされる

View File

@@ -1,6 +1,6 @@
# syntax = docker/dockerfile:1.23
ARG NODE_VERSION=22.22.2-bookworm
ARG NODE_VERSION=26.4.0-trixie
# build assets & compile TypeScript

View File

@@ -1,7 +0,0 @@
import { defineConfig } from 'cypress'
export default defineConfig({
e2e: {
baseUrl: 'http://localhost:61812',
},
})

View File

@@ -1,265 +0,0 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
describe('Before setup instance', () => {
beforeEach(() => {
cy.resetState();
});
afterEach(() => {
// テスト終了直前にページ遷移するようなテストケース(例えばアカウント作成)だと、たぶんCypressのバグでブラウザの内容が次のテストケースに引き継がれてしまう(例えばアカウントが作成し終わった段階からテストが始まる)。
// waitを入れることでそれを防止できる
cy.wait(1000);
});
it('successfully loads', () => {
cy.visitHome();
});
it('setup instance', () => {
cy.visitHome();
cy.intercept('POST', '/api/admin/accounts/create').as('signup');
cy.get('[data-cy-admin-initial-password] input').type('example_password_please_change_this_or_you_will_get_hacked');
cy.get('[data-cy-admin-username] input').type('admin');
cy.get('[data-cy-admin-password] input').type('admin1234');
cy.get('[data-cy-admin-ok]').click();
// なぜか動かない
//cy.wait('@signup').should('have.property', 'response.statusCode');
cy.wait('@signup');
cy.intercept('POST', '/api/admin/update-meta').as('update-meta');
cy.get('[data-cy-next]').click();
cy.get('[data-cy-server-name] input').type('Testskey');
cy.get('[data-cy-server-setup-wizard-apply]').click();
cy.wait('@update-meta');
});
});
describe('After setup instance', () => {
beforeEach(() => {
cy.resetState();
// インスタンス初期セットアップ
cy.registerUser('admin', 'pass', true);
});
afterEach(() => {
// テスト終了直前にページ遷移するようなテストケース(例えばアカウント作成)だと、たぶんCypressのバグでブラウザの内容が次のテストケースに引き継がれてしまう(例えばアカウントが作成し終わった段階からテストが始まる)。
// waitを入れることでそれを防止できる
cy.wait(1000);
});
it('successfully loads', () => {
cy.visitHome();
});
it('signup', () => {
cy.visitHome();
cy.intercept('POST', '/api/signup').as('signup');
cy.get('[data-cy-signup]').click();
cy.get('[data-cy-signup-rules-continue]').should('be.disabled');
cy.get('[data-cy-signup-rules-notes-agree] [data-cy-switch-toggle]').click();
cy.get('[data-cy-modal-dialog-ok]').click();
cy.get('[data-cy-signup-rules-continue]').should('not.be.disabled');
cy.get('[data-cy-signup-rules-continue]').click();
cy.get('[data-cy-signup-submit]').should('be.disabled');
cy.get('[data-cy-signup-username] input').type('alice');
cy.get('[data-cy-signup-submit]').should('be.disabled');
cy.get('[data-cy-signup-password] input').type('alice1234');
cy.get('[data-cy-signup-submit]').should('be.disabled');
cy.get('[data-cy-signup-password-retype] input').type('alice1234');
cy.get('[data-cy-signup-submit]').should('be.disabled');
cy.get('[data-cy-signup-invitation-code] input').type('test-invitation-code');
cy.get('[data-cy-signup-submit]').should('not.be.disabled');
cy.get('[data-cy-signup-submit]').click();
cy.wait('@signup');
});
it('signup with duplicated username', () => {
cy.registerUser('alice', 'alice1234');
cy.visitHome();
// ユーザー名が重複している場合の挙動確認
cy.get('[data-cy-signup]').click();
cy.get('[data-cy-signup-rules-continue]').should('be.disabled');
cy.get('[data-cy-signup-rules-notes-agree] [data-cy-switch-toggle]').click();
cy.get('[data-cy-modal-dialog-ok]').click();
cy.get('[data-cy-signup-rules-continue]').should('not.be.disabled');
cy.get('[data-cy-signup-rules-continue]').click();
cy.get('[data-cy-signup-username] input').type('alice');
cy.get('[data-cy-signup-password] input').type('alice1234');
cy.get('[data-cy-signup-password-retype] input').type('alice1234');
cy.get('[data-cy-signup-submit]').should('be.disabled');
});
});
describe('After user signup', () => {
beforeEach(() => {
cy.resetState();
// インスタンス初期セットアップ
cy.registerUser('admin', 'pass', true);
// ユーザー作成
cy.registerUser('alice', 'alice1234');
});
afterEach(() => {
// テスト終了直前にページ遷移するようなテストケース(例えばアカウント作成)だと、たぶんCypressのバグでブラウザの内容が次のテストケースに引き継がれてしまう(例えばアカウントが作成し終わった段階からテストが始まる)。
// waitを入れることでそれを防止できる
cy.wait(1000);
});
it('successfully loads', () => {
cy.visitHome();
});
it('signin', () => {
cy.visitHome();
cy.intercept('POST', '/api/signin-flow').as('signin');
cy.get('[data-cy-signin]').click();
cy.get('[data-cy-signin-page-input]').should('be.visible', { timeout: 1000 });
// Enterキーで続行できるかの確認も兼ねる
cy.get('[data-cy-signin-username] input').type('alice{enter}');
cy.get('[data-cy-signin-page-password]').should('be.visible', { timeout: 10000 });
// Enterキーで続行できるかの確認も兼ねる
cy.get('[data-cy-signin-password] input').type('alice1234{enter}');
cy.wait('@signin');
});
it('suspend', function() {
cy.request('POST', '/api/admin/suspend-user', {
i: this.admin.token,
userId: this.alice.id,
});
cy.visitHome();
cy.get('[data-cy-signin]').click();
cy.get('[data-cy-signin-page-input]').should('be.visible', { timeout: 1000 });
cy.get('[data-cy-signin-username] input').type('alice{enter}');
// TODO: cypressにブラウザの言語指定できる機能が実装され次第英語のみテストするようにする
cy.contains(/アカウントが凍結されています|This account has been suspended due to/gi);
});
});
describe('After user signed in', () => {
beforeEach(() => {
cy.resetState();
// インスタンス初期セットアップ
cy.registerUser('admin', 'pass', true);
// ユーザー作成
cy.registerUser('alice', 'alice1234');
cy.login('alice', 'alice1234');
});
afterEach(() => {
// テスト終了直前にページ遷移するようなテストケース(例えばアカウント作成)だと、たぶんCypressのバグでブラウザの内容が次のテストケースに引き継がれてしまう(例えばアカウントが作成し終わった段階からテストが始まる)。
// waitを入れることでそれを防止できる
cy.wait(1000);
});
it('successfully loads', () => {
// 表示に時間がかかるのでデフォルト秒数だとタイムアウトする
cy.get('[data-cy-user-setup-continue]', { timeout: 30000 }).should('be.visible');
});
it('account setup wizard', () => {
// 表示に時間がかかるのでデフォルト秒数だとタイムアウトする
cy.get('[data-cy-user-setup-continue]', { timeout: 30000 }).click();
cy.get('[data-cy-user-setup-user-name] input').type('ありす');
cy.get('[data-cy-user-setup-user-description] textarea').type('ほげ');
// TODO: アイコン設定テスト
cy.get('[data-cy-user-setup-continue]').click();
// プライバシー設定
cy.get('[data-cy-user-setup-continue]').click();
// フォローはスキップ
cy.get('[data-cy-user-setup-continue]').click();
// プッシュ通知設定はスキップ
cy.get('[data-cy-user-setup-continue]').click();
cy.get('[data-cy-user-setup-continue]').click();
});
});
describe('After user setup', () => {
beforeEach(() => {
cy.resetState();
// インスタンス初期セットアップ
cy.registerUser('admin', 'pass', true);
// ユーザー作成
cy.registerUser('alice', 'alice1234');
cy.login('alice', 'alice1234');
// アカウント初期設定ウィザード
// 表示に時間がかかるのでデフォルト秒数だとタイムアウトする
cy.get('[data-cy-user-setup] [data-cy-modal-window-close]', { timeout: 30000 }).click();
cy.get('[data-cy-modal-dialog-ok]').click();
});
afterEach(() => {
// テスト終了直前にページ遷移するようなテストケース(例えばアカウント作成)だと、たぶんCypressのバグでブラウザの内容が次のテストケースに引き継がれてしまう(例えばアカウントが作成し終わった段階からテストが始まる)。
// waitを入れることでそれを防止できる
cy.wait(1000);
});
it('note', () => {
cy.get('[data-cy-open-post-form]').should('be.visible');
cy.get('[data-cy-open-post-form]').click();
cy.get('[data-cy-post-form-text]').type('Hello, Misskey!');
cy.get('[data-cy-open-post-form-submit]').click();
cy.contains('Hello, Misskey!', { timeout: 15000 });
});
it('open note form with hotkey', () => {
// Wait until the page loads
cy.get('[data-cy-open-post-form]').should('be.visible');
// Use trigger() to give different `code` to test if hotkeys also work on non-QWERTY keyboards.
cy.document().trigger("keydown", { eventConstructor: 'KeyboardEvent', key: "n", code: "KeyL" });
// See if the form is opened
cy.get('[data-cy-post-form-text]').should('be.visible');
// Close it
cy.focused().trigger("keydown", { eventConstructor: 'KeyboardEvent', key: "Escape", code: "Escape" });
// See if the form is closed
cy.get('[data-cy-post-form-text]').should('not.be.visible');
});
});
// TODO: 投稿フォームの公開範囲指定のテスト
// TODO: 投稿フォームのファイル添付のテスト
// TODO: 投稿フォームのハッシュタグ保持フィールドのテスト

View File

@@ -1,35 +0,0 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
describe('Router transition', () => {
describe('Redirect', () => {
// サーバの初期化。ルートのテストに関しては各describeごとに1度だけ実行で十分だと思う使いまわした方が早い
before(() => {
cy.resetState();
// インスタンス初期セットアップ
cy.registerUser('admin', 'pass', true);
// ユーザー作成
cy.registerUser('alice', 'alice1234');
cy.login('alice', 'alice1234');
// アカウント初期設定ウィザード
// 表示に時間がかかるのでデフォルト秒数だとタイムアウトする
cy.get('[data-cy-user-setup] [data-cy-modal-window-close]', { timeout: 30000 }).click();
cy.wait(500);
cy.get('[data-cy-modal-dialog-ok]').click();
});
it('redirect to user profile', () => {
// テストのためだけに用意されたリダイレクト用ルートに飛ぶ
cy.visit('/redirect-test');
// プロフィールページのURLであることを確認する
cy.url().should('include', '/@alice')
});
});
});

View File

@@ -1,76 +0,0 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
/* flaky
describe('After user signed in', () => {
beforeEach(() => {
cy.resetState();
cy.viewport('macbook-16');
// インスタンス初期セットアップ
cy.registerUser('admin', 'pass', true);
// ユーザー作成
cy.registerUser('alice', 'alice1234');
cy.login('alice', 'alice1234');
// アカウント初期設定ウィザード
cy.get('[data-cy-user-setup] [data-cy-modal-window-close]').click();
cy.get('[data-cy-modal-dialog-ok]').click();
});
afterEach(() => {
// テスト終了直前にページ遷移するようなテストケース(例えばアカウント作成)だと、たぶんCypressのバグでブラウザの内容が次のテストケースに引き継がれてしまう(例えばアカウントが作成し終わった段階からテストが始まる)。
// waitを入れることでそれを防止できる
cy.wait(1000);
});
it('widget edit toggle is visible', () => {
cy.get('[data-cy-widget-edit]').should('be.visible');
});
it('widget select should be visible in edit mode', () => {
cy.get('[data-cy-widget-edit]').click();
cy.get('[data-cy-widget-select]').should('be.visible');
});
it('first widget should be removed', () => {
cy.get('[data-cy-widget-edit]').click();
cy.get('[data-cy-customize-container]:first-child [data-cy-customize-container-remove]._button').click();
cy.get('[data-cy-customize-container]').should('have.length', 2);
});
function buildWidgetTest(widgetName) {
it(`${widgetName} widget should get added`, () => {
cy.get('[data-cy-widget-edit]').click();
cy.get('[data-cy-widget-select] select').select(widgetName, { force: true });
cy.get('[data-cy-bg]._modalBg[data-cy-transparent]').click({ multiple: true, force: true });
cy.get('[data-cy-widget-add]').click({ force: true });
cy.get(`[data-cy-mkw-${widgetName}]`).should('exist');
});
}
buildWidgetTest('memo');
buildWidgetTest('notifications');
buildWidgetTest('timeline');
buildWidgetTest('calendar');
buildWidgetTest('rss');
buildWidgetTest('trends');
buildWidgetTest('clock');
buildWidgetTest('activity');
buildWidgetTest('photos');
buildWidgetTest('digitalClock');
buildWidgetTest('federation');
buildWidgetTest('postForm');
buildWidgetTest('slideshow');
buildWidgetTest('serverMetric');
buildWidgetTest('onlineUsers');
buildWidgetTest('jobQueue');
buildWidgetTest('button');
buildWidgetTest('aiscript');
buildWidgetTest('aichan');
});
*/

View File

@@ -1,5 +0,0 @@
{
"name": "Using fixtures to represent data",
"email": "hello@cypress.io",
"body": "Fixtures are a great way to mock data for responses to routes"
}

View File

@@ -1,67 +0,0 @@
// ***********************************************
// This example commands.js shows you how to
// create various custom commands and overwrite
// existing commands.
//
// For more comprehensive examples of custom
// commands please read more here:
// https://on.cypress.io/custom-commands
// ***********************************************
//
//
// -- This is a parent command --
// Cypress.Commands.add('login', (email, password) => { ... })
//
//
// -- This is a child command --
// Cypress.Commands.add('drag', { prevSubject: 'element'}, (subject, options) => { ... })
//
//
// -- This is a dual command --
// Cypress.Commands.add('dismiss', { prevSubject: 'optional'}, (subject, options) => { ... })
//
//
// -- This will overwrite an existing command --
// Cypress.Commands.overwrite('visit', (originalFn, url, options) => { ... })
Cypress.Commands.add('visitHome', () => {
cy.visit('/');
cy.get('button', { timeout: 30000 }).should('be.visible');
})
Cypress.Commands.add('resetState', () => {
// iframe.contentWindow.indexedDB.deleteDatabase() がchromeのバグで使用できないため、indexedDBを無効化している。
// see https://github.com/misskey-dev/misskey/issues/13605#issuecomment-2053652123
/*
cy.window().then(win => {
win.indexedDB.deleteDatabase('keyval-store');
});
*/
cy.request('POST', '/api/reset-db', {}).as('reset');
cy.get('@reset').its('status').should('equal', 204);
cy.reload(true);
});
Cypress.Commands.add('registerUser', (username, password, isAdmin = false) => {
const route = isAdmin ? '/api/admin/accounts/create' : '/api/signup';
cy.request('POST', route, {
username: username,
password: password,
...(isAdmin ? { setupPassword: 'example_password_please_change_this_or_you_will_get_hacked' } : {}),
}).its('body').as(username);
});
Cypress.Commands.add('login', (username, password) => {
cy.visitHome();
cy.intercept('POST', '/api/signin-flow').as('signin');
cy.get('[data-cy-signin]').click();
cy.get('[data-cy-signin-page-input]').should('be.visible', { timeout: 1000 });
cy.get('[data-cy-signin-username] input').type(`${username}{enter}`);
cy.get('[data-cy-signin-page-password]').should('be.visible', { timeout: 10000 });
cy.get('[data-cy-signin-password] input').type(`${password}{enter}`);
cy.wait('@signin').as('signedIn');
});

View File

@@ -1,34 +0,0 @@
// ***********************************************************
// This example support/index.js is processed and
// loaded automatically before your test files.
//
// This is a great place to put global configuration and
// behavior that modifies Cypress.
//
// You can change the location of this file or turn off
// automatically serving support files with the
// 'supportFile' configuration option.
//
// You can read more here:
// https://on.cypress.io/configuration
// ***********************************************************
// Import commands.js using ES2015 syntax:
import './commands'
// Alternatively you can use CommonJS syntax:
// require('./commands')
Cypress.on('uncaught:exception', (err, runnable) => {
if ([
'The source image cannot be decoded',
// Chrome
'ResizeObserver loop limit exceeded',
// Firefox
'ResizeObserver loop completed with undelivered notifications',
].some(msg => err.message.includes(msg))) {
return false;
}
});

View File

@@ -1,19 +0,0 @@
declare global {
namespace Cypress {
interface Chainable {
login(username: string, password: string): Chainable<void>;
registerUser(
username: string,
password: string,
isAdmin?: boolean
): Chainable<void>;
resetState(): Chainable<void>;
visitHome(): Chainable<void>;
}
}
}
export {}

View File

@@ -1,8 +0,0 @@
{
"compilerOptions": {
"lib": ["dom"],
"target": "esnext",
"types": ["cypress", "node"]
},
"include": ["./**/*.ts"]
}

View File

@@ -1419,6 +1419,8 @@ addToEmojiPalette: "絵文字パレットに追加"
emojiPaletteAlreadyAddedConfirm: "この絵文字はすでにこの絵文字パレットに含まれています。追加しなおしますか?"
append: "末尾に追加"
prepend: "先頭に追加"
urlPreviewSensitiveList: "サムネイルの表示を制限するURL"
urlPreviewSensitiveListDescription: "スペースで区切るとAND指定になり、改行で区切るとOR指定になります。スラッシュで囲むと正規表現になります。一致した場合、サムネイルが表示されなくなります。"
_imageEditing:
_vars:

View File

@@ -1,12 +1,12 @@
{
"name": "misskey",
"version": "2026.6.1-alpha.0",
"version": "2026.7.0-alpha.2",
"codename": "nasubi",
"repository": {
"type": "git",
"url": "https://github.com/misskey-dev/misskey.git"
},
"packageManager": "pnpm@11.8.0",
"packageManager": "pnpm@11.9.0",
"workspaces": [
"packages/misskey-js",
"packages/i18n",
@@ -39,11 +39,10 @@
"migrateandstart": "pnpm migrate && pnpm start",
"watch": "pnpm dev",
"dev": "node scripts/dev.mjs",
"lint": "pnpm --no-bail -r lint",
"cy:open": "pnpm cypress open --config-file=cypress.config.ts",
"cy:run": "pnpm cypress run",
"e2e": "pnpm start-server-and-test start:test http://localhost:61812 cy:run",
"e2e-dev-container": "ncp ./.config/cypress-devcontainer.yml ./.config/test.yml && pnpm start-server-and-test start:test http://localhost:61812 cy:run",
"check-dts": "node scripts/check-dts.mjs",
"lint": "pnpm --no-bail -r lint && pnpm check-dts",
"e2e": "pnpm start-server-and-test start:test http://localhost:61812 \"pnpm --filter frontend test:e2e\"",
"e2e-dev-container": "ncp ./.config/playwright-devcontainer.yml ./.config/test.yml && pnpm start-server-and-test start:test http://localhost:61812 \"pnpm --filter frontend test:e2e\"",
"backend-unit-test": "cd packages/backend && pnpm test",
"backend-unit-test-and-coverage": "cd packages/backend && pnpm test-and-coverage",
"test": "pnpm -r test",
@@ -53,30 +52,25 @@
"cleanall": "pnpm clean-all"
},
"dependencies": {
"cssnano": "8.0.1",
"esbuild": "0.28.1",
"execa": "9.6.1",
"ignore-walk": "8.0.0",
"js-yaml": "4.2.0",
"postcss": "8.5.15",
"tar": "7.5.16",
"terser": "5.48.0"
"ignore-walk": "9.0.0",
"js-yaml": "5.2.0",
"tar": "7.5.19"
},
"devDependencies": {
"@eslint/js": "9.39.4",
"@misskey-dev/eslint-plugin": "2.1.0",
"@types/js-yaml": "4.0.9",
"@types/node": "24.13.1",
"@typescript-eslint/eslint-plugin": "8.61.0",
"@typescript-eslint/parser": "8.61.0",
"@types/node": "26.0.1",
"@typescript-eslint/eslint-plugin": "8.62.0",
"@typescript-eslint/parser": "8.62.0",
"@typescript/native-preview": "7.0.0-dev.20260426.1",
"cross-env": "10.1.0",
"cypress": "15.17.0",
"eslint": "9.39.4",
"globals": "17.6.0",
"globals": "17.7.0",
"ncp": "2.0.0",
"pnpm": "11.8.0",
"start-server-and-test": "3.0.9",
"pnpm": "11.9.0",
"start-server-and-test": "3.0.11",
"typescript": "5.9.3"
}
}

View File

@@ -0,0 +1,16 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
export class UrlPreviewSensitiveList1782581064131 {
name = 'UrlPreviewSensitiveList1782581064131'
async up(queryRunner) {
await queryRunner.query(`ALTER TABLE "meta" ADD "urlPreviewSensitiveList" character varying(3072) array NOT NULL DEFAULT '{}'`);
}
async down(queryRunner) {
await queryRunner.query(`ALTER TABLE "meta" DROP COLUMN "urlPreviewSensitiveList"`);
}
}

View File

@@ -4,7 +4,7 @@
"private": true,
"type": "module",
"engines": {
"node": "^22.15.0 || ^24.10.0"
"node": "^22.22.2 || ^24.17.0 || ^26.4.0"
},
"scripts": {
"start": "pnpm compile-config && node ./built/entry.js",
@@ -51,8 +51,8 @@
"utf-8-validate": "6.0.6"
},
"dependencies": {
"@aws-sdk/client-s3": "3.1065.0",
"@aws-sdk/lib-storage": "3.1065.0",
"@aws-sdk/client-s3": "3.1075.0",
"@aws-sdk/lib-storage": "3.1075.0",
"@fastify/accepts": "5.0.4",
"@fastify/cors": "11.2.0",
"@fastify/http-proxy": "11.5.0",
@@ -61,25 +61,24 @@
"@kitajs/html": "4.2.13",
"@misskey-dev/emoji-assets": "17.0.3",
"@misskey-dev/emoji-data": "17.0.3",
"@misskey-dev/sharp-read-bmp": "1.2.0",
"@misskey-dev/sharp-read-bmp": "1.3.1",
"@misskey-dev/summaly": "5.5.1",
"@napi-rs/canvas": "1.0.0",
"@nestjs/common": "11.1.26",
"@nestjs/core": "11.1.26",
"@nestjs/testing": "11.1.26",
"@oxc-project/runtime": "0.135.0",
"@napi-rs/canvas": "1.0.1",
"@nestjs/common": "11.1.27",
"@nestjs/core": "11.1.27",
"@nestjs/testing": "11.1.27",
"@oxc-project/runtime": "0.137.0",
"@peertube/http-signature": "1.7.0",
"@sentry/node": "10.57.0",
"@sentry/profiling-node": "10.57.0",
"@simplewebauthn/server": "13.3.1",
"@sinonjs/fake-timers": "15.4.0",
"@smithy/node-http-handler": "4.7.7",
"@sentry/node": "10.62.0",
"@sentry/profiling-node": "10.62.0",
"@simplewebauthn/server": "13.3.2",
"@smithy/node-http-handler": "4.8.2",
"accepts": "1.3.8",
"ajv": "8.20.0",
"archiver": "8.0.0",
"bcryptjs": "3.0.3",
"blurhash": "2.0.5",
"bullmq": "5.78.0",
"bullmq": "5.79.2",
"cacheable-lookup": "7.0.0",
"chalk": "5.6.2",
"chalk-template": "1.1.2",
@@ -88,12 +87,12 @@
"content-disposition": "2.0.1",
"date-fns": "4.4.0",
"deep-email-validator": "0.1.27",
"fastify": "5.8.5",
"fastify": "5.9.0",
"fastify-raw-body": "5.0.0",
"feed": "5.2.1",
"file-type": "22.0.1",
"fluent-ffmpeg": "2.1.3",
"got": "15.0.5",
"got": "15.0.7",
"hpagent": "1.2.0",
"http-link-header": "1.1.3",
"i18n": "workspace:*",
@@ -103,40 +102,39 @@
"is-svg": "6.1.0",
"json5": "2.2.3",
"jsonld": "9.0.0",
"juice": "12.1.0",
"juice": "12.1.1",
"meilisearch": "0.58.0",
"mfm-js": "0.26.0",
"mime-types": "3.0.2",
"misskey-js": "workspace:*",
"misskey-reversi": "workspace:*",
"ms": "3.0.0-canary.202508261828",
"nanoid": "5.1.11",
"nanoid": "5.1.16",
"nested-property": "4.0.0",
"node-fetch": "3.3.2",
"node-html-parser": "7.1.0",
"nodemailer": "8.0.10",
"node-html-parser": "8.0.3",
"nodemailer": "9.0.1",
"os-utils": "0.0.14",
"otpauth": "9.5.1",
"pg": "8.21.0",
"pg": "8.22.0",
"pkce-challenge": "6.0.0",
"probe-image-size": "7.3.0",
"promise-limit": "2.7.0",
"qrcode": "1.5.4",
"random-seed": "0.3.0",
"ratelimiter": "3.4.1",
"re2": "1.24.1",
"re2": "1.25.0",
"reflect-metadata": "0.2.2",
"rename": "1.0.4",
"rss-parser": "3.13.0",
"rxjs": "7.8.2",
"sanitize-html": "2.17.4",
"sanitize-html": "2.17.5",
"secure-json-parse": "4.1.0",
"semver": "7.8.4",
"sharp": "0.33.5",
"semver": "7.8.5",
"sharp": "0.35.2",
"slacc": "0.1.5",
"strict-event-emitter-types": "2.0.0",
"stringz": "2.1.0",
"systeminformation": "5.31.7",
"systeminformation": "5.31.11",
"tinycolor2": "1.6.0",
"tmp": "0.2.7",
"tsc-alias": "1.8.17",
@@ -149,19 +147,19 @@
},
"devDependencies": {
"@kitajs/ts-html-plugin": "4.1.4",
"@nestjs/platform-express": "11.1.26",
"@nestjs/platform-express": "11.1.27",
"@rollup/plugin-esm-shim": "0.1.8",
"@sentry/vue": "10.57.0",
"@sentry/vue": "10.62.0",
"@sinonjs/fake-timers": "15.4.0",
"@types/accepts": "1.3.7",
"@types/archiver": "8.0.0",
"@types/fluent-ffmpeg": "2.1.28",
"@types/http-link-header": "1.0.7",
"@types/js-yaml": "4.0.9",
"@types/jsonld": "1.5.15",
"@types/mime-types": "3.0.1",
"@types/ms": "2.1.0",
"@types/node": "24.13.1",
"@types/nodemailer": "8.0.0",
"@types/node": "26.0.1",
"@types/nodemailer": "8.0.1",
"@types/pg": "8.20.0",
"@types/qrcode": "1.5.6",
"@types/random-seed": "0.3.5",
@@ -171,28 +169,26 @@
"@types/semver": "7.7.1",
"@types/simple-oauth2": "5.0.8",
"@types/sinonjs__fake-timers": "15.0.1",
"@types/supertest": "7.2.0",
"@types/tinycolor2": "1.4.6",
"@types/tmp": "0.2.6",
"@types/vary": "1.1.3",
"@types/web-push": "3.6.4",
"@types/ws": "8.18.1",
"@typescript-eslint/eslint-plugin": "8.61.0",
"@typescript-eslint/parser": "8.61.0",
"@vitest/coverage-v8": "4.1.8",
"@typescript-eslint/eslint-plugin": "8.62.0",
"@typescript-eslint/parser": "8.62.0",
"@vitest/coverage-v8": "4.1.9",
"aws-sdk-client-mock": "4.1.0",
"cbor": "10.0.12",
"cbor2": "2.3.0",
"cross-env": "10.1.0",
"eslint-plugin-import": "2.32.0",
"execa": "9.6.1",
"fkill": "10.0.3",
"js-yaml": "4.2.0",
"js-yaml": "5.2.0",
"pid-port": "2.1.1",
"rolldown": "1.1.0",
"rolldown": "1.1.3",
"simple-oauth2": "5.1.0",
"supertest": "7.2.2",
"vite": "8.0.16",
"vitest": "4.1.8",
"vite": "8.1.0",
"vitest": "4.1.9",
"vitest-mock-extended": "4.0.0"
}
}

View File

@@ -75,6 +75,8 @@ export default defineConfig((args) => {
're2',
'ipaddr.js',
'file-type',
// バンドルするとSentryの自動計装が正しく行われなくなるため外しておく
'pg',
];
const define: Record<string, string> = {

View File

@@ -11,7 +11,7 @@
import fs from 'node:fs';
import { resolve, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import yaml from 'js-yaml';
import { load as loadYaml } from 'js-yaml';
const _filename = fileURLToPath(import.meta.url);
const _dirname = dirname(_filename);
@@ -34,7 +34,7 @@ function yamlToJson(ymlPath) {
console.log(`${ymlPath}${OUTPUT_PATH}`);
const yamlContent = fs.readFileSync(ymlPath, 'utf-8');
const jsonContent = yaml.load(yamlContent);
const jsonContent = loadYaml(yamlContent);
if (!fs.existsSync(dirname(OUTPUT_PATH))) {
fs.mkdirSync(dirname(OUTPUT_PATH), { recursive: true });
}

View File

@@ -46,33 +46,20 @@ const HEAP_SNAPSHOT_SAVE_PATH = process.env.MK_MEMORY_HEAP_SNAPSHOT_SAVE_PATH;
const procStatusKeys = ['VmPeak', 'VmSize', 'VmHWM', 'VmRSS', 'VmData', 'VmStk', 'VmExe', 'VmLib', 'VmPTE', 'VmSwap'] as const;
const smapsRollupKeys = ['Pss', 'Shared_Clean', 'Shared_Dirty', 'Private_Clean', 'Private_Dirty', 'Swap', 'SwapPss'] as const;
const typedArrayNames = new Set([
'ArrayBuffer',
'SharedArrayBuffer',
'DataView',
'Int8Array',
'Uint8Array',
'Uint8ClampedArray',
'Int16Array',
'Uint16Array',
'Int32Array',
'Uint32Array',
'Float16Array',
'Float32Array',
'Float64Array',
'BigInt64Array',
'BigUint64Array',
'system / JSArrayBufferData',
]);
const otherJsNodeTypes = new Set([
'object',
'closure',
'regexp',
'number',
'symbol',
'bigint',
]);
type GcMessage = 'gc ok' | 'gc unavailable';
type RuntimeMemoryUsageMessage = {
type: 'memory usage';
value: NodeJS.MemoryUsage;
};
type HeapSnapshotMessage = {
type: 'heap snapshot';
path?: string;
};
type HeapSnapshotErrorMessage = {
type: 'heap snapshot error';
message: string;
};
type HeapSnapshotResponseMessage = HeapSnapshotMessage | HeapSnapshotErrorMessage;
function parseMemoryFile<KS extends readonly string[]>(content: string, keys: KS, path: string, required: boolean): Record<KS[number], number> {
const result = {} as Record<KS[number], number>;
@@ -92,29 +79,6 @@ function bytesToKiB(value: number) {
return Math.round(value / 1024);
}
function isTypedArrayNode(type, name) {
return typedArrayNames.has(name) ||
(type === 'native' && (name.includes('ArrayBuffer') || name.includes('TypedArray')));
}
function isSystemNode(type, name) {
return type === 'hidden' ||
type === 'synthetic' ||
type === 'object shape' ||
name.startsWith('system /') ||
name.startsWith('(system ');
}
function classifyHeapSnapshotNode(type, name): keyof typeof heapSnapshotCategory {
if (type === 'code') return 'code';
if (type === 'string' || type === 'concatenated string' || type === 'sliced string') return 'strings';
if (isTypedArrayNode(type, name)) return 'typedArrays';
if (type === 'array' || (type === 'object' && name === 'Array')) return 'jsArrays';
if (isSystemNode(type, name)) return 'systemObjects';
if (otherJsNodeTypes.has(type)) return 'otherJsObjects';
return 'otherNonJsObjects';
}
function sanitizeHeapSnapshotBreakdownLabel(value, fallback = 'unknown') {
const label = String(value ?? '').replace(/\s+/g, ' ').trim();
if (label === '') return fallback;
@@ -126,17 +90,13 @@ function classifyHeapSnapshotBreakdown(category: keyof typeof heapSnapshotCatego
if (category === 'strings') return type;
if (category === 'jsArrays') {
if (type === 'array') return 'array nodes';
if (type === 'array elements') return 'Array elements';
if (type === 'object' && name === 'Array') return 'Array objects';
return sanitizeHeapSnapshotBreakdownLabel(`${type}: ${name}`);
}
if (category === 'typedArrays') {
if (name === 'system / JSArrayBufferData') return 'ArrayBuffer data';
if (name === 'Uint8Array') return 'Uint8Array / Buffer';
if (typedArrayNames.has(name)) return name;
if (type === 'native' && name.includes('ArrayBuffer')) return 'native ArrayBuffer';
if (type === 'native' && name.includes('TypedArray')) return 'native TypedArray';
return sanitizeHeapSnapshotBreakdownLabel(`${type}: ${name}`);
}
@@ -152,6 +112,7 @@ function classifyHeapSnapshotBreakdown(category: keyof typeof heapSnapshotCatego
}
if (category === 'otherNonJsObjects') {
if (type === 'extra native bytes') return 'Extra native bytes';
if (type === 'native') return sanitizeHeapSnapshotBreakdownLabel(`native: ${name}`, 'native: unknown');
return sanitizeHeapSnapshotBreakdownLabel(`${type}: ${name}`, type);
}
@@ -189,32 +150,56 @@ function collapseHeapSnapshotBreakdown(breakdowns: Record<string, Record<string,
return collapsed;
}
// Keep these buckets aligned with Chrome DevTools' heap snapshot Statistics view.
function analyzeHeapSnapshot(snapshot) {
const meta = snapshot?.snapshot?.meta;
const nodes = snapshot?.nodes;
const edges = snapshot?.edges;
const strings = snapshot?.strings;
if (meta == null || !Array.isArray(nodes) || !Array.isArray(strings)) {
if (meta == null || !Array.isArray(nodes) || !Array.isArray(edges) || !Array.isArray(strings)) {
throw new Error('Invalid heap snapshot format');
}
const nodeFields = meta.node_fields;
if (!Array.isArray(nodeFields)) throw new Error('Invalid heap snapshot node fields');
const edgeFields = meta.edge_fields;
if (!Array.isArray(edgeFields)) throw new Error('Invalid heap snapshot edge fields');
const typeOffset = nodeFields.indexOf('type');
const nameOffset = nodeFields.indexOf('name');
const selfSizeOffset = nodeFields.indexOf('self_size');
if (typeOffset < 0 || nameOffset < 0 || selfSizeOffset < 0) {
const edgeCountOffset = nodeFields.indexOf('edge_count');
if (typeOffset < 0 || nameOffset < 0 || selfSizeOffset < 0 || edgeCountOffset < 0) {
throw new Error('Heap snapshot is missing required node fields');
}
const edgeTypeOffset = edgeFields.indexOf('type');
const edgeNameOffset = edgeFields.indexOf('name_or_index');
const edgeToNodeOffset = edgeFields.indexOf('to_node');
if (edgeTypeOffset < 0 || edgeNameOffset < 0 || edgeToNodeOffset < 0) {
throw new Error('Heap snapshot is missing required edge fields');
}
const nodeTypeNames = meta.node_types?.[typeOffset];
if (!Array.isArray(nodeTypeNames)) throw new Error('Invalid heap snapshot node types');
const edgeTypeNames = meta.edge_types?.[edgeTypeOffset];
if (!Array.isArray(edgeTypeNames)) throw new Error('Invalid heap snapshot edge types');
function createEmptyHeapSnapshotCategoryMap() {
return Object.fromEntries(Object.keys(heapSnapshotCategory).map(category => [category, 0])) as Record<keyof typeof heapSnapshotCategory, number>;
}
const fieldCount = nodeFields.length;
const nodeFieldCount = nodeFields.length;
const edgeFieldCount = edgeFields.length;
const nativeType = nodeTypeNames.indexOf('native');
const codeType = nodeTypeNames.indexOf('code');
const hiddenType = nodeTypeNames.indexOf('hidden');
const stringTypes = new Set([
nodeTypeNames.indexOf('string'),
nodeTypeNames.indexOf('concatenated string'),
nodeTypeNames.indexOf('sliced string'),
]);
const internalEdgeType = edgeTypeNames.indexOf('internal');
const extraNativeBytes = Number.isFinite(snapshot.snapshot.extra_native_bytes) ? snapshot.snapshot.extra_native_bytes : 0;
const categories = createEmptyHeapSnapshotCategoryMap();
const nodeCounts = createEmptyHeapSnapshotCategoryMap();
const breakdowns = Object.fromEntries(
@@ -227,17 +212,104 @@ function analyzeHeapSnapshot(snapshot) {
map[key] = (map[key] ?? 0) + value;
}
for (let offset = 0; offset < nodes.length; offset += fieldCount) {
const type = nodeTypeNames[nodes[offset + typeOffset]] ?? 'unknown';
const name = strings[nodes[offset + nameOffset]] ?? '';
const selfSize = nodes[offset + selfSizeOffset] ?? 0;
const category = classifyHeapSnapshotNode(type, name);
const edgeStartIndexes = new Map<number, number>();
const retainerCounts = new Map<number, number>();
let edgeIndex = 0;
for (let nodeIndex = 0; nodeIndex < nodes.length; nodeIndex += nodeFieldCount) {
edgeStartIndexes.set(nodeIndex, edgeIndex);
const edgeCount = nodes[nodeIndex + edgeCountOffset] ?? 0;
for (let i = 0; i < edgeCount; i++, edgeIndex += edgeFieldCount) {
const toNodeIndex = edges[edgeIndex + edgeToNodeOffset];
retainerCounts.set(toNodeIndex, (retainerCounts.get(toNodeIndex) ?? 0) + 1);
}
}
categories[category] += selfSize;
const jsArrayElementNodeIndexes = new Set<number>();
function addCategoryValue(category: keyof typeof heapSnapshotCategory, value: number, type: string, name: string, nodeIndex: number | null = null) {
if (value <= 0) return;
categories[category] += value;
addValue(breakdowns[category], classifyHeapSnapshotBreakdown(category, type, name), value);
if (nodeIndex != null) nodeCounts[category]++;
}
function addJsArrayElementSize(nodeIndex: number) {
const beginEdgeIndex = edgeStartIndexes.get(nodeIndex) ?? 0;
const edgeCount = nodes[nodeIndex + edgeCountOffset] ?? 0;
for (let i = 0, currentEdgeIndex = beginEdgeIndex; i < edgeCount; i++, currentEdgeIndex += edgeFieldCount) {
const edgeType = edges[currentEdgeIndex + edgeTypeOffset];
if (edgeType !== internalEdgeType) continue;
const edgeName = strings[edges[currentEdgeIndex + edgeNameOffset]];
if (edgeName !== 'elements') continue;
const elementsNodeIndex = edges[currentEdgeIndex + edgeToNodeOffset];
if ((retainerCounts.get(elementsNodeIndex) ?? 0) === 1) {
const elementsSize = nodes[elementsNodeIndex + selfSizeOffset] ?? 0;
addCategoryValue('jsArrays', elementsSize, 'array elements', 'Array elements', elementsNodeIndex);
jsArrayElementNodeIndexes.add(elementsNodeIndex);
}
break;
}
}
if (extraNativeBytes > 0) {
addCategoryValue('otherNonJsObjects', extraNativeBytes, 'extra native bytes', 'extra native bytes');
}
for (let nodeIndex = 0; nodeIndex < nodes.length; nodeIndex += nodeFieldCount) {
const typeId = nodes[nodeIndex + typeOffset];
const type = nodeTypeNames[typeId] ?? 'unknown';
const name = strings[nodes[nodeIndex + nameOffset]] ?? '';
const selfSize = nodes[nodeIndex + selfSizeOffset] ?? 0;
categories.total += selfSize;
nodeCounts[category]++;
nodeCounts.total++;
addValue(breakdowns[category], classifyHeapSnapshotBreakdown(category, type, name), selfSize);
if (typeId === hiddenType) {
addCategoryValue('systemObjects', selfSize, type, name, nodeIndex);
continue;
}
if (typeId === nativeType) {
if (name === 'system / JSArrayBufferData') {
addCategoryValue('typedArrays', selfSize, type, name, nodeIndex);
} else {
addCategoryValue('otherNonJsObjects', selfSize, type, name, nodeIndex);
}
continue;
}
if (typeId === codeType) {
addCategoryValue('code', selfSize, type, name, nodeIndex);
continue;
}
if (stringTypes.has(typeId)) {
addCategoryValue('strings', selfSize, type, name, nodeIndex);
continue;
}
if (name === 'Array') {
addCategoryValue('jsArrays', selfSize, type, name, nodeIndex);
addJsArrayElementSize(nodeIndex);
continue;
}
}
categories.total += extraNativeBytes;
for (let nodeIndex = 0; nodeIndex < nodes.length; nodeIndex += nodeFieldCount) {
if (jsArrayElementNodeIndexes.has(nodeIndex)) continue;
const typeId = nodes[nodeIndex + typeOffset];
if (typeId === hiddenType || typeId === nativeType || typeId === codeType || stringTypes.has(typeId)) continue;
const name = strings[nodes[nodeIndex + nameOffset]] ?? '';
if (name === 'Array') continue;
const type = nodeTypeNames[typeId] ?? 'unknown';
const selfSize = nodes[nodeIndex + selfSizeOffset] ?? 0;
addCategoryValue('otherJsObjects', selfSize, type, name, nodeIndex);
}
return {
@@ -259,14 +331,32 @@ async function getSmapsRollupMemoryUsage(pid: number) {
return parseMemoryFile(smapsRollup, smapsRollupKeys, path, false);
}
function waitForMessage(serverProcess: ChildProcess, predicate: (message: any) => boolean, description: string, timeout = IPC_TIMEOUT) {
return new Promise((resolve, reject) => {
function isRecord(value: unknown): value is Record<string, unknown> {
return value != null && typeof value === 'object';
}
function isGcMessage(message: unknown): message is GcMessage {
return message === 'gc ok' || message === 'gc unavailable';
}
function isRuntimeMemoryUsageMessage(message: unknown): message is RuntimeMemoryUsageMessage {
return isRecord(message) && message.type === 'memory usage' && isRecord(message.value);
}
function isHeapSnapshotResponseMessage(message: unknown): message is HeapSnapshotResponseMessage {
if (!isRecord(message)) return false;
if (message.type === 'heap snapshot') return true;
return message.type === 'heap snapshot error' && typeof message.message === 'string';
}
function waitForMessage<T>(serverProcess: ChildProcess, predicate: (message: unknown) => message is T, description: string, timeout = IPC_TIMEOUT) {
return new Promise<T>((resolve, reject) => {
const timer = globalThis.setTimeout(() => {
serverProcess.off('message', onMessage);
reject(new Error(`Timed out waiting for ${description}`));
}, timeout);
const onMessage = (message: any) => {
const onMessage = (message: unknown) => {
if (!predicate(message)) return;
globalThis.clearTimeout(timer);
serverProcess.off('message', onMessage);
@@ -280,7 +370,7 @@ function waitForMessage(serverProcess: ChildProcess, predicate: (message: any) =
async function getRuntimeMemoryUsage(serverProcess: ChildProcess) {
const response = waitForMessage(
serverProcess,
message => message != null && typeof message === 'object' && message.type === 'memory usage',
isRuntimeMemoryUsageMessage,
'memory usage',
);
@@ -303,7 +393,7 @@ async function getHeapSnapshotStatistics(serverProcess: ChildProcess): Promise<H
const snapshotPath = join(tmpdir(), `misskey-backend-heap-${process.pid}-${serverProcess.pid}-${Date.now()}.heapsnapshot`);
const response = waitForMessage(
serverProcess,
message => message != null && typeof message === 'object' && (message.type === 'heap snapshot' || message.type === 'heap snapshot error'),
isHeapSnapshotResponseMessage,
'heap snapshot',
HEAP_SNAPSHOT_TIMEOUT,
);
@@ -385,7 +475,7 @@ async function measureMemory() {
async function triggerGc() {
const ok = waitForMessage(
serverProcess,
message => message === 'gc ok' || message === 'gc unavailable',
isGcMessage,
'GC completion',
);

View File

@@ -26,7 +26,6 @@ declare module 'probe-image-size' {
function probeImageSize(src: string | ReadStream, callback: (err: Error | null, result?: ProbeResult) => void): void;
function probeImageSize(src: string | ReadStream, options: ProbeOptions, callback: (err: Error | null, result?: ProbeResult) => void): void;
namespace probeImageSize {} // Hack
export = probeImageSize;
// eslint-disable-next-line import/no-default-export
export { probeImageSize as default };
}

View File

@@ -13,6 +13,7 @@ import { loadConfig } from '@/config.js';
import type { Config } from '@/config.js';
import { showMachineInfo } from '@/misc/show-machine-info.js';
import { envOption } from '@/env.js';
import { initTelemetry } from '@/core/telemetry/telemetry-registry.js';
import { initExtraThreadPool, jobQueue, server } from './common.js';
const logger = new Logger('core', 'cyan');
@@ -66,26 +67,7 @@ export async function masterMain() {
initExtraThreadPool(config);
if (config.sentryForBackend) {
const Sentry = await import('@sentry/node');
const { nodeProfilingIntegration } = await import('@sentry/profiling-node');
Sentry.init({
integrations: [
...(config.sentryForBackend.enableNodeProfiling ? [nodeProfilingIntegration()] : []),
],
// Performance Monitoring
tracesSampleRate: 1.0, // Capture 100% of the transactions
// Set sampling rate for profiling - this is relative to tracesSampleRate
profilesSampleRate: 1.0,
maxBreadcrumbs: 0,
...config.sentryForBackend.options,
});
}
await initTelemetry(config);
bootLogger.info(
`mode: [disableClustering: ${envOption.disableClustering}, onlyServer: ${envOption.onlyServer}, onlyQueue: ${envOption.onlyQueue}]`,

View File

@@ -6,6 +6,7 @@
import cluster from 'node:cluster';
import { envOption } from '@/env.js';
import { loadConfig } from '@/config.js';
import { initTelemetry } from '@/core/telemetry/telemetry-registry.js';
import { initExtraThreadPool, jobQueue, server } from './common.js';
/**
@@ -16,26 +17,7 @@ export async function workerMain() {
initExtraThreadPool(config);
if (config.sentryForBackend) {
const Sentry = await import('@sentry/node');
const { nodeProfilingIntegration } = await import('@sentry/profiling-node');
Sentry.init({
integrations: [
...(config.sentryForBackend.enableNodeProfiling ? [nodeProfilingIntegration()] : []),
],
// Performance Monitoring
tracesSampleRate: 1.0, // Capture 100% of the transactions
// Set sampling rate for profiling - this is relative to tracesSampleRate
profilesSampleRate: 1.0,
maxBreadcrumbs: 0,
...config.sentryForBackend.options,
});
}
await initTelemetry(config);
if (envOption.onlyServer) {
await server();

View File

@@ -20,6 +20,12 @@ type RedisOptionsSource = Partial<RedisOptions> & {
prefix?: string;
};
type SentryBackendConfig = {
options: Partial<Sentry.NodeOptions>;
enableNodeProfiling: boolean;
disabledIntegrations?: string[];
};
/**
* 設定ファイルの型
*/
@@ -64,7 +70,7 @@ type Source = {
index: string;
scope?: 'local' | 'global' | string[];
};
sentryForBackend?: { options: Partial<Sentry.NodeOptions>; enableNodeProfiling: boolean; };
sentryForBackend?: SentryBackendConfig;
sentryForFrontend?: {
options: Partial<SentryVue.BrowserOptions> & { dsn: string };
vueIntegration?: SentryVue.VueIntegrationOptions | null;
@@ -201,7 +207,7 @@ export type Config = {
redisForJobQueue: RedisOptions & RedisOptionsSource;
redisForTimelines: RedisOptions & RedisOptionsSource;
redisForReactions: RedisOptions & RedisOptionsSource;
sentryForBackend: { options: Partial<Sentry.NodeOptions>; enableNodeProfiling: boolean; } | undefined;
sentryForBackend: SentryBackendConfig | undefined;
sentryForFrontend: {
options: Partial<SentryVue.BrowserOptions> & { dsn: string };
vueIntegration?: SentryVue.VueIntegrationOptions | null;

View File

@@ -137,7 +137,8 @@ export class AiService {
try {
const form = new FormData();
for (let i = 0; i < chunk.length; i++) {
form.append(`image${i}`, new Blob([chunk[i]], { type: 'image/png' }), `${i}.png`);
const image = Uint8Array.from(chunk[i]);
form.append(`image${i}`, new Blob([image], { type: 'image/png' }), `${i}.png`);
}
// Content-Type は FormData から boundary 付きで自動設定させるため、手動設定はしない。

View File

@@ -154,10 +154,12 @@ import { ApQuestionService } from './activitypub/models/ApQuestionService.js';
import { QueueModule } from './QueueModule.js';
import { QueueService } from './QueueService.js';
import { LoggerService } from './LoggerService.js';
import { TelemetryService } from './telemetry/TelemetryService.js';
import type { Provider } from '@nestjs/common';
//#region 文字列ベースでのinjection用(循環参照対応のため)
const $LoggerService: Provider = { provide: 'LoggerService', useExisting: LoggerService };
const $TelemetryService: Provider = { provide: 'TelemetryService', useExisting: TelemetryService };
const $AbuseReportService: Provider = { provide: 'AbuseReportService', useExisting: AbuseReportService };
const $AbuseReportNotificationService: Provider = { provide: 'AbuseReportNotificationService', useExisting: AbuseReportNotificationService };
const $AccountMoveService: Provider = { provide: 'AccountMoveService', useExisting: AccountMoveService };
@@ -458,6 +460,7 @@ const $ApQuestionService: Provider = { provide: 'ApQuestionService', useExisting
ApPersonService,
ApQuestionService,
QueueService,
TelemetryService,
//#region 文字列ベースでのinjection用(循環参照対応のため)
$LoggerService,
@@ -606,6 +609,7 @@ const $ApQuestionService: Provider = { provide: 'ApQuestionService', useExisting
$ApNoteService,
$ApPersonService,
$ApQuestionService,
$TelemetryService,
//#endregion
],
exports: [
@@ -757,6 +761,7 @@ const $ApQuestionService: Provider = { provide: 'ApQuestionService', useExisting
ApPersonService,
ApQuestionService,
QueueService,
TelemetryService,
//#region 文字列ベースでのinjection用(循環参照対応のため)
$LoggerService,
@@ -903,6 +908,7 @@ const $ApQuestionService: Provider = { provide: 'ApQuestionService', useExisting
$ApNoteService,
$ApPersonService,
$ApQuestionService,
$TelemetryService,
//#endregion
],
})

View File

@@ -7,6 +7,7 @@ import { randomUUID } from 'node:crypto';
import * as fs from 'node:fs';
import { Inject, Injectable } from '@nestjs/common';
import sharp from 'sharp';
import type { Sharp } from 'sharp';
import { sharpBmp } from '@misskey-dev/sharp-read-bmp';
import { In, IsNull } from 'typeorm';
import { DeleteObjectCommandInput, PutObjectCommandInput, NoSuchKey } from '@aws-sdk/client-s3';
@@ -300,7 +301,7 @@ export class DriveService {
};
}
let img: sharp.Sharp | null = null;
let img: Sharp | null = null;
let satisfyWebpublic: boolean;
let isAnimated: boolean;

View File

@@ -3,11 +3,9 @@
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { URLSearchParams } from 'node:url';
import * as nodemailer from 'nodemailer';
import juice from 'juice';
import { Inject, Injectable } from '@nestjs/common';
import { validate as validateEmail } from 'deep-email-validator';
import { UtilityService } from '@/core/UtilityService.js';
import { DI } from '@/di-symbols.js';
import type { Config } from '@/config.js';
@@ -197,6 +195,7 @@ export class EmailService {
} else if (this.meta.enableTruemailApi && this.meta.truemailInstance && this.meta.truemailAuthKey != null) {
validated = await this.trueMail(this.meta.truemailInstance, emailAddress, this.meta.truemailAuthKey);
} else {
const { validate: validateEmail } = await import('deep-email-validator');
validated = await validateEmail({
email: emailAddress,
validateRegex: true,

View File

@@ -5,20 +5,28 @@
import { Inject, Injectable } from '@nestjs/common';
import * as Redis from 'ioredis';
import { DataSource } from 'typeorm';
import { DI } from '@/di-symbols.js';
import type { MiUser } from '@/models/User.js';
import { normalizeForSearch } from '@/misc/normalize-for-search.js';
import { IdService } from '@/core/IdService.js';
import type { MiHashtag } from '@/models/Hashtag.js';
import { MiHashtag } from '@/models/Hashtag.js';
import type { HashtagsRepository, MiMeta } from '@/models/_.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { bindThis } from '@/decorators.js';
import { FeaturedService } from '@/core/FeaturedService.js';
import { UtilityService } from '@/core/UtilityService.js';
import { isDuplicateKeyValueError } from '@/misc/is-duplicate-key-value-error.js';
import Logger from '../logger.js';
const logger = new Logger('hashtag/create');
@Injectable()
export class HashtagService {
constructor(
@Inject(DI.db)
private db: DataSource,
@Inject(DI.meta)
private meta: MiMeta,
@@ -60,19 +68,74 @@ export class HashtagService {
// TODO: サンプリング
this.updateHashtagsRanking(tag, user.id);
const index = await this.hashtagsRepository.findOneBy({ name: tag });
{
const index = await this.hashtagsRepository.findOneBy({ name: tag });
if (index == null && !inc) return;
if (index == null && inc) {
try {
if (isUserAttached) {
await this.hashtagsRepository.insert({
id: this.idService.gen(),
name: tag,
mentionedUserIds: [],
mentionedUsersCount: 0,
mentionedLocalUserIds: [],
mentionedLocalUsersCount: 0,
mentionedRemoteUserIds: [],
mentionedRemoteUsersCount: 0,
attachedUserIds: [user.id],
attachedUsersCount: 1,
attachedLocalUserIds: this.userEntityService.isLocalUser(user) ? [user.id] : [],
attachedLocalUsersCount: this.userEntityService.isLocalUser(user) ? 1 : 0,
attachedRemoteUserIds: this.userEntityService.isRemoteUser(user) ? [user.id] : [],
attachedRemoteUsersCount: this.userEntityService.isRemoteUser(user) ? 1 : 0,
} as MiHashtag);
} else {
await this.hashtagsRepository.insert({
id: this.idService.gen(),
name: tag,
mentionedUserIds: [user.id],
mentionedUsersCount: 1,
mentionedLocalUserIds: this.userEntityService.isLocalUser(user) ? [user.id] : [],
mentionedLocalUsersCount: this.userEntityService.isLocalUser(user) ? 1 : 0,
mentionedRemoteUserIds: this.userEntityService.isRemoteUser(user) ? [user.id] : [],
mentionedRemoteUsersCount: this.userEntityService.isRemoteUser(user) ? 1 : 0,
attachedUserIds: [],
attachedUsersCount: 0,
attachedLocalUserIds: [],
attachedLocalUsersCount: 0,
attachedRemoteUserIds: [],
attachedRemoteUsersCount: 0,
} as MiHashtag);
}
return;
} catch (err) {
if (isDuplicateKeyValueError(err)) {
logger.info(`Duplicate insertion detected. Falling back to update. #${tag}`);
} else {
throw err;
}
}
}
}
if (index != null) {
const q = this.hashtagsRepository.createQueryBuilder('tag').update()
.where('name = :name', { name: tag });
await this.db.transaction(async transactionalEntityManager => {
const transactionalHashtagRepository = transactionalEntityManager
.getRepository(MiHashtag);
const index = await transactionalHashtagRepository
.createQueryBuilder()
.setLock('pessimistic_write')
.where('name = :name', { name: tag })
.getOne();
if (index == null) return;
const set = {} as any;
if (isUserAttached) {
if (inc) {
// 自分が初めてこのタグを使ったなら
// 自分が初めてこのタグを使ったなら
if (!index.attachedUserIds.some(id => id === user.id)) {
set.attachedUserIds = () => `array_append("attachedUserIds", '${user.id}')`;
set.attachedUsersCount = () => '"attachedUsersCount" + 1';
@@ -117,46 +180,14 @@ export class HashtagService {
}
if (Object.keys(set).length > 0) {
q.set(set);
q.execute();
await transactionalHashtagRepository
.createQueryBuilder()
.update()
.where('id = :id', { id: index.id })
.set(set)
.execute();
}
} else {
if (isUserAttached) {
this.hashtagsRepository.insert({
id: this.idService.gen(),
name: tag,
mentionedUserIds: [],
mentionedUsersCount: 0,
mentionedLocalUserIds: [],
mentionedLocalUsersCount: 0,
mentionedRemoteUserIds: [],
mentionedRemoteUsersCount: 0,
attachedUserIds: [user.id],
attachedUsersCount: 1,
attachedLocalUserIds: this.userEntityService.isLocalUser(user) ? [user.id] : [],
attachedLocalUsersCount: this.userEntityService.isLocalUser(user) ? 1 : 0,
attachedRemoteUserIds: this.userEntityService.isRemoteUser(user) ? [user.id] : [],
attachedRemoteUsersCount: this.userEntityService.isRemoteUser(user) ? 1 : 0,
} as MiHashtag);
} else {
this.hashtagsRepository.insert({
id: this.idService.gen(),
name: tag,
mentionedUserIds: [user.id],
mentionedUsersCount: 1,
mentionedLocalUserIds: this.userEntityService.isLocalUser(user) ? [user.id] : [],
mentionedLocalUsersCount: this.userEntityService.isLocalUser(user) ? 1 : 0,
mentionedRemoteUserIds: this.userEntityService.isRemoteUser(user) ? [user.id] : [],
mentionedRemoteUsersCount: this.userEntityService.isRemoteUser(user) ? 1 : 0,
attachedUserIds: [],
attachedUsersCount: 0,
attachedLocalUserIds: [],
attachedLocalUsersCount: 0,
attachedRemoteUserIds: [],
attachedRemoteUsersCount: 0,
} as MiHashtag);
}
}
});
}
@bindThis

View File

@@ -5,6 +5,7 @@
import { Injectable } from '@nestjs/common';
import sharp from 'sharp';
import type { Sharp, WebpOptions, AvifOptions } from 'sharp';
export type IImage = {
data: Buffer;
@@ -19,14 +20,14 @@ export type IImageStream = {
};
export type IImageSharp = {
data: sharp.Sharp;
data: Sharp;
ext: string | null;
type: string;
};
export type IImageStreamable = IImage | IImageStream | IImageSharp;
export const webpDefault: sharp.WebpOptions = {
export const webpDefault: WebpOptions = {
quality: 77,
alphaQuality: 95,
lossless: false,
@@ -37,7 +38,7 @@ export const webpDefault: sharp.WebpOptions = {
loop: 0,
};
export const avifDefault: sharp.AvifOptions = {
export const avifDefault: AvifOptions = {
quality: 60,
lossless: false,
effort: 2,
@@ -57,12 +58,12 @@ export class ImageProcessingService {
* with resize, remove metadata, resolve orientation, stop animation
*/
@bindThis
public async convertToWebp(path: string, width: number, height: number, options: sharp.WebpOptions = webpDefault): Promise<IImage> {
public async convertToWebp(path: string, width: number, height: number, options: WebpOptions = webpDefault): Promise<IImage> {
return this.convertSharpToWebp(sharp(path), width, height, options);
}
@bindThis
public async convertSharpToWebp(sharp: sharp.Sharp, width: number, height: number, options: sharp.WebpOptions = webpDefault): Promise<IImage> {
public async convertSharpToWebp(sharp: Sharp, width: number, height: number, options: WebpOptions = webpDefault): Promise<IImage> {
const result = this.convertSharpToWebpStream(sharp, width, height, options);
return {
@@ -73,12 +74,12 @@ export class ImageProcessingService {
}
@bindThis
public convertToWebpStream(path: string, width: number, height: number, options: sharp.WebpOptions = webpDefault): IImageSharp {
public convertToWebpStream(path: string, width: number, height: number, options: WebpOptions = webpDefault): IImageSharp {
return this.convertSharpToWebpStream(sharp(path), width, height, options);
}
@bindThis
public convertSharpToWebpStream(sharp: sharp.Sharp, width: number, height: number, options: sharp.WebpOptions = webpDefault): IImageSharp {
public convertSharpToWebpStream(sharp: Sharp, width: number, height: number, options: WebpOptions = webpDefault): IImageSharp {
const data = sharp
.resize(width, height, {
fit: 'inside',
@@ -99,12 +100,12 @@ export class ImageProcessingService {
* with resize, remove metadata, resolve orientation, stop animation
*/
@bindThis
public async convertToAvif(path: string, width: number, height: number, options: sharp.AvifOptions = avifDefault): Promise<IImage> {
public async convertToAvif(path: string, width: number, height: number, options: AvifOptions = avifDefault): Promise<IImage> {
return this.convertSharpToAvif(sharp(path), width, height, options);
}
@bindThis
public async convertSharpToAvif(sharp: sharp.Sharp, width: number, height: number, options: sharp.AvifOptions = avifDefault): Promise<IImage> {
public async convertSharpToAvif(sharp: Sharp, width: number, height: number, options: AvifOptions = avifDefault): Promise<IImage> {
const result = this.convertSharpToAvifStream(sharp, width, height, options);
return {
@@ -115,12 +116,12 @@ export class ImageProcessingService {
}
@bindThis
public convertToAvifStream(path: string, width: number, height: number, options: sharp.AvifOptions = avifDefault): IImageSharp {
public convertToAvifStream(path: string, width: number, height: number, options: AvifOptions = avifDefault): IImageSharp {
return this.convertSharpToAvifStream(sharp(path), width, height, options);
}
@bindThis
public convertSharpToAvifStream(sharp: sharp.Sharp, width: number, height: number, options: sharp.AvifOptions = avifDefault): IImageSharp {
public convertSharpToAvifStream(sharp: Sharp, width: number, height: number, options: AvifOptions = avifDefault): IImageSharp {
const data = sharp
.resize(width, height, {
fit: 'inside',
@@ -146,7 +147,7 @@ export class ImageProcessingService {
}
@bindThis
public async convertSharpToPng(sharp: sharp.Sharp, width: number, height: number): Promise<IImage> {
public async convertSharpToPng(sharp: Sharp, width: number, height: number): Promise<IImage> {
const data = await sharp
.resize(width, height, {
fit: 'inside',

View File

@@ -0,0 +1,28 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { Injectable } from '@nestjs/common';
import { bindThis } from '@/decorators.js';
import { captureMessage, shutdownTelemetry, startSpan } from './telemetry-registry.js';
import type { OnApplicationShutdown } from '@nestjs/common';
import type { TelemetryCaptureMessageOptions } from './adapters/TelemetryAdapter.js';
@Injectable()
export class TelemetryService implements OnApplicationShutdown {
@bindThis
public captureMessage(message: string, opts: TelemetryCaptureMessageOptions): void {
captureMessage(message, opts);
}
@bindThis
public startSpan<T>(name: string, fn: () => T): T {
return startSpan(name, fn);
}
@bindThis
public async onApplicationShutdown(_signal?: string): Promise<void> {
await shutdownTelemetry();
}
}

View File

@@ -0,0 +1,96 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import type { Config } from '@/config.js';
import type { TelemetryAdapter, TelemetryCaptureMessageOptions } from './TelemetryAdapter.js';
type SentryIntegrationsOption = NonNullable<import('@sentry/node').NodeOptions['integrations']>;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
type SentryIntegrationFactory = Extract<SentryIntegrationsOption, (integrations: any[]) => any[]>;
type SentryIntegration = Parameters<SentryIntegrationFactory>[0][number];
type SentryNodeOptions = import('@sentry/node').NodeOptions;
type BuildSentryIntegrationsOptions = {
disabledIntegrations?: string[];
enableNodeProfiling: boolean;
nodeProfilingIntegration?: () => SentryIntegration;
warn?: (message: string) => void;
};
export function buildSentryIntegrations(options: BuildSentryIntegrationsOptions): SentryIntegrationFactory {
return (defaults) => {
const disabledIntegrations = new Set(options.disabledIntegrations ?? []);
const defaultIntegrationNames = new Set(defaults.map((integration) => integration.name));
const unknownIntegrations = [...disabledIntegrations].filter((name) => !defaultIntegrationNames.has(name));
if (unknownIntegrations.length > 0) {
(options.warn ?? console.warn)(`Unknown Sentry integration configured in sentryForBackend.disabledIntegrations: ${unknownIntegrations.join(', ')}`);
}
return [
...defaults.filter((integration) => !disabledIntegrations.has(integration.name)),
...(options.enableNodeProfiling && options.nodeProfilingIntegration != null ? [options.nodeProfilingIntegration()] : []),
];
};
}
export function buildSentryNodeOptions(
config: NonNullable<Config['sentryForBackend']>,
nodeProfilingIntegration?: () => SentryIntegration,
): SentryNodeOptions {
return {
// Do not send Sentry trace headers to remote ActivityPub/Webhook/etc. hosts by default.
// Admins can opt in for trusted internal services via sentryForBackend.options.
tracePropagationTargets: [],
// Performance Monitoring
tracesSampleRate: 1.0, // Capture 100% of the transactions
// Set sampling rate for profiling - this is relative to tracesSampleRate
profilesSampleRate: 1.0,
maxBreadcrumbs: 0,
...config.options,
integrations: buildSentryIntegrations({
disabledIntegrations: config.disabledIntegrations,
enableNodeProfiling: config.enableNodeProfiling,
nodeProfilingIntegration,
}),
};
}
export class SentryTelemetryAdapter implements TelemetryAdapter {
private constructor(
private readonly Sentry: typeof import('@sentry/node'),
) {
}
public static async create(config: NonNullable<Config['sentryForBackend']>): Promise<SentryTelemetryAdapter> {
const Sentry = await import('@sentry/node');
const { nodeProfilingIntegration } = await import('@sentry/profiling-node');
Sentry.init(buildSentryNodeOptions(config, nodeProfilingIntegration));
return new SentryTelemetryAdapter(Sentry);
}
public captureMessage(message: string, opts: TelemetryCaptureMessageOptions): void {
this.Sentry.captureMessage(message, {
level: opts.level,
...(opts.userId != null ? { user: { id: opts.userId } } : {}),
extra: opts.extra,
});
}
public startSpan<T>(name: string, fn: () => T): T {
return this.Sentry.startSpan({ name }, fn);
}
public async shutdown(): Promise<void> {
await this.Sentry.close();
}
}

View File

@@ -0,0 +1,21 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
export interface TelemetryCaptureMessageOptions {
level: 'error';
userId?: string;
extra?: Record<string, unknown>;
}
/**
* Sentry・OpenTelemetryなど、エラートラッキング/APMサービスごとの実装差異を隠蔽するための抽象。
* 新しいサービスを追加する場合はこのインターフェースを実装するアダプタをこのディレクトリに追加し、
* telemetry-registry.tsのinitTelemetry内で登録する。
*/
export interface TelemetryAdapter {
captureMessage(message: string, opts: TelemetryCaptureMessageOptions): void;
startSpan<T>(name: string, fn: () => T): T;
shutdown(): Promise<void>;
}

View File

@@ -0,0 +1,39 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import type { Config } from '@/config.js';
import { SentryTelemetryAdapter } from './adapters/SentryTelemetryAdapter.js';
import type { TelemetryAdapter, TelemetryCaptureMessageOptions } from './adapters/TelemetryAdapter.js';
/**
* NestのDIコンテナが構築される前(boot処理内)で初期化する必要があるため、
* DIを介さないモジュールレベルの状態として有効なアダプタを保持する。
* TelemetryServiceはこの状態への薄いラッパーとして振る舞う。
*/
const adapters: TelemetryAdapter[] = [];
export async function initTelemetry(config: Config): Promise<void> {
if (config.sentryForBackend) {
adapters.push(await SentryTelemetryAdapter.create(config.sentryForBackend));
}
}
export function captureMessage(message: string, opts: TelemetryCaptureMessageOptions): void {
for (const adapter of adapters) {
adapter.captureMessage(message, opts);
}
}
export function startSpan<T>(name: string, fn: () => T): T {
const wrapped = adapters.reduceRight<() => T>(
(inner, adapter) => () => adapter.startSpan(name, inner),
fn,
);
return wrapped();
}
export async function shutdownTelemetry(): Promise<void> {
await Promise.all(adapters.map(adapter => adapter.shutdown()));
}

View File

@@ -11,6 +11,7 @@ const envOption = {
verbose: false,
withLogTime: false,
quiet: false,
enableCrossOriginIsolation: false,
};
for (const key of Object.keys(envOption) as (keyof typeof envOption)[]) {

View File

@@ -672,6 +672,11 @@ export class MiMeta {
})
public urlPreviewUserAgent: string | null;
@Column('varchar', {
length: 3072, array: true, default: '{}',
})
public urlPreviewSensitiveList: string[];
@Column('varchar', {
length: 128,
default: 'none',

View File

@@ -9,6 +9,7 @@ import type { Config } from '@/config.js';
import { DI } from '@/di-symbols.js';
import type Logger from '@/logger.js';
import { bindThis } from '@/decorators.js';
import { TelemetryService } from '@/core/telemetry/TelemetryService.js';
import { CheckModeratorsActivityProcessorService } from '@/queue/processors/CheckModeratorsActivityProcessorService.js';
import { UserWebhookDeliverProcessorService } from './processors/UserWebhookDeliverProcessorService.js';
import { SystemWebhookDeliverProcessorService } from './processors/SystemWebhookDeliverProcessorService.js';
@@ -92,6 +93,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
private config: Config,
private queueLoggerService: QueueLoggerService,
private telemetryService: TelemetryService,
private userWebhookDeliverProcessorService: UserWebhookDeliverProcessorService,
private systemWebhookDeliverProcessorService: SystemWebhookDeliverProcessorService,
private endedPollNotificationProcessorService: EndedPollNotificationProcessorService,
@@ -156,13 +158,6 @@ export class QueueProcessorService implements OnApplicationShutdown {
};
}
let Sentry: typeof import('@sentry/node') | undefined;
if (this.config.sentryForBackend) {
import('@sentry/node').then((mod) => {
Sentry = mod;
});
}
//#region system
{
const processer = (job: Bull.Job) => {
@@ -181,11 +176,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
};
this.systemQueueWorker = new Bull.Worker(QUEUE.SYSTEM, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: System: ' + job.name }, () => processer(job));
} else {
return processer(job);
}
return this.telemetryService.startSpan('Queue: System: ' + job.name, () => processer(job));
}, {
...baseWorkerOptions(this.config, QUEUE.SYSTEM),
autorun: false,
@@ -198,12 +189,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`))
.on('failed', (job, err: Error) => {
logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) });
if (Sentry != null) {
Sentry.captureMessage(`Queue: System: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: System: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -238,11 +227,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
};
this.dbQueueWorker = new Bull.Worker(QUEUE.DB, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: DB: ' + job.name }, () => processer(job));
} else {
return processer(job);
}
return this.telemetryService.startSpan('Queue: DB: ' + job.name, () => processer(job));
}, {
...baseWorkerOptions(this.config, QUEUE.DB),
autorun: false,
@@ -255,12 +240,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) });
if (Sentry != null) {
Sentry.captureMessage(`Queue: DB: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: DB: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -270,11 +253,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
//#region deliver
{
this.deliverQueueWorker = new Bull.Worker(QUEUE.DELIVER, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: Deliver' }, () => this.deliverProcessorService.process(job));
} else {
return this.deliverProcessorService.process(job);
}
return this.telemetryService.startSpan('Queue: Deliver', () => this.deliverProcessorService.process(job));
}, {
...baseWorkerOptions(this.config, QUEUE.DELIVER),
autorun: false,
@@ -295,12 +274,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} to=${job ? job.data.to : '-'}`);
if (Sentry != null) {
Sentry.captureMessage(`Queue: Deliver: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: Deliver: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -310,11 +287,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
//#region inbox
{
this.inboxQueueWorker = new Bull.Worker(QUEUE.INBOX, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: Inbox' }, () => this.inboxProcessorService.process(job));
} else {
return this.inboxProcessorService.process(job);
}
return this.telemetryService.startSpan('Queue: Inbox', () => this.inboxProcessorService.process(job));
}, {
...baseWorkerOptions(this.config, QUEUE.INBOX),
autorun: false,
@@ -335,12 +308,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} activity=${job ? (job.data.activity ? job.data.activity.id : 'none') : '-'}`, { job: renderJob(job), e: renderError(err) });
if (Sentry != null) {
Sentry.captureMessage(`Queue: Inbox: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: Inbox: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -350,11 +321,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
//#region user-webhook deliver
{
this.userWebhookDeliverQueueWorker = new Bull.Worker(QUEUE.USER_WEBHOOK_DELIVER, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: UserWebhookDeliver' }, () => this.userWebhookDeliverProcessorService.process(job));
} else {
return this.userWebhookDeliverProcessorService.process(job);
}
return this.telemetryService.startSpan('Queue: UserWebhookDeliver', () => this.userWebhookDeliverProcessorService.process(job));
}, {
...baseWorkerOptions(this.config, QUEUE.USER_WEBHOOK_DELIVER),
autorun: false,
@@ -375,12 +342,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} to=${job ? job.data.to : '-'}`);
if (Sentry != null) {
Sentry.captureMessage(`Queue: UserWebhookDeliver: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: UserWebhookDeliver: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -390,11 +355,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
//#region system-webhook deliver
{
this.systemWebhookDeliverQueueWorker = new Bull.Worker(QUEUE.SYSTEM_WEBHOOK_DELIVER, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: SystemWebhookDeliver' }, () => this.systemWebhookDeliverProcessorService.process(job));
} else {
return this.systemWebhookDeliverProcessorService.process(job);
}
return this.telemetryService.startSpan('Queue: SystemWebhookDeliver', () => this.systemWebhookDeliverProcessorService.process(job));
}, {
...baseWorkerOptions(this.config, QUEUE.SYSTEM_WEBHOOK_DELIVER),
autorun: false,
@@ -415,12 +376,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) ${getJobInfo(job, true)} to=${job.data.to}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) ${getJobInfo(job)} to=${job ? job.data.to : '-'}`);
if (Sentry != null) {
Sentry.captureMessage(`Queue: SystemWebhookDeliver: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: SystemWebhookDeliver: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -440,11 +399,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
};
this.relationshipQueueWorker = new Bull.Worker(QUEUE.RELATIONSHIP, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: Relationship: ' + job.name }, () => processer(job));
} else {
return processer(job);
}
return this.telemetryService.startSpan('Queue: Relationship: ' + job.name, () => processer(job));
}, {
...baseWorkerOptions(this.config, QUEUE.RELATIONSHIP),
autorun: false,
@@ -462,12 +417,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) });
if (Sentry != null) {
Sentry.captureMessage(`Queue: Relationship: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: Relationship: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -485,11 +438,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
};
this.objectStorageQueueWorker = new Bull.Worker(QUEUE.OBJECT_STORAGE, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: ObjectStorage: ' + job.name }, () => processer(job));
} else {
return processer(job);
}
return this.telemetryService.startSpan('Queue: ObjectStorage: ' + job.name, () => processer(job));
}, {
...baseWorkerOptions(this.config, QUEUE.OBJECT_STORAGE),
autorun: false,
@@ -503,12 +452,10 @@ export class QueueProcessorService implements OnApplicationShutdown {
.on('completed', (job, result) => logger.debug(`completed(${result}) id=${job.id}`))
.on('failed', (job, err) => {
logger.error(`failed(${err.name}: ${err.message}) id=${job?.id ?? '?'}`, { job: renderJob(job), e: renderError(err) });
if (Sentry != null) {
Sentry.captureMessage(`Queue: ObjectStorage: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
}
this.telemetryService.captureMessage(`Queue: ObjectStorage: ${job?.name ?? '?'}: ${err.name}: ${err.message}`, {
level: 'error',
extra: { job, err },
});
})
.on('error', (err: Error) => logger.error(`error ${err.name}: ${err.message}`, { e: renderError(err) }))
.on('stalled', (jobId) => logger.warn(`stalled id=${jobId}`));
@@ -518,11 +465,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
//#region ended poll notification
{
this.endedPollNotificationQueueWorker = new Bull.Worker(QUEUE.ENDED_POLL_NOTIFICATION, (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: EndedPollNotification' }, () => this.endedPollNotificationProcessorService.process(job));
} else {
return this.endedPollNotificationProcessorService.process(job);
}
return this.telemetryService.startSpan('Queue: EndedPollNotification', () => this.endedPollNotificationProcessorService.process(job));
}, {
...baseWorkerOptions(this.config, QUEUE.ENDED_POLL_NOTIFICATION),
autorun: false,
@@ -533,11 +476,7 @@ export class QueueProcessorService implements OnApplicationShutdown {
//#region post scheduled note
{
this.postScheduledNoteQueueWorker = new Bull.Worker(QUEUE.POST_SCHEDULED_NOTE, async (job) => {
if (Sentry != null) {
return Sentry.startSpan({ name: 'Queue: PostScheduledNote' }, () => this.postScheduledNoteProcessorService.process(job));
} else {
return this.postScheduledNoteProcessorService.process(job);
}
return this.telemetryService.startSpan('Queue: PostScheduledNote', () => this.postScheduledNoteProcessorService.process(job));
}, {
...baseWorkerOptions(this.config, QUEUE.POST_SCHEDULED_NOTE),
autorun: false,

View File

@@ -21,6 +21,7 @@ import { genIdenticon } from '@/misc/gen-identicon.js';
import { UserEntityService } from '@/core/entities/UserEntityService.js';
import { LoggerService } from '@/core/LoggerService.js';
import { bindThis } from '@/decorators.js';
import { envOption } from '@/env.js';
import { ActivityPubServerService } from './ActivityPubServerService.js';
import { NodeinfoServerService } from './NodeinfoServerService.js';
import { ApiServerService } from './api/ApiServerService.js';
@@ -89,6 +90,15 @@ export class ServerService implements OnApplicationShutdown {
});
}
// for test
if (envOption.enableCrossOriginIsolation) {
fastify.addHook('onRequest', (request, reply, done) => {
reply.header('Cross-Origin-Opener-Policy', 'same-origin');
reply.header('Cross-Origin-Embedder-Policy', 'credentialless');
done();
});
}
// Register raw-body parser for ActivityPub HTTP signature validation.
await fastify.register(fastifyRawBody, {
global: false,

View File

@@ -16,6 +16,7 @@ import type { MiMeta, UserIpsRepository } from '@/models/_.js';
import { createTemp } from '@/misc/create-temp.js';
import { bindThis } from '@/decorators.js';
import { RoleService } from '@/core/RoleService.js';
import { TelemetryService } from '@/core/telemetry/TelemetryService.js';
import type { Config } from '@/config.js';
import { ApiError } from './error.js';
import { RateLimiterService } from './RateLimiterService.js';
@@ -36,7 +37,6 @@ export class ApiCallService implements OnApplicationShutdown {
private logger: Logger;
private userIpHistories: Map<MiUser['id'], Set<string>>;
private userIpHistoriesClearIntervalId: NodeJS.Timeout;
private Sentry: typeof import('@sentry/node') | null = null;
constructor(
@Inject(DI.meta)
@@ -52,6 +52,7 @@ export class ApiCallService implements OnApplicationShutdown {
private rateLimiterService: RateLimiterService,
private roleService: RoleService,
private apiLoggerService: ApiLoggerService,
private telemetryService: TelemetryService,
) {
this.logger = this.apiLoggerService.logger;
this.userIpHistories = new Map<MiUser['id'], Set<string>>();
@@ -59,12 +60,6 @@ export class ApiCallService implements OnApplicationShutdown {
this.userIpHistoriesClearIntervalId = setInterval(() => {
this.userIpHistories.clear();
}, 1000 * 60 * 60);
if (this.config.sentryForBackend) {
import('@sentry/node').then((Sentry) => {
this.Sentry = Sentry;
});
}
}
#sendApiError(reply: FastifyReply, err: ApiError): void {
@@ -126,24 +121,20 @@ export class ApiCallService implements OnApplicationShutdown {
},
});
if (this.Sentry != null) {
this.Sentry.captureMessage(`Internal error occurred in ${ep.name}: ${err.message}`, {
level: 'error',
user: {
id: userId,
this.telemetryService.captureMessage(`Internal error occurred in ${ep.name}: ${err.message}`, {
level: 'error',
userId,
extra: {
ep: ep.name,
ps: data,
e: {
message: err.message,
code: err.name,
stack: err.stack,
id: errId,
},
extra: {
ep: ep.name,
ps: data,
e: {
message: err.message,
code: err.name,
stack: err.stack,
id: errId,
},
},
});
}
},
});
throw new ApiError(null, {
e: {
@@ -441,15 +432,8 @@ export class ApiCallService implements OnApplicationShutdown {
}
// API invoking
if (this.Sentry != null) {
return await this.Sentry.startSpan({
name: 'API: ' + ep.name,
}, () => ep.exec(data, user, token, file, request.ip, request.headers)
.catch((err: Error) => this.#onExecError(ep, data, err, user?.id)));
} else {
return await ep.exec(data, user, token, file, request.ip, request.headers)
.catch((err: Error) => this.#onExecError(ep, data, err, user?.id));
}
return await this.telemetryService.startSpan('API: ' + ep.name, () => ep.exec(data, user, token, file, request.ip, request.headers)
.catch((err: Error) => this.#onExecError(ep, data, err, user?.id)));
}
@bindThis

View File

@@ -544,6 +544,14 @@ export const meta = {
type: 'string',
optional: false, nullable: true,
},
urlPreviewSensitiveList: {
type: 'array',
optional: false, nullable: false,
items: {
type: 'string',
optional: false, nullable: false,
},
},
federation: {
type: 'string',
enum: ['all', 'specified', 'none'],
@@ -760,6 +768,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
urlPreviewRequireContentLength: instance.urlPreviewRequireContentLength,
urlPreviewUserAgent: instance.urlPreviewUserAgent,
urlPreviewSummaryProxyUrl: instance.urlPreviewSummaryProxyUrl,
urlPreviewSensitiveList: instance.urlPreviewSensitiveList,
federation: instance.federation,
federationHosts: instance.federationHosts,
deliverSuspendedSoftware: instance.deliverSuspendedSoftware,

View File

@@ -189,6 +189,12 @@ export const paramDef = {
urlPreviewRequireContentLength: { type: 'boolean' },
urlPreviewUserAgent: { type: 'string', nullable: true },
urlPreviewSummaryProxyUrl: { type: 'string', nullable: true },
urlPreviewSensitiveList: {
type: 'array', nullable: true,
items: {
type: 'string',
}
},
federation: {
type: 'string',
enum: ['all', 'none', 'specified'],
@@ -734,6 +740,10 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
set.urlPreviewSummaryProxyUrl = value === '' ? null : value;
}
if (Array.isArray(ps.urlPreviewSensitiveList)) {
set.urlPreviewSensitiveList = ps.urlPreviewSensitiveList.filter(Boolean);
}
if (ps.federation !== undefined) {
set.federation = ps.federation;
}

View File

@@ -35,6 +35,14 @@ export const meta = {
type: 'number',
optional: false, nullable: false,
},
reactionsCount: {
type: 'number',
optional: false, nullable: false,
},
//originalReactionsCount: {
// type: 'number',
// optional: false, nullable: false,
//},
instances: {
type: 'number',
optional: false, nullable: false,

View File

@@ -11,6 +11,7 @@ import { HttpRequestService } from '@/core/HttpRequestService.js';
import type Logger from '@/logger.js';
import { query } from '@/misc/prelude/url.js';
import { LoggerService } from '@/core/LoggerService.js';
import { UtilityService } from '@/core/UtilityService.js';
import { bindThis } from '@/decorators.js';
import { ApiError } from '@/server/api/error.js';
import { MiMeta } from '@/models/Meta.js';
@@ -29,6 +30,7 @@ export class UrlPreviewService {
private meta: MiMeta,
private httpRequestService: HttpRequestService,
private utilityService: UtilityService,
private loggerService: LoggerService,
) {
this.logger = this.loggerService.getLogger('url-preview');
@@ -95,6 +97,10 @@ export class UrlPreviewService {
summary.icon = this.wrap(summary.icon);
summary.thumbnail = this.wrap(summary.thumbnail);
if (summary.sensitive !== true) {
summary.sensitive = this.utilityService.isKeyWordIncluded(summary.url, this.meta.urlPreviewSensitiveList);
}
// Cache 1day
reply.header('Cache-Control', 'max-age=86400, immutable');

View File

@@ -7,7 +7,7 @@ process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import * as crypto from 'node:crypto';
import cbor from 'cbor';
import { encode as encodeToCbor } from 'cbor2';
import * as OTPAuth from 'otpauth';
import { loadConfig } from '@/config.js';
import { api, signup, sendEnvUpdateRequest } from '../utils.js';
@@ -61,7 +61,7 @@ describe('2要素認証', () => {
const keyDoneParam = (param: {
token: string,
keyName: string,
credentialId: Buffer,
credentialId: Uint8Array,
creationOptions: PublicKeyCredentialCreationOptionsJSON,
}): {
token: string,
@@ -70,10 +70,10 @@ describe('2要素認証', () => {
credential: RegistrationResponseJSON,
} => {
// A COSE encoded public key
const credentialPublicKey = cbor.encode(new Map<number, unknown>([
const credentialPublicKey = encodeToCbor(new Map<number, unknown>([
[-1, coseEc2CrvP256],
[-2, Buffer.from(coseEc2X, 'hex')],
[-3, Buffer.from(coseEc2Y, 'hex')],
[-2, Uint8Array.from(Buffer.from(coseEc2X, 'hex'))],
[-3, Uint8Array.from(Buffer.from(coseEc2Y, 'hex'))],
[1, coseKtyEc2],
[2, coseKid],
[3, coseAlgEs256],
@@ -85,21 +85,23 @@ describe('2要素認証', () => {
credentialIdLength.writeUInt16BE(param.credentialId.length, 0);
const authData = Buffer.concat([
rpIdHash(), // rpIdHash(32)
Buffer.from([0x45]), // flags(1)
Buffer.from([0x00, 0x00, 0x00, 0x00]), // signCount(4)
Buffer.from([0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]), // AAGUID(16)
new Uint8Array([0x45]), // flags(1)
new Uint8Array(4), // signCount(4)
new Uint8Array(16), // AAGUID(16)
credentialIdLength,
param.credentialId,
credentialPublicKey,
]);
const credentialIdBase64url = Buffer.from(param.credentialId).toString('base64url');
return {
password,
token: param.token,
name: param.keyName,
credential: <RegistrationResponseJSON>{
id: param.credentialId.toString('base64url'),
rawId: param.credentialId.toString('base64url'),
id: credentialIdBase64url,
rawId: credentialIdBase64url,
response: <AuthenticatorAttestationResponseJSON>{
clientDataJSON: Buffer.from(JSON.stringify({
type: 'webauthn.create',
@@ -107,11 +109,11 @@ describe('2要素認証', () => {
origin: config.scheme + '://' + config.host,
androidPackageName: 'org.mozilla.firefox',
}), 'utf-8').toString('base64url'),
attestationObject: cbor.encode({
attestationObject: Buffer.from(encodeToCbor({
fmt: 'none',
attStmt: {},
authData,
}).toString('base64url'),
authData: new Uint8Array(authData),
})).toString('base64url'),
},
clientExtensionResults: {},
type: 'public-key',

View File

@@ -6,10 +6,12 @@
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { describe, beforeAll, beforeEach, test } from 'vitest';
import { describe, beforeAll, beforeEach, test, vi } from 'vitest';
import { UserToken, api, post, signup } from '../utils.js';
import type * as misskey from 'misskey-js';
const waitForPushToTlOptions = { timeout: 3000, interval: 25 };
describe('API visibility', () => {
describe('Note visibility', () => {
//#region vars
@@ -409,10 +411,12 @@ describe('API visibility', () => {
//#region HTL
test('[HTL] public-post が 自分が見れる', async () => {
const res = await api('notes/timeline', { limit: 100 }, alice);
assert.strictEqual(res.status, 200);
const notes = res.body.filter(n => n.id === pub.id);
assert.strictEqual(notes[0].text, 'x');
await vi.waitFor(async () => {
const res = await api('notes/timeline', { limit: 100 }, alice);
assert.strictEqual(res.status, 200);
const notes = res.body.filter(n => n.id === pub.id);
assert.strictEqual(notes[0].text, 'x');
}, waitForPushToTlOptions);
});
test('[HTL] public-post が 非フォロワーから見れない', async () => {
@@ -423,10 +427,12 @@ describe('API visibility', () => {
});
test('[HTL] followers-post が フォロワーから見れる', async () => {
const res = await api('notes/timeline', { limit: 100 }, follower);
assert.strictEqual(res.status, 200);
const notes = res.body.filter(n => n.id === fol.id);
assert.strictEqual(notes[0].text, 'x');
await vi.waitFor(async () => {
const res = await api('notes/timeline', { limit: 100 }, follower);
assert.strictEqual(res.status, 200);
const notes = res.body.filter(n => n.id === fol.id);
assert.strictEqual(notes[0].text, 'x');
}, waitForPushToTlOptions);
});
//#endregion

View File

@@ -6,10 +6,12 @@
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { describe, beforeAll, test } from 'vitest';
import { describe, beforeAll, test, vi } from 'vitest';
import { api, castAsError, post, signup } from '../utils.js';
import type * as misskey from 'misskey-js';
const waitForPushToTlOptions = { timeout: 3000, interval: 25 };
describe('Block', () => {
// alice blocks bob
let alice: misskey.entities.SignupResponse;
@@ -75,13 +77,15 @@ describe('Block', () => {
const bobNote = await post(bob, { text: 'hi' });
const carolNote = await post(carol, { text: 'hi' });
const res = await api('notes/local-timeline', {}, bob);
const body = res.body as misskey.entities.Note[];
await vi.waitFor(async () => {
const res = await api('notes/local-timeline', {}, bob);
const body = res.body as misskey.entities.Note[];
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(body.some(note => note.id === aliceNote.id), false);
assert.strictEqual(body.some(note => note.id === bobNote.id), true);
assert.strictEqual(body.some(note => note.id === carolNote.id), true);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(body.some(note => note.id === aliceNote.id), false);
assert.strictEqual(body.some(note => note.id === bobNote.id), true);
assert.strictEqual(body.some(note => note.id === carolNote.id), true);
}, waitForPushToTlOptions);
});
});

View File

@@ -6,7 +6,7 @@
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { describe, beforeAll, test, expect } from 'vitest';
import { describe, beforeAll, test, expect, vi } from 'vitest';
// node-fetch only supports it's own Blob yet
// https://github.com/node-fetch/node-fetch/pull/1664
import { Blob } from 'node-fetch';
@@ -14,6 +14,8 @@ import { api, castAsError, initTestDb, post, role, signup, simpleGet, uploadFile
import type * as misskey from 'misskey-js';
import { MiUser } from '@/models/_.js';
const waitForPushToTlOptions = { timeout: 3000, interval: 25 };
describe('Endpoints', () => {
let alice: misskey.entities.SignupResponse;
let bob: misskey.entities.SignupResponse;
@@ -1149,12 +1151,14 @@ describe('Endpoints', () => {
visibility: 'followers',
});
const res = await api('notes/timeline', {}, dave);
await vi.waitFor(async () => {
const res = await api('notes/timeline', {}, dave);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.length, 1);
assert.strictEqual(res.body[0].id, carolPost.id);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.length, 1);
assert.strictEqual(res.body[0].id, carolPost.id);
}, waitForPushToTlOptions);
});
});

View File

@@ -7,9 +7,8 @@ import { INestApplicationContext } from '@nestjs/common';
process.env.NODE_ENV = 'test';
import { setTimeout } from 'node:timers/promises';
import * as assert from 'assert';
import { afterAll, beforeAll, afterEach, describe, test } from 'vitest';
import { afterAll, beforeAll, afterEach, describe, test, vi } from 'vitest';
import { loadConfig } from '@/config.js';
import { MiRepository, MiUser, UsersRepository, miRepository } from '@/models/_.js';
import { secureRndstr } from '@/misc/secure-rndstr.js';
@@ -17,6 +16,11 @@ import { jobQueue } from '@/boot/common.js';
import { api, castAsError, initTestDb, signup, successfulApiCall, uploadFile } from '../utils.js';
import type * as misskey from 'misskey-js';
// i/move 呼び出しで大部分が同期的に完了する後続ジョブ (フォロー解除/ブロック・ミュート引き継ぎ/リスト更新等) を待つ
const waitForMoveJobOptions = { timeout: 5000, interval: 50 };
// AccountMoveService がテスト環境向けに 10000ms に短縮している遅延アンフォロージョブを待つ (安全マージンを載せた上限)
const waitForDelayedUnfollowJobOptions = { timeout: 15000, interval: 100 };
describe('Account Move', () => {
let jq: INestApplicationContext;
let url: URL;
@@ -273,53 +277,63 @@ describe('Account Move', () => {
assert.strictEqual(move.status, 200);
await setTimeout(1000 * 3); // wait for jobs to finish
// Unfollow delayed?
const aliceFollowings = await api('users/following', {
userId: alice.id,
}, alice);
assert.strictEqual(aliceFollowings.status, 200);
assert.ok(aliceFollowings);
assert.strictEqual(aliceFollowings.body.length, 3);
await vi.waitFor(async () => {
const aliceFollowings = await api('users/following', {
userId: alice.id,
}, alice);
assert.strictEqual(aliceFollowings.status, 200);
assert.ok(aliceFollowings);
assert.strictEqual(aliceFollowings.body.length, 3);
}, waitForMoveJobOptions);
const carolFollowings = await api('users/following', {
userId: carol.id,
}, carol);
assert.strictEqual(carolFollowings.status, 200);
assert.ok(carolFollowings);
assert.strictEqual(carolFollowings.body.length, 2);
assert.strictEqual(carolFollowings.body[0].followeeId, bob.id);
assert.strictEqual(carolFollowings.body[1].followeeId, alice.id);
await vi.waitFor(async () => {
const carolFollowings = await api('users/following', {
userId: carol.id,
}, carol);
assert.strictEqual(carolFollowings.status, 200);
assert.ok(carolFollowings);
assert.strictEqual(carolFollowings.body.length, 2);
assert.strictEqual(carolFollowings.body[0].followeeId, bob.id);
assert.strictEqual(carolFollowings.body[1].followeeId, alice.id);
}, waitForMoveJobOptions);
const blockings = await api('blocking/list', {}, dave);
assert.strictEqual(blockings.status, 200);
assert.ok(blockings);
assert.strictEqual(blockings.body.length, 2);
assert.strictEqual(blockings.body[0].blockeeId, bob.id);
assert.strictEqual(blockings.body[1].blockeeId, alice.id);
await vi.waitFor(async () => {
const blockings = await api('blocking/list', {}, dave);
assert.strictEqual(blockings.status, 200);
assert.ok(blockings);
assert.strictEqual(blockings.body.length, 2);
assert.strictEqual(blockings.body[0].blockeeId, bob.id);
assert.strictEqual(blockings.body[1].blockeeId, alice.id);
}, waitForMoveJobOptions);
const mutings = await api('mute/list', {}, dave);
assert.strictEqual(mutings.status, 200);
assert.ok(mutings);
assert.strictEqual(mutings.body.length, 2);
assert.strictEqual(mutings.body[0].muteeId, bob.id);
assert.strictEqual(mutings.body[1].muteeId, alice.id);
await vi.waitFor(async () => {
const mutings = await api('mute/list', {}, dave);
assert.strictEqual(mutings.status, 200);
assert.ok(mutings);
assert.strictEqual(mutings.body.length, 2);
assert.strictEqual(mutings.body[0].muteeId, bob.id);
assert.strictEqual(mutings.body[1].muteeId, alice.id);
}, waitForMoveJobOptions);
const rootLists = await api('users/lists/list', {}, root);
assert.strictEqual(rootLists.status, 200);
assert.ok(rootLists);
assert.ok(rootLists.body[0].userIds);
assert.strictEqual(rootLists.body[0].userIds.length, 2);
assert.ok(rootLists.body[0].userIds.find((id: string) => id === bob.id));
assert.ok(rootLists.body[0].userIds.find((id: string) => id === alice.id));
await vi.waitFor(async () => {
const rootLists = await api('users/lists/list', {}, root);
assert.strictEqual(rootLists.status, 200);
assert.ok(rootLists);
assert.ok(rootLists.body[0].userIds);
assert.strictEqual(rootLists.body[0].userIds.length, 2);
assert.ok(rootLists.body[0].userIds.find((id: string) => id === bob.id));
assert.ok(rootLists.body[0].userIds.find((id: string) => id === alice.id));
}, waitForMoveJobOptions);
const eveLists = await api('users/lists/list', {}, eve);
assert.strictEqual(eveLists.status, 200);
assert.ok(eveLists);
assert.ok(eveLists.body[0].userIds);
assert.strictEqual(eveLists.body[0].userIds.length, 1);
assert.ok(eveLists.body[0].userIds.find((id: string) => id === bob.id));
await vi.waitFor(async () => {
const eveLists = await api('users/lists/list', {}, eve);
assert.strictEqual(eveLists.status, 200);
assert.ok(eveLists);
assert.ok(eveLists.body[0].userIds);
assert.strictEqual(eveLists.body[0].userIds.length, 1);
assert.ok(eveLists.body[0].userIds.find((id: string) => id === bob.id));
}, waitForMoveJobOptions);
});
test('A locked account automatically accept the follow request if it had already accepted the old account.', async () => {
@@ -340,14 +354,14 @@ describe('Account Move', () => {
});
test('Unfollowed after 10 sec (24 hours in production).', async () => {
await setTimeout(1000 * 8);
await vi.waitFor(async () => {
const following = await api('users/following', {
userId: alice.id,
}, alice);
const following = await api('users/following', {
userId: alice.id,
}, alice);
assert.strictEqual(following.status, 200);
assert.strictEqual(following.body.length, 0);
assert.strictEqual(following.status, 200);
assert.strictEqual(following.body.length, 0);
}, waitForDelayedUnfollowJobOptions);
});
test('Unable to move if the destination account has already moved.', async () => {

View File

@@ -6,10 +6,12 @@
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { beforeAll, describe, test } from 'vitest';
import { beforeAll, describe, test, vi } from 'vitest';
import { api, post, react, signup, waitFire } from '../utils.js';
import type * as misskey from 'misskey-js';
const waitForPushToTlOptions = { timeout: 3000, interval: 25 };
describe('Mute', () => {
// alice mutes carol
let alice: misskey.entities.SignupResponse;
@@ -67,13 +69,15 @@ describe('Mute', () => {
const bobNote = await post(bob, { text: 'hi' });
const carolNote = await post(carol, { text: 'hi' });
const res = await api('notes/local-timeline', {}, alice);
await vi.waitFor(async () => {
const res = await api('notes/local-timeline', {}, alice);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === aliceNote.id), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), false);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === aliceNote.id), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), false);
}, waitForPushToTlOptions);
});
test('タイムラインにミュートしているユーザーの投稿のRenoteが含まれない', async () => {
@@ -83,13 +87,15 @@ describe('Mute', () => {
renoteId: carolNote.id,
});
const res = await api('notes/local-timeline', {}, alice);
await vi.waitFor(async () => {
const res = await api('notes/local-timeline', {}, alice);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === aliceNote.id), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), false);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), false);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === aliceNote.id), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), false);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), false);
}, waitForPushToTlOptions);
});
});

View File

@@ -6,11 +6,12 @@
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { beforeAll, describe, test } from 'vitest';
import { setTimeout } from 'node:timers/promises';
import { beforeAll, describe, test, vi } from 'vitest';
import { api, post, signup, waitFire } from '../utils.js';
import type * as misskey from 'misskey-js';
const waitForPushToTlOptions = { timeout: 3000, interval: 25 };
describe('Renote Mute', () => {
// alice mutes carol
let alice: misskey.entities.SignupResponse;
@@ -36,16 +37,15 @@ describe('Renote Mute', () => {
const carolRenote = await post(carol, { renoteId: bobNote.id });
const carolNote = await post(carol, { text: 'hi' });
// redisに追加されるのを待つ
await setTimeout(100);
await vi.waitFor(async () => {
const res = await api('notes/local-timeline', {}, alice);
const res = await api('notes/local-timeline', {}, alice);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolRenote.id), false);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), true);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolRenote.id), false);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), true);
}, waitForPushToTlOptions);
});
test('タイムラインにリノートミュートしているユーザーの引用が含まれる', async () => {
@@ -53,16 +53,15 @@ describe('Renote Mute', () => {
const carolRenote = await post(carol, { renoteId: bobNote.id, text: 'kore' });
const carolNote = await post(carol, { text: 'hi' });
// redisに追加されるのを待つ
await setTimeout(100);
await vi.waitFor(async () => {
const res = await api('notes/local-timeline', {}, alice);
const res = await api('notes/local-timeline', {}, alice);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolRenote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), true);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === bobNote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolRenote.id), true);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), true);
}, waitForPushToTlOptions);
});
// #12956
@@ -70,15 +69,14 @@ describe('Renote Mute', () => {
const carolNote = await post(carol, { text: 'hi' });
const bobRenote = await post(bob, { renoteId: carolNote.id });
// redisに追加されるのを待つ
await setTimeout(100);
await vi.waitFor(async () => {
const res = await api('notes/local-timeline', {}, alice);
const res = await api('notes/local-timeline', {}, alice);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), true);
assert.strictEqual(res.body.some(note => note.id === bobRenote.id), true);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.some(note => note.id === carolNote.id), true);
assert.strictEqual(res.body.some(note => note.id === bobRenote.id), true);
}, waitForPushToTlOptions);
});
test('ストリームにリノートミュートしているユーザーのリノートが流れない', async () => {

View File

@@ -8,9 +8,9 @@ process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { describe, beforeAll, test } from 'vitest';
import { WebSocket } from 'ws';
import { MiFollowing } from '@/models/Following.js';
import { api, createAppToken, initTestDb, port, post, signup, waitFire } from '../utils.js';
import { api, connectStream, createAppToken, initTestDb, port, post, signup, waitFire } from '../utils.js';
import type * as misskey from 'misskey-js';
import { MiFollowing } from '@/models/Following.js';
describe('Streaming', () => {
let Followings: any;
@@ -510,7 +510,7 @@ describe('Streaming', () => {
test('withReplies: true のとき自分のfollowers投稿に対するリプライが流れる', async () => {
const erinNote = await post(erin, { text: 'hi', visibility: 'followers' });
const fired = await waitFire(
erin, 'homeTimeline', // erin:home
erin, 'hybridTimeline', // erin:Hybrid
() => api('notes/create', { text: 'hello', replyId: erinNote.id }, ayano), // ayano reply to erin's followers post
msg => msg.type === 'note' && msg.body.userId === ayano.id, // wait ayano
);
@@ -521,7 +521,7 @@ describe('Streaming', () => {
test('withReplies: false でも自分の投稿に対するリプライが流れる', async () => {
const ayanoNote = await post(ayano, { text: 'hi', visibility: 'followers' });
const fired = await waitFire(
ayano, 'homeTimeline', // ayano:home
ayano, 'hybridTimeline', // ayano:Hybrid
() => api('notes/create', { text: 'hello', replyId: ayanoNote.id }, erin), // erin reply to ayano's followers post
msg => msg.type === 'note' && msg.body.userId === erin.id, // wait erin
);
@@ -530,9 +530,12 @@ describe('Streaming', () => {
});
test('withReplies: true のフォローしていない人のfollowersートに対するリプライが流れない', async () => {
// ayano は kyoko をフォローしているため kyoko の followers 投稿にリプライできるが、
// erin は kyoko をフォローしていないため、そのリプライは erin の Hybrid Timeline には流れないはず
const kyokoFollowersNote = await post(kyoko, { text: 'hi', visibility: 'followers' });
const fired = await waitFire(
erin, 'homeTimeline', // erin:home
() => api('notes/create', { text: 'hello', replyId: chitose.id }, ayano), // ayano reply to chitose's post
erin, 'hybridTimeline', // erin:Hybrid
() => api('notes/create', { text: 'hello', replyId: kyokoFollowersNote.id }, ayano), // ayano reply to kyoko's followers post
msg => msg.type === 'note' && msg.body.userId === ayano.id, // wait ayano
);
@@ -680,7 +683,7 @@ describe('Streaming', () => {
const fired = await waitFire(
chitose, 'userList',
() => api('notes/create', { text: 'foo' }, takumi),
msg => msg.type === 'note' && msg.body.userId === kyoko.id,
msg => msg.type === 'note' && msg.body.userId === takumi.id,
{ listId: list.id },
);
@@ -744,164 +747,83 @@ describe('Streaming', () => {
assert.strictEqual(fired, true);
});
// XXX: QueryFailedError: duplicate key value violates unique constraint "IDX_347fec870eafea7b26c8a73bac"
/*
describe('Hashtag Timeline', () => {
test('指定したハッシュタグの投稿が流れる', () => new Promise<void>(async done => {
const ws = await connectStream(chitose, 'hashtag', ({ type, body }) => {
if (type === 'note') {
assert.deepStrictEqual(body.text, '#foo');
ws.close();
done();
}
}, {
q: [
['foo'],
],
});
test('指定したハッシュタグの投稿が流れる', async () => {
const fired = await waitFire(
chitose, 'hashtag',
() => api('notes/create', { text: '#foo' }, chitose),
msg => msg.type === 'note' && msg.body.text === '#foo',
{ q: [['foo']] },
);
post(chitose, {
text: '#foo',
});
}));
assert.strictEqual(fired, true);
});
test('指定したハッシュタグの投稿が流れる (AND)', () => new Promise<void>(async done => {
let fooCount = 0;
let barCount = 0;
let fooBarCount = 0;
test('指定したハッシュタグの投稿が流れる (AND)', async () => {
const received: string[] = [];
const ws = await connectStream(chitose, 'hashtag', (msg) => {
if (msg.type === 'note') received.push(msg.body.text);
}, { q: [['foo', 'bar']] });
const ws = await connectStream(chitose, 'hashtag', ({ type, body }) => {
if (type === 'note') {
if (body.text === '#foo') fooCount++;
if (body.text === '#bar') barCount++;
if (body.text === '#foo #bar') fooBarCount++;
}
}, {
q: [
['foo', 'bar'],
],
});
await Promise.all([
await api('notes/create', { text: '#foo' }, chitose),
await api('notes/create', { text: '#bar' }, chitose),
await api('notes/create', { text: '#foo #bar' }, chitose),
]);
post(chitose, {
text: '#foo',
});
await new Promise(r => setTimeout(r, 1000));
ws.close();
post(chitose, {
text: '#bar',
});
assert.strictEqual(received.includes('#foo'), false);
assert.strictEqual(received.includes('#bar'), false);
assert.strictEqual(received.includes('#foo #bar'), true);
});
post(chitose, {
text: '#foo #bar',
});
test('指定したハッシュタグの投稿が流れる (OR)', async () => {
const received: string[] = [];
const ws = await connectStream(chitose, 'hashtag', (msg) => {
if (msg.type === 'note') received.push(msg.body.text);
}, { q: [['foo'], ['bar']] });
setTimeout(() => {
assert.strictEqual(fooCount, 0);
assert.strictEqual(barCount, 0);
assert.strictEqual(fooBarCount, 1);
ws.close();
done();
}, 3000);
}));
await Promise.all([
await api('notes/create', { text: '#foo' }, chitose),
await api('notes/create', { text: '#bar' }, chitose),
await api('notes/create', { text: '#foo #bar' }, chitose),
await api('notes/create', { text: '#piyo' }, chitose),
]);
test('指定したハッシュタグの投稿が流れる (OR)', () => new Promise<void>(async done => {
let fooCount = 0;
let barCount = 0;
let fooBarCount = 0;
let piyoCount = 0;
await new Promise(r => setTimeout(r, 1000));
ws.close();
const ws = await connectStream(chitose, 'hashtag', ({ type, body }) => {
if (type === 'note') {
if (body.text === '#foo') fooCount++;
if (body.text === '#bar') barCount++;
if (body.text === '#foo #bar') fooBarCount++;
if (body.text === '#piyo') piyoCount++;
}
}, {
q: [
['foo'],
['bar'],
],
});
assert.strictEqual(received.includes('#foo'), true);
assert.strictEqual(received.includes('#bar'), true);
assert.strictEqual(received.includes('#foo #bar'), true);
assert.strictEqual(received.includes('#piyo'), false);
});
post(chitose, {
text: '#foo',
});
test('指定したハッシュタグの投稿が流れる (AND + OR)', async () => {
const received: string[] = [];
const ws = await connectStream(chitose, 'hashtag', (msg) => {
if (msg.type === 'note') received.push(msg.body.text);
}, { q: [['foo', 'bar'], ['piyo']] });
post(chitose, {
text: '#bar',
});
await Promise.all([
api('notes/create', { text: '#foo' }, chitose),
api('notes/create', { text: '#bar' }, chitose),
api('notes/create', { text: '#foo #bar' }, chitose),
api('notes/create', { text: '#piyo' }, chitose),
api('notes/create', { text: '#waaa' }, chitose),
]);
post(chitose, {
text: '#foo #bar',
});
await new Promise(r => setTimeout(r, 1000));
ws.close();
post(chitose, {
text: '#piyo',
});
setTimeout(() => {
assert.strictEqual(fooCount, 1);
assert.strictEqual(barCount, 1);
assert.strictEqual(fooBarCount, 1);
assert.strictEqual(piyoCount, 0);
ws.close();
done();
}, 3000);
}));
test('指定したハッシュタグの投稿が流れる (AND + OR)', () => new Promise<void>(async done => {
let fooCount = 0;
let barCount = 0;
let fooBarCount = 0;
let piyoCount = 0;
let waaaCount = 0;
const ws = await connectStream(chitose, 'hashtag', ({ type, body }) => {
if (type === 'note') {
if (body.text === '#foo') fooCount++;
if (body.text === '#bar') barCount++;
if (body.text === '#foo #bar') fooBarCount++;
if (body.text === '#piyo') piyoCount++;
if (body.text === '#waaa') waaaCount++;
}
}, {
q: [
['foo', 'bar'],
['piyo'],
],
});
post(chitose, {
text: '#foo',
});
post(chitose, {
text: '#bar',
});
post(chitose, {
text: '#foo #bar',
});
post(chitose, {
text: '#piyo',
});
post(chitose, {
text: '#waaa',
});
setTimeout(() => {
assert.strictEqual(fooCount, 0);
assert.strictEqual(barCount, 0);
assert.strictEqual(fooBarCount, 1);
assert.strictEqual(piyoCount, 1);
assert.strictEqual(waaaCount, 0);
ws.close();
done();
}, 3000);
}));
assert.strictEqual(received.includes('#foo'), false);
assert.strictEqual(received.includes('#bar'), false);
assert.strictEqual(received.includes('#foo #bar'), true);
assert.strictEqual(received.includes('#piyo'), true);
assert.strictEqual(received.includes('#waaa'), false);
});
});
*/
});
});

File diff suppressed because it is too large Load Diff

View File

@@ -6,10 +6,12 @@
process.env.NODE_ENV = 'test';
import * as assert from 'assert';
import { beforeAll, describe, test } from 'vitest';
import { beforeAll, describe, test, vi } from 'vitest';
import { api, post, signup, uploadUrl } from '../utils.js';
import type * as misskey from 'misskey-js';
const waitForPushToTlOptions = { timeout: 3000, interval: 25 };
describe('users/notes', () => {
let alice: misskey.entities.SignupResponse;
let jpgNote: misskey.entities.Note;
@@ -32,16 +34,18 @@ describe('users/notes', () => {
}, 1000 * 60 * 2);
test('withFiles', async () => {
const res = await api('users/notes', {
userId: alice.id,
withFiles: true,
}, alice);
await vi.waitFor(async () => {
const res = await api('users/notes', {
userId: alice.id,
withFiles: true,
}, alice);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.length, 3);
assert.strictEqual(res.body.some((note: any) => note.id === jpgNote.id), true);
assert.strictEqual(res.body.some((note: any) => note.id === pngNote.id), true);
assert.strictEqual(res.body.some((note: any) => note.id === jpgPngNote.id), true);
assert.strictEqual(res.status, 200);
assert.strictEqual(Array.isArray(res.body), true);
assert.strictEqual(res.body.length, 3);
assert.strictEqual(res.body.some((note: any) => note.id === jpgNote.id), true);
assert.strictEqual(res.body.some((note: any) => note.id === pngNote.id), true);
assert.strictEqual(res.body.some((note: any) => note.id === jpgPngNote.id), true);
}, waitForPushToTlOptions);
});
});

View File

@@ -7,6 +7,9 @@ import { beforeAll } from 'vitest';
import { initTestDb, sendEnvResetRequest } from './utils.js';
beforeAll(async () => {
await initTestDb(false);
// 前ファイルのNestJSアプリをdispose(env-reset)した後にスキーマをdrop & 再作成する。
// 逆順だと、前ファイルの最後のテストが投げっぱなしにした非同期処理(cacheServiceのrefresh等)が
// dispose前のdrop中に発火し、Unhandled Rejection (relation does not exist) でクラッシュしうる。
await sendEnvResetRequest();
await initTestDb(false);
});

View File

@@ -0,0 +1,60 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { describe, expect, test, vi } from 'vitest';
import { buildSentryIntegrations } from '@/core/telemetry/adapters/SentryTelemetryAdapter.js';
type TestIntegration = Parameters<ReturnType<typeof buildSentryIntegrations>>[0][number];
function testIntegration(name: string): TestIntegration {
return { name };
}
describe('SentryTelemetryAdapter', () => {
test('removes disabled integrations from Sentry defaults', () => {
const integrations = buildSentryIntegrations({
disabledIntegrations: ['Postgres'],
enableNodeProfiling: false,
});
const result = integrations([
testIntegration('Http'),
testIntegration('Postgres'),
testIntegration('Redis'),
]);
expect(result.map((integration: TestIntegration) => integration.name)).toEqual(['Http', 'Redis']);
});
test('keeps profiling integration when enabled', () => {
const integrations = buildSentryIntegrations({
disabledIntegrations: [],
enableNodeProfiling: true,
nodeProfilingIntegration: () => testIntegration('ProfilingIntegration'),
});
const result = integrations([
testIntegration('Http'),
]);
expect(result.map((integration: TestIntegration) => integration.name)).toEqual(['Http', 'ProfilingIntegration']);
});
test('warns about unknown disabled integration names without removing defaults', () => {
const warn = vi.fn();
const integrations = buildSentryIntegrations({
disabledIntegrations: ['Unknown'],
enableNodeProfiling: false,
warn,
});
const result = integrations([
testIntegration('Http'),
]);
expect(result.map((integration: TestIntegration) => integration.name)).toEqual(['Http']);
expect(warn).toHaveBeenCalledWith('Unknown Sentry integration configured in sentryForBackend.disabledIntegrations: Unknown');
});
});

View File

@@ -0,0 +1,31 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
process.env.NODE_ENV = 'test';
import { describe, expect, test } from 'vitest';
import { buildSentryNodeOptions } from '../../../../../src/core/telemetry/adapters/SentryTelemetryAdapter.js';
describe('buildSentryNodeOptions', () => {
test('disables outbound trace propagation by default', () => {
const options = buildSentryNodeOptions({
enableNodeProfiling: false,
options: {},
});
expect(options.tracePropagationTargets).toEqual([]);
});
test('allows explicit tracePropagationTargets to override the default', () => {
const options = buildSentryNodeOptions({
enableNodeProfiling: false,
options: {
tracePropagationTargets: ['^https://internal\\.example/'],
},
});
expect(options.tracePropagationTargets).toEqual(['^https://internal\\.example/']);
});
});

View File

@@ -9,10 +9,10 @@ import { basename, isAbsolute } from 'node:path';
import { randomUUID } from 'node:crypto';
import { inspect } from 'node:util';
import WebSocket, { ClientOptions } from 'ws';
import fetch, { RequestInit, type Headers } from 'node-fetch';
import fetch, { Blob, FormData } from 'node-fetch';
import type { RequestInit, Headers, Response } from 'node-fetch';
import * as htmlParser from 'node-html-parser';
import { DataSource } from 'typeorm';
import { type Response } from 'node-fetch';
import Fastify from 'fastify';
import { entities } from '@/postgres.js';
import { loadConfig } from '@/config.js';

Some files were not shown because too many files have changed in this diff Show More