1
0
mirror of https://github.com/misskey-dev/misskey.git synced 2026-06-23 20:34:46 +02:00

Compare commits

..

38 Commits

Author SHA1 Message Date
syuilo
2b59956092 Update backend-memory-report.mjs 2026-06-23 18:45:56 +09:00
syuilo
713b9dc40c wip 2026-06-23 13:07:33 +09:00
syuilo
6e4380f11d enhance(dev): tweak report-backend-memory 2026-06-23 12:37:47 +09:00
syuilo
cb1d1d651a enhance(dev): tweak report-backend-memory 2026-06-23 12:18:47 +09:00
syuilo
c899aafeef enhance(dev): tweak report-backend-memory 2026-06-23 11:54:16 +09:00
syuilo
72d91ce3da refactor(dev): report-backend-memoryのmarkdown生成ロジックを分離 2026-06-23 11:29:45 +09:00
syuilo
09b761e4d1 enhance(dev): tweak report-backend-memory 2026-06-23 11:17:03 +09:00
syuilo
6d11f572b3 enhance(dev): improve backend memory usage comparison workflow (#17591)
* wip

* Update get-backend-memory.yml

* [ci skip] tweak table
2026-06-23 11:01:27 +09:00
syuilo
d54b948085 enhance(dev): tweak Frontend Chunk Report 2026-06-22 22:22:36 +09:00
syuilo
f5806a0560 enhance(dev): tweak Frontend Chunk Report 2026-06-22 22:15:32 +09:00
syuilo
5d8c31b6e5 fix(dev): tweak frontend-bundle-report-comment 2026-06-22 22:11:46 +09:00
syuilo
fff87f6604 enhance(dev): tweak Frontend Chunk Report 2026-06-22 21:59:54 +09:00
syuilo
7a3e03411f Merge branch 'develop' of https://github.com/misskey-dev/misskey into develop 2026-06-22 20:49:16 +09:00
syuilo
6d89d479e2 fix(dev): tweak frontend-bundle-report 2026-06-22 20:49:03 +09:00
github-actions[bot]
ab73b8abe3 [skip ci] Update CHANGELOG.md (prepend template) 2026-06-22 11:47:30 +00:00
github-actions[bot]
2954dee108 Release: 2026.6.0 2026-06-22 11:47:20 +00:00
かっこかり
00c6210a59 fix(backend): fix tests (#17606)
* fix(backend): fix tests

* attempt to fix test

* Revert "attempt to fix test"

This reverts commit ebe92c9dd9.

* fix

* fix

* test: fix test failure

---------

Co-authored-by: anatawa12 <anatawa12@icloud.com>
2026-06-22 20:18:40 +09:00
syuilo
2b87748537 Update CHANGELOG.md 2026-06-22 19:56:40 +09:00
syuilo
266a3c473b enhance(dev): improve frontend bundle report (#17600)
* wip

* Update package.json

* wip

* Update pnpm-lock.yaml

* wip

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* Update frontend-bundle-visualizer-report.mjs

* refactor

* Update frontend-js-size.yml

* refactor

* Update package.json
2026-06-22 19:41:03 +09:00
かっこかり
1eac4ccf51 Merge commit from fork 2026-06-22 16:34:48 +09:00
かっこかり
d323fe00d0 Merge commit from fork
* fix(backend): Prevent the reuse of used TOTP tokens

* fix

* fix

* tighten totp window
2026-06-22 16:34:15 +09:00
かっこかり
053e244582 fix(backend): summalyのバージョンを埋め込んでビルドするようにし、同一チャンクに巻き込まれないように (#17595)
* fix(backend): summalyのバージョンを埋め込んでビルドするようにし、同一チャンクに巻き込まれないように

* fix
2026-06-20 22:46:38 +09:00
かっこかり
c0a8c7f93a enhance(backend): SummalyのUser Agentを改善 (#17589)
* enhance(backend): SummalyのUser Agentを改善

* Update Changelog

* update summaly
2026-06-20 21:33:15 +09:00
syuilo
1d0b27b4c5 Update frontend-js-size.yml 2026-06-20 20:19:54 +09:00
syuilo
3c003d73c4 Update frontend-js-size.yml 2026-06-20 20:03:57 +09:00
syuilo
36d78a788d Update frontend-js-size.yml 2026-06-20 19:58:49 +09:00
syuilo
09f058f29a Update frontend-js-size.yml 2026-06-20 15:27:43 +09:00
syuilo
ad8b194643 Update frontend-js-size.yml 2026-06-20 15:10:59 +09:00
syuilo
4c9dd0e5ff Update frontend-js-size.yml 2026-06-20 14:55:54 +09:00
syuilo
0ced35ae6c Update frontend-js-size.yml 2026-06-20 14:33:47 +09:00
syuilo
dcced940af Update frontend-js-size.yml 2026-06-20 13:41:01 +09:00
syuilo
dc97a72fdb Update frontend-js-size.yml 2026-06-20 13:26:07 +09:00
syuilo
cc7f1e7366 enhance(dev/frontend-js-size): diffが大きい順にソートした上位10チャンクの表を追加 2026-06-20 11:49:15 +09:00
syuilo
77878256a8 fix(dev): follow up of 0956da49e9 2026-06-20 11:36:44 +09:00
syuilo
662129f414 fix(dev): follow up of 0956da49e9 2026-06-20 11:29:58 +09:00
syuilo
4457a75d22 fix(dev): follow up of 0956da49e9 2026-06-20 11:12:39 +09:00
syuilo
0956da49e9 feat(dev): フロントエンドのバンドルサイズ比較のAction (#17586)
Create frontend-js-size.yml
2026-06-20 11:00:58 +09:00
anatawa12
21a4f95bd6 fix: the script contains locale json is prefetched (#17585)
This commit upgrades rolldown used by vite to 1.1.0 and set
includeDependenciesRecursively instead of maxSize for
i18n code splitting group.

We unexpectedly prefetched the script file includes locale JSON
before this fix because locale inliner did not remove prefetch
for transitive dependency of i18n global variable.

Current locale inliner assumes the file contains i18n global
variable and the file contains i18n global variable are same file
and only removes prefetch for the file for i18n global variable
and leaves dependency files of the file.
However, in the previous fix for rolldown migration regression,
we set `maxSize: 1` for manual chunk of i18n.
This makes the chunk for i18n global variable (@/i18n.js) and
the chunk includes locale JSON (@@/js/locale.js) distinct chunks.
As a result, only prefetch for i18n global is removed and local
JSON remain in the prefetch file name dictionary (__vite__mapDeps).

There is two ways to fix this problem: 1) make rolldown to bundle
i18n related files into one but leave unrelated files separated
module or 2) update locale inliner to remove transitive dependency
of i18n of __vite__mapDeps.
2nd way is prune to rolldown changes, and it's possible by parsing
each .js file to (re)create module graph in inliner, it's complex.
Therefore, this commit fixes this with 1st way with
includeDependenciesRecursively option on `codeSplitting.groups`
newly added in rolldown 1.1.0.
Since latest vite as of writing (8.0.16) strictly depends on
rolldown 1.0.3, we cannot use it normally. We use overrides
to work around this problem. As far as I checked the vite
repository, upgrading rolldown to 1.1.x includes no code changes
except for package.json, so I hope this upgrade is safe.
2026-06-20 08:59:01 +09:00
30 changed files with 3257 additions and 520 deletions

View File

@@ -0,0 +1,46 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { appendFileSync, statSync } from 'node:fs';
import { extname } from 'node:path';
import { fileURLToPath } from 'node:url';
const traceFile = process.env.MK_BACKEND_JS_FOOTPRINT_TRACE;
const jsExtensions = new Set(['.js', '.mjs', '.cjs']);
function recordLoadedFile(kind, url, format) {
if (traceFile == null || !url.startsWith('file:')) return;
let filePath;
try {
filePath = fileURLToPath(url);
} catch {
return;
}
const extension = extname(filePath);
if (!jsExtensions.has(extension)) return;
let size = null;
try {
size = statSync(filePath).size;
} catch {
return;
}
appendFileSync(traceFile, `${JSON.stringify({
kind,
format,
path: filePath,
size,
timestamp: Date.now(),
})}\n`);
}
export async function load(url, context, nextLoad) {
const result = await nextLoad(url, context);
recordLoadedFile('esm', url, result.format ?? context.format ?? null);
return result;
}

View File

@@ -0,0 +1,46 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
'use strict';
const { appendFileSync, statSync } = require('node:fs');
const Module = require('node:module');
const { extname } = require('node:path');
const traceFile = process.env.MK_BACKEND_JS_FOOTPRINT_TRACE;
const jsExtensions = new Set(['.js', '.mjs', '.cjs']);
function recordLoadedFile(kind, filePath, request) {
if (traceFile == null || typeof filePath !== 'string') return;
const extension = extname(filePath);
if (!jsExtensions.has(extension) && extension !== '.node') return;
let size = null;
try {
size = statSync(filePath).size;
} catch {
return;
}
appendFileSync(traceFile, `${JSON.stringify({
kind,
format: extension === '.node' ? 'native' : 'commonjs',
path: filePath,
request,
size,
timestamp: Date.now(),
})}\n`);
}
const originalLoad = Module._load;
const originalResolveFilename = Module._resolveFilename;
Module._load = function load(request, parent, isMain) {
const resolved = originalResolveFilename.call(this, request, parent, isMain);
const result = originalLoad.apply(this, arguments);
recordLoadedFile('cjs', resolved, request);
return result;
};

530
.github/scripts/backend-js-footprint.mjs vendored Normal file
View File

@@ -0,0 +1,530 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { fork, spawn } from 'node:child_process';
import { createRequire } from 'node:module';
import { cpus, tmpdir } from 'node:os';
import { dirname, extname, join, relative, resolve, sep } from 'node:path';
import { setTimeout } from 'node:timers/promises';
import { fileURLToPath, pathToFileURL } from 'node:url';
import { gzipSync } from 'node:zlib';
import * as fs from 'node:fs/promises';
import * as fsSync from 'node:fs';
import * as http from 'node:http';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const [repoDirArg, outputFileArg] = process.argv.slice(2);
if (repoDirArg == null || outputFileArg == null) {
console.error('Usage: node .github/scripts/backend-js-footprint.mjs <repo-dir> <output.json>');
process.exit(1);
}
const STARTUP_TIMEOUT = readIntegerEnv('MK_JS_FOOTPRINT_STARTUP_TIMEOUT_MS', 120000, 1);
const SETTLE_TIME = readIntegerEnv('MK_JS_FOOTPRINT_SETTLE_TIME_MS', 10000, 0);
const REQUEST_COUNT = readIntegerEnv('MK_JS_FOOTPRINT_REQUEST_COUNT', 10, 0);
const MAX_TABLE_ITEMS = readIntegerEnv('MK_JS_FOOTPRINT_MAX_ITEMS', 20, 1);
const repoDir = resolve(repoDirArg);
const outputFile = resolve(outputFileArg);
const backendDir = join(repoDir, 'packages/backend');
const backendBuiltDir = join(backendDir, 'built');
const traceFile = join(tmpdir(), `misskey-backend-js-footprint-${process.pid}-${Date.now()}.jsonl`);
const require = createRequire(join(repoDir, 'package.json'));
const ts = require('typescript');
const jsExtensions = new Set(['.js', '.mjs', '.cjs']);
const fileMetricCache = new Map();
const packageInfoCache = new Map();
const nativePackageNames = new Set();
function readIntegerEnv(name, defaultValue, min) {
const rawValue = process.env[name];
if (rawValue == null || rawValue === '') return defaultValue;
if (!/^\d+$/.test(rawValue)) throw new Error(`${name} must be an integer`);
const value = Number(rawValue);
if (!Number.isSafeInteger(value) || value < min) throw new Error(`${name} must be >= ${min}`);
return value;
}
function commandName(command) {
if (process.platform !== 'win32') return command;
if (command === 'pnpm') return 'pnpm.cmd';
return command;
}
function isInside(parent, child) {
const rel = relative(parent, child);
return rel === '' || (!rel.startsWith('..') && !rel.includes(`..${sep}`));
}
function normalizePath(filePath) {
return filePath.split(sep).join('/');
}
function bytesToKiB(value) {
return Math.round(value / 1024);
}
function run(command, args, options = {}) {
return new Promise((resolvePromise, reject) => {
const child = spawn(commandName(command), args, {
cwd: options.cwd,
env: options.env,
stdio: ['ignore', 'pipe', 'pipe'],
});
let stdout = '';
let stderr = '';
child.stdout.on('data', data => {
stdout += data;
if (options.logStdout) process.stderr.write(data);
});
child.stderr.on('data', data => {
stderr += data;
process.stderr.write(data);
});
child.on('error', reject);
child.on('close', code => {
if (code === 0) {
resolvePromise(stdout);
} else {
reject(new Error(`${command} ${args.join(' ')} failed with exit code ${code}\n${stderr}`));
}
});
});
}
async function resetState() {
const backendRequire = createRequire(join(backendDir, 'package.json'));
const pg = backendRequire('pg');
const Redis = backendRequire('ioredis');
const postgres = new pg.Client({
host: '127.0.0.1',
port: 54312,
database: 'postgres',
user: 'postgres',
});
await postgres.connect();
try {
await postgres.query('DROP DATABASE IF EXISTS "test-misskey" WITH (FORCE)');
await postgres.query('CREATE DATABASE "test-misskey"');
} finally {
await postgres.end();
}
const redis = new Redis({ host: '127.0.0.1', port: 56312 });
try {
await redis.flushall();
} finally {
redis.disconnect();
}
}
function createRequest() {
return new Promise((resolvePromise, reject) => {
const req = http.request({
host: 'localhost',
port: 61812,
path: '/api/meta',
method: 'POST',
}, res => {
res.on('data', () => { });
res.on('end', () => resolvePromise());
});
req.on('error', reject);
req.end();
});
}
async function waitForServerReady(serverProcess) {
let serverReady = false;
serverProcess.on('message', message => {
if (message === 'ok') serverReady = true;
});
const startupStartTime = Date.now();
while (!serverReady) {
if (Date.now() - startupStartTime > STARTUP_TIMEOUT) {
serverProcess.kill('SIGTERM');
throw new Error('Server startup timeout');
}
await setTimeout(100);
}
}
async function stopServer(serverProcess) {
serverProcess.kill('SIGTERM');
let exited = false;
await new Promise(resolvePromise => {
serverProcess.on('exit', () => {
exited = true;
resolvePromise(undefined);
});
setTimeout(10000).then(() => {
if (!exited) serverProcess.kill('SIGKILL');
resolvePromise(undefined);
});
});
}
function getPackageNameFromPath(filePath) {
const normalized = normalizePath(filePath);
const marker = '/node_modules/';
const index = normalized.lastIndexOf(marker);
if (index === -1) return null;
const rest = normalized.slice(index + marker.length).split('/');
if (rest[0] === '.pnpm') {
const nestedNodeModulesIndex = rest.indexOf('node_modules');
if (nestedNodeModulesIndex === -1) return null;
const packageParts = rest.slice(nestedNodeModulesIndex + 1);
if (packageParts.length === 0) return null;
return packageParts[0].startsWith('@') ? packageParts.slice(0, 2).join('/') : packageParts[0];
}
return rest[0]?.startsWith('@') ? rest.slice(0, 2).join('/') : rest[0] ?? null;
}
function findPackageDir(filePath, packageName) {
const normalizedPackageName = packageName.split('/').join(sep);
let current = dirname(filePath);
while (current !== dirname(current)) {
if (current.endsWith(`${sep}${normalizedPackageName}`) && fsSync.existsSync(join(current, 'package.json'))) {
return current;
}
const parent = dirname(current);
if (parent === current) break;
current = parent;
}
return null;
}
function readPackageInfo(filePath) {
const externalPackageName = getPackageNameFromPath(filePath);
if (externalPackageName != null) {
const packageDir = findPackageDir(filePath, externalPackageName);
const cacheKey = packageDir ?? externalPackageName;
if (packageInfoCache.has(cacheKey)) return packageInfoCache.get(cacheKey);
let version = null;
if (packageDir != null) {
try {
const packageJson = JSON.parse(fsSync.readFileSync(join(packageDir, 'package.json'), 'utf8'));
version = typeof packageJson.version === 'string' ? packageJson.version : null;
} catch { }
}
const info = {
category: 'external',
name: externalPackageName,
version,
dir: packageDir,
};
packageInfoCache.set(cacheKey, info);
return info;
}
if (isInside(backendBuiltDir, filePath)) {
return {
category: 'internal',
name: 'backend',
version: null,
dir: backendDir,
};
}
return {
category: 'internal',
name: 'workspace',
version: null,
dir: repoDir,
};
}
function analyzeSource(filePath, source) {
const sourceFile = ts.createSourceFile(filePath, source, ts.ScriptTarget.Latest, true, ts.ScriptKind.JS);
const metrics = {
astNodeCount: 0,
functionCount: 0,
classCount: 0,
stringLiteralBytes: 0,
};
function visit(node) {
metrics.astNodeCount += 1;
if (
ts.isFunctionDeclaration(node) ||
ts.isFunctionExpression(node) ||
ts.isArrowFunction(node) ||
ts.isMethodDeclaration(node) ||
ts.isConstructorDeclaration(node) ||
ts.isGetAccessorDeclaration(node) ||
ts.isSetAccessorDeclaration(node)
) {
metrics.functionCount += 1;
} else if (ts.isClassDeclaration(node) || ts.isClassExpression(node)) {
metrics.classCount += 1;
} else if (ts.isStringLiteral(node) || ts.isNoSubstitutionTemplateLiteral(node)) {
metrics.stringLiteralBytes += Buffer.byteLength(node.text);
}
ts.forEachChild(node, visit);
}
visit(sourceFile);
return metrics;
}
function readFileMetrics(filePath) {
if (fileMetricCache.has(filePath)) return fileMetricCache.get(filePath);
const source = fsSync.readFileSync(filePath);
const sourceText = source.toString('utf8');
const astMetrics = analyzeSource(filePath, sourceText);
const packageInfo = readPackageInfo(filePath);
const metric = {
path: filePath,
displayPath: normalizePath(relative(repoDir, filePath)),
sourceBytes: source.byteLength,
gzipBytes: gzipSync(source).byteLength,
...astMetrics,
package: packageInfo,
};
fileMetricCache.set(filePath, metric);
return metric;
}
async function readTraceRecords() {
let content = '';
try {
content = await fs.readFile(traceFile, 'utf8');
} catch (err) {
if (err.code === 'ENOENT') return [];
throw err;
}
const records = [];
for (const line of content.split('\n')) {
if (line.trim() === '') continue;
try {
records.push(JSON.parse(line));
} catch { }
}
return records;
}
function emptyTotals() {
return {
loadedJsModules: 0,
loadedJsSourceBytes: 0,
loadedJsGzipBytes: 0,
astNodeCount: 0,
functionCount: 0,
classCount: 0,
stringLiteralBytes: 0,
externalPackageCount: 0,
nativeAddonPackageCount: 0,
};
}
function addFileMetrics(target, metric) {
target.loadedJsModules += 1;
target.loadedJsSourceBytes += metric.sourceBytes;
target.loadedJsGzipBytes += metric.gzipBytes;
target.astNodeCount += metric.astNodeCount;
target.functionCount += metric.functionCount;
target.classCount += metric.classCount;
target.stringLiteralBytes += metric.stringLiteralBytes;
}
function summarizeRecords(records, phase) {
const jsPaths = new Set();
const nativePaths = new Set();
for (const record of records) {
if (typeof record.path !== 'string') continue;
const extension = extname(record.path);
if (jsExtensions.has(extension)) {
jsPaths.add(resolve(record.path));
} else if (extension === '.node') {
nativePaths.add(resolve(record.path));
}
}
for (const nativePath of nativePaths) {
const packageInfo = readPackageInfo(nativePath);
if (packageInfo.category === 'external') nativePackageNames.add(packageInfo.name);
}
const totals = emptyTotals();
const packages = new Map();
const modules = [];
for (const filePath of [...jsPaths].toSorted()) {
let metric;
try {
metric = readFileMetrics(filePath);
} catch (err) {
process.stderr.write(`Failed to analyze ${filePath}: ${err.message}\n`);
continue;
}
addFileMetrics(totals, metric);
const packageKey = metric.package.name;
if (!packages.has(packageKey)) {
packages.set(packageKey, {
name: metric.package.name,
version: metric.package.version,
category: metric.package.category,
sourceBytes: 0,
gzipBytes: 0,
modules: 0,
astNodeCount: 0,
functionCount: 0,
classCount: 0,
stringLiteralBytes: 0,
nativeAddon: false,
});
}
const packageSummary = packages.get(packageKey);
packageSummary.sourceBytes += metric.sourceBytes;
packageSummary.gzipBytes += metric.gzipBytes;
packageSummary.modules += 1;
packageSummary.astNodeCount += metric.astNodeCount;
packageSummary.functionCount += metric.functionCount;
packageSummary.classCount += metric.classCount;
packageSummary.stringLiteralBytes += metric.stringLiteralBytes;
modules.push({
path: metric.displayPath,
package: metric.package.name,
category: metric.package.category,
sourceBytes: metric.sourceBytes,
gzipBytes: metric.gzipBytes,
astNodeCount: metric.astNodeCount,
functionCount: metric.functionCount,
classCount: metric.classCount,
stringLiteralBytes: metric.stringLiteralBytes,
});
}
for (const packageName of nativePackageNames) {
const packageSummary = packages.get(packageName);
if (packageSummary != null) packageSummary.nativeAddon = true;
}
const externalPackages = [...packages.values()].filter(packageSummary => packageSummary.category === 'external');
totals.externalPackageCount = externalPackages.length;
totals.nativeAddonPackageCount = externalPackages.filter(packageSummary => packageSummary.nativeAddon).length;
return {
phase,
totals: {
...totals,
loadedJsSourceKiB: bytesToKiB(totals.loadedJsSourceBytes),
loadedJsGzipKiB: bytesToKiB(totals.loadedJsGzipBytes),
stringLiteralKiB: bytesToKiB(totals.stringLiteralBytes),
},
packages: [...packages.values()].toSorted((a, b) => b.sourceBytes - a.sourceBytes),
modules: modules.toSorted((a, b) => b.sourceBytes - a.sourceBytes).slice(0, MAX_TABLE_ITEMS),
};
}
async function measureFootprint() {
await fs.writeFile(traceFile, '');
process.stderr.write('Resetting database and Redis\n');
await resetState();
process.stderr.write('Running migrations\n');
await run('pnpm', ['--filter', 'backend', 'migrate'], {
cwd: repoDir,
env: process.env,
logStdout: true,
});
const serverProcess = fork(join(backendBuiltDir, 'entry.js'), [], {
cwd: backendDir,
env: {
...process.env,
NODE_ENV: 'production',
MK_DISABLE_CLUSTERING: '1',
MK_ONLY_SERVER: '1',
MK_NO_DAEMONS: '1',
MK_BACKEND_JS_FOOTPRINT_TRACE: traceFile,
},
stdio: ['pipe', 'pipe', 'pipe', 'ipc'],
execArgv: [
'--require',
join(__dirname, 'backend-js-footprint-require.cjs'),
'--experimental-loader',
pathToFileURL(join(__dirname, 'backend-js-footprint-loader.mjs')).href,
],
});
serverProcess.stdout?.on('data', data => {
process.stderr.write(`[server stdout] ${data}`);
});
serverProcess.stderr?.on('data', data => {
process.stderr.write(`[server stderr] ${data}`);
});
serverProcess.on('error', err => {
process.stderr.write(`[server error] ${err}\n`);
});
try {
await waitForServerReady(serverProcess);
await setTimeout(SETTLE_TIME);
const startup = summarizeRecords(await readTraceRecords(), 'startup');
await Promise.all(
Array.from({ length: REQUEST_COUNT }).map(() => createRequest()),
);
await setTimeout(1000);
const afterRequest = summarizeRecords(await readTraceRecords(), 'afterRequest');
return {
timestamp: new Date().toISOString(),
measurement: {
strategy: 'runtime-loader-trace',
startupTimeoutMs: STARTUP_TIMEOUT,
settleTimeMs: SETTLE_TIME,
requestCount: REQUEST_COUNT,
cpus: cpus().length,
},
startup,
afterRequest,
};
} finally {
await stopServer(serverProcess);
await fs.rm(traceFile, { force: true });
}
}
const result = await measureFootprint();
await fs.writeFile(outputFile, `${JSON.stringify(result, null, 2)}\n`);

View File

@@ -0,0 +1,424 @@
import { readFile, writeFile } from 'node:fs/promises';
const [baseFile, headFile, outputFile, baseJsFootprintFile, headJsFootprintFile] = process.argv.slice(2);
if (baseFile == null || headFile == null || outputFile == null) {
console.error('Usage: node .github/scripts/backend-memory-report.mjs <base-memory.json> <head-memory.json> <report.md> [base-js-footprint.json head-js-footprint.json]');
process.exit(1);
}
const numberFormatter = new Intl.NumberFormat('en-US', {
maximumFractionDigits: 2,
});
const phases = [
{
key: 'afterGc',
title: 'After GC',
},
];
const metrics = [
'HeapUsed',
'Pss',
'Private_Dirty',
'VmRSS',
'External',
];
function formatNumber(value) {
return numberFormatter.format(value);
}
function formatMemory(valueKiB) {
return `${formatNumber(valueKiB / 1024)} MB`;
}
function formatBytes(value) {
if (!Number.isFinite(value)) return '-';
if (value < 1024) return `${formatNumber(value)} B`;
if (value < 1024 * 1024) return `${formatNumber(value / 1024)} KiB`;
return `${formatNumber(value / 1024 / 1024)} MiB`;
}
function formatPercent(value) {
return `${formatNumber(value)}%`;
}
function formatMathText(text) {
return text
.replaceAll('\\', '\\\\')
.replaceAll('{', '\\{')
.replaceAll('}', '\\}')
.replaceAll('%', '\\%');
}
function formatColoredDiff(text, diff) {
const color = diff > 0 ? 'orange' : 'green';
return `$\\color{${color}}{\\text{${formatMathText(text).replaceAll('\\%', '\\\\%')}}}$`;
}
function formatDiff(baseKiB, headKiB) {
const diff = headKiB - baseKiB;
if (diff === 0) return formatMemory(0);
const sign = diff > 0 ? '+' : '-';
return formatColoredDiff(`${sign}${formatMemory(Math.abs(diff))}`, diff);
}
function formatDiffPercent(baseKiB, headKiB) {
const diff = headKiB - baseKiB;
if (diff === 0) return '0%';
if (baseKiB <= 0) return '-';
const sign = diff > 0 ? '+' : '-';
return formatColoredDiff(`${sign}${formatPercent(Math.abs((diff * 100) / baseKiB))}`, diff);
}
function getMemoryValue(report, phase, metric) {
const value = report?.[phase]?.[metric];
return Number.isFinite(value) ? value : null;
}
function median(values) {
const sorted = values.toSorted((a, b) => a - b);
const center = Math.floor(sorted.length / 2);
if (sorted.length % 2 === 1) return sorted[center];
return Math.round((sorted[center - 1] + sorted[center]) / 2);
}
function getSampleValues(report, phase, metric) {
if (!Array.isArray(report?.samples)) return [];
return report.samples
.map(sample => getMemoryValue(sample, phase, metric))
.filter(value => Number.isFinite(value));
}
function getSampleSpread(report, phase, metric) {
const values = getSampleValues(report, phase, metric);
if (values.length < 2) return null;
const center = median(values);
return median(values.map(value => Math.abs(value - center)));
}
function renderTable(base, head, phase) {
const lines = [
'| Metric | Base | Head | Δ | Δ (%) |',
'| --- | ---: | ---: | ---: | ---: |',
];
for (const metric of metrics) {
const baseValue = getMemoryValue(base, phase, metric);
const headValue = getMemoryValue(head, phase, metric);
if (baseValue == null || headValue == null) continue;
const baseSpread = getSampleSpread(base, phase, metric);
const headSpread = getSampleSpread(head, phase, metric);
lines.push(`| ${metric} | ${formatMemory(baseValue)} <br> ± ${formatMemory(baseSpread)} | ${formatMemory(headValue)} <br> ± ${formatMemory(headSpread)} | ${formatDiff(baseValue, headValue)} | ${formatDiffPercent(baseValue, headValue)} |`);
}
return lines.join('\n');
}
function getDiffPercent(base, head, phase, metric) {
const baseValue = getMemoryValue(base, phase, metric);
const headValue = getMemoryValue(head, phase, metric);
if (baseValue == null || headValue == null || baseValue <= 0) return null;
return ((headValue - baseValue) * 100) / baseValue;
}
function getWarningMetric(base, head) {
for (const metric of ['Pss', 'Private_Dirty', 'VmRSS']) {
if (getMemoryValue(base, 'afterGc', metric) != null && getMemoryValue(head, 'afterGc', metric) != null) {
return metric;
}
}
return null;
}
function isBeyondSampleNoise(base, head, phase, metric) {
const baseValue = getMemoryValue(base, phase, metric);
const headValue = getMemoryValue(head, phase, metric);
if (baseValue == null || headValue == null) return false;
const diff = headValue - baseValue;
if (diff <= 0) return false;
const baseSpread = getSampleSpread(base, phase, metric);
const headSpread = getSampleSpread(head, phase, metric);
if (baseSpread == null || headSpread == null) return true;
const combinedSpread = Math.hypot(baseSpread, headSpread);
return diff > combinedSpread * 3;
}
function workflowFooter() {
const repository = process.env.GITHUB_REPOSITORY;
const runId = process.env.GITHUB_RUN_ID;
if (repository == null || runId == null) {
return 'See workflow logs for details.';
}
return `[See workflow logs for details](https://github.com/${repository}/actions/runs/${runId})`;
}
function measurementSummary(base, head) {
const baseCount = base?.sampleCount;
const headCount = head?.sampleCount;
const strategy = base?.comparison?.strategy;
if (baseCount == null || headCount == null) return null;
if (strategy === 'interleaved-pairs') {
const rounds = base?.comparison?.rounds ?? baseCount;
const warmupRounds = base?.comparison?.warmupRounds ?? 0;
return `_Measured as ${rounds} interleaved base/head pairs after ${warmupRounds} warmup pair(s). Values are medians; ± is median absolute deviation._`;
}
return `_Sample count: base ${baseCount}, head ${headCount}. Values are medians; ± is median absolute deviation._`;
}
function formatPlainDiff(baseValue, headValue, formatter = formatNumber) {
const diff = headValue - baseValue;
if (diff === 0) return formatter(0);
const sign = diff > 0 ? '+' : '-';
return `${sign}${formatter(Math.abs(diff))}`;
}
function formatPlainDiffPercent(baseValue, headValue) {
const diff = headValue - baseValue;
if (diff === 0) return '0%';
if (baseValue <= 0) return '-';
const sign = diff > 0 ? '+' : '-';
return `${sign}${formatPercent(Math.abs((diff * 100) / baseValue))}`;
}
function getJsFootprintValue(report, phase, key) {
const value = report?.[phase]?.totals?.[key];
return Number.isFinite(value) ? value : null;
}
function renderJsFootprintMetricTable(base, head) {
const metricRows = [
['Loaded JS modules', 'loadedJsModules', formatNumber],
['Loaded JS source', 'loadedJsSourceBytes', formatBytes],
//['Loaded JS gzip estimate', 'loadedJsGzipBytes', formatBytes],
//['AST nodes', 'astNodeCount', formatNumber],
//['Functions', 'functionCount', formatNumber],
//['Classes', 'classCount', formatNumber],
//['String literals', 'stringLiteralBytes', formatBytes],
['External packages loaded', 'externalPackageCount', formatNumber],
['Native addon packages', 'nativeAddonPackageCount', formatNumber],
];
const lines = [
'| Metric | Base | Head | Δ | Δ (%) |',
'| --- | ---: | ---: | ---: | ---: |',
];
for (const [title, key, formatter] of metricRows) {
const baseValue = getJsFootprintValue(base, 'afterRequest', key);
const headValue = getJsFootprintValue(head, 'afterRequest', key);
if (baseValue == null || headValue == null) continue;
lines.push(`| ${title} | ${formatter(baseValue)} | ${formatter(headValue)} | ${formatPlainDiff(baseValue, headValue, formatter)} | ${formatPlainDiffPercent(baseValue, headValue)} |`);
}
return lines.join('\n');
}
function renderJsFootprintPhaseTable(base, head) {
const lines = [
'| Phase | Base modules | Head modules | Δ modules | Base source | Head source | Δ source |',
'| --- | ---: | ---: | ---: | ---: | ---: | ---: |',
];
for (const [phase, title] of [['startup', 'Startup'], ['afterRequest', 'After warmup requests']]) {
const baseModules = getJsFootprintValue(base, phase, 'loadedJsModules');
const headModules = getJsFootprintValue(head, phase, 'loadedJsModules');
const baseSource = getJsFootprintValue(base, phase, 'loadedJsSourceBytes');
const headSource = getJsFootprintValue(head, phase, 'loadedJsSourceBytes');
if (baseModules == null || headModules == null || baseSource == null || headSource == null) continue;
lines.push(`| ${title} | ${formatNumber(baseModules)} | ${formatNumber(headModules)} | ${formatPlainDiff(baseModules, headModules)} | ${formatBytes(baseSource)} | ${formatBytes(headSource)} | ${formatPlainDiff(baseSource, headSource, formatBytes)} |`);
}
return lines.join('\n');
}
function packageMap(report) {
const map = new Map();
for (const packageSummary of report?.afterRequest?.packages ?? []) {
if (packageSummary?.category !== 'external' || typeof packageSummary.name !== 'string') continue;
map.set(packageSummary.name, packageSummary);
}
return map;
}
function packageDisplayName(packageSummary) {
if (packageSummary.version == null) return packageSummary.name;
return `${packageSummary.name} ${packageSummary.version}`;
}
function renderNewExternalPackages(base, head) {
const basePackages = packageMap(base);
const headPackages = packageMap(head);
const newPackages = [...headPackages.values()]
.filter(packageSummary => !basePackages.has(packageSummary.name))
.toSorted((a, b) => b.sourceBytes - a.sourceBytes)
.slice(0, 10);
if (newPackages.length === 0) return null;
const lines = [
'#### Newly Loaded External Packages',
'',
'| Package | Loaded JS | Modules | Notes |',
'| --- | ---: | ---: | --- |',
];
for (const packageSummary of newPackages) {
lines.push(`| ${packageDisplayName(packageSummary)} | ${formatBytes(packageSummary.sourceBytes)} | ${formatNumber(packageSummary.modules)} | ${packageSummary.nativeAddon ? 'native addon' : ''} |`);
}
return lines.join('\n');
}
function renderLargestPackageIncreases(base, head) {
const basePackages = packageMap(base);
const headPackages = packageMap(head);
const increases = [...headPackages.values()]
.map(headPackage => {
const basePackage = basePackages.get(headPackage.name);
const baseSourceBytes = basePackage?.sourceBytes ?? 0;
const baseModules = basePackage?.modules ?? 0;
return {
...headPackage,
baseSourceBytes,
baseModules,
sourceDiff: headPackage.sourceBytes - baseSourceBytes,
moduleDiff: headPackage.modules - baseModules,
};
})
.filter(packageSummary => packageSummary.sourceDiff > 0)
.toSorted((a, b) => b.sourceDiff - a.sourceDiff)
.slice(0, 10);
if (increases.length === 0) return null;
const lines = [
'#### Largest Package Increases',
'',
'| Package | Base | Head | Δ | Modules Δ |',
'| --- | ---: | ---: | ---: | ---: |',
];
for (const packageSummary of increases) {
lines.push(`| ${packageDisplayName(packageSummary)} | ${formatBytes(packageSummary.baseSourceBytes)} | ${formatBytes(packageSummary.sourceBytes)} | ${formatPlainDiff(packageSummary.baseSourceBytes, packageSummary.sourceBytes, formatBytes)} | ${formatPlainDiff(packageSummary.baseModules, packageSummary.modules)} |`);
}
return lines.join('\n');
}
function moduleMap(report) {
const map = new Map();
for (const moduleSummary of report?.afterRequest?.modules ?? []) {
if (typeof moduleSummary.path !== 'string') continue;
map.set(moduleSummary.path, moduleSummary);
}
return map;
}
function renderNewLoadedModules(base, head) {
const baseModules = moduleMap(base);
const headModules = moduleMap(head);
const newModules = [...headModules.values()]
.filter(moduleSummary => !baseModules.has(moduleSummary.path))
.toSorted((a, b) => b.sourceBytes - a.sourceBytes)
.slice(0, 10);
if (newModules.length === 0) return null;
const lines = [
'#### Largest Newly Loaded Modules',
'',
'| Module | Package | Loaded JS |',
'| --- | --- | ---: |',
];
for (const moduleSummary of newModules) {
lines.push(`| \`${moduleSummary.path}\` | ${moduleSummary.package} | ${formatBytes(moduleSummary.sourceBytes)} |`);
}
return lines.join('\n');
}
function renderJsFootprintSection(base, head) {
if (base == null || head == null) return null;
const lines = [
'### Runtime Loaded JS Footprint',
'',
renderJsFootprintMetricTable(base, head),
'',
'#### Load Phase Breakdown',
'',
renderJsFootprintPhaseTable(base, head),
'',
];
for (const block of [
renderNewExternalPackages(base, head),
renderLargestPackageIncreases(base, head),
renderNewLoadedModules(base, head),
]) {
if (block == null) continue;
lines.push(block);
lines.push('');
}
return lines.join('\n');
}
const base = JSON.parse(await readFile(baseFile, 'utf8'));
const head = JSON.parse(await readFile(headFile, 'utf8'));
const baseJsFootprint = baseJsFootprintFile == null ? null : JSON.parse(await readFile(baseJsFootprintFile, 'utf8'));
const headJsFootprint = headJsFootprintFile == null ? null : JSON.parse(await readFile(headJsFootprintFile, 'utf8'));
const lines = [
'## Backend Memory Usage Report',
'',
];
const summary = measurementSummary(base, head);
if (summary != null) {
lines.push(summary);
lines.push('');
}
for (const phase of phases) {
lines.push(`### ${phase.title}`);
lines.push(renderTable(base, head, phase.key));
lines.push('');
}
const jsFootprintSection = renderJsFootprintSection(baseJsFootprint, headJsFootprint);
if (jsFootprintSection != null) {
lines.push(jsFootprintSection);
lines.push('');
}
const warningMetric = getWarningMetric(base, head);
const warningDiffPercent = warningMetric == null ? null : getDiffPercent(base, head, 'afterGc', warningMetric);
if (warningMetric != null && warningDiffPercent != null && warningDiffPercent > 5 && isBeyondSampleNoise(base, head, 'afterGc', warningMetric)) {
lines.push(`⚠️ **Warning**: Memory usage (${warningMetric}) has increased by more than 5% and exceeds the observed sample noise. Please verify this is not an unintended change.`);
lines.push('');
}
lines.push(workflowFooter());
await writeFile(outputFile, `${lines.join('\n')}\n`);

View File

@@ -0,0 +1,276 @@
import { readFile, writeFile } from 'node:fs/promises';
const [beforeFile, afterFile, outputFile] = process.argv.slice(2);
if (beforeFile == null || afterFile == null || outputFile == null) {
console.error('Usage: node .github/scripts/frontend-bundle-visualizer-report.mjs <before-stats.json> <after-stats.json> <report.md>');
process.exit(1);
}
const byteFormatter = new Intl.NumberFormat('en-US');
const numberFormatter = new Intl.NumberFormat('en-US');
function formatBytes(value) {
if (!Number.isFinite(value) || value <= 0) return '0 B';
const units = ['B', 'KiB', 'MiB', 'GiB'];
let unitIndex = 0;
let size = value;
while (size >= 1024 && unitIndex < units.length - 1) {
size /= 1024;
unitIndex += 1;
}
const maximumFractionDigits = size >= 10 || unitIndex === 0 ? 0 : 1;
return `${byteFormatter.format(Number(size.toFixed(maximumFractionDigits)))} ${units[unitIndex]}`;
}
function formatNumber(value) {
return numberFormatter.format(value);
}
function formatPercent(value) {
return `${Math.round(value)}%`;
}
function sharePercent(value, total) {
if (total === 0) return '0%';
return formatPercent((value / total) * 100);
}
function formatMathText(text) {
return text
.replaceAll('\\', '\\\\')
.replaceAll('{', '\\{')
.replaceAll('}', '\\}')
.replaceAll('%', '\\%');
}
function formatColoredDiff(text, diff) {
const color = diff > 0 ? 'orange' : 'green';
return `$\\color{${color}}{\\text{${formatMathText(text)}}}$`;
}
function formatDiff(before, after, formatter) {
const diff = after - before;
if (diff === 0) return formatter(0);
const sign = diff > 0 ? '+' : '-';
return formatColoredDiff(`${sign}${formatter(Math.abs(diff))}`, diff);
}
function formatDiffPercent(before, after) {
if (before === 0 && after === 0) return '0%';
if (before === 0) return '-';
const diff = after - before;
if (diff === 0) return '0%';
const sign = diff > 0 ? '+' : '-';
return formatColoredDiff(`${sign}${formatPercent(Math.abs(diff / before) * 100)}`, diff);
}
function tableCell(value) {
return String(value).replaceAll('|', '\\|').replaceAll('\r', ' ').replaceAll('\n', ' ');
}
function code(value) {
const sanitized = String(value).replaceAll('\r', ' ').replaceAll('\n', ' ');
const backtickRuns = sanitized.match(/`+/g) ?? [];
const fenceLength = Math.max(1, ...backtickRuns.map((run) => run.length + 1));
const fence = '`'.repeat(fenceLength);
const padding = sanitized.startsWith('`') || sanitized.endsWith('`') ? ' ' : '';
return `${fence}${padding}${sanitized}${padding}${fence}`;
}
function tableCode(value) {
return tableCell(code(value));
}
function collectReport(data) {
const nodeParts = data.nodeParts ?? {};
const nodeMetas = Object.values(data.nodeMetas ?? {});
const moduleRows = [];
const bundleMap = new Map();
for (const meta of nodeMetas) {
const row = {
id: meta.id,
bundles: 0,
renderedLength: 0,
gzipLength: 0,
brotliLength: 0,
importedByCount: meta.importedBy?.length ?? 0,
importedCount: meta.imported?.length ?? 0,
};
for (const [bundleId, partUid] of Object.entries(meta.moduleParts ?? {})) {
const part = nodeParts[partUid];
if (part == null) continue;
row.bundles += 1;
row.renderedLength += part.renderedLength;
row.gzipLength += part.gzipLength;
row.brotliLength += part.brotliLength;
const bundle = bundleMap.get(bundleId) ?? {
id: bundleId,
modules: 0,
renderedLength: 0,
gzipLength: 0,
brotliLength: 0,
};
bundle.modules += 1;
bundle.renderedLength += part.renderedLength;
bundle.gzipLength += part.gzipLength;
bundle.brotliLength += part.brotliLength;
bundleMap.set(bundleId, bundle);
}
if (row.bundles > 0) {
moduleRows.push(row);
}
}
let staticImports = 0;
let dynamicImports = 0;
for (const meta of nodeMetas) {
for (const imported of meta.imported ?? []) {
if (imported.dynamic) {
dynamicImports += 1;
} else {
staticImports += 1;
}
}
}
const bundleRows = [...bundleMap.values()].sort((a, b) => b.renderedLength - a.renderedLength);
const hotModules = [...moduleRows].sort((a, b) => b.renderedLength - a.renderedLength);
const totalRendered = moduleRows.reduce((sum, row) => sum + row.renderedLength, 0);
const totalGzip = moduleRows.reduce((sum, row) => sum + row.gzipLength, 0);
const totalBrotli = moduleRows.reduce((sum, row) => sum + row.brotliLength, 0);
return {
options: data.options ?? {},
summary: {
bundles: bundleRows.length,
modules: moduleRows.length,
entries: nodeMetas.filter((meta) => meta.isEntry).length,
externals: nodeMetas.filter((meta) => meta.isExternal).length,
staticImports,
dynamicImports,
},
metrics: {
renderedLength: totalRendered,
gzipLength: totalGzip,
brotliLength: totalBrotli,
},
hotModules,
};
}
function renderSummaryTable(before, after) {
const summary = [
'bundles',
'modules',
'entries',
//'externals',
'staticImports',
'dynamicImports',
];
const metrics = [
'renderedLength',
'gzipLength',
'brotliLength',
];
return [
`<table>`,
`<thead>`,
`<tr>`,
`<th rowspan="2"></th>`,
`<th rowspan="2">Bundles</th>`,
`<th rowspan="2">Modules</th>`,
`<th rowspan="2">Entries</th>`,
`<th colspan="2">Imports</th>`,
`<th colspan="3">Size</th>`,
`</tr>`,
`<tr>`,
`<th>Static</th>`,
`<th>Dynamic</th>`,
`<th>Rendered</th>`,
`<th>Gzip</th>`,
`<th>Brotli</th>`,
`</tr>`,
`</thead>`,
`<tbody>`,
`<tr>`,
`<th><b>Before</b></th>`,
...summary.map((key) => `<td>${formatNumber(before.summary[key])}</td>`),
...metrics.map((key) => `<td>${formatBytes(before.metrics[key])}</td>`),
`</tr>`,
`<tr>`,
`<th><b>After</b></th>`,
...summary.map((key) => `<td>${formatNumber(after.summary[key])}</td>`),
...metrics.map((key) => `<td>${formatBytes(after.metrics[key])}</td>`),
`</tr>`,
`<tr><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr>`,
`<tr>`,
`<th><b>Δ</b></th>`,
...summary.map((key) => `<td>${formatDiff(before.summary[key], after.summary[key], formatNumber)}</td>`),
...metrics.map((key) => `<td>${formatDiff(before.metrics[key], after.metrics[key], formatBytes)}</td>`),
`</tr>`,
`<tr>`,
`<th><b>Δ (%)</b></th>`,
...summary.map((key) => `<td>${formatDiffPercent(before.summary[key], after.summary[key])}</td>`),
...metrics.map((key) => `<td>${formatDiffPercent(before.metrics[key], after.metrics[key])}</td>`),
`</tr>`,
`</tbody>`,
`</table>`,
];
}
const beforeData = JSON.parse(await readFile(beforeFile, 'utf8'));
const afterData = JSON.parse(await readFile(afterFile, 'utf8'));
const before = collectReport(beforeData);
const after = collectReport(afterData);
const lines = [
'## Frontend Bundle Report',
'',
...renderSummaryTable(before, after),
'',
'<details>',
'<summary>Top 10</summary>',
'',
];
for (const row of after.hotModules.slice(0, 10)) {
lines.push(`- ${code(row.id)}: ${sharePercent(row.renderedLength, after.metrics.renderedLength)} (${formatBytes(row.renderedLength)})`);
}
lines.push(
'',
'</details>',
);
lines.push(
'',
'<details>',
'<summary>Hot Modules (Self Size)</summary>',
'',
'| Module | Bundles | Rendered | Share | Gzip | Brotli | Imports | Imported By |',
'|---|---:|---:|---:|---:|---:|---:|---:|',
);
for (const row of after.hotModules.slice(0, 15)) {
lines.push(`| ${tableCode(row.id)} | ${row.bundles} | ${formatBytes(row.renderedLength)} | ${sharePercent(row.renderedLength, after.metrics.renderedLength)} | ${formatBytes(row.gzipLength)} | ${formatBytes(row.brotliLength)} | ${row.importedCount} | ${row.importedByCount} |`);
}
lines.push(
'',
'</details>',
);
await writeFile(outputFile, `${lines.join('\n')}\n`);

331
.github/scripts/frontend-js-size.mjs vendored Normal file
View File

@@ -0,0 +1,331 @@
import { promises as fs } from 'node:fs';
import path from 'node:path';
const marker = '<!-- misskey-frontend-js-size -->';
const locale = process.env.FRONTEND_JS_SIZE_LOCALE || 'ja-JP';
function normalizePath(filePath) {
return filePath.split(path.sep).join('/');
}
async function exists(filePath) {
try {
await fs.access(filePath);
return true;
} catch {
return false;
}
}
async function fileSize(filePath) {
const stat = await fs.stat(filePath);
return stat.size;
}
async function* walk(dir) {
for (const entry of await fs.readdir(dir, { withFileTypes: true })) {
const fullPath = path.join(dir, entry.name);
if (entry.isDirectory()) {
yield* walk(fullPath);
} else if (entry.isFile()) {
yield fullPath;
}
}
}
function formatBytes(size) {
if (size == null) return '-';
if (size < 1024) return `${size} B`;
if (size < 1024 * 1024) return `${stripTrailingZeros((size / 1024).toFixed(1))} KiB`;
return `${stripTrailingZeros((size / 1024 / 1024).toFixed(2))} MiB`;
}
function stripTrailingZeros(value) {
return value.replace(/\.0+$/, '').replace(/(\.\d*?)0+$/, '$1');
}
function formatMathText(text) {
return text
.replaceAll('\\', '\\\\')
.replaceAll('{', '\\{')
.replaceAll('}', '\\}')
.replaceAll('%', '\\\\%');
}
function formatDiff(diff) {
if (diff == null) return '-';
if (diff === 0) return '0 B';
const sign = diff > 0 ? '+' : '-';
const text = `${sign}${formatBytes(Math.abs(diff))}`;
const color = diff > 0 ? 'orange' : 'green';
return `$\\color{${color}}{\\text{${formatMathText(text)}}}$`;
}
function formatDiffPercent(beforeSize, afterSize) {
if (beforeSize == null || beforeSize === 0 || afterSize == null || afterSize === 0) return '-';
const diff = afterSize - beforeSize;
if (diff === 0) return `0%`;
const percent = Math.round(diff / beforeSize * 100);
const color = diff > 0 ? 'orange' : 'green';
return `$\\color{${color}}{\\text{${formatMathText(percent.toString() + '%')}}}$`;
}
function escapeCell(value) {
return String(value).replaceAll('|', '\\|').replaceAll('\n', '<br>');
}
function entryDisplayName(entry) {
if (!entry) return '';
return entry.displayName || entry.file;
}
function findEntryKey(manifest) {
const entries = Object.entries(manifest);
return entries.find(([key, chunk]) => key === 'src/_boot_.ts' || chunk.src === 'src/_boot_.ts')?.[0]
?? entries.find(([, chunk]) => chunk.name === 'entry' && chunk.isEntry)?.[0]
?? entries.find(([, chunk]) => chunk.isEntry)?.[0]
?? null;
}
function stableChunkKey(manifestKey, chunk) {
return chunk.src ?? (chunk.name ? `chunk:${chunk.name}` : manifestKey);
}
function collectStartupKeys(manifest) {
const entryKey = findEntryKey(manifest);
const keys = new Set();
if (entryKey == null) return keys;
function visit(key) {
if (keys.has(key)) return;
const chunk = manifest[key];
if (!chunk || !chunk.file?.endsWith('.js')) return;
keys.add(stableChunkKey(key, chunk));
for (const importKey of chunk.imports ?? []) {
visit(importKey);
}
}
visit(entryKey);
return keys;
}
async function resolveBuiltFile(outDir, file) {
if (file.startsWith('scripts/')) {
const localizedFile = file.slice('scripts/'.length);
const localizedPath = path.join(outDir, locale, localizedFile);
if (await exists(localizedPath)) {
return {
absolutePath: localizedPath,
relativePath: `${locale}/${localizedFile}`,
};
}
throw new Error(`Expected ${locale} localized chunk for ${file}, but ${localizedPath} was not found.`);
}
return {
absolutePath: path.join(outDir, file),
relativePath: file,
};
}
async function collectReport(repoDir) {
const outDir = path.join(repoDir, 'built/_frontend_vite_');
const manifestPath = path.join(outDir, 'manifest.json');
const manifest = JSON.parse(await fs.readFile(manifestPath, 'utf8'));
const byKey = new Map();
const byFile = new Set();
for (const [key, chunk] of Object.entries(manifest)) {
if (!chunk.file?.endsWith('.js')) continue;
const builtFile = await resolveBuiltFile(outDir, chunk.file);
const size = await fileSize(builtFile.absolutePath);
const stableKey = stableChunkKey(key, chunk);
const displayName = chunk.src ?? chunk.name ?? key;
byKey.set(stableKey, {
key: stableKey,
displayName,
file: builtFile.relativePath,
size,
});
byFile.add(builtFile.relativePath);
}
const localeDir = path.join(outDir, locale);
if (await exists(localeDir)) {
for await (const fullPath of walk(localeDir)) {
if (!fullPath.endsWith('.js')) continue;
const relativePath = normalizePath(path.relative(outDir, fullPath));
if (byFile.has(relativePath)) continue;
const size = await fileSize(fullPath);
byKey.set(relativePath, {
key: relativePath,
displayName: relativePath,
file: relativePath,
size,
});
}
}
return {
manifest,
chunks: Object.fromEntries(byKey),
startupKeys: [...collectStartupKeys(manifest)],
};
}
function commonKeys(before, after) {
return Object.keys(before.chunks)
.filter((key) => after.chunks[key] != null);
}
function addedKeys(before, after) {
return Object.keys(after.chunks)
.filter((key) => before.chunks[key] == null);
}
function removedKeys(before, after) {
return Object.keys(before.chunks)
.filter((key) => after.chunks[key] == null);
}
function rowChangeType(beforeEntry, afterEntry, beforeSize, afterSize) {
if (beforeEntry == null) return 'added';
if (afterEntry == null) return 'removed';
if (beforeSize !== afterSize) return 'updated';
return 'unchanged';
}
function getChunkComparisonRows(keys, before, after) {
return keys.map((key) => {
const beforeEntry = before.chunks[key];
const afterEntry = after.chunks[key];
const beforeSize = beforeEntry?.size ?? 0;
const afterSize = afterEntry?.size ?? 0;
return {
key,
name: entryDisplayName(beforeEntry ?? afterEntry),
chunkFile: beforeEntry?.file ?? afterEntry?.file,
beforeSize,
afterSize,
changeType: rowChangeType(beforeEntry, afterEntry, beforeSize, afterSize),
sortSize: Math.max(beforeSize, afterSize),
};
});
}
function summarizeChanges(rows) {
return {
updated: rows.filter((row) => row.changeType === 'updated').length,
added: rows.filter((row) => row.changeType === 'added').length,
removed: rows.filter((row) => row.changeType === 'removed').length,
};
}
function formatChangeSummary(label, summary) {
return `${label} (${summary.updated} updated, ${summary.added} added, ${summary.removed} removed)`;
}
function compareComparisonRows(a, b) {
return Math.abs(b.afterSize - b.beforeSize) - Math.abs(a.afterSize - a.beforeSize)
|| (b.afterSize - b.beforeSize) - (a.afterSize - a.beforeSize)
|| b.sortSize - a.sortSize
|| a.name.localeCompare(b.name);
}
function markdownTable(rows, total) {
if (rows.length === 0) return '_No data_';
const lines = [
'| Chunk | Before | After | Δ | Δ (%) |',
'| --- | ---: | ---: | ---: | ---: |',
];
if (total != null) {
lines.push(`| (total) | ${formatBytes(total.beforeSize)} | ${formatBytes(total.afterSize)} | ${formatDiff(total.afterSize - total.beforeSize)} | ${formatDiffPercent(total.beforeSize, total.afterSize)} |`);
lines.push('| | | | | |');
}
for (const row of rows) {
if (row.changeType === 'added') {
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${formatBytes(row.beforeSize)} | ${formatBytes(row.afterSize)} | ${formatDiff(row.afterSize - row.beforeSize)} | $\\color{orange}{\\text{(+)}}$ |`);
} else if (row.changeType === 'removed') {
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${formatBytes(row.beforeSize)} | ${formatBytes(row.afterSize)} | ${formatDiff(row.afterSize - row.beforeSize)} | $\\color{green}{\\text{(-)}}$ |`);
} else {
lines.push(`| <details><summary>\`${escapeCell(row.name)}\`</summary> \`${escapeCell(row.chunkFile)}\` </details> | ${formatBytes(row.beforeSize)} | ${formatBytes(row.afterSize)} | ${formatDiff(row.afterSize - row.beforeSize)} | ${formatDiffPercent(row.beforeSize, row.afterSize)} |`);
}
}
return lines.join('\n');
}
const beforeDir = process.argv[2];
const afterDir = process.argv[3];
const outFile = process.argv[4];
const beforeSha = process.env.BASE_SHA;
const afterSha = process.env.HEAD_SHA;
const before = await collectReport(beforeDir);
const after = await collectReport(afterDir);
const commonChunkKeys = commonKeys(before, after);
const allChunkKeys = [
...commonChunkKeys,
...addedKeys(before, after),
...removedKeys(before, after),
];
//const comparisonRows = getChunkComparisonRows(commonChunkKeys, before, after);
const allComparisonRows = getChunkComparisonRows(allChunkKeys, before, after);
const changedRows = allComparisonRows.filter((row) => row.changeType !== 'unchanged');
const diffSummary = summarizeChanges(changedRows);
const diffTotal = {
beforeSize: allComparisonRows.reduce((sum, row) => sum + row.beforeSize, 0),
afterSize: allComparisonRows.reduce((sum, row) => sum + row.afterSize, 0),
};
const diffRows = changedRows.sort(compareComparisonRows).slice(0, 30); // TODO: 実際に30を超えて切り捨てられたrowがあった場合はその旨をmarkdown内に表示するようにする
const startupKeys = new Set([
...before.startupKeys,
...after.startupKeys,
]);
const startupComparisonRows = getChunkComparisonRows([...startupKeys], before, after);
const startupRows = startupComparisonRows
.sort(compareComparisonRows);
const startupSummary = summarizeChanges(startupComparisonRows);
const startupTotal = {
beforeSize: startupComparisonRows.reduce((sum, row) => sum + row.beforeSize, 0),
afterSize: startupComparisonRows.reduce((sum, row) => sum + row.afterSize, 0),
};
//const largeRows = comparisonRows
// .sort((a, b) => b.sortSize - a.sortSize || a.name.localeCompare(b.name))
// .slice(0, 30);
const body = [
marker,
`## Frontend Chunk Report`,
'',
'<details open>',
`<summary>${formatChangeSummary('Diffs', diffSummary)}</summary>`,
'',
markdownTable(diffRows, diffTotal),
'',
'</details>',
'',
'<details>',
`<summary>${formatChangeSummary('Startup', startupSummary)}</summary>`,
'',
markdownTable(startupRows, startupTotal),
'',
`_Startup chunks are the Vite entry for \`src/_boot_.ts\` and its static imports._`,
'',
'</details>',
'',
//'<details>',
//`<summary>Largest</summary>`,
//'',
//markdownTable(largeRows),
//'',
//'</details>',
//'',
].join('\n');
await fs.writeFile(outFile, body);

View File

@@ -0,0 +1,224 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
import { spawn } from 'node:child_process';
import { createRequire } from 'node:module';
import { writeFile } from 'node:fs/promises';
import { join, resolve } from 'node:path';
const phases = ['beforeGc', 'afterGc', 'afterRequest'];
const [baseDirArg, headDirArg, baseOutputArg, headOutputArg] = process.argv.slice(2);
if (baseDirArg == null || headDirArg == null || baseOutputArg == null || headOutputArg == null) {
console.error('Usage: node .github/scripts/measure-backend-memory-comparison.mjs <base-dir> <head-dir> <base-output.json> <head-output.json>');
process.exit(1);
}
function readIntegerEnv(name, defaultValue, min) {
const rawValue = process.env[name];
if (rawValue == null || rawValue === '') return defaultValue;
if (!/^\d+$/.test(rawValue)) throw new Error(`${name} must be an integer`);
const value = Number(rawValue);
if (!Number.isSafeInteger(value) || value < min) throw new Error(`${name} must be >= ${min}`);
return value;
}
function commandName(command) {
if (process.platform !== 'win32') return command;
if (command === 'pnpm') return 'pnpm.cmd';
return command;
}
function run(command, args, options = {}) {
return new Promise((resolvePromise, reject) => {
const child = spawn(commandName(command), args, {
cwd: options.cwd,
env: options.env,
stdio: ['ignore', 'pipe', 'pipe'],
});
let stdout = '';
let stderr = '';
child.stdout.on('data', data => {
stdout += data;
if (options.logStdout) process.stderr.write(data);
});
child.stderr.on('data', data => {
stderr += data;
process.stderr.write(data);
});
child.on('error', reject);
child.on('close', code => {
if (code === 0) {
resolvePromise(stdout);
} else {
reject(new Error(`${command} ${args.join(' ')} failed with exit code ${code}\n${stderr}`));
}
});
});
}
async function resetState(repoDir) {
const require = createRequire(join(repoDir, 'packages/backend/package.json'));
const pg = require('pg');
const Redis = require('ioredis');
const postgres = new pg.Client({
host: '127.0.0.1',
port: 54312,
database: 'postgres',
user: 'postgres',
});
await postgres.connect();
try {
await postgres.query('DROP DATABASE IF EXISTS "test-misskey" WITH (FORCE)');
await postgres.query('CREATE DATABASE "test-misskey"');
} finally {
await postgres.end();
}
const redis = new Redis({ host: '127.0.0.1', port: 56312 });
try {
await redis.flushall();
} finally {
redis.disconnect();
}
}
function median(values) {
const sorted = values.toSorted((a, b) => a - b);
const center = Math.floor(sorted.length / 2);
if (sorted.length % 2 === 1) return sorted[center];
return Math.round((sorted[center - 1] + sorted[center]) / 2);
}
function summarizeSamples(samples) {
const summary = {};
for (const phase of phases) {
summary[phase] = {};
const metricKeys = new Set();
for (const sample of samples) {
for (const key of Object.keys(sample[phase] ?? {})) {
metricKeys.add(key);
}
}
for (const key of metricKeys) {
const values = samples
.map(sample => sample[phase]?.[key])
.filter(value => Number.isFinite(value));
if (values.length > 0) summary[phase][key] = median(values);
}
}
return summary;
}
async function measureRepo(label, repoDir, round, orderIndex) {
process.stderr.write(`[${label}] Resetting database and Redis\n`);
await resetState(repoDir);
process.stderr.write(`[${label}] Running migrations\n`);
await run('pnpm', ['--filter', 'backend', 'migrate'], {
cwd: repoDir,
env: process.env,
logStdout: true,
});
process.stderr.write(`[${label}] Measuring memory\n`);
const stdout = await run('node', ['packages/backend/scripts/measure-memory.mjs'], {
cwd: repoDir,
env: {
...process.env,
MK_MEMORY_SAMPLE_COUNT: '1',
},
});
const report = JSON.parse(stdout);
const sample = report.samples?.[0] ?? {
timestamp: report.timestamp,
beforeGc: report.beforeGc,
afterGc: report.afterGc,
afterRequest: report.afterRequest,
};
return {
...sample,
label,
round,
orderIndex,
};
}
async function main() {
const baseDir = resolve(baseDirArg);
const headDir = resolve(headDirArg);
const baseOutput = resolve(baseOutputArg);
const headOutput = resolve(headOutputArg);
const rounds = readIntegerEnv('MK_MEMORY_COMPARE_ROUNDS', 5, 1);
const warmupRounds = readIntegerEnv('MK_MEMORY_COMPARE_WARMUP_ROUNDS', 1, 0);
const startedAt = new Date().toISOString();
const repos = {
base: {
dir: baseDir,
samples: [],
},
head: {
dir: headDir,
samples: [],
},
};
for (let round = 1; round <= warmupRounds; round++) {
process.stderr.write(`Starting warmup round ${round}/${warmupRounds}\n`);
for (const label of ['base', 'head']) {
await measureRepo(label, repos[label].dir, -round, 0);
}
}
for (let round = 1; round <= rounds; round++) {
const order = round % 2 === 1 ? ['base', 'head'] : ['head', 'base'];
process.stderr.write(`Starting measurement round ${round}/${rounds}: ${order.join(' -> ')}\n`);
for (const [orderIndex, label] of order.entries()) {
const sample = await measureRepo(label, repos[label].dir, round, orderIndex);
repos[label].samples.push(sample);
}
}
for (const label of ['base', 'head']) {
const report = {
timestamp: new Date().toISOString(),
sampleCount: repos[label].samples.length,
aggregation: 'median',
comparison: {
strategy: 'interleaved-pairs',
rounds,
warmupRounds,
startedAt,
},
...summarizeSamples(repos[label].samples),
samples: repos[label].samples,
};
await writeFile(label === 'base' ? baseOutput : headOutput, `${JSON.stringify(report, null, 2)}\n`);
}
}
main().catch(err => {
console.error(err);
process.exit(1);
});

View File

@@ -0,0 +1,327 @@
name: frontend-bundle-report-comment
on:
workflow_run:
workflows:
- frontend-bundle-report
types:
- completed
pull_request_target:
types:
- opened
- synchronize
- reopened
- ready_for_review
paths:
- packages/frontend/**
- packages/frontend-shared/**
- packages/frontend-builder/**
- packages/i18n/**
- packages/icons-subsetter/**
- packages/misskey-js/**
- packages/misskey-reversi/**
- packages/misskey-bubble-game/**
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- .node-version
- .github/scripts/frontend-js-size.mjs
- .github/scripts/frontend-bundle-visualizer-report.mjs
- .github/workflows/frontend-bundle-report.yml
- .github/workflows/frontend-bundle-report-comment.yml
permissions:
actions: read
contents: read
issues: write
pull-requests: write
jobs:
comment:
name: Comment frontend bundle report
if: github.event_name == 'pull_request_target' || (github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion == 'success')
runs-on: ubuntu-latest
concurrency:
group: frontend-bundle-report-comment-${{ github.event.pull_request.number || github.event.workflow_run.id }}
cancel-in-progress: true
steps:
- name: Find bundle report run
if: github.event_name == 'pull_request_target'
id: find-report-run
uses: actions/github-script@v9
with:
script: |
const workflow_id = 'frontend-bundle-report.yml';
const artifactName = 'frontend-bundle-report';
const headSha = context.payload.pull_request.head.sha;
const prNumber = context.payload.pull_request.number;
const pollIntervalMs = 30_000;
const timeoutMs = 90 * 60_000;
const startedAt = Date.now();
const { owner, repo } = context.repo;
async function listReportWorkflowRuns() {
const runsForHead = await github.paginate(github.rest.actions.listWorkflowRuns, {
owner,
repo,
workflow_id,
event: 'pull_request',
head_sha: headSha,
per_page: 100,
});
if (runsForHead.length > 0) {
return runsForHead;
}
const recentRuns = await github.paginate(github.rest.actions.listWorkflowRuns, {
owner,
repo,
workflow_id,
event: 'pull_request',
per_page: 100,
});
return recentRuns.filter((run) =>
run.pull_requests?.some((pullRequest) => pullRequest.number === prNumber));
}
async function findReportRun() {
const runs = (await listReportWorkflowRuns())
.sort((a, b) => new Date(b.updated_at) - new Date(a.updated_at));
for (const run of runs) {
if (run.status !== 'completed') continue;
if (run.conclusion !== 'success') {
core.warning(`Frontend bundle report run ${run.id} completed with conclusion: ${run.conclusion}`);
return { done: true, run: null };
}
const artifacts = await github.paginate(github.rest.actions.listWorkflowRunArtifacts, {
owner,
repo,
run_id: run.id,
per_page: 100,
});
const report = artifacts.find((artifact) => artifact.name === artifactName && !artifact.expired);
if (report) return { done: true, run };
core.info(`Frontend bundle report run ${run.id} did not produce ${artifactName}.`);
return { done: true, run: null };
}
return { done: false, run: null };
}
while (Date.now() - startedAt < timeoutMs) {
const { done, run } = await findReportRun();
if (run) {
core.info(`Found frontend bundle report on workflow run ${run.id}.`);
core.setOutput('run-id', String(run.id));
return;
}
if (done) {
return;
}
core.info('Waiting for frontend bundle report artifact...');
await new Promise((resolve) => setTimeout(resolve, pollIntervalMs));
}
core.warning(`Timed out waiting for ${artifactName} from ${workflow_id} for ${headSha}.`);
- name: Find bundle report artifact
if: github.event_name == 'workflow_run'
id: find-report-artifact
uses: actions/github-script@v9
with:
script: |
const artifactName = 'frontend-bundle-report';
const { owner, repo } = context.repo;
const artifacts = await github.paginate(github.rest.actions.listWorkflowRunArtifacts, {
owner,
repo,
run_id: context.payload.workflow_run.id,
per_page: 100,
});
const report = artifacts.find((artifact) => artifact.name === artifactName && !artifact.expired);
if (report) {
core.setOutput('exists', 'true');
} else {
core.info(`Workflow run ${context.payload.workflow_run.id} did not produce ${artifactName}.`);
core.setOutput('exists', 'false');
}
- name: Download bundle report from workflow_run
if: github.event_name == 'workflow_run' && steps.find-report-artifact.outputs.exists == 'true'
uses: actions/download-artifact@v8
with:
name: frontend-bundle-report
path: ${{ runner.temp }}/frontend-bundle-report
github-token: ${{ github.token }}
repository: ${{ github.repository }}
run-id: ${{ github.event.workflow_run.id }}
- name: Download bundle report from pull_request_target
if: github.event_name == 'pull_request_target' && steps.find-report-run.outputs.run-id != ''
uses: actions/download-artifact@v8
with:
name: frontend-bundle-report
path: ${{ runner.temp }}/frontend-bundle-report
github-token: ${{ github.token }}
repository: ${{ github.repository }}
run-id: ${{ steps.find-report-run.outputs.run-id }}
- name: Comment on pull request
if: (github.event_name == 'workflow_run' && steps.find-report-artifact.outputs.exists == 'true') || steps.find-report-run.outputs.run-id != ''
uses: actions/github-script@v9
with:
github-token: ${{ secrets.FRONTEND_BUNDLE_REPORT_COMMENT_TOKEN || secrets.FRONTEND_JS_SIZE_COMMENT_TOKEN || secrets.FRONTEND_BUNDLE_VISUALIZER_COMMENT_TOKEN || github.token }}
script: |
const fs = require('node:fs');
const path = require('node:path');
const jsSizeMarker = '<!-- misskey-frontend-js-size -->';
const visualizerMarker = '<!-- misskey-frontend-bundle-visualizer -->';
const reportMarkers = [jsSizeMarker, visualizerMarker];
const reportDir = path.join(process.env.RUNNER_TEMP, 'frontend-bundle-report');
const jsSizeReportPath = path.join(reportDir, 'frontend-js-size-report.md');
const visualizerReportPath = path.join(reportDir, 'frontend-bundle-visualizer-report.md');
const prNumberPath = path.join(reportDir, 'pr-number.txt');
const headShaPath = path.join(reportDir, 'head-sha.txt');
const workflowRun = context.payload.workflow_run;
const pullRequest = context.payload.pull_request;
const eventHeadSha = workflowRun?.head_sha ?? pullRequest?.head?.sha ?? null;
const { owner, repo } = context.repo;
if (!fs.existsSync(jsSizeReportPath)) {
core.setFailed('The frontend bundle report artifact does not contain frontend-js-size-report.md.');
return;
}
if (!fs.existsSync(visualizerReportPath)) {
core.setFailed('The frontend bundle report artifact does not contain frontend-bundle-visualizer-report.md.');
return;
}
const artifactHeadSha = fs.existsSync(headShaPath)
? fs.readFileSync(headShaPath, 'utf8').trim()
: null;
if (eventHeadSha != null && artifactHeadSha != null && artifactHeadSha !== eventHeadSha) {
core.info(`The artifact head SHA (${artifactHeadSha}) differs from the event head SHA (${eventHeadSha}). Using artifact metadata for PR validation.`);
}
const reportHeadSha = artifactHeadSha ?? eventHeadSha;
const artifactPrNumber = fs.existsSync(prNumberPath)
? Number(fs.readFileSync(prNumberPath, 'utf8').trim())
: null;
let issue_number = null;
if (pullRequest != null) {
issue_number = pullRequest.number;
if (Number.isInteger(artifactPrNumber) && artifactPrNumber !== issue_number) {
core.setFailed(`The artifact pull request number (${artifactPrNumber}) does not match the event pull request number (${issue_number}).`);
return;
}
} else if (workflowRun != null) {
const associatedPullRequests = new Map();
for (const pullRequest of workflowRun.pull_requests ?? []) {
if (Number.isInteger(pullRequest.number)) {
associatedPullRequests.set(pullRequest.number, pullRequest);
}
}
if (reportHeadSha != null) {
const pullRequestsForCommit = await github.paginate(github.rest.repos.listPullRequestsAssociatedWithCommit, {
owner,
repo,
commit_sha: reportHeadSha,
per_page: 100,
});
for (const pullRequest of pullRequestsForCommit) {
associatedPullRequests.set(pullRequest.number, pullRequest);
}
}
if (Number.isInteger(artifactPrNumber) && associatedPullRequests.has(artifactPrNumber)) {
issue_number = artifactPrNumber;
} else if (Number.isInteger(artifactPrNumber) && associatedPullRequests.size === 0) {
issue_number = artifactPrNumber;
} else if (!Number.isInteger(artifactPrNumber) && associatedPullRequests.size === 1) {
issue_number = [...associatedPullRequests.keys()][0];
} else if (Number.isInteger(artifactPrNumber)) {
core.setFailed(`The artifact pull request number (${artifactPrNumber}) is not associated with ${reportHeadSha}.`);
return;
} else {
core.setFailed(`Could not determine the pull request associated with ${reportHeadSha}.`);
return;
}
} else {
core.setFailed('Could not determine the pull request event for this report.');
return;
}
const currentPullRequest = await github.rest.pulls.get({
owner,
repo,
pull_number: issue_number,
});
const currentHeadSha = currentPullRequest.data.head?.sha;
if (reportHeadSha != null && currentHeadSha != null && reportHeadSha !== currentHeadSha) {
core.info(`The report head SHA (${reportHeadSha}) is not the current pull request head SHA (${currentHeadSha}). Skipping stale frontend bundle report.`);
return;
}
const jsSizeReport = fs.readFileSync(jsSizeReportPath, 'utf8').trim();
if (!jsSizeReport.includes(jsSizeMarker)) {
core.setFailed('The frontend JS size report is missing the expected marker.');
return;
}
const visualizerReport = fs.readFileSync(visualizerReportPath, 'utf8').trim();
let body = [
jsSizeReport,
visualizerReport,
].join('\n\n') + '\n';
const maxCommentLength = 65_000;
if (body.length > maxCommentLength) {
const reportLocation = workflowRun?.html_url != null
? `[workflow run](${workflowRun.html_url})`
: 'workflow artifact';
const footer = [
'',
'',
`_Report truncated because it exceeded ${maxCommentLength.toLocaleString('en-US')} characters. See the ${reportLocation} for the full report._`,
].join('\n');
body = `${body.slice(0, maxCommentLength - footer.length)}${footer}`;
}
const comments = await github.paginate(github.rest.issues.listComments, {
owner,
repo,
issue_number,
per_page: 100,
});
const previousReports = comments.filter((comment) =>
comment.user?.type === 'Bot' && reportMarkers.some((reportMarker) => comment.body?.includes(reportMarker)));
if (previousReports.length > 0) {
const [previous, ...duplicates] = previousReports;
await github.rest.issues.updateComment({
owner,
repo,
comment_id: previous.id,
body,
});
for (const duplicate of duplicates) {
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: duplicate.id,
});
}
} else {
await github.rest.issues.createComment({
owner,
repo,
issue_number,
body,
});
}

View File

@@ -0,0 +1,163 @@
name: frontend-bundle-report
on:
pull_request:
types:
- opened
- synchronize
- reopened
- ready_for_review
paths:
- packages/frontend/**
- packages/frontend-shared/**
- packages/frontend-builder/**
- packages/i18n/**
- packages/icons-subsetter/**
- packages/misskey-js/**
- packages/misskey-reversi/**
- packages/misskey-bubble-game/**
- package.json
- pnpm-lock.yaml
- pnpm-workspace.yaml
- .node-version
- .github/scripts/frontend-js-size.mjs
- .github/scripts/frontend-bundle-visualizer-report.mjs
- .github/workflows/frontend-bundle-report.yml
- .github/workflows/frontend-bundle-report-comment.yml
permissions:
contents: read
pull-requests: read
concurrency:
group: frontend-bundle-report-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
report:
name: Build frontend bundle report
runs-on: ubuntu-latest
env:
FRONTEND_JS_SIZE_LOCALE: ja-JP
steps:
- name: Checkout base
uses: actions/checkout@v6.0.2
with:
repository: ${{ github.event.pull_request.base.repo.full_name }}
ref: ${{ github.event.pull_request.base.sha }}
path: before
submodules: true
- name: Checkout pull request
uses: actions/checkout@v6.0.2
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha }}
path: after
submodules: true
- name: Check base visualizer support
id: check-base-visualizer
shell: bash
run: |
if grep -q 'FRONTEND_BUNDLE_VISUALIZER' before/packages/frontend/vite.config.ts; then
echo 'supported=true' >> "$GITHUB_OUTPUT"
else
echo 'supported=false' >> "$GITHUB_OUTPUT"
echo 'Base commit does not support frontend bundle visualizer. Skipping frontend bundle report.' >> "$GITHUB_STEP_SUMMARY"
fi
- name: Setup pnpm
if: steps.check-base-visualizer.outputs.supported == 'true'
uses: pnpm/action-setup@v6.0.3
with:
package_json_file: after/package.json
- name: Setup Node.js
if: steps.check-base-visualizer.outputs.supported == 'true'
uses: actions/setup-node@v6.4.0
with:
node-version-file: after/.node-version
cache: pnpm
cache-dependency-path: |
before/pnpm-lock.yaml
after/pnpm-lock.yaml
- name: Install dependencies for base
if: steps.check-base-visualizer.outputs.supported == 'true'
working-directory: before
run: pnpm i --frozen-lockfile
- name: Build frontend dependencies for base
if: steps.check-base-visualizer.outputs.supported == 'true'
working-directory: before
run: pnpm --filter "frontend^..." run build
- name: Prepare report output
if: steps.check-base-visualizer.outputs.supported == 'true'
run: mkdir -p "$RUNNER_TEMP/frontend-bundle-report"
- name: Build frontend report for base
if: steps.check-base-visualizer.outputs.supported == 'true'
working-directory: before
env:
FRONTEND_BUNDLE_VISUALIZER: 'true'
FRONTEND_BUNDLE_VISUALIZER_TEMPLATE: raw-data
FRONTEND_BUNDLE_VISUALIZER_FILE: ${{ runner.temp }}/frontend-bundle-report/before-stats.json
run: pnpm --filter frontend run build
- name: Install dependencies for pull request
if: steps.check-base-visualizer.outputs.supported == 'true'
working-directory: after
run: pnpm i --frozen-lockfile
- name: Build frontend dependencies for pull request
if: steps.check-base-visualizer.outputs.supported == 'true'
working-directory: after
run: pnpm --filter "frontend^..." run build
- name: Build frontend report for pull request
if: steps.check-base-visualizer.outputs.supported == 'true'
working-directory: after
env:
FRONTEND_BUNDLE_VISUALIZER: 'true'
FRONTEND_BUNDLE_VISUALIZER_TEMPLATE: raw-data
FRONTEND_BUNDLE_VISUALIZER_FILE: ${{ runner.temp }}/frontend-bundle-report/after-stats.json
run: pnpm --filter frontend run build
- name: Generate report markdown
if: steps.check-base-visualizer.outputs.supported == 'true'
shell: bash
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
REPORT_DIR="$RUNNER_TEMP/frontend-bundle-report"
node after/.github/scripts/frontend-js-size.mjs before after "$REPORT_DIR/frontend-js-size-report.md"
node after/.github/scripts/frontend-bundle-visualizer-report.mjs "$REPORT_DIR/before-stats.json" "$REPORT_DIR/after-stats.json" "$REPORT_DIR/frontend-bundle-visualizer-report.md"
printf '%s\n' "$PR_NUMBER" > "$REPORT_DIR/pr-number.txt"
printf '%s\n' "$BASE_SHA" > "$REPORT_DIR/base-sha.txt"
printf '%s\n' "$HEAD_SHA" > "$REPORT_DIR/head-sha.txt"
printf '%s\n' "${{ github.event.pull_request.html_url }}" > "$REPORT_DIR/pr-url.txt"
- name: Check report
if: steps.check-base-visualizer.outputs.supported == 'true'
run: |
REPORT_DIR="$RUNNER_TEMP/frontend-bundle-report"
test -s "$REPORT_DIR/before-stats.json"
test -s "$REPORT_DIR/after-stats.json"
test -s "$REPORT_DIR/frontend-js-size-report.md"
test -s "$REPORT_DIR/frontend-bundle-visualizer-report.md"
cat "$REPORT_DIR/frontend-js-size-report.md" >> "$GITHUB_STEP_SUMMARY"
printf '\n\n' >> "$GITHUB_STEP_SUMMARY"
cat "$REPORT_DIR/frontend-bundle-visualizer-report.md" >> "$GITHUB_STEP_SUMMARY"
- name: Upload bundle report
if: steps.check-base-visualizer.outputs.supported == 'true'
uses: actions/upload-artifact@v7
with:
name: frontend-bundle-report
path: ${{ runner.temp }}/frontend-bundle-report/
if-no-files-found: error
retention-days: 7

View File

@@ -9,7 +9,13 @@ on:
paths:
- packages/backend/**
- packages/misskey-js/**
- .github/scripts/backend-memory-report.mjs
- .github/scripts/measure-backend-memory-comparison.mjs
- .github/scripts/backend-js-footprint.mjs
- .github/scripts/backend-js-footprint-loader.mjs
- .github/scripts/backend-js-footprint-require.cjs
- .github/workflows/get-backend-memory.yml
- .github/workflows/report-backend-memory.yml
jobs:
get-memory-usage:
@@ -17,15 +23,6 @@ jobs:
permissions:
contents: read
strategy:
matrix:
memory-json-name: [memory-base.json, memory-head.json]
include:
- memory-json-name: memory-base.json
ref: ${{ github.base_ref }}
- memory-json-name: memory-head.json
ref: refs/pull/${{ github.event.number }}/merge
services:
postgres:
image: postgres:18
@@ -40,37 +37,76 @@ jobs:
- 56312:6379
steps:
- uses: actions/checkout@v6.0.2
- name: Checkout base
uses: actions/checkout@v6.0.2
with:
ref: ${{ matrix.ref }}
ref: ${{ github.base_ref }}
path: base
submodules: true
- name: Checkout head
uses: actions/checkout@v6.0.2
with:
ref: refs/pull/${{ github.event.number }}/merge
path: head
submodules: true
- name: Setup pnpm
uses: pnpm/action-setup@v6.0.3
with:
package_json_file: head/package.json
- name: Use Node.js
uses: actions/setup-node@v6.4.0
with:
node-version-file: '.node-version'
node-version-file: 'head/.node-version'
cache: 'pnpm'
- run: pnpm i --frozen-lockfile
- name: Check pnpm-lock.yaml
cache-dependency-path: |
base/pnpm-lock.yaml
head/pnpm-lock.yaml
- name: Install base dependencies
working-directory: base
run: pnpm i --frozen-lockfile
- name: Check base pnpm-lock.yaml
working-directory: base
run: git diff --exit-code pnpm-lock.yaml
- name: Copy Configure
run: cp .github/misskey/test.yml .config/default.yml
- name: Compile Configure
run: pnpm compile-config
- name: Build
run: pnpm build
- name: Run migrations
run: pnpm --filter backend migrate
- name: Measure memory usage
- name: Configure base
working-directory: base
run: |
# Start the server and measure memory usage
node packages/backend/scripts/measure-memory.mjs > ${{ matrix.memory-json-name }}
cp .github/misskey/test.yml .config/default.yml
pnpm compile-config
- name: Build base
working-directory: base
run: pnpm build
- name: Install head dependencies
working-directory: head
run: pnpm i --frozen-lockfile
- name: Check head pnpm-lock.yaml
working-directory: head
run: git diff --exit-code pnpm-lock.yaml
- name: Configure head
working-directory: head
run: |
cp .github/misskey/test.yml .config/default.yml
pnpm compile-config
- name: Build head
working-directory: head
run: pnpm build
- name: Measure backend memory usage
env:
MK_MEMORY_COMPARE_ROUNDS: 5
MK_MEMORY_COMPARE_WARMUP_ROUNDS: 1
run: node head/.github/scripts/measure-backend-memory-comparison.mjs base head memory-base.json memory-head.json
- name: Measure backend loaded JS footprint
run: |
node head/.github/scripts/backend-js-footprint.mjs base js-footprint-base.json
node head/.github/scripts/backend-js-footprint.mjs head js-footprint-head.json
- name: Upload Artifact
uses: actions/upload-artifact@v7
with:
name: memory-artifact-${{ matrix.memory-json-name }}
path: ${{ matrix.memory-json-name }}
name: memory-artifact-results
path: |
memory-base.json
memory-head.json
js-footprint-base.json
js-footprint-head.json
save-pr-number:
runs-on: ubuntu-latest

View File

@@ -11,9 +11,14 @@ jobs:
runs-on: ubuntu-latest
if: ${{ github.event.workflow_run.conclusion == 'success' }}
permissions:
actions: read
contents: read
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
- name: Download artifact
uses: actions/github-script@v9
with:
@@ -48,120 +53,13 @@ jobs:
run: cat ./artifacts/memory-base.json
- name: Output head
run: cat ./artifacts/memory-head.json
- name: Compare memory usage
id: compare
run: |
BASE_MEMORY=$(cat ./artifacts/memory-base.json)
HEAD_MEMORY=$(cat ./artifacts/memory-head.json)
variation() {
calc() {
BASE=$(echo "$BASE_MEMORY" | jq -r ".${1}.${2} // 0")
HEAD=$(echo "$HEAD_MEMORY" | jq -r ".${1}.${2} // 0")
DIFF=$((HEAD - BASE))
if [ "$BASE" -gt 0 ]; then
DIFF_PERCENT=$(echo "scale=2; ($DIFF * 100) / $BASE" | bc)
else
DIFF_PERCENT=0
fi
# Convert KB to MB for readability
BASE_MB=$(echo "scale=2; $BASE / 1024" | bc)
HEAD_MB=$(echo "scale=2; $HEAD / 1024" | bc)
DIFF_MB=$(echo "scale=2; $DIFF / 1024" | bc)
JSON=$(jq -c -n \
--argjson base "$BASE_MB" \
--argjson head "$HEAD_MB" \
--argjson diff "$DIFF_MB" \
--argjson diff_percent "$DIFF_PERCENT" \
'{base: $base, head: $head, diff: $diff, diff_percent: $diff_percent}')
echo "$JSON"
}
JSON=$(jq -c -n \
--argjson VmRSS "$(calc $1 VmRSS)" \
--argjson VmHWM "$(calc $1 VmHWM)" \
--argjson VmSize "$(calc $1 VmSize)" \
--argjson VmData "$(calc $1 VmData)" \
'{VmRSS: $VmRSS, VmHWM: $VmHWM, VmSize: $VmSize, VmData: $VmData}')
echo "$JSON"
}
JSON=$(jq -c -n \
--argjson beforeGc "$(variation beforeGc)" \
--argjson afterGc "$(variation afterGc)" \
--argjson afterRequest "$(variation afterRequest)" \
'{beforeGc: $beforeGc, afterGc: $afterGc, afterRequest: $afterRequest}')
echo "res=$JSON" >> "$GITHUB_OUTPUT"
- name: Output base JS footprint
run: cat ./artifacts/js-footprint-base.json
- name: Output head JS footprint
run: cat ./artifacts/js-footprint-head.json
- id: build-comment
name: Build memory comment
env:
RES: ${{ steps.compare.outputs.res }}
run: |
HEADER="## Backend memory usage comparison"
FOOTER="[See workflow logs for details](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID})"
echo "$HEADER" > ./output.md
echo >> ./output.md
table() {
echo "| Metric | base (MB) | head (MB) | Diff (MB) | Diff (%) |" >> ./output.md
echo "|--------|------:|------:|------:|------:|" >> ./output.md
line() {
METRIC=$2
BASE=$(echo "$RES" | jq -r ".${1}.${2}.base")
HEAD=$(echo "$RES" | jq -r ".${1}.${2}.head")
DIFF=$(echo "$RES" | jq -r ".${1}.${2}.diff")
DIFF_PERCENT=$(echo "$RES" | jq -r ".${1}.${2}.diff_percent")
if (( $(echo "$DIFF_PERCENT > 0" | bc -l) )); then
DIFF="+$DIFF"
DIFF_PERCENT="+$DIFF_PERCENT"
fi
# highlight VmRSS
if [ "$2" = "VmRSS" ]; then
METRIC="**${METRIC}**"
BASE="**${BASE}**"
HEAD="**${HEAD}**"
DIFF="**${DIFF}**"
DIFF_PERCENT="**${DIFF_PERCENT}**"
fi
echo "| ${METRIC} | ${BASE} MB | ${HEAD} MB | ${DIFF} MB | ${DIFF_PERCENT}% |" >> ./output.md
}
line $1 VmRSS
line $1 VmHWM
line $1 VmSize
line $1 VmData
}
echo "### Before GC" >> ./output.md
table beforeGc
echo >> ./output.md
echo "### After GC" >> ./output.md
table afterGc
echo >> ./output.md
echo "### After Request" >> ./output.md
table afterRequest
echo >> ./output.md
# Determine if this is a significant change (more than 5% increase)
if [ "$(echo "$RES" | jq -r '.afterGc.VmRSS.diff_percent | tonumber > 5')" = "true" ]; then
echo "⚠️ **Warning**: Memory usage has increased by more than 5%. Please verify this is not an unintended change." >> ./output.md
echo >> ./output.md
fi
echo "$FOOTER" >> ./output.md
run: node .github/scripts/backend-memory-report.mjs ./artifacts/memory-base.json ./artifacts/memory-head.json ./output.md ./artifacts/js-footprint-base.json ./artifacts/js-footprint-head.json
- uses: thollander/actions-comment-pull-request@v3
with:
pr-number: ${{ steps.load-pr-num.outputs.pr-number }}

View File

@@ -1,3 +1,15 @@
## Unreleased
### General
-
### Client
-
### Server
-
## 2026.6.0
### General
@@ -25,6 +37,7 @@
- Enhance: リモートノートクリーニングジョブのスキップ処理のパフォーマンス改善
- Enhance: リモートノートクリーニングジョブの削除対象検索処理のパフォーマンス改善
- Enhance: ActivityPub の画像添付に width/height を含めるように
- Enhance: URLプレビューのデフォルトの User Agent に Misskey サーバーのURLを含めるように
- Fix: backend バンドルで `@tensorflow/tfjs-node` を external に含めず、起動時に `@mapbox/node-pre-gyp``find()` が backend の package.json を誤検出して `is not node-pre-gyp ready` エラーを永続的に吐く問題を修正
- Fix: MemoryKVCacheのキャッシュGC処理において、更新されたキャッシュが期限切れにならないことがある問題を修正
- Fix: PerUserDriveChart がシステム所有ファイル (userId が null) の更新で `"group"` の非NULL制約違反によりクラッシュする問題を修正 (#17498)
@@ -33,6 +46,7 @@
- Fix: `actor` を持たない不正なInboxアクティビティを受信した際に配送ジョブが `TypeError` でクラッシュする問題を修正 (受信時に検証して400で返し、ジョブを積まないように変更)
- Fix: Startup and shutdown failures (port-in-use, socket permission denied, plugin timeouts, leaked WebSocket connections) are now reported through the misskey logger instead of an UnhandledPromiseRejectionWarning stack trace
- Fix: リモートのノートに対するメンション数制限が、サーバーが解決できたユーザー数ベースで行われていた問題を修正
- Fix: セキュリティに関する修正
## 2026.5.4

View File

@@ -1,6 +1,6 @@
{
"name": "misskey",
"version": "2026.6.0-beta.1",
"version": "2026.6.0",
"codename": "nasubi",
"repository": {
"type": "git",

View File

@@ -64,7 +64,7 @@
"@misskey-dev/emoji-assets": "17.0.3",
"@misskey-dev/emoji-data": "17.0.3",
"@misskey-dev/sharp-read-bmp": "1.2.0",
"@misskey-dev/summaly": "5.3.0",
"@misskey-dev/summaly": "5.5.1",
"@napi-rs/canvas": "1.0.0",
"@nestjs/common": "11.1.26",
"@nestjs/core": "11.1.26",

View File

@@ -1,4 +1,5 @@
import { defineConfig } from 'rolldown';
import { version as summalyVersion } from '@misskey-dev/summaly';
import type { Plugin, ExternalOption } from 'rolldown';
import { execa, execaNode } from 'execa';
import type { ResultPromise } from 'execa';
@@ -84,6 +85,11 @@ export default defineConfig((args) => {
'file-type',
];
const define: Record<string, string> = {
// Summalyのバージョンを埋め込む
'_SUMMALY_VERSION_': JSON.stringify(summalyVersion),
};
if (isE2E) {
return {
input: './test-server/entry.ts',
@@ -92,6 +98,9 @@ export default defineConfig((args) => {
plugins: [
esmShim(),
],
transform: {
define,
},
output: {
keepNames: true,
sourcemap: true,
@@ -116,6 +125,9 @@ export default defineConfig((args) => {
esmShim(),
(isWatchMode ? backendDevServerPlugin() : undefined),
],
transform: {
define,
},
output: {
keepNames: true,
minify: !isWatchMode,

View File

@@ -20,11 +20,23 @@ import * as fs from 'node:fs/promises';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const SAMPLE_COUNT = 3; // Number of samples to measure
const STARTUP_TIMEOUT = 120000; // 120 seconds timeout for server startup
const MEMORY_SETTLE_TIME = 10000; // Wait 10 seconds after startup for memory to settle
function readIntegerEnv(name, defaultValue, min) {
const rawValue = process.env[name];
if (rawValue == null || rawValue === '') return defaultValue;
if (!/^\d+$/.test(rawValue)) throw new Error(`${name} must be an integer`);
const keys = {
const value = Number(rawValue);
if (!Number.isSafeInteger(value) || value < min) throw new Error(`${name} must be >= ${min}`);
return value;
}
const SAMPLE_COUNT = readIntegerEnv('MK_MEMORY_SAMPLE_COUNT', 3, 1); // Number of samples to measure
const STARTUP_TIMEOUT = readIntegerEnv('MK_MEMORY_STARTUP_TIMEOUT_MS', 120000, 1); // Timeout for server startup
const MEMORY_SETTLE_TIME = readIntegerEnv('MK_MEMORY_SETTLE_TIME_MS', 10000, 0); // Wait after startup for memory to settle
const IPC_TIMEOUT = readIntegerEnv('MK_MEMORY_IPC_TIMEOUT_MS', 30000, 1); // Timeout for IPC responses
const REQUEST_COUNT = readIntegerEnv('MK_MEMORY_REQUEST_COUNT', 10, 0);
const procStatusKeys = {
VmPeak: 0,
VmSize: 0,
VmHWM: 0,
@@ -37,30 +49,152 @@ const keys = {
VmSwap: 0,
};
async function getMemoryUsage(pid) {
const status = await fs.readFile(`/proc/${pid}/status`, 'utf-8');
const smapsRollupKeys = {
Pss: 0,
Shared_Clean: 0,
Shared_Dirty: 0,
Private_Clean: 0,
Private_Dirty: 0,
Swap: 0,
SwapPss: 0,
};
const runtimeKeys = {
HeapTotal: 0,
HeapUsed: 0,
External: 0,
ArrayBuffers: 0,
};
const memoryKeys = {
...procStatusKeys,
...smapsRollupKeys,
...runtimeKeys,
};
const phases = ['beforeGc', 'afterGc', 'afterRequest'];
function parseMemoryFile(content, keys, path, required) {
const result = {};
for (const key of Object.keys(keys)) {
const match = status.match(new RegExp(`${key}:\\s+(\\d+)\\s+kB`));
const match = content.match(new RegExp(`${key}:\\s+(\\d+)\\s+kB`));
if (match) {
result[key] = parseInt(match[1], 10);
} else {
throw new Error(`Failed to parse ${key} from /proc/${pid}/status`);
} else if (required) {
throw new Error(`Failed to parse ${key} from ${path}`);
}
}
return result;
}
function bytesToKiB(value) {
return Math.round(value / 1024);
}
async function getMemoryUsage(pid) {
const path = `/proc/${pid}/status`;
const status = await fs.readFile(path, 'utf-8');
return parseMemoryFile(status, procStatusKeys, path, true);
}
async function getSmapsRollupMemoryUsage(pid) {
const path = `/proc/${pid}/smaps_rollup`;
try {
const smapsRollup = await fs.readFile(path, 'utf-8');
return parseMemoryFile(smapsRollup, smapsRollupKeys, path, false);
} catch (err) {
if (err.code === 'ENOENT' || err.code === 'EACCES') {
process.stderr.write(`Failed to read ${path}: ${err.message}\n`);
return {};
}
throw err;
}
}
function waitForMessage(serverProcess, predicate, description, timeout = IPC_TIMEOUT) {
return new Promise((resolve, reject) => {
const timer = globalThis.setTimeout(() => {
serverProcess.off('message', onMessage);
reject(new Error(`Timed out waiting for ${description}`));
}, timeout);
const onMessage = (message) => {
if (!predicate(message)) return;
globalThis.clearTimeout(timer);
serverProcess.off('message', onMessage);
resolve(message);
};
serverProcess.on('message', onMessage);
});
}
async function getRuntimeMemoryUsage(serverProcess) {
const response = waitForMessage(
serverProcess,
message => message != null && typeof message === 'object' && message.type === 'memory usage',
'memory usage',
);
serverProcess.send('memory usage');
const message = await response;
const memoryUsage = message.value;
return {
HeapTotal: bytesToKiB(memoryUsage.heapTotal),
HeapUsed: bytesToKiB(memoryUsage.heapUsed),
External: bytesToKiB(memoryUsage.external),
ArrayBuffers: bytesToKiB(memoryUsage.arrayBuffers),
};
}
async function getAllMemoryUsage(serverProcess) {
const pid = serverProcess.pid;
return {
...await getMemoryUsage(pid),
...await getSmapsRollupMemoryUsage(pid),
...await getRuntimeMemoryUsage(serverProcess),
};
}
function median(values) {
const sorted = values.toSorted((a, b) => a - b);
const center = Math.floor(sorted.length / 2);
if (sorted.length % 2 === 1) return sorted[center];
return Math.round((sorted[center - 1] + sorted[center]) / 2);
}
function summarizeResults(results) {
const summary = {};
for (const phase of phases) {
summary[phase] = {};
for (const key of Object.keys(memoryKeys)) {
const values = results
.map(result => result[phase][key])
.filter(value => Number.isFinite(value));
if (values.length > 0) {
summary[phase][key] = median(values);
}
}
}
return result;
return summary;
}
async function measureMemory() {
// Start the Misskey backend server using fork to enable IPC
const serverProcess = fork(join(__dirname, '../built/entry.js'), ['expose-gc'], {
const serverProcess = fork(join(__dirname, '../built/entry.js'), [], {
cwd: join(__dirname, '..'),
env: {
...process.env,
NODE_ENV: 'production',
MK_DISABLE_CLUSTERING: '1',
MK_ONLY_SERVER: '1',
MK_NO_DAEMONS: '1',
},
stdio: ['pipe', 'pipe', 'pipe', 'ipc'],
execArgv: [...process.execArgv, '--expose-gc'],
@@ -90,15 +224,18 @@ async function measureMemory() {
});
async function triggerGc() {
const ok = new Promise((resolve) => {
serverProcess.once('message', (message) => {
if (message === 'gc ok') resolve();
});
});
const ok = waitForMessage(
serverProcess,
message => message === 'gc ok' || message === 'gc unavailable',
'GC completion',
);
serverProcess.send('gc');
await ok;
const message = await ok;
if (message === 'gc unavailable') {
throw new Error('GC is unavailable. Start the process with --expose-gc to enable this feature.');
}
await setTimeout(1000);
}
@@ -139,23 +276,20 @@ async function measureMemory() {
// Wait for memory to settle
await setTimeout(MEMORY_SETTLE_TIME);
const pid = serverProcess.pid;
const beforeGc = await getMemoryUsage(pid);
const beforeGc = await getAllMemoryUsage(serverProcess);
await triggerGc();
const afterGc = await getMemoryUsage(pid);
const afterGc = await getAllMemoryUsage(serverProcess);
// create some http requests to simulate load
const REQUEST_COUNT = 10;
await Promise.all(
Array.from({ length: REQUEST_COUNT }).map(() => createRequest()),
);
await triggerGc();
const afterRequest = await getMemoryUsage(pid);
const afterRequest = await getAllMemoryUsage(serverProcess);
// Stop the server
serverProcess.kill('SIGTERM');
@@ -187,35 +321,27 @@ async function measureMemory() {
}
async function main() {
// 直列の方が時間的に分散されて正確そうだから直列でやる
const results = [];
for (let i = 0; i < SAMPLE_COUNT; i++) {
process.stderr.write(`Starting sample ${i + 1}/${SAMPLE_COUNT}\n`);
const res = await measureMemory();
results.push(res);
}
// Calculate averages
const beforeGc = structuredClone(keys);
const afterGc = structuredClone(keys);
const afterRequest = structuredClone(keys);
for (const res of results) {
for (const key of Object.keys(keys)) {
beforeGc[key] += res.beforeGc[key];
afterGc[key] += res.afterGc[key];
afterRequest[key] += res.afterRequest[key];
}
}
for (const key of Object.keys(keys)) {
beforeGc[key] = Math.round(beforeGc[key] / SAMPLE_COUNT);
afterGc[key] = Math.round(afterGc[key] / SAMPLE_COUNT);
afterRequest[key] = Math.round(afterRequest[key] / SAMPLE_COUNT);
}
const summary = summarizeResults(results);
const result = {
timestamp: new Date().toISOString(),
beforeGc,
afterGc,
afterRequest,
sampleCount: SAMPLE_COUNT,
aggregation: 'median',
measurement: {
startupTimeoutMs: STARTUP_TIMEOUT,
memorySettleTimeMs: MEMORY_SETTLE_TIME,
ipcTimeoutMs: IPC_TIMEOUT,
requestCount: REQUEST_COUNT,
},
...summary,
samples: results,
};
// Output as JSON to stdout

View File

@@ -0,0 +1,6 @@
/*
* SPDX-FileCopyrightText: syuilo and misskey-project
* SPDX-License-Identifier: AGPL-3.0-only
*/
declare const _SUMMALY_VERSION_: string;

View File

@@ -6,6 +6,7 @@
import { NestFactory } from '@nestjs/core';
import { init } from 'slacc';
import { NestLogger } from '@/NestLogger.js';
import { envOption } from '@/env.js';
import type { Config } from '@/config.js';
let slaccInitialized = false;
@@ -31,7 +32,7 @@ export async function server() {
const serverService = app.get(ServerService);
await serverService.launch();
if (process.env.NODE_ENV !== 'test') {
if (process.env.NODE_ENV !== 'test' && !envOption.noDaemons) {
const { ChartManagementService } = await import('../core/chart/ChartManagementService.js');
const { QueueStatsService } = await import('../daemons/QueueStatsService.js');
const { ServerStatsService } = await import('../daemons/ServerStatsService.js');
@@ -54,7 +55,9 @@ export async function jobQueue() {
});
jobQueue.get(QueueProcessorService).start();
jobQueue.get(ChartManagementService).start();
if (!envOption.noDaemons) {
jobQueue.get(ChartManagementService).start();
}
return jobQueue;
}

View File

@@ -91,10 +91,20 @@ process.on('message', msg => {
if (msg === 'gc') {
if (global.gc != null) {
logger.info('Manual GC triggered');
global.gc();
for (let i = 0; i < 3; i++) {
global.gc();
}
if (process.send != null) process.send('gc ok');
} else {
logger.warn('Manual GC requested but gc is not available. Start the process with --expose-gc to enable this feature.');
if (process.send != null) process.send('gc unavailable');
}
} else if (msg === 'memory usage') {
if (process.send != null) {
process.send({
type: 'memory usage',
value: process.memoryUsage(),
});
}
}
});

View File

@@ -4,17 +4,19 @@
*/
import { Inject, Injectable } from '@nestjs/common';
import { QueryFailedError } from 'typeorm';
import * as Redis from 'ioredis';
import * as OTPAuth from 'otpauth';
import { createHash } from 'node:crypto';
import { DI } from '@/di-symbols.js';
import type { MiUserProfile, UserProfilesRepository, UsersRepository } from '@/models/_.js';
import { bindThis } from '@/decorators.js';
import { isDuplicateKeyValueError } from '@/misc/is-duplicate-key-value-error.js';
import type { MiLocalUser } from '@/models/User.js';
@Injectable()
export class UserAuthService {
constructor(
@Inject(DI.redis)
private redisClient: Redis.Redis,
@Inject(DI.usersRepository)
private usersRepository: UsersRepository,
@@ -30,16 +32,58 @@ export class UserAuthService {
twoFactorBackupSecret: profile.twoFactorBackupSecret.filter((secret) => secret !== token),
});
} else {
const delta = OTPAuth.TOTP.validate({
secret: OTPAuth.Secret.fromBase32(profile.twoFactorSecret!),
digits: 6,
token,
window: 5,
});
if (delta === null) {
if (!await this.validateOtp(profile.userId, profile.twoFactorSecret!, token)) {
throw new Error('authentication failed');
}
}
}
public async validateOtp(
userId: MiUserProfile['userId'],
twoFactorSecret: string,
token: string,
) {
if (process.env.NODE_ENV === 'test' && process.env.MISSKEY_TEST_CHECK_DUPLICATED_TOTP !== '1') {
return true;
}
// 1. 判定に用いるタイムスタンプを固定
const now = Date.now();
const normalizedToken = token.trim();
const validationWindow = 1;
const timeStep = 30; // TOTPの周期
// 2. TOTPインスタンスを生成設定を一元管理するため
const totp = new OTPAuth.TOTP({
secret: OTPAuth.Secret.fromBase32(twoFactorSecret),
digits: 6,
period: timeStep,
});
// 3. 固定したタイムスタンプを使って検証
const delta = totp.validate({
token: normalizedToken,
window: validationWindow,
timestamp: now,
});
if (delta === null) {
throw new Error('authentication failed');
}
// 4. totp.counter() を用い、同じタイムスタンプから基準ステップを取得
const currentStep = totp.counter({ timestamp: now });
const step = currentStep + delta;
const secretFingerprint = createHash('sha256')
.update(twoFactorSecret ?? '')
.digest('base64url');
const usedTokenRedisKey = `2fa:used:${userId}:${secretFingerprint}:${step}`;
// 5. TTL有効期限を設定いてredis set
const ttl = timeStep * (validationWindow * 2 + 1);
const setResult = await this.redisClient.set(usedTokenRedisKey, normalizedToken, 'EX', ttl, 'NX');
return setResult === 'OK';
}
}

View File

@@ -10,6 +10,7 @@ import { UserEntityService } from '@/core/entities/UserEntityService.js';
import type { UserProfilesRepository } from '@/models/_.js';
import { GlobalEventService } from '@/core/GlobalEventService.js';
import { DI } from '@/di-symbols.js';
import { UserAuthService } from "@/core/UserAuthService.js";
export const meta = {
requireCredential: true,
@@ -45,6 +46,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
private userProfilesRepository: UserProfilesRepository,
private userEntityService: UserEntityService,
private userAuthService: UserAuthService,
private globalEventService: GlobalEventService,
) {
super(meta, paramDef, async (ps, me) => {
@@ -56,14 +58,7 @@ export default class extends Endpoint<typeof meta, typeof paramDef> { // eslint-
throw new Error('二段階認証の設定が開始されていません');
}
const delta = OTPAuth.TOTP.validate({
secret: OTPAuth.Secret.fromBase32(profile.twoFactorTempSecret),
digits: 6,
token,
window: 5,
});
if (delta === null) {
if (!await this.userAuthService.validateOtp(profile.userId, profile.twoFactorTempSecret, token)) {
throw new Error('not verified');
}

View File

@@ -19,6 +19,7 @@ import type { FastifyRequest, FastifyReply } from 'fastify';
@Injectable()
export class UrlPreviewService {
private logger: Logger;
private readonly summalyDefaultUserAgent: string;
constructor(
@Inject(DI.config)
@@ -31,6 +32,7 @@ export class UrlPreviewService {
private loggerService: LoggerService,
) {
this.logger = this.loggerService.getLogger('url-preview');
this.summalyDefaultUserAgent = `SummalyBot/${_SUMMALY_VERSION_} (${this.config.url}; +https://github.com/misskey-dev/summaly/blob/master/README.md)`;
}
@bindThis
@@ -113,20 +115,16 @@ export class UrlPreviewService {
}
private async fetchSummary(url: string, meta: MiMeta, lang?: string): Promise<SummalyResult> {
const agent = this.config.proxy
? {
http: this.httpRequestService.httpAgent,
https: this.httpRequestService.httpsAgent,
}
: undefined;
const { summaly } = await import('@misskey-dev/summaly');
return summaly(url, {
followRedirects: this.meta.urlPreviewAllowRedirect,
lang: lang ?? 'ja-JP',
agent: agent,
userAgent: meta.urlPreviewUserAgent ?? undefined,
agent: {
http: this.httpRequestService.httpAgent,
https: this.httpRequestService.httpsAgent,
},
userAgent: meta.urlPreviewUserAgent ?? this.summalyDefaultUserAgent,
operationTimeout: meta.urlPreviewTimeout,
contentLengthLimit: meta.urlPreviewMaximumContentLength,
contentLengthRequired: meta.urlPreviewRequireContentLength,
@@ -139,7 +137,7 @@ export class UrlPreviewService {
url: url,
lang: lang ?? 'ja-JP',
followRedirects: this.meta.urlPreviewAllowRedirect,
userAgent: meta.urlPreviewUserAgent ?? undefined,
userAgent: meta.urlPreviewUserAgent ?? this.summalyDefaultUserAgent,
operationTimeout: meta.urlPreviewTimeout,
contentLengthLimit: meta.urlPreviewMaximumContentLength,
contentLengthRequired: meta.urlPreviewRequireContentLength,

View File

@@ -10,7 +10,7 @@ import * as crypto from 'node:crypto';
import cbor from 'cbor';
import * as OTPAuth from 'otpauth';
import { loadConfig } from '@/config.js';
import { api, signup } from '../utils.js';
import { api, signup, sendEnvUpdateRequest } from '../utils.js';
import type {
AuthenticationResponseJSON,
AuthenticatorAssertionResponseJSON,
@@ -20,7 +20,7 @@ import type {
RegistrationResponseJSON,
} from '@simplewebauthn/server';
import type * as misskey from 'misskey-js';
import { describe, beforeAll, test } from 'vitest';
import { describe, beforeAll, beforeEach, test } from 'vitest';
describe('2要素認証', () => {
let alice: misskey.entities.SignupResponse;
@@ -181,6 +181,10 @@ describe('2要素認証', () => {
alice = await signup({ username, password });
}, 1000 * 60 * 2);
beforeEach(async () => {
await sendEnvUpdateRequest({ key: 'MISSKEY_TEST_CHECK_DUPLICATED_TOTP', value: '' });
});
test('が設定でき、OTPでログインできる。', async () => {
const registerResponse = await api('i/2fa/register', {
password,
@@ -487,4 +491,33 @@ describe('2要素認証', () => {
token: otpToken(registerResponse.body.secret),
}, alice);
});
test('のTOTPトークンは一度使うと同じトークンは再利用できない。', async () => {
await sendEnvUpdateRequest({ key: 'MISSKEY_TEST_CHECK_DUPLICATED_TOTP', value: '1' });
const registerResponse = await api('i/2fa/register', {
password,
}, alice);
assert.strictEqual(registerResponse.status, 200);
const sharedOtpToken = otpToken(registerResponse.body.secret);
const doneResponse = await api('i/2fa/done', {
token: sharedOtpToken,
}, alice);
assert.strictEqual(doneResponse.status, 200);
const signinResponse = await api('signin-flow', {
...signinParam(),
token: sharedOtpToken,
});
assert.strictEqual(signinResponse.status, 403);
await sendEnvUpdateRequest({ key: 'MISSKEY_TEST_CHECK_DUPLICATED_TOTP', value: '' });
// 後片付け
await api('i/2fa/unregister', {
password,
token: otpToken(registerResponse.body.secret),
}, alice);
});
});

View File

@@ -29,7 +29,7 @@
},
"devDependencies": {
"@misskey-dev/emoji-assets": "17.0.3",
"@misskey-dev/summaly": "5.3.0",
"@misskey-dev/summaly": "5.5.1",
"@tabler/icons-webfont": "3.35.0",
"@testing-library/vue": "8.1.0",
"@types/estree": "1.0.9",

View File

@@ -153,11 +153,10 @@ export function getConfig(): UserConfig {
name: 'vue',
test: /node_modules[\\/]vue/,
}, {
// split each i18n related module to each distinct module, deny hoisting
// split i18n related module to distinct module
name: 'i18n',
test: /i18n\.ts/,
minSize: 0,
maxSize: 1,
includeDependenciesRecursively: false,
test: /i18n\.ts|locale\.ts/,
}],
},
entryFileNames: `scripts/${localesHash}-[hash:8].js`,

View File

@@ -72,7 +72,7 @@
},
"devDependencies": {
"@misskey-dev/emoji-assets": "17.0.3",
"@misskey-dev/summaly": "5.3.0",
"@misskey-dev/summaly": "5.5.1",
"@rollup/plugin-json": "6.1.0",
"@rollup/pluginutils": "5.4.0",
"@storybook/addon-essentials": "8.6.18",
@@ -129,6 +129,7 @@
"react": "19.2.7",
"react-dom": "19.2.7",
"rolldown": "1.1.0",
"rollup-plugin-visualizer": "7.0.1",
"sass-embedded": "1.100.0",
"seedrandom": "3.0.5",
"start-server-and-test": "3.0.9",

View File

@@ -2,7 +2,8 @@ import path from 'path';
import pluginVue from '@vitejs/plugin-vue';
import pluginGlsl from 'vite-plugin-glsl';
import { replacePlugin } from 'rolldown/plugins';
import type { UserConfig } from 'vite';
import { visualizer } from 'rollup-plugin-visualizer';
import type { PluginOption, UserConfig } from 'vite';
import { defineConfig } from 'vite';
import * as yaml from 'js-yaml';
import { promises as fsp } from 'fs';
@@ -23,6 +24,32 @@ const host = url ? (new URL(url)).hostname : undefined;
const extensions = ['.ts', '.tsx', '.js', '.jsx', '.mjs', '.json', '.json5', '.svg', '.sass', '.scss', '.css', '.vue'];
function getBundleVisualizerPlugin(): PluginOption[] {
if (process.env.FRONTEND_BUNDLE_VISUALIZER !== 'true') return [];
const template = process.env.FRONTEND_BUNDLE_VISUALIZER_TEMPLATE === 'markdown'
? 'markdown'
: process.env.FRONTEND_BUNDLE_VISUALIZER_TEMPLATE === 'raw-data'
? 'raw-data'
: 'treemap';
const defaultFilename = template === 'markdown'
? path.resolve(__dirname, '../../built/_frontend_bundle_visualizer_/report.md')
: template === 'raw-data'
? path.resolve(__dirname, '../../built/_frontend_bundle_visualizer_/stats.json')
: path.resolve(__dirname, '../../built/_frontend_bundle_visualizer_/stats.html');
return [
visualizer({
filename: process.env.FRONTEND_BUNDLE_VISUALIZER_FILE ?? defaultFilename,
title: 'Misskey frontend bundle visualizer',
template,
gzipSize: true,
brotliSize: true,
projectRoot: path.resolve(__dirname, '../..'),
}) as PluginOption,
];
}
/**
* 検索インデックスの生成設定
*/
@@ -129,6 +156,7 @@ export function getConfig(): UserConfig {
}),
]
: [],
...getBundleVisualizerPlugin(),
],
resolve: {
@@ -194,11 +222,10 @@ export function getConfig(): UserConfig {
name: 'photoswipe',
test: /node_modules[\\/]photoswipe/,
}, {
// split each i18n related module to each distinct module, deny hoisting
// split i18n related module to distinct module
name: 'i18n',
test: /i18n\.ts/,
minSize: 0,
maxSize: 1,
includeDependenciesRecursively: false,
test: /i18n\.ts|locale\.ts/,
}],
},
entryFileNames: `scripts/${localesHash}-[hash:8].js`,

View File

@@ -1,7 +1,7 @@
{
"type": "module",
"name": "misskey-js",
"version": "2026.6.0-beta.1",
"version": "2026.6.0",
"description": "Misskey SDK for JavaScript",
"license": "MIT",
"main": "./built/index.js",

746
pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

View File

@@ -63,6 +63,8 @@ overrides:
'@aiscript-dev/aiscript-languageserver': '-'
chokidar: 5.0.0
lodash: 4.18.1
# remove when vite is updated to versions with rolldown > 1.1.0
vite>rolldown: "~1.1.0"
engineStrict: true
saveExact: true
shellEmulator: true