Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,30 @@ The following environment variables can be used optionally:
- `TOKEN`: Your GitHub token (for private repos)
- `PRIVATE_BASE_URL`: The server's URL (for private repos - when running on [Vercel](https://vercel.com), this field is filled with the URL of the deployment automatically)

### Release mirror (optional)

Mirrors each release asset to an S3-compatible bucket so the Windows installer
can be served with a caller-supplied tag in its filename. Without these
variables the mirror stays off and every download route behaves exactly as
before.

- `BUCKET_NAME_CLOUDFLARE_R2`, `AWS_ACCESS_KEY_ID_CLOUDFLARE_R2`,
`AWS_SECRET_ACCESS_KEY_CLOUDFLARE_R2`: bucket and credentials. All three are
required to turn the mirror on
- `AWS_ENDPOINT_CLOUDFLARE_R2`, `AWS_DEFAULT_REGION_CLOUDFLARE_R2`: S3 endpoint
and region (region defaults to `auto`)
- `RELEASES_MIRROR_PREFIX`: key prefix in the bucket (defaults to `releases`)
- `INSTALLER_TAG_PREFIX`: prefix placed before the tag in the filename
(defaults to `build-`). The client that reads the filename must be configured
with the same value

Mirroring is one copy per release, not per download, and it is kicked off in the
background from the traffic the service already receives.

Object keys are `<prefix>/<REPOSITORY>/<version>/<asset name>`. The repository is
part of the key so that several deployments, each reading a different release
repo, can safely share one bucket.

## Statistics

Since Nutela routes all the traffic for downloading the actual application files to [GitHub Releases](https://help.github.com/articles/creating-releases/), you can use their API to determine the download count for a certain release.
Expand All @@ -63,6 +87,20 @@ Accepts a platform (like "darwin" or "win32") to download the appropriate copy y

If the cache isn't filled yet or doesn't contain a download link for the specified platform, it will respond like `/`.

Both `/download` and `/download/:platform` accept an optional `?t=<tag>` for the
Windows installer. When the release mirror is configured and the asset has
already been copied, the response redirects to a presigned URL that forces the
filename `<asset> [<prefix><tag>].exe`, letting the installed client correlate
the download it came from. Any failure — mirror off, malformed tag, release not
copied yet — silently falls back to the normal redirect, so the user still gets
the same binary.

`?t=` is ignored when `?update=true`, since a Squirrel update must keep the
canonical filename.

**The tag is opaque to this service and may carry sensitive caller data. Never
log it, and prefer a short-lived, single-use value.**

### /update/:platform/:version

Checks if there is an update available by reading from the cache.
Expand Down
3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
]
},
"dependencies": {
"@aws-sdk/client-s3": "3.1096.0",
"@aws-sdk/lib-storage": "3.1096.0",
"@aws-sdk/s3-request-presigner": "3.1096.0",
"async-retry": "1.3.3",
"date-fns": "3.6.0",
"dotenv": "16.4.5",
Expand Down
223 changes: 223 additions & 0 deletions src/lib/mirror.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
import {
GetObjectCommand,
HeadObjectCommand,
S3Client,
} from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { Readable } from 'node:stream';

import { Asset, type StorageConfig } from './types';

/**
* Espelho das releases num bucket S3/R2.
*
* Existe por um motivo só: as URLs do GitHub carregam o `response-content-
* disposition` DENTRO da assinatura, então não há como servir o mesmo binário
* com um nome de arquivo próprio redirecionando para lá. Com o arquivo num
* bucket próprio, uma presigned URL resolve isso sem que nenhum byte passe por
* este servidor.
*
* É uma cópia por release, não por download.
*/

/** Validade da URL assinada. Curta: ela é gerada no clique do usuário. */
const SIGNED_URL_TTL_SECONDS = 15 * 60;

/** Formato aceito para a etiqueta: opaca, sem espaço nem caractere de path. */
const TAG_PATTERN = /^[A-Za-z0-9_-]{16,64}$/;

export const isValidTag = (value: string) => TAG_PATTERN.test(value);

/**
* Prefixo da etiqueta no nome do arquivo. Configurável porque o cliente que lê
* esse nome precisa combinar com ele, e o par pode ser rotacionado sem release.
*/
export const tagPrefix = () => process.env.INSTALLER_TAG_PREFIX ?? 'build-';

/**
* `Meu.App.-.v1.2.3.exe` + etiqueta → `Meu.App.-.v1.2.3 [<prefixo><etiqueta>].exe`
*/
export const buildTaggedFilename = (assetName: string, tag: string) => {
const safeName = assetName.replace(/["\r\n]/g, '');
const extensionAt = safeName.lastIndexOf('.');

const base = extensionAt === -1 ? safeName : safeName.slice(0, extensionAt);
const extension = extensionAt === -1 ? '' : safeName.slice(extensionAt);

return `${base} [${tagPrefix()}${tag}]${extension}`;
};

/**
* O repositório entra na chave para que múltiplos deploys deste serviço, cada um
* lendo um repositório de releases diferente, possam dividir o mesmo bucket sem
* risco de um sobrescrever o outro.
*/
export const buildObjectKey = (
prefix: string | undefined,
repository: string,
version: string,
assetName: string,
) => `${prefix ?? 'releases'}/${repository}/${version}/${assetName}`;

export default class Mirror {
private readonly config: StorageConfig;

private readonly repository: string;

private readonly githubToken?: string;

private readonly s3?: S3Client;

/** Chaves já confirmadas no bucket — evita um HeadObject por request. */
private readonly mirrored = new Set<string>();

/** Cópias em andamento, para duas requisições não subirem o mesmo arquivo. */
private readonly inFlight = new Set<string>();

constructor(
config: StorageConfig,
repository?: string,
githubToken?: string,
) {
this.config = config;
this.repository = repository ?? 'unknown';
this.githubToken = githubToken;

if (!this.isConfigured) return;

this.s3 = new S3Client({
region: config.region ?? 'auto',
endpoint: config.endpoint,
credentials: {
accessKeyId: config.accessKeyId!,
secretAccessKey: config.secretAccessKey!,
},
});
}

/**
* Sem bucket configurado o espelho fica inteiro desligado e as rotas caem no
* redirect de sempre. Permite subir o código antes de existir a infra.
*/
get isConfigured() {
const { bucket, accessKeyId, secretAccessKey } = this.config;
return Boolean(bucket && accessKeyId && secretAccessKey);
}

private objectKey(version: string, assetName: string) {
return buildObjectKey(
this.config.prefix,
this.repository,
version,
assetName,
);
}

private async exists(key: string) {
if (this.mirrored.has(key)) return true;

try {
await this.s3!.send(
new HeadObjectCommand({ Bucket: this.config.bucket!, Key: key }),
);
this.mirrored.add(key);
return true;
} catch {
return false;
}
}

/**
* Copia o asset para o bucket se ainda não estiver lá. Chamada sem await
* pelas rotas: a primeira pessoa a baixar depois de um release novo cai no
* redirect normal (sem etiqueta) enquanto a cópia acontece.
*/
async ensureMirrored(version: string, asset: Asset): Promise<void> {
if (!this.isConfigured) return;

const key = this.objectKey(version, asset.name);

// Reserva ANTES de qualquer await. Se o `exists()` viesse primeiro, todas as
// requisições concorrentes cederiam o event loop no HeadObject e passariam
// pelo guard juntas — cada uma baixando ~150 MB do GitHub e abrindo um
// upload multipart. No dia de uma release isso derruba o dyno, que é o mesmo
// que serve o canal de update de toda a base instalada.
if (this.inFlight.has(key)) return;

this.inFlight.add(key);

try {
if (await this.exists(key)) return;

const headers: Record<string, string> = {
Accept: 'application/octet-stream',
};

if (this.githubToken) {
headers.Authorization = `token ${this.githubToken}`;
}

const response = await fetch(
this.githubToken ? asset.api_url : asset.url,
{ headers },
);

if (!response.ok || !response.body) {
// Sem cancelar, cada falha repetida (rate limit, token expirado) deixa
// uma conexão pendurada.
await response.body?.cancel();
throw new Error(`GitHub respondeu ${response.status}`);
}

await new Upload({
client: this.s3!,
params: {
Bucket: this.config.bucket!,
Key: key,
Body: Readable.fromWeb(response.body as never),
ContentType: asset.content_type || 'application/octet-stream',
},
}).done();

this.mirrored.add(key);
console.log(`[mirror] espelhado ${key}`);
} catch (err) {
console.error(`[mirror] falha ao espelhar ${key}:`, err);
} finally {
this.inFlight.delete(key);
}
}

/**
* URL assinada que força o download com a etiqueta no nome do arquivo.
* Devolve null quando o espelho não está pronto — quem chama cai no redirect
* normal e o usuário recebe o mesmo binário, só sem a etiqueta.
*
* A etiqueta é opaca para este serviço e pode carregar dado sensível de quem
* chamou: NÃO logar.
*/
async getTaggedUrl(
version: string,
asset: Asset,
tag: string,
): Promise<string | null> {
if (!this.isConfigured || !isValidTag(tag)) return null;

const key = this.objectKey(version, asset.name);

if (!(await this.exists(key))) return null;

const filename = buildTaggedFilename(asset.name, tag);

return getSignedUrl(
this.s3!,
new GetObjectCommand({
Bucket: this.config.bucket!,
Key: key,
ResponseContentDisposition: `attachment; filename="${filename}"`,
}),
{ expiresIn: SIGNED_URL_TTL_SECONDS },
);
}
}
Loading
Loading