Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ conflated how the provider resolves the value with how the key hashes it.

**Cross.**

- feat(cross): modules are now built during `--prepare` (staged into
`<buildRoot>/modules/`) and during `--build` without `--app` (standalone
module build). Modules whose path references `${app_root}` are skipped with
an informational message when `--app` is absent. `build.app_path` is
deprecated; pass `--app` on the command line instead.

- feat(cross): `sdk_path` and `sdk_env_setup` may be written relative to the
manifest that declares them, on the same rule `augment[].patches`,
augment `path:` and `modules[].path` already follow. Absolute values are
Expand Down
21 changes: 14 additions & 7 deletions examples/cross/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,8 +176,10 @@ cross:
tree. Versioned outputs (`libhello.so.1.2.3` with `libhello.so` / `.so.1`
symlinks) are staged whole — the real file plus every soname link — so both
the bare name and any `DT_NEEDED` SONAME resolve at load time.
- Modules build (and stage) only when a runnable bundle is produced — i.e. with
`--app`. Declared without `--app`, they are skipped with a warning.
- Without `--app`, modules whose path does not reference `${app_root}` are
still built and staged into `<buildRoot>/modules/`. Modules whose path
contains `${app_root}` require `--app` to resolve that variable and are
skipped with an informational message when `--app` is absent.
- `build: cargo` cross-compiles a Rust crate: emb derives the Rust target from
the GNU triple, synthesizes the cargo/cc-rs env from the profile (target-
scoped `CC_<triple>` / `CARGO_TARGET_<TRIPLE>_LINKER` / `CFLAGS_<triple>` /
Expand All @@ -186,11 +188,16 @@ cross:
`cargo` on `PATH`; targets the `arm-gnu` provider (the compiler is a real
cross `gcc`). `features:` map to `--features`.

**Where modules ship.** They land in the app bundle's `lib/`, so the runnable
bundle (`--app`, `--tar`, `--deploy`) and the flatpak packager carry them
automatically. The deb/ipk/rpm/targz packagers package the *embedder binary*
plus `cross.package.files:` — not the app bundle — so a module reaches those
only if the bundle tree is listed under `files:` (the same as `libapp.so`).
**Where modules ship.** With `--app` they land in the app bundle's `lib/`,
so the runnable bundle (`--app`, `--tar`, `--deploy`) and the flatpak packager
carry them automatically. Without `--app` (standalone or `--prepare` build)
they are staged into `<buildRoot>/modules/`. Nothing in a manifest reaches that
directory — the path variables are `${embedder_root}`, `${app_root}` and
`${runnable}`, none of which names the build root — so those artifacts are for
tooling outside emb, not for `files:` to pick up.
The deb/ipk/rpm/targz packagers package the *embedder binary* plus
`cross.package.files:` — not the app bundle — so a module reaches those only
if the bundle tree is listed under `files:` (the same as `libapp.so`).

## Local augment sources (`path:`)

Expand Down
113 changes: 100 additions & 13 deletions lib/src/commands/cross_command.dart
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import 'package:emb_cli/src/host/host_info.dart';
import 'package:emb_cli/src/host/interactivity.dart';
import 'package:emb_cli/src/host/preflight.dart';
import 'package:emb_cli/src/json_output.dart';
import 'package:emb_cli/src/manifest/build_config.dart';
import 'package:emb_cli/src/manifest/manifest_loader.dart';
import 'package:emb_cli/src/manifest/source_repo.dart';
import 'package:emb_cli/src/repo/git_repo.dart';
Expand Down Expand Up @@ -174,6 +175,7 @@ class CrossCommand extends Command<int> {
}) : _logger = logger,
_environment = environment ?? Platform.environment,
_host = host,
_loader = loader,
_project = CrossProjectResolver(loader),
_aotFactoryInjected = aotFactory,
_bundleFactory = bundleFactory ?? BundleBuilder.new,
Expand Down Expand Up @@ -470,6 +472,7 @@ class CrossCommand extends Command<int> {
final Logger _logger;
final Map<String, String> _environment;
final HostInfo? _host;
final ManifestLoader _loader;
final CrossProjectResolver _project;
final AotBuilder Function(Workspace ws, HostInfo host)? _aotFactoryInjected;
final BundleBuilder Function(Workspace ws) _bundleFactory;
Expand Down Expand Up @@ -705,6 +708,26 @@ class CrossCommand extends Command<int> {
var resolved = resolveTarget();
if (resolved == null) return ExitCode.usage.code;

// `build.app_path` has no effect here: BuildConfig is read by `emb build`.
// Checked in both manifests this build read, because a `.emb/` project can
// declare a shared `build:` block in its base while the selected target
// comes from a sibling file — the two are the same path for a flat
// manifest, which the set collapses.
if (building) {
for (final path in {
selection.sourcePath,
project.nativeSourcePath,
}.whereType<String>()) {
final raw = _loader.loadManifestFile(File(path))?.raw;
if (declaresAppPath(raw?['build'])) {
_logger.warn(
'$path: build.app_path is deprecated and has no effect; '
'pass --app <path> on the command line instead.',
);
}
}
}

if (building &&
args['flatpak'] == true &&
appDirArg == null &&
Expand Down Expand Up @@ -1053,6 +1076,22 @@ class CrossCommand extends Command<int> {
selectedBackends: selectedBackends,
);
if (preparedEmbedder == null) return ExitCode.software.code;
if (target.modules.isNotEmpty) {
final libDir = Directory(
p.join(preparedEmbedder.buildRoot.path, 'modules'),
)..createSync(recursive: true);
final ok = await _buildModules(
Comment thread
MEverett90 marked this conversation as resolved.
Resolved
preparedEmbedder.builder,
profile,
host,
target,
manifestDir,
preparedEmbedder.buildRoot,
libDir,
appDir: _appDir(appDirArg),
);
if (!ok) return ExitCode.software.code;
}
}

// --deploy/--run without --build: ship what is already built rather than
Expand Down Expand Up @@ -1510,11 +1549,24 @@ class CrossCommand extends Command<int> {
);
return ExitCode.usage.code;
}
if (appPath == null && target.modules.isNotEmpty) {
_logger.warn(
' modules : ${target.modules.map((m) => m.name).join(", ")} '
'declared but no --app bundle to stage them into (add --app).',
// Build modules standalone when no app bundle is being assembled.
// Skipped when preparedEmbedder is non-null: --prepare already built them
// into <buildRoot>/modules/ in this invocation.
if (appPath == null &&
target.modules.isNotEmpty &&
preparedEmbedder == null) {
final libDir = Directory(p.join(buildRoot.path, 'modules'))
..createSync(recursive: true);
final ok = await _buildModules(
builder,
profile,
host,
target,
manifestDir,
buildRoot,
libDir,
);
if (!ok) return ExitCode.software.code;
}
if (appPath != null) {
final rc = await _runnable(
Expand Down Expand Up @@ -3663,10 +3715,12 @@ class CrossCommand extends Command<int> {
return true;
}

/// Build each app-owned [CrossTarget.modules] entry against the embedder's
/// cross toolchain ([builder]) and stage its declared `.so` artifacts into
/// [libDir] (the app bundle's `lib/`). Returns false — with an error logged —
/// on any build failure or missing declared artifact.
/// Build each [CrossTarget.modules] entry against the embedder's cross
/// toolchain ([builder]) and stage its declared `.so` artifacts into
/// [libDir]. Skips the compile step when the module source is unchanged
/// since the last successful build (stamp file matches source fingerprint).
/// Returns false — with an error logged — on any build failure or missing
/// declared artifact.
Comment thread
MEverett90 marked this conversation as resolved.
Resolved
Future<bool> _buildModules(
CrossBuilder builder,
CrossProfile profile,
Expand All @@ -3682,9 +3736,15 @@ class CrossCommand extends Command<int> {
if (appDir != null) 'app_root': appDir,
};
for (final m in target.modules) {
final src = Directory(
p.join(manifestDir.path, expandManifestVars(m.path, moduleVars)),
);
final expanded = expandManifestVars(m.path, moduleVars);
if (appDir == null && hasUnresolvedVars(expanded)) {
_logger.info(
' module ${m.name}: skipped'
r' (path has ${app_root} — pass --app to include)',
);
continue;
}
final src = Directory(p.join(manifestDir.path, expanded));
if (!src.existsSync()) {
_logger.err(' module ${m.name}: source dir not found: ${src.path}');
return false;
Expand All @@ -3693,8 +3753,30 @@ class CrossCommand extends Command<int> {
final sw = Stopwatch()..start();

final gen = m.build.generator;

// Artifact dir a prior build would have produced — computed without
// running the build so we can short-circuit when source is unchanged.
final cachedArtifactDir = gen != null
? buildDir
: Directory(
p.join(
buildDir.path,
rustTriple(profile.targetTriple),
'release',
),
);

final stampFile = File(p.join(buildDir.path, '.module-stamp'));
final fingerprint = await _sourceFingerprint(src);
final upToDate =
stampFile.existsSync() &&
stampFile.readAsStringSync().trim() == fingerprint &&
cachedArtifactDir.existsSync();

final Directory artifactDir;
if (gen != null) {
if (upToDate) {
artifactDir = cachedArtifactDir;
} else if (gen != null) {
// cmake/meson: reuse the embedder's cross toolchain via CrossBuilder.
final r = await builder.build(
sourceDir: src,
Expand All @@ -3707,11 +3789,13 @@ class CrossCommand extends Command<int> {
return false;
}
artifactDir = Directory(r.buildDir);
stampFile.writeAsStringSync(fingerprint);
} else {
// cargo: synthesize the cross env from the profile and run cargo.
final dir = await _cargoModule(profile, host, m, src, buildDir);
if (dir == null) return false;
artifactDir = dir;
stampFile.writeAsStringSync(fingerprint);
}

for (final soname in m.artifacts) {
Expand All @@ -3734,7 +3818,10 @@ class CrossCommand extends Command<int> {
}
}
_logger.info(
' module ${m.name}: ${m.artifacts.join(", ")} → lib/ (${_secs(sw)})',
upToDate
? ' module ${m.name}: up-to-date (cached)'
: ' module ${m.name}: ${m.artifacts.join(", ")} '
'→ ${p.basename(libDir.path)}/ (${_secs(sw)})',
);
}
return true;
Expand Down
8 changes: 8 additions & 0 deletions lib/src/commands/matrix_command.dart
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import 'package:emb_cli/src/cross/cross_project.dart';
import 'package:emb_cli/src/cross/cross_provider.dart';
import 'package:emb_cli/src/cross/cross_target.dart';
import 'package:emb_cli/src/host/host_info.dart';
import 'package:emb_cli/src/manifest/build_config.dart';
import 'package:emb_cli/src/manifest/manifest_loader.dart';
import 'package:emb_cli/src/workspace/workspace.dart';
import 'package:mason_logger/mason_logger.dart';
Expand Down Expand Up @@ -107,6 +108,13 @@ class MatrixCommand extends Command<int> {
: hostFilter.where(supported.contains).toList();
if (hosts.isEmpty) continue;

if (declaresAppPath(manifest.raw['build'])) {
_logger.warn(
'${file.path}: build.app_path is deprecated and has no effect; '
'pass --app <path> on the command line instead.',
);
}

for (final entry in _targets(crossMap)) {
if (targetFilter != null && entry.name != targetFilter) continue;
final CrossTarget target;
Expand Down
10 changes: 10 additions & 0 deletions lib/src/manifest/build_config.dart
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,13 @@ class BuildConfig {
String toString() =>
'BuildConfig(app: $appPath, archs: $archs, modes: $modes)';
}

/// True when a manifest's raw `build:` block declares `app_path`.
///
/// Presence, not value: [BuildConfig.appPath] defaults to `.`, so once parsed a
/// manifest that wrote the default is indistinguishable from one that omitted
/// the key. The cross commands warn on the former, so they have to ask the raw
/// map.
bool declaresAppPath(Object? buildBlock) =>
buildBlock is Map &&
(buildBlock.containsKey('app_path') || buildBlock.containsKey('appPath'));
Loading
Loading