You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the undici package is quite heavy with a lot of stuff that you might not need, e.g. web api. Would be good to separate undici "core" into its own package.
I think it's inevitable that we adopt a monorepo at this point, this project has grown too big, and there are separate subsystems that could be used separately.
This is also my preference. Does anyone feel extra-strongly about using npm?
pnpm is now included in corepack so it should be very easy for everyone to use it without any additional install or set up (minus a corepack enable call).
I've not been able to make npm workspace work in any reasonable way for OSS development
I think the most useful part would be if somebody could figure out how to set it up with changesets so that we could ship independent versions of the packages. Alternatively, they will all have the same version.
Create PR that moves everything into packages/undici and documentation
No code will split up just yet
CI should run as normal
Docs site should work as normal
Other tooling should continue to work (build, commit hooks, etc.), and run from root as normal
This should only introduce npm workspaces. Changesets should not be required.
Even though the code will exist in packages/undici directory, the package should still be named and published as undici (for now).
(In nodejs/node) Update Node.js -> Undici vendor process
Step 1 is simply the first step in migrating to a monorepo. This will reduce impact on contributors and should not disrupt development.
Code split (incremental)
Create new packages packages/core, packages/fetch, and packages/types
Start publishing packages as @undici/core, @undici/fetch, and @undici/types
Retain packages/undici as undici and make it roll-up and export all of the other packages
Introduce changesets for version and publishing management.
(Optional) Move linting and formatting to root
Step 2 serves as an incremental step to a true monorepo organization. By retaining undici package we don't have to affect Node.js or users. Everything will appear as normal.
Migrate Node.js from undici to @undici/fetch
Make necessary change in nodejs/node to vendor @undici/fetch instead of undici
More code splitting!
Intentionally not-well-defined at this time, consider other parts of Undici that we can split out into its own separate packages.
My experience with npm workspaces is that they are not fit for purpose. Specifically, they are missing the workspace protocol, https://pnpm.io/workspaces#referencing-workspace-packages-through-aliases, which makes it impossible to maintain an OSS library without it. (Or I couldn’t figure out how to handle versioning).
I've found more success where versions are updated by a tool such as changesets. That said, switching to pnpm has other great benefits -- and everyone can do it since it's supported in corepack. Ill make sure to present options for us to consider
I dont like monorepos. They make managing code complicated. I would rather have one package released on npm with everything than having to manage many packages where each package has different versions.
Then I would suggest to split it into two different repos. undici core and undici web.
I'd prefer this over a monorepo.
Sure the code lives in separate folders, but yet we still have a lot of it. It is difficult to know where to make changes, how to add new apis, etc.
@Uzlopak was able to add EventSource without any help from me or anyone else on how to organize code. If we're talking about the lower leveled dispatcher api/mocks/ProxyAgent/etc., that simply isn't an issue with the web stuff and wouldn't be solved by moving it. Moving the web code to their own folder (in the PR linked here) will let people use tooling to ignore those files.
Not only that, others have made breaking changes relatively quickly lately (such as the removal of some dispatcher handler options and interceptors).
possibly axios, got etc...
axios and got both use node:http and axios will use XMLHttpRequest in browsers. If axios ever moved to fetch then they likely use undici's fetch instead of undici core (or just global fetch). I know this is very shitty proof but I looked at the most downloaded dependents of undici on npm and every single one of them used fetch. I would go so far as to say that undici.fetch is just as fundamentally important as undici.request, etc., as they both serve two entirely separate use cases in a similar api.
I haven't thought too much about what a minimal @undici/core would contain. I find the current Dispatcher implementation to be confusing. The indirection in particular is the worst part (such as the makeDispatcher piece in index.js). This obviously was done for good reason (such as extending/implementing the interface in an easy way across the repo). But if for example these pieces lived in their own module, it could force us to come up with a better API for other pieces like the Mock interfaces or custom Handlers.
I'm very confused as to how you came to the conclusion that the core api being confusing can be solved by moving the web stuff. The web stuff doesn't need docs, and this seems like a documentation issue w.r.t. core.
I can't speak on information density because I've never had issues with it honestly. Of course we could move MockAgent or ProxyAgent to their own packages, and I wouldn't have an issue with that, because they are higher leveled than even fetch, and do not rely on core.
Fetch relies on core internals and core relies on fetch internals (specifically undici.request w/ a FormData body). There is no easy way of decoupling fetch from core with a benefit that seems to boil down to '[core/web] is too [confusing/big]' ("big" in a more abstract way than package size). From what I've seen @Ethan-Arrowood mostly has an issue with documentation in core, because the web stuff simply has enough documentation on MDN or a million other places.
So I would like to leave a few questions:
can these issues be solved with better/more documentation?
In short, yes many of these issues can be solved with better/more documentation AND improved code organization (not necessarily meaning monorepo). Let's start there and iterate.
Activity
metcoder95 commentedon Feb 12, 2024
I think @Ethan-Arrowood was managing this idea for a while, after the split of
undici-typesEthan-Arrowood commentedon Feb 12, 2024
There's some old threads about monorepo ideas. Happy to revisit
mcollina commentedon Feb 12, 2024
I think it's inevitable that we adopt a monorepo at this point, this project has grown too big, and there are separate subsystems that could be used separately.
Ethan-Arrowood commentedon Feb 12, 2024
This is also my preference. Does anyone feel extra-strongly about using npm?
pnpm is now included in corepack so it should be very easy for everyone to use it without any additional install or set up (minus a
corepack enablecall).Ethan-Arrowood commentedon Feb 12, 2024
I will also start by experimenting with npm workspaces to start. They've supposedly come along way so maybe its worth trying again.
mcollina commentedon Feb 12, 2024
I've not been able to make npm workspace work in any reasonable way for OSS development
I think the most useful part would be if somebody could figure out how to set it up with changesets so that we could ship independent versions of the packages. Alternatively, they will all have the same version.
Ethan-Arrowood commentedon Feb 12, 2024
I have experience with changesets. Do you want me to try npm+changesets or go straight to pnpm+changesets?
Ethan-Arrowood commentedon Feb 13, 2024
Rough plan:
packages/undicianddocumentationpackages/undicidirectory, the package should still be named and published asundici(for now).nodejs/node) Update Node.js -> Undici vendor processpackages/core,packages/fetch, andpackages/types@undici/core,@undici/fetch, and@undici/typespackages/undiciasundiciand make it roll-up and export all of the other packagesMigrate Node.js from
undicito@undici/fetchnodejs/nodeto vendor@undici/fetchinstead ofundiciMore code splitting!
mcollina commentedon Feb 13, 2024
I'm +1
metcoder95 commentedon Feb 13, 2024
LGTM 🚀
Ethan-Arrowood commentedon Feb 19, 2024
I'll have a PR up for step 1 very soon. I'd like to land #2766 first as that will help simplify things.
mcollina commentedon Feb 20, 2024
My experience with npm workspaces is that they are not fit for purpose. Specifically, they are missing the workspace protocol, https://pnpm.io/workspaces#referencing-workspace-packages-through-aliases, which makes it impossible to maintain an OSS library without it. (Or I couldn’t figure out how to handle versioning).
@KhafraDev wdyt?
Ethan-Arrowood commentedon Feb 20, 2024
I've found more success where versions are updated by a tool such as changesets. That said, switching to pnpm has other great benefits -- and everyone can do it since it's supported in corepack. Ill make sure to present options for us to consider
Uzlopak commentedon Feb 20, 2024
I dont like monorepos. They make managing code complicated. I would rather have one package released on npm with everything than having to manage many packages where each package has different versions.
20 remaining items
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder
refactor: move web stuff into their own folder (#2793)
KhafraDev commentedon Feb 21, 2024
I'd prefer this over a monorepo.
@Uzlopak was able to add EventSource without any help from me or anyone else on how to organize code. If we're talking about the lower leveled dispatcher api/mocks/ProxyAgent/etc., that simply isn't an issue with the web stuff and wouldn't be solved by moving it. Moving the web code to their own folder (in the PR linked here) will let people use tooling to ignore those files.
Not only that, others have made breaking changes relatively quickly lately (such as the removal of some dispatcher handler options and interceptors).
axios and got both use node:http and axios will use XMLHttpRequest in browsers. If axios ever moved to fetch then they likely use undici's fetch instead of undici core (or just global fetch). I know this is very shitty proof but I looked at the most downloaded dependents of undici on npm and every single one of them used fetch. I would go so far as to say that undici.fetch is just as fundamentally important as undici.request, etc., as they both serve two entirely separate use cases in a similar api.
I'm very confused as to how you came to the conclusion that the core api being confusing can be solved by moving the web stuff. The web stuff doesn't need docs, and this seems like a documentation issue w.r.t. core.
I can't speak on
information densitybecause I've never had issues with it honestly. Of course we could move MockAgent or ProxyAgent to their own packages, and I wouldn't have an issue with that, because they are higher leveled than even fetch, and do not rely on core.Fetch relies on core internals and core relies on fetch internals (specifically undici.request w/ a FormData body). There is no easy way of decoupling fetch from core with a benefit that seems to boil down to '[core/web] is too [confusing/big]' ("big" in a more abstract way than package size). From what I've seen @Ethan-Arrowood mostly has an issue with documentation in core, because the web stuff simply has enough documentation on MDN or a million other places.
So I would like to leave a few questions:
lib/webfrom code search results.Ethan-Arrowood commentedon Feb 21, 2024
In short, yes many of these issues can be solved with better/more documentation AND improved code organization (not necessarily meaning monorepo). Let's start there and iterate.
refactor: move web stuff into their own folder (#2793)
refactor: move web stuff into their own folder (nodejs#2793)
refactor: move web stuff into their own folder (nodejs#2793)