Remove SoftwareSerial as submodule

[d-a-v]

This is a proposal.

A quick fix can be added here to remove EspSoftwareSerial as a submodule and instead rely on a specific stable version of it, available for users through the arduino library manager, not downloaded by default except in CI and release process (<= questionnable too)

This is also proposed because EspSoftwareSerial release process seems to be recurrently slowed down by this repository state or status.

More insights:

• SoftwareSerial.h ssize_t undefined #6482 (comment)
• Errors while initializing SoftwareSerial.h & declaring object of SoftwareSerial class plerup/espsoftwareserial#103 (comment)
• SoftwareSerial doesn't read data after a certain time being idle plerup/espsoftwareserial#100 (comment)
• Compilation error in Arduino v1.8.5, ESP8266 v2.4.0 plerup/espsoftwareserial#99 (comment)
• Netdump library #6518 (comment)

and probably others.

[earlephilhower]

I'm not a SoftwareSerial user, but I've been observing this vicariously.

As I understand things:

• Initially SWSerial was a separate library that people installed via the Arduino Library Manager or manually
• At some point it was brought into the main core as a GIT submodule, due to its utility and stability
• It's recently gotten a lot more active with updates and changes, but with a submodule we're pinning in the 8266 to a specific version
• Users who use library manager to install a newer version may have issues since they'd have 2 copies available (in the 8266 core and in their own libs dir)
• There's a double-whammy delay for SW serial updates, first to get a PR into the 8266 core, then to get an 8266 release (which can be a multi-month timeframe)

So, if I understand things, it seems to me that the best way forward, allowing for SW serial updates on their own timeframe, would be for us to decouple SW serial from the core by removing the submodule from the 8266 core. Users will then be required to install it from the lib manager or their own git clone, on their own time, and at whatever latest version makes sense for them.

In a nutshell

1. Drop the submodule and let both projects move at their own speed.
2. Add a note to the README/CHANGELOG on next release letting people know to how to grab the library if they need it.

Does that sum up things and sound reasonable? I think SoftwareSerial a valuable project and want to make sure it's able to move at its own pace, with minimum friction on both side.

[devyte]

Initially SWSerial was a separate library that people installed via the Arduino Library Manager or manually
At some point it was brought into the main core as a GIT submodule, due to its utility and stability

That's not quite correct. Initially, swserial was implemented and maintained in a separate repo, but was packaged as part of the core during our release process. That created several problems, including the fact that the lib was missing when latest git was installed.
In order to address those issues, it was brought in as a submodule, which allowed consistency for all installations.
That is the current state.

My understanding is that it was originally packaged with our core, despite its being external to it, because it is considered too important for users. I believe it was @igrr who explained that to me, precisely because I asked about it when I installed latest git for the first time ages ago.

I don't think it's a good idea to break it off from our submodules. I think it's likely that mismatches between releases will happen, which will cause more harm than good.

There is another good reason to have it as a submodule, and that is precisely to allow binding our core to a specific commit in swserial. When an issue is found in our core's swserial it can get fixed in the swserial repo, then our submodule is updated to point to the fixed commit.

What doesn't make much sense to me, at least from our core's pov, is to have independent versioned releases of swserial. I think that can confuse users. I think it should be either aligned versioning, like we donwith our toolchain, lwip2, etc, or none at all.
Also, I don't think users should install odd snapshots of swserial, at least not unless they know what they're doing, e. g. propose PRs etc. It's unreasonable to expect that all combinations between core/swserial will work.

[devyte]

Alright, to paraphrase my countrymen: somebody has to cut the cake.
CC @dok-net as current maintainer of the swserial repo

The swserial lib will remain as a submodule for now, the reasons for that are those stated above.
Core code changes proposed which are meant for enhancements of swserial will be evaluated on a case by case basis, as has been done so far. It has been the case that such evaluations have been slow to come, mostly due to a conservative approach when faced with code changes that are not easy to understand or evaluate for impact. The request moving forward is to please make those changes in several smaller PRs rather than single PRs with big changes, in order to make them easier to to understand and evaluate.
I don't think it's a good idea to have tagged releases for the swserial lib, but I'm not really opposed to it, and in the end that is up to the repo maintainer. Whatever is done there for fixes and enhancements, once changes are merged and status is determined to be stable, a proposal should be made here in order to update the submodule to point to the updated lib commit or version in the swserial repo. To be clear, any particular version of the core, whether released or git, should be tied to a specific commit or version of swserial. A user shouldn't be using random combinations of versions of the two, that is not supported at either end, and any user who tries it is on his own.

If there are any further questions or comments, please look me up on gitter and we'll discuss directly.

Closing.