Duplicate enrollment via orbit and osquery

[michalnicp]

Fleet version: fleet-v4.21.1

---

🧑‍💻  Expected behavior

After installing orbit, only a single host should be enrolled in fleet.

💥  Actual behavior

2 hosts are enrolled in fleet. Orbit and osquery enroll and results in 2 hosts being created.

More info

Set the following agent options before installing orbit

command_line_flags:
  host_identifier: hostname

> select id, hostname, computer_name, osquery_host_id, orbit_node_key, node_key from hosts\G
***************************[ 1. row ]***************************
id              | 1
hostname        |
computer_name   |
osquery_host_id | fda60f00-e302-4780-aab4-1860903e0128
orbit_node_key  | A9TKh/lr+95snXUHR/WhUpNOmTYJ0Qwp
node_key        | A9TKh/lr+95snXUHR/WhUpNOmTYJ0Qwp
***************************[ 2. row ]***************************
id              | 2
hostname        | 85fbefa0010c
computer_name   | 85fbefa0010c
osquery_host_id | 85fbefa0010c
orbit_node_key  | <null>
node_key        | Y28ZqS1Clbjo2LcoGFLBfktbU6K9qQW+

When orbit enrolls, we assume that the osquery host identifier will be uuid. But, this can be overridden by setting host_identifier in the agent options. This may not be the only way that we can end up in this state.

As a result, we also end up in a reenroll loop because fleet will eventually delete the host created by the orbit enrollment. After it is deleted, orbit will enroll again. This may also have performance impacts due to increased writes to the db if there are many hosts in this state.

See https://osquery.slack.com/archives/C01DXJL16D8/p1670845293461979.

[zwass]

A few options come to mind:

1. Handle this on the server. Is there some way we can correlate the two "hosts"?

2. Disallow that setting (override it or error?)

3. Document that this is an unsupported configuration.

Thoughts?

[chiiph]

Maybe we should timebox figuring out 1 to a day, and then see if we can log an error to detect this case easily in troubleshooting. And as the plan B in case both 1 and 2 are tricky, default to 3?

[michalnicp]

When troubleshooting this issue with the affected user, I was unable to determine exactly how the host ended up in this state. The user did not change the agent options. Maybe osquery generated a random uuid for the host identifier, even though it was set to uuid.

Another option I would propose is to expose the host identifier in an osquery table eg osquery_info. Then, orbit could retrieve the host identifier using osquery during orbit enrollment, instead of assuming it will be always be uuid. This also means we could remove the code for retrieving the hardware uuid from orbit because it would now use osquery for that.