Skip to content

NetBox: integrate into Malcolm for asset inventory/management #17

Closed

Description

@mmguero

From Malcolm created by robefernandez: cisagov#113

Congratulations for the project, it's really useful and easy to setup in just minutes using the scripts and docker compose.

I've just deployed the solution for testing it so I'm actually a newbie and I have to spend more time to discover all the features but I have a question that will be decisive to continue using it or not by the moment:
Does it have asset inventory capabilities to list all the devices on the network?

I set to true the property LOGSTASH_OUI_LOOKUP (Logstash will map MAC addresses to vendors for all source and destination MAC addresses when analyzing Zeek logs).

Is there any dashboard or any place where we can obtain a list of the network devices?

Best regards.

Activity

  1. added
    enhancementNew feature or request
    logstashRelating to Malcolm's use of Logstash
    on Sep 9, 2020
  2. mmguero commented on Sep 9, 2020

    @mmguero
    CollaboratorAuthor

    The other way I could do it to preserve the mapping of MAC->OUI as I talked about in the prevoius comment would be to do this:

    ...
        "network": {
          "type": "ipv4",
          "mac_oui": {
            "00:10:db:ff:10:01": "Juniper Networks",
            "8c:85:90:65:85:8f": "Apple, Inc."
          }
        },
    ...
    

    Basically network.mac_oui could be a hash of MAC addresses to OUIs... but I'm not sure how well this works in visualizations

  3. mmguero commented on Sep 9, 2020

    @mmguero
    CollaboratorAuthor

    Currently MAC addresses and OUIs are stored in separate source/destination (or originator/responder if you prefer) fields:

    srcMac/dstMac
    srcOui/dstOui

    There are visualizations in the Connections dashboard in Kibana, or by going to the SPIView in Moloch and selecting these fields you could CSV-export a list of these separately and combine them.

    You are not the first person to ask about aggregated src/dst fields to do asset inventory, though. I am going to use this github issue as an enhancement request to create an aggregated field that contains both (probably using the Elastic Common Schema as that's where things will probably end up).

  4. mmguero commented on Sep 9, 2020

    @mmguero
    CollaboratorAuthor

    Hmmm... at that point I Think you're better off just running two reports yourself and combining the results for source and destination. I don't know that it would be desirable to aggregate all of an events IPs into a single IP field. But you could create a kibana visualization that's a data table, with

    srcIp, zeek.orig_hostname, srcMac, srcOui, and service

    and then another one for destination IPs, then export and combine the results.

  5. mmguero commented on Sep 9, 2020

    @mmguero
    CollaboratorAuthor

    Commit ae60cf2 creates network.oui and network.mac fields which are arrays made of source and dst OUI and source and dest mac, respectively. This will make it easier to get lists of MAC addresses for asset inventory purposes.

    The only gotcha I can see here is is if you create like a table visualization with split rows, you'd end up with something like this:

    Say you have a single record with ["Xerox", "Apple"] for network.oui and ["9c:93:4e:01:02:03", "d8:30:62:01:02:03"]. If you create a table visualization with split rows I guess you'd end up with something like this?

    OUI            MAC Address
    -----------     -------------------
    Apple         d8:30:62:01:02:03
    Apple         9c:93:4e:01:02:03
    Xerox         d8:30:62:01:02:03 
    Xerox         9c:93:4e:01:02:03
    

    I think that's how it would work anyway, once they're merged into their arrays you lose the association between them like you have with srcMac/srcOui dstMac/dstOui.

  6. mmguero commented on Sep 9, 2020

    @mmguero
    CollaboratorAuthor

    Thanks @mmguero for your quick response.

    What do you think if IP address and hostname are also shown?

    Let me give you an example:

    Hostname IP MAC OUI
    Router 10.224.0.1 00:10:DB:FF:00:11 Juniper Networks
    HMIfloor2 10.224.0.102 88:5D:90:7F:D3:67 Schmidt & Co.
    HMIfloor3 10.224.0.103 1D:34:FE:9A:12:7A Schmidt & Co.
    PLC001 10.224.10.11 00:11:00:7F:5D:34 Schneider Electric
    PLC002 10.224.10.12 00:11:00:7F:89:67 Schneider Electric
    PLC003 10.224.10.13 00:11:00:7F:34:98 Schneider Electric
    Scada001 10.224.10.10 1C:21:D1:5F:BA:76 B-Scada Inc.
    Srv-GBC9 10.224.10.15 A0:AF:BD:FF:AB:13 Intel

    And just to go one step ahead, what do you think if we can get what services are exposed?

    Hostname IP MAC OUI Services
    Srv-GBC9 10.224.10.15 A0:AF:BD:FF:AB:13 Intel 68/udp ( dhcpc )
    517/udp ( talk )
    518/udp ( ntalk )
    631/udp ( ipp )
    1719/udp ( h323gatestat )
    5353/udp ( zeroconf )

    Thanks.

  7. moved this to Todo (design) in Malcolmon May 10, 2022
  8. moved this from Todo (design) to No status in Malcolmon Sep 19, 2022
  9. moved this to Todo (develop) in Malcolmon Sep 19, 2022
  10. mmguero commented on Sep 19, 2022

    @mmguero
    CollaboratorAuthor

    After doing some more research, I think the direction we're going to go is to integrate NetBox into Malcolm, for several reasons:

    • It's a very featureful asset management solution
    • It's actively developed and maintained
    • It's Apache License 2.0 license is compatible with Malcolm's
    • I've worked out how to spin it up in Docker and integrate it with Malcolm's authentication mechanisms
    • It's got an easy REST API that Malcolm (or whatever else) can interface with
    • It provides a modern full-featured GUI a user could use to go beyond the stuff Malcolm could auto-populate
  11. changed the title [-]Asset inventory capabilities[/-] [+]NetBox: integrate into Malcolm for asset inventory/management[/+] on Sep 19, 2022
  12. added
    externalDepends on a bug or feature external to this project
    apiRelated to issues dealing with the Malcolm REST API
    and removed
    logstashRelating to Malcolm's use of Logstash
    on Sep 19, 2022
  13. self-assigned this
    on Sep 19, 2022
  14. moved this from Todo (develop) to In Progress in Malcolmon Oct 11, 2022
  15. added 2 commits that reference this issue on Oct 17, 2022
  16. mmguero commented on Nov 21, 2022

    @mmguero
    CollaboratorAuthor
  17. mmguero commented on Dec 7, 2022

    @mmguero
    CollaboratorAuthor

    Closing this issue as the initial integration of netbox is done. But I will open another issue (and reference this one) that will be a meta-issue outlining the steps for data enrichment and other uses for the feature between netbox and the rest of the Malcolm stack.

  18. Repository owner moved this from In Progress to Done in Malcolmon Dec 7, 2022
  19. moved this from Done to Released in Malcolmon Dec 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

apiRelated to issues dealing with the Malcolm REST APIenhancementNew feature or requestexternalDepends on a bug or feature external to this project

Type

No type

Projects

  • Status
    Released

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions