Skip to content

[BUG] head on struct removes other fields in that struct #5507

Description

@Swiddis

What is the bug?
For a struct field "a": { "b": 1, "c": 2 }, a query with fields a, a.b, a.c | head only returns an a column.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Index a single big5 doc (here's one with irrelevant fields removed):
{
   "@timestamp":"2023-01-01T00:57:41.000Z",
   "agent":{
      "id":"baac7358-a449-4c36-bf0f-befb211f1d38",
      "name":"crystaldrifter",
      "type":"filebeat",
      "version":"8.8.0",
      "ephemeral_id":"baac7358-a449-4c36-bf0f-befb211f1d38"
   }
}
  1. Try a query that mixes struct sub-fields and the struct field. This correctly returns all 3 referenced fields:
// source=big5 | fields agent.id agent.name agent
{
  "schema": [
    {
      "name": "agent.id",
      "type": "string"
    },
    {
      "name": "agent.name",
      "type": "string"
    },
    {
      "name": "agent",
      "type": "struct"
    }
  ],
  "datarows": [
    [
      "baac7358-a449-4c36-bf0f-befb211f1d38",
      "crystaldrifter",
      {
        "name": "crystaldrifter",
        "id": "baac7358-a449-4c36-bf0f-befb211f1d38",
        "type": "filebeat",
        "ephemeral_id": "baac7358-a449-4c36-bf0f-befb211f1d38",
        "version": "8.8.0"
      }
    ]
  ],
  "total": 1,
  "size": 1
}
  1. Now add a head: The first two fields disappear.
// source=big5 | fields agent.id agent.name agent | head
{
  "schema": [
    {
      "name": "agent",
      "type": "struct"
    }
  ],
  "datarows": [
    [
      {
        "name": "crystaldrifter",
        "id": "baac7358-a449-4c36-bf0f-befb211f1d38",
        "type": "filebeat",
        "ephemeral_id": "baac7358-a449-4c36-bf0f-befb211f1d38",
        "version": "8.8.0"
      }
    ]
  ],
  "total": 1,
  "size": 1
}

What is the expected behavior?
fields should return all columns listed. Extracting redundant sub-fields from a struct is sometimes useful for presentation ("I just want to scan this one key before digging into the whole struct")

What is your host/environment?

  • mainline (JDK 25)

Do you have any screenshots?
N/A

Do you have any additional context?
N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    PPLPiped processing languagebugSomething isn't working

    Type

    No type

    Projects

    Status
    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions