Skip to content

object-ID alignment mismatch between the public caption JSON files and the instance IDs produced by the official ScanNet++ toolbox preprocessing #9

Open

Description

@zhaoyibo0422-code

Hi, thanks for open-sourcing ExCap3D.

I am trying to reproduce the ScanNet++ captioning pipeline using the public code and the public ExCap3D caption release. I found what looks like an object-ID alignment mismatch between the public caption JSON files and the instance IDs produced by the official ScanNet++ toolbox preprocessing.

What I did:

  1. Prepared full .pth files using the official ScanNet++ toolbox:
    python -m semantic.prep.prepare_training_data semantic/configs/prepare_training_data.yml
  2. Sampled points with sample_pth.py
  3. Converted sampled .pth to Mask3D .npy with datasets.preprocessing.scannetpp_pth_preprocessing.py

From the ScanNet++ toolbox code (semantic/transforms/mesh.py), vtx_instance_anno_id is assigned from:

instance_anno_id_multi[inst_mask, new_label_position] = instance['objectId']
sample['vtx_instance_anno_id'] = instance_anno_id
I verified that:

full_pth_data_normals and pth_data_sampled_normals have the same instance IDs
sampled .pth and final Mask3D .npy also have the same instance IDs
However, the public ExCap3D caption JSON keys do not match these IDs.

Concrete example: scene 286b55a2bf

In segments_anno.json, relevant objectIds are:
toilet -> 51
sink -> 50
soap dispenser -> 49
door -> 63
cabinet -> 72
In the public ExCap3D caption JSON, the corresponding captions are keyed by:
13 -> toilet caption
14 -> sink caption
15 -> soap dispenser caption
32 -> door caption
33 -> cabinet caption
I also checked:

PTH vs NPY for this scene: shared=23, pth_only=0, npy_only=0
PTH vs caption JSON for this scene: shared=0, pth_only=23, caption_only=13
On the validation split, this leads to many mismatches:

caption-only inst ids: 191 / 1135
suspicious overlap inst ids: 218 / 944
My questions are:

Are the public ExCap3D caption JSON keys expected to match ScanNet++ segGroups[].objectId?
If not, what ID do the public caption JSON keys correspond to?
Is there an official mapping script or a specific ScanNet++ preprocessing version needed to align the public captions with the released code?
Are the internal scene_captions files used in the training scripts different from the public excap3d-release JSON files?
If helpful, I can attach the single-scene evidence for 286b55a2bf.

Thanks.

Activity

  1. cy94 commented on Mar 31, 2026

    @cy94
    Owner

    Hi, the instance IDs should be the same as in the released data. Are you using the latest v2 version of the ScanNet++ data? And is the issue only in the validation set?

  2. zhaoyibo0422-code commented on Mar 31, 2026

    @zhaoyibo0422-code
    Author

    Thanks for the quick reply.

    Yes, I am using the latest public ScanNet++ v2 release and the official toolbox pipeline:

    • data_root: /mnt/Datasets/scannetpp/data
    • labels_path: /mnt/Datasets/scannetpp/metadata/semantic_benchmark/top100.txt
    • instance_labels_path: /mnt/Datasets/scannetpp/metadata/semantic_benchmark/top100_instance.txt
    • mapping_file: /mnt/Datasets/scannetpp/metadata/semantic_benchmark/map_benchmark.csv
    • list_path: /mnt/Datasets/scannetpp/splits/nvs_sem_train_val.txt

    I generated the full .pth with:
    python -m semantic.prep.prepare_training_data semantic/configs/prepare_training_data.yml

    Then I ran sample_pth.py and converted to Mask3D .npy.

    From the toolbox code, vtx_instance_anno_id is assigned from segGroups[].objectId, and I verified:

    • full .pth and sampled .pth have the same instance IDs
    • sampled .pth and final .npy also have the same instance IDs

    So the mismatch seems to be between the public ExCap3D caption JSON keys and segGroups[].objectId in the public v2 ScanNet++ data.

    Concrete example: scene 286b55a2bf

    • in scans/segments_anno.json:
      • toilet -> objectId=51
      • sink -> objectId=50
      • soap dispenser -> objectId=49
      • door -> objectId=63
      • cabinet -> objectId=72
    • but in the public ExCap3D caption JSON for the same scene, the corresponding captions are keyed as:
      • 13 -> toilet caption
      • 14 -> sink caption
      • 15 -> soap dispenser caption
      • 32 -> door caption
      • 33 -> cabinet caption

    So at least for this scene, the caption keys do not match objectId.

    I first noticed it on val, but it does not seem val-only. Across the public train+val caption release, I found mixed scene-level key patterns rather than a single consistent mapping.

    Could you clarify whether the public caption JSON keys are expected to match segGroups[].objectId, or whether they correspond to another scene-local ID (e.g. segGroup index / instance_ndx)? If a remapping step is required for the public release, could you share the official procedure?Duplicate of #

  3. zhaoyibo0422-code commented on Apr 1, 2026

    @zhaoyibo0422-code
    Author

    From my checks, the caption JSON keys do not behave like a single global ID field from the raw public ScanNet++ v2 annotations. Across scenes, they look mixed: some match objectId, some match segGroup index, and some match segGroup index + 1.

    So I suspect “the released data” in your comment may refer to the released processed ExCap3D/Mask3D data rather than raw ScanNet++ v2 + self-generated preprocessing.

    Could you clarify which exact instance-id field in the released processed data is expected to match the public caption JSON keys?

    • segGroups[].objectId / vtx_instance_anno_id
    • segGroup index / vtx_instance_labels
    • or another released remapped scene-local instance id
  4. SpeedyGonzales949 commented on May 8, 2026

    @SpeedyGonzales949

    I have the same issue. e.g.: Ids also do not match for scene ef69d58016.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions