Inconsistency between feature_info layer names and PyTorch child module names causes IntermediateLayerGetter error when using the timm-based HGNetv2 through TimmModel

[yuki-inaho]

Overview

When creating a model via timm.create_model(...), model.feature_info.module_name() returns layer names in a dot notation such as "stages.0", "stages.1". However, PyTorch’s named_children() for the same model yields underscore notation like "stages_0", "stages_1".

IntermediateLayerGetter relies on named_children(), meaning if a user specifies return_layers=["stages.0", "stages.1", ...] (following dot notation from timm’s feature_info), PyTorch cannot find the corresponding modules (which are actually named "stages_0", etc.) and raises the following error:

ValueError: return_layers are not present in model. ['stem', 'stages_0', 'stages_1', 'stages_2', 'stages_3']

From this message, I can see that IntermediateLayerGetter is looking for "stages_0", but timm’s feature info suggests the user should use "stages.0". This mismatch leads to a conflict.

---

Steps to Reproduce

Commit: df742232c6e046e763e57573a78039828f19fb6b

1. Create a TimmModel instance that internally calls timm.create_model(...) and check the module names for the return layers.
2. Pass return_layers containing the layer names as returned by model.feature_info.module_name() (e.g., "stages.0").
3. IntermediateLayerGetter references named_children() under the hood, which cannot find "stages.0". This raises a ValueError.

Example code:

# Sample code
import sys
sys.path.append("RT-DETR/rtdetrv2_pytorch")

import timm
from src.nn.backbone.timm_model import TimmModel

model_timm_hgnetv2 = timm.create_model("hgnetv2_b4.ssld_stage2_ft_in1k", pretrained=False, features_only=True)
model_timm_hgnetv2 = model_timm_hgnetv2.eval()
for i, info in enumerate(model_timm_hgnetv2.feature_info):
    print(f"Index {i}: Module {info['module']} - Channels: {info['num_chs']}, Reduction: {info['reduction']}")
# Index 0: Module stages.0 - Channels: 128, Reduction: 4
# Index 1: Module stages.1 - Channels: 512, Reduction: 8
# Index 2: Module stages.2 - Channels: 1024, Reduction: 16
# Index 3: Module stages.3 - Channels: 2048, Reduction: 32

model_timm = TimmModel(
    name="hgnetv2_b4.ssld_stage2_ft_in1k",
    return_layers=["stages.0", "stages.1", "stages.2", "stages.3"],
    features_only=True,
)

# Traceback (most recent call last):
#  File "/home/yuki-inaho/Project/rt-detr_sandbox/test.py", line 16, in <module>
#    model_timm = TimmModel(
#  File "/home/yuki-inaho/Project/rt-detr_sandbox/RT-DETR/rtdetrv2_pytorch/src/nn/backbone/timm_model.py", line 42, in __init__
#    self.model = IntermediateLayerGetter(model, return_layers)
#  File "/home/yuki-inaho/Project/rt-detr_sandbox/RT-DETR/rtdetrv2_pytorch/src/nn/backbone/utils.py", line 32, in __init__
#    raise ValueError("return_layers are not present in model. {}"\
# ValueError: return_layers are not present in model. ['stem', 'stages_0', 'stages_1', 'stages_2', 'stages_3']
#
# from:
# https://github.com/lyuwenyu/RT-DETR/blob/b8957b30431abc938db16016f6b5e395b562c5dd/rtdetrv2_pytorch/src/nn/backbone/utils.py

As explained, PyTorch sees "stages_0" while timm returns "stages.0", causing a naming mismatch.

---

Possible Fix

1. Provide a mapping to convert dot notation to underscore notation

Here’s an example fix:
Create a dictionary to replace timm’s feature_info layer names ("stages.0" etc.) with PyTorch’s underscore names ("stages_0") before passing them to IntermediateLayerGetter.

+++ rtdetrv2_pytorch/src/nn/backbone/timm_model_copy.py	2025-02-14 17:38:21.648183784 +0900
@@ -37,9 +37,23 @@
 
         assert set(return_layers).issubset(model.feature_info.module_name()), \
             f'return_layers should be a subset of {model.feature_info.module_name()}'
-        
+
+        # Create a mapping to replace layer names written in feature_info with layer names in PyTorch
+        feature_to_child_map = {}
+        for m_name in model.feature_info.module_name():
+            # Example: 'stages.0' -> 'stages_0'
+            #          'stages.1' -> 'stages_1' ...
+            mapped_name = m_name.replace(".", "_")
+            feature_to_child_map[m_name] = mapped_name
+
+        # Replace 'stages.0', 'stages.1', etc. in the received return_layers
+        # with 'stages_0', 'stages_1', etc. and pass them to IntermediateLayerGetter
+        new_return_layers = []
+        for layer in return_layers:
+            new_return_layers.append(feature_to_child_map[layer])
+
         # self.model = model
-        self.model = IntermediateLayerGetter(model, return_layers)
+        self.model = IntermediateLayerGetter(model, return_layers=new_return_layers)
 
         return_idx = [model.feature_info.module_name().index(name) for name in return_layers]
         self.strides = [model.feature_info.reduction()[i] for i in return_idx]

By using the new mapping, TimmModel can accept ["stages.0", "stages.1"] and internally convert them to ["stages_0", "stages_1"], thus eliminating the error.

---

Background

I are using RT-DETR with an HGNetv2 backbone. timm provides pretrained HGNetv2 which could be conveniently reused for weight initialization and module structures. During experimentation, I discovered that timm’s HGNetv2 returns dotted layer names (e.g. "stages.0"), conflicting with PyTorch’s underscore names ("stages_0") in IntermediateLayerGetter.

By applying the patch above, I managed to train RT-DETR successfully with timm-based HGNetv2 using a config like:

__include__:
  [
    "../dataset/coco_detection.yml",
    "../runtime.yml",
    "./include/dataloader.yml",
    "./include/optimizer.yml",
    "./include/rtdetrv2_r50vd.yml",
  ]

output_dir: ./output/rtdetrv2_hgnetv2_l_6x_coco

RTDETR:
  backbone: TimmModel

TimmModel:
  name: "hgnetv2_b4.ssld_stage2_ft_in1k"
  return_layers: ["stages.1", "stages.2", "stages.3"]
  pretrained: True
  exportable: True
  features_only: True

optimizer:
  type: AdamW
  params:
    - params: "^(?=.*backbone)(?!.*norm|bn).*$"
      lr: 0.000005
    - params: "^(?=.*(?:encoder|decoder))(?=.*(?:norm|bn)).*$"
      weight_decay: 0.

  lr: 0.0001
  betas: [0.9, 0.999]
  weight_decay: 0.0001

I hope this information helps others.