Changed code from line 178 to 181

[hungryGeek16]

Context:

• So here I wanted to train a dataset that is relatively smaller than imagenet , on a lightweight architecture.
• I found Mobilenet variants suitable for this task and did the required setup for the suitable environment on colab, and executed th given command below to start training as directed for normal CNN architectures:

!python -m torch.distributed.launch --nproc_per_node=1 main_dino.py --arch mobilenet_v2 --optimizer sgd --lr 0.03 --weight_decay 1e-4 --weight_decay_end 1e-4 --global_crops_scale 0.14 1 --local_crops_scale 0.05 0.14 --data_path /path/to/my/dataset --output_dir /path/to/my/output/folder

• After running this, the script gave an error as shown below:

# ERROR
/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See 
https://pytorch.org/docs/stable/distributed.html#launch-utility for 
further instructions

  FutureWarning,
Using cache found in /root/.cache/torch/hub/facebookresearch_xcit_main
| distributed init (rank 0): env://
git:
  sha: cb711401860da580817918b9167ed73e3eef3dcf, status: has uncommited changes, branch: main

arch: mobilenet_v2
batch_size_per_gpu: 64
clip_grad: 3.0
data_path: /content/raw-img
dist_url: env://
drop_path_rate: 0.1
epochs: 100
freeze_last_layer: 1
global_crops_scale: [0.14, 1.0]
gpu: 0
local_crops_number: 8
local_crops_scale: [0.05, 0.14]
local_rank: 0
lr: 0.03
min_lr: 1e-06
momentum_teacher: 0.996
norm_last_layer: True
num_workers: 10
optimizer: sgd
out_dim: 65536
output_dir: output
patch_size: 16
rank: 0
saveckp_freq: 20
seed: 0
teacher_temp: 0.04
use_bn_in_head: False
use_fp16: True
warmup_epochs: 10
warmup_teacher_temp: 0.04
warmup_teacher_temp_epochs: 0
weight_decay: 0.0001
weight_decay_end: 0.0001
world_size: 1
/usr/local/lib/python3.7/dist-packages/torchvision/transforms/transforms.py:853: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py:481: UserWarning: This DataLoader will create 10 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  cpuset_checked))
Data loaded: there are 26179 images.
Using cache found in /root/.cache/torch/hub/facebookresearch_xcit_main
Traceback (most recent call last):
  File "main_dino.py", line 472, in <module>
    train_dino(args)
  File "main_dino.py", line 179, in train_dino
    embed_dim = student.fc.weight.shape[1]
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1178, in __getattr__
    type(self).__name__, name))
AttributeError: 'MobileNetV2' object has no attribute 'fc'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 755) of binary: /usr/bin/python3
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/usr/local/lib/python3.7/dist-packages/torch/distributed/run.py", line 713, in run
    )(*cmd_args)
  File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
main_dino.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2021-12-02_05:33:46
  host      : 1563bdaf9696
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 755)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

Solution:

• After a while, I understood the error was due to the fact that Mobilenets doesn't have fc attribute, so instead I used embed_dim = student.classifier[1].out_features at line 178 of main_dino.py.

• So this is how I changed it ,

if args.arch.find("mobile") == -1:
    embed_dim = student.fc.weight.shape[1]
else:
    embed_dim = student.classifier[1].out_features

[facebook-github-bot]

Hi @rahulmangalampalli!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at [email protected]. Thanks!

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[woctezuma]

#162

[HollrayChan]

@rahulmangalampalli Hi，I am conducting related experiments. How does your MobileNet or v2 v3 perform on dino ? the parameters in the comments work well?