Score_MicroNet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Counting the Prameters and Operations for MicroNet\n",
    "\n",
    "Here is the plan.\n",
    "\n",
    "You can select the version of micronet. There are two versions and the checkpoints we uploaded on github are from ver2 network,\n",
    "so if you want to verify our model, please choose the network version as 'ver2'.\n",
    "\n",
    "Our network was trained with FP-32 precision, but we will calculate our score with freebie quantization rule.\n",
    "Therefore, we calculate the number of parameters and multiplication operations with the assumption of 16bit not on addition.\n",
    "\n",
    "Our network has below operations:\n",
    "\n",
    "- Module: `Convolution`, `AveragePool`, `FullyConnected`\n",
    "- Activation: `HardSwish`, `Sigmoid`\n",
    "\n",
    "We also used the batch normalization, but we heard that we can count except BN in terms of storage and for math operations even BN is not merged just as the code shows.\n",
    "\n",
    "***\n",
    "We calculate the math operations with our method from `Counting` as considering the sparsity.\n",
    "we count the `Hard Swish` function as below:\n",
    "\n",
    "addition: 1, multiplication: 4\n",
    "\n",
    "Hard swish function have this formula: x * relu_6(x + 3.0) / 6.0. From your previous annoucnments,\n",
    "relu_6 function have two multiplicaiton operations, so you can easily get above results.\n",
    "\n",
    "In our counting code, actually this doesnot capture our hard swish fuction, but count with relu 6 function.\n",
    "However, we only use the relu6 function in Hard swish activation, so it is proper to get exact flops count."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import argparse\n",
    "import shutil\n",
    "import math\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import numpy as np\n",
    "from Counting import count\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from Utils import *\n",
    "from Models import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### These are for Score Calculation.\n",
    "- count_nonzero: count the number of nonzero parameters except BN\n",
    "- micro_score: get score. Flops are already calculated with sparsity, so we multiply the non zero ratio only at params. In addition, we did not use any quantization so that we can assume freebie quantization rule. Therefore, we calculate the score with multiplying 0.5 both on multiplications and params."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def count_nonzero(net):\n",
    "    num = 0 \n",
    "    for module in net.parameters():\n",
    "        if module.ndimension() != 1:\n",
    "            num += torch.sum(torch.abs(module.flatten()) != 0.)\n",
    "    return num.item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def micro_score(net, precision = 'Freebie'):\n",
    "    input = torch.randn(1, 3, 32, 32).to(net.device)\n",
    "    addflops, multflops, params = count(net, inputs=(input, ))\n",
    "    non_zero_ratio = count_nonzero(net) / params\n",
    "\n",
    "    params = params * non_zero_ratio\n",
    "    #use fp-16bit\n",
    "    if precision == 'Freebie':\n",
    "        multflops = multflops / 2\n",
    "        params = params / 2\n",
    "    \n",
    "    score = (params/36500000 + (addflops + multflops)/10490000000)\n",
    "    print('Non zero ratio: {}'.format(non_zero_ratio))\n",
    "    print('Score: {}, flops: {}, params: {}'.format(score, addflops + multflops, params))\n",
    "    return score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Model\n",
    "\n",
    "To verify our checkpoints, here is the details of checkpoints.\n",
    "Here, we will verify our ver1, ver2 checkpoints. They are in the `Checkpoint_ver1` and`Checkpoint_ver2` folder, respectively.\n",
    "\n",
    "#### Ver1\n",
    "- micronet_v1.t7: version 1 model with no pruning\n",
    "- micronet_v1_pr_60.t7: version 1 model with pruning 60%. --> This is our ver1 checkpoint to submit.\n",
    "\n",
    "#### Ver2\n",
    "- micronet_v2.t7: version 2 model with no pruning\n",
    "- micronet_v2_pr_15.t7: version 2 model with 15% pruning\n",
    "- micronet_v2_pr_30.t7: version 2 model with 30% pruning\n",
    "- micronet_v2_pr_45.t7: version 2 model with 45% pruning\n",
    "- micronet_v2_pr_60.t7: version 2 model with 60% pruning --> This is our ver2 checkpoint to submit.\n",
    "\n",
    "The `micronet_v1.t7` and `micronet_v2.t7`  files have four checkpoints with dictionary type, so if you want to load the checkpoints for verification, load the checkpoints with the key <strong>'net4'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|| Prepare dataset with FastAutoAugmentation ||\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "train_loader, test_loader, num_classes = transform_data_set('CIFAR100', batch_size = 128, augmentation = 'FastAuto')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the test accuracy\n",
    "We here load the final pruned network with 60%, and evaluate with <strong>CIFAR100 test dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Micronet Version 1 (with 60% prune)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 100/100 [00:02<00:00, 49.52it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss: 0.748 | Acc1: 80.070% | Acc5: 95.820%\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(80.07, 0.7479651126265526)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net1 = MicroNet(ver = 'ver1', num_classes = 100, add_se = True, Activation = 'HSwish')\n",
    "checkpoint = torch.load('./Checkpoint_ver1/micronet_v1_pr_60.t7')\n",
    "net1.load_state_dict(checkpoint, strict = False)\n",
    "net1.to(net1.device)\n",
    "eval_32bit(net1, test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Micronet Version 2 (with 60% prune)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 100/100 [00:02<00:00, 45.44it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss: 0.743 | Acc1: 80.040% | Acc5: 95.510%\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(80.04, 0.7427933803200721)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net2 = MicroNet(ver = 'ver2', num_classes = 100, add_se = True, Activation = 'HSwish')\n",
    "checkpoint = torch.load('./Checkpoint_ver2/micronet_v2_pr_60.t7')\n",
    "net2.load_state_dict(checkpoint, strict = False)\n",
    "net2.to(net2.device)\n",
    "eval_32bit(net2, test_loader) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Score\n",
    "Then, you can get the score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Count has not implemented counting method for  CrossEntropyLoss()\n",
      "Register FLOP counter for module Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(32, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(32, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=96, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(96, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(16, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(32, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(16, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(16, 4, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(4, 16, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(16, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(48, 48, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=48, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(48, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(16, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(16, 4, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(4, 16, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(16, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(48, 48, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=48, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(48, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(32, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(8, 32, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(32, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=96, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(96, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(32, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(32, 8, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(8, 32, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(32, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(96, 96, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=96, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(96, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(32, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(12, 48, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(48, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(144, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(144, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(12, 48, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(48, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(144, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(144, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(144, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(48, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(48, 12, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(12, 48, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(48, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(144, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(144, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(144, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(72, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(72, 18, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(18, 72, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(72, 216, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=216, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(72, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(72, 18, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(18, 72, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(72, 216, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=216, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(72, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(72, 18, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(18, 72, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(72, 216, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=216, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(72, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(72, 18, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(18, 72, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(72, 216, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=216, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 72, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(72, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(72, 18, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(18, 72, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(72, 216, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 216, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=216, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(216, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(216, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(20, 80, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(80, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(240, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(240, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(240, 88, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(88, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(80, 88, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(88, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(88, 22, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(22, 88, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(88, 264, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(264, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(264, 264, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=264, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(264, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(264, 88, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(88, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(88, 22, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(22, 88, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(88, 264, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(264, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(264, 264, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=264, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(264, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(264, 106, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(106, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(88, 106, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(106, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(106, 26, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(26, 106, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Dropout(p=0.3)\n",
      "Register FLOP counter for module Linear(in_features=106, out_features=100, bias=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Non zero ratio: 0.39999911201094\n",
      "Score: 0.004901393953798138, flops: 25523716.0, params: 90091.0\n",
      "Count has not implemented counting method for  CrossEntropyLoss()\n",
      "Register FLOP counter for module Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(32, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(32, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(80, 80, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=80, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(20, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(32, 20, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(20, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(20, 5, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(5, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(20, 50, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(50, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(50, 50, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=50, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(50, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(50, 20, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(20, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(20, 5, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(5, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(20, 50, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(50, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(50, 50, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=50, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(50, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(50, 36, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(36, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(36, 9, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(9, 36, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(36, 90, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(90, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(90, 90, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=90, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(90, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(90, 36, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(36, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(36, 9, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(9, 36, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(36, 90, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(90, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(90, 90, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=90, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(90, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(90, 56, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(56, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(36, 56, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(56, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(56, 14, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(14, 56, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(56, 140, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(140, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(140, 140, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=140, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(140, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(140, 56, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(56, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(56, 14, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(14, 56, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(56, 140, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(140, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(140, 140, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=140, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(140, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(140, 56, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(56, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(56, 14, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(14, 56, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(56, 140, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(140, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(140, 140, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=140, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(140, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(140, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(20, 80, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(80, 200, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 200, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=200, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(20, 80, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(80, 200, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 200, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=200, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(20, 80, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(80, 200, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 200, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=200, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(20, 80, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(80, 200, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 200, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=200, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 80, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(80, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(80, 20, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(20, 80, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(80, 200, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 200, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=200, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(200, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(200, 88, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(88, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(88, 22, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(22, 88, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(88, 220, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(220, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(220, 220, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=220, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(220, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(220, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(88, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(24, 96, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(96, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(240, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(240, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(240, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(96, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Count has not implemented counting method for  Sequential()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(96, 24, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(24, 96, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module Conv2d(96, 240, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(240, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(240, 240, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=240, bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(240, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module Conv2d(240, 114, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(114, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Register FLOP counter for module Conv2d(96, 114, kernel_size=(1, 1), stride=(1, 1), bias=False)\n",
      "Register FLOP counter for module BatchNorm2d(114, eps=1e-05, momentum=0.01, affine=True, track_running_stats=True)\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Conv2d(114, 28, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Conv2d(28, 114, kernel_size=(1, 1), stride=(1, 1))\n",
      "Register FLOP counter for module Sigmoid()\n",
      "Register FLOP counter for module AdaptiveAvgPool2d(output_size=1)\n",
      "Register FLOP counter for module Dropout(p=0.3)\n",
      "Register FLOP counter for module Linear(in_features=114, out_features=100, bias=True)\n",
      "Register FLOP counter for module ReLU6(inplace)\n",
      "Non zero ratio: 0.4000008489252606\n",
      "Score: 0.005064351727019863, flops: 26041594.0, params: 94237.0\n"
     ]
    }
   ],
   "source": [
    "ver1_score = micro_score(net1)\n",
    "ver2_score = micro_score(net2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "micronet version1 score: 0.0049, micronet version2 score: 0.0051\n"
     ]
    }
   ],
   "source": [
    "print('micronet version1 score: {:.4f}, micronet version2 score: {:.4f}'.format(ver1_score, ver2_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submission\n",
    "\n",
    "#### As above results, we would like to submit version1 checkpoint. Then, Our score is <strong> `0.0049`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}