2022/09/26 10:38:04 - mmengine - INFO - ------------------------------------------------------------ System environment: sys.platform: linux Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0] CUDA available: True numpy_random_seed: 445508762 GPU 0,1,2,3,4,5,6,7: NVIDIA A100-SXM4-80GB CUDA_HOME: /mnt/lustre/share/cuda-11.3 NVCC: Cuda compilation tools, release 11.3, V11.3.109 GCC: gcc (GCC) 5.4.0 PyTorch: 1.12.1 PyTorch compiling details: PyTorch built with: - GCC 9.3 - C++ Version: 201402 - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815) - OpenMP 201511 (a.k.a. OpenMP 4.5) - LAPACK is enabled (usually provided by MKL) - NNPACK is enabled - CPU capability usage: AVX2 - CUDA Runtime 11.3 - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37 - CuDNN 8.3.2 (built against CUDA 11.5) - Magma 2.5.2 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, TorchVision: 0.13.1 OpenCV: 4.6.0 MMEngine: 0.1.0 Runtime environment: cudnn_benchmark: False mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: None Distributed launcher: slurm Distributed training: True GPU number: 8 ------------------------------------------------------------ 2022/09/26 10:38:05 - mmengine - INFO - Config: default_scope = 'mmpose' default_hooks = dict( timer=dict(type='IterTimerHook'), logger=dict(type='LoggerHook', interval=50), param_scheduler=dict(type='ParamSchedulerHook'), checkpoint=dict( type='CheckpointHook', interval=10, max_keep_ckpts=1, save_best='coco/AP', rule='greater'), sampler_seed=dict(type='DistSamplerSeedHook'), visualization=dict(type='PoseVisualizationHook', enable=False)) custom_hooks = [dict(type='SyncBuffersHook')] env_cfg = dict( cudnn_benchmark=False, mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0), dist_cfg=dict(backend='nccl')) vis_backends = [dict(type='LocalVisBackend')] visualizer = dict( type='PoseLocalVisualizer', vis_backends=[dict(type='LocalVisBackend')], name='visualizer') log_processor = dict( type='LogProcessor', window_size=50, by_epoch=True, num_digits=6) log_level = 'INFO' load_from = None resume = False file_client_args = dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' })) train_cfg = dict(by_epoch=True, max_epochs=210, val_interval=10) val_cfg = dict() test_cfg = dict() optim_wrapper = dict(optimizer=dict(type='Adam', lr=0.0005)) param_scheduler = [ dict( type='LinearLR', begin=0, end=500, start_factor=0.001, by_epoch=False), dict( type='MultiStepLR', begin=0, end=210, milestones=[170, 200], gamma=0.1, by_epoch=True) ] auto_scale_lr = dict(base_batch_size=512) codec = dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2, unbiased=True) model = dict( type='TopdownPoseEstimator', data_preprocessor=dict( type='PoseDataPreprocessor', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], bgr_to_rgb=True), backbone=dict( type='ResNet', depth=101, init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet101')), head=dict( type='HeatmapHead', in_channels=2048, out_channels=17, loss=dict(type='KeypointMSELoss', use_target_weight=True), decoder=dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2, unbiased=True)), test_cfg=dict(flip_test=True, flip_mode='heatmap', shift_heatmap=True)) dataset_type = 'CocoDataset' data_mode = 'topdown' data_root = 'data/coco/' train_pipeline = [ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='RandomFlip', direction='horizontal'), dict(type='RandomHalfBody'), dict(type='RandomBBoxTransform'), dict(type='TopdownAffine', input_size=(192, 256)), dict( type='GenerateTarget', target_type='heatmap', encoder=dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2, unbiased=True)), dict(type='PackPoseInputs') ] val_pipeline = [ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(192, 256)), dict(type='PackPoseInputs') ] train_dataloader = dict( batch_size=64, num_workers=2, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=True), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_train2017.json', data_prefix=dict(img='train2017/'), pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='RandomFlip', direction='horizontal'), dict(type='RandomHalfBody'), dict(type='RandomBBoxTransform'), dict(type='TopdownAffine', input_size=(192, 256)), dict( type='GenerateTarget', target_type='heatmap', encoder=dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2, unbiased=True)), dict(type='PackPoseInputs') ])) val_dataloader = dict( batch_size=32, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_val2017.json', bbox_file= 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json', data_prefix=dict(img='val2017/'), test_mode=True, pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(192, 256)), dict(type='PackPoseInputs') ])) test_dataloader = dict( batch_size=32, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_val2017.json', bbox_file= 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json', data_prefix=dict(img='val2017/'), test_mode=True, pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(192, 256)), dict(type='PackPoseInputs') ])) val_evaluator = dict( type='CocoMetric', ann_file='data/coco/annotations/person_keypoints_val2017.json') test_evaluator = dict( type='CocoMetric', ann_file='data/coco/annotations/person_keypoints_val2017.json') launcher = 'slurm' work_dir = '/mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/' 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "data sampler" registry tree. As a workaround, the current "data sampler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optimizer wrapper constructor" registry tree. As a workaround, the current "optimizer wrapper constructor" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optimizer" registry tree. As a workaround, the current "optimizer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optim_wrapper" registry tree. As a workaround, the current "optim_wrapper" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:40 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:44 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "data sampler" registry tree. As a workaround, the current "data sampler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:46 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:48 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 10:38:48 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. Name of parameter - Initialization information backbone.conv1.weight - torch.Size([64, 3, 7, 7]): PretrainedInit: load from torchvision://resnet101 backbone.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.conv1.weight - torch.Size([64, 64, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.downsample.0.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.downsample.1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.0.downsample.1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.1.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer1.2.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.conv1.weight - torch.Size([128, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.downsample.1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.0.downsample.1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.1.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.2.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer2.3.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.conv1.weight - torch.Size([256, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.downsample.0.weight - torch.Size([1024, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.downsample.1.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.0.downsample.1.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.1.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.2.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.3.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.4.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.5.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.6.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.7.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.8.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.9.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.10.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.11.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.12.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.13.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.14.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.15.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.16.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.17.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.18.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.19.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.20.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.21.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer3.22.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.conv1.weight - torch.Size([512, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.downsample.0.weight - torch.Size([2048, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.downsample.1.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.0.downsample.1.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.1.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 backbone.layer4.2.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet101 head.deconv_layers.0.weight - torch.Size([2048, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.1.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.1.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.3.weight - torch.Size([256, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.4.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.4.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.6.weight - torch.Size([256, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.7.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.7.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.final_layer.weight - torch.Size([17, 256, 1, 1]): NormalInit: mean=0, std=0.001, bias=0 head.final_layer.bias - torch.Size([17]): NormalInit: mean=0, std=0.001, bias=0 2022/09/26 10:38:48 - mmengine - INFO - Checkpoints will be saved to /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256 by HardDiskBackend. 2022/09/26 10:39:38 - mmengine - INFO - Epoch(train) [1][50/293] lr: 4.954910e-05 eta: 16:53:00 time: 0.988621 data_time: 0.295220 memory: 9504 loss_kpt: 0.002156 acc_pose: 0.213289 loss: 0.002156 2022/09/26 10:40:10 - mmengine - INFO - Epoch(train) [1][100/293] lr: 9.959920e-05 eta: 13:56:43 time: 0.645871 data_time: 0.166869 memory: 9504 loss_kpt: 0.001748 acc_pose: 0.377474 loss: 0.001748 2022/09/26 10:40:47 - mmengine - INFO - Epoch(train) [1][150/293] lr: 1.496493e-04 eta: 13:31:09 time: 0.744256 data_time: 0.141651 memory: 9504 loss_kpt: 0.001453 acc_pose: 0.484978 loss: 0.001453 2022/09/26 10:41:20 - mmengine - INFO - Epoch(train) [1][200/293] lr: 1.996994e-04 eta: 12:53:20 time: 0.647536 data_time: 0.320973 memory: 9504 loss_kpt: 0.001318 acc_pose: 0.511024 loss: 0.001318 2022/09/26 10:41:53 - mmengine - INFO - Epoch(train) [1][250/293] lr: 2.497495e-04 eta: 12:35:59 time: 0.674706 data_time: 0.131474 memory: 9504 loss_kpt: 0.001253 acc_pose: 0.587814 loss: 0.001253 2022/09/26 10:42:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:42:52 - mmengine - INFO - Epoch(train) [2][50/293] lr: 3.428427e-04 eta: 10:43:41 time: 0.629127 data_time: 0.132896 memory: 9504 loss_kpt: 0.001174 acc_pose: 0.609098 loss: 0.001174 2022/09/26 10:43:21 - mmengine - INFO - Epoch(train) [2][100/293] lr: 3.928928e-04 eta: 10:35:27 time: 0.571699 data_time: 0.084539 memory: 9504 loss_kpt: 0.001151 acc_pose: 0.615625 loss: 0.001151 2022/09/26 10:43:51 - mmengine - INFO - Epoch(train) [2][150/293] lr: 4.429429e-04 eta: 10:31:10 time: 0.590913 data_time: 0.082368 memory: 9504 loss_kpt: 0.001134 acc_pose: 0.690881 loss: 0.001134 2022/09/26 10:44:20 - mmengine - INFO - Epoch(train) [2][200/293] lr: 4.929930e-04 eta: 10:27:45 time: 0.591773 data_time: 0.082843 memory: 9504 loss_kpt: 0.001124 acc_pose: 0.617206 loss: 0.001124 2022/09/26 10:44:49 - mmengine - INFO - Epoch(train) [2][250/293] lr: 5.000000e-04 eta: 10:23:02 time: 0.572194 data_time: 0.129734 memory: 9504 loss_kpt: 0.001103 acc_pose: 0.602779 loss: 0.001103 2022/09/26 10:45:15 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:45:47 - mmengine - INFO - Epoch(train) [3][50/293] lr: 5.000000e-04 eta: 9:42:02 time: 0.638139 data_time: 0.154690 memory: 9504 loss_kpt: 0.001084 acc_pose: 0.632150 loss: 0.001084 2022/09/26 10:46:17 - mmengine - INFO - Epoch(train) [3][100/293] lr: 5.000000e-04 eta: 9:44:18 time: 0.610612 data_time: 0.095166 memory: 9504 loss_kpt: 0.001072 acc_pose: 0.616794 loss: 0.001072 2022/09/26 10:46:46 - mmengine - INFO - Epoch(train) [3][150/293] lr: 5.000000e-04 eta: 9:43:12 time: 0.567328 data_time: 0.082174 memory: 9504 loss_kpt: 0.001058 acc_pose: 0.626484 loss: 0.001058 2022/09/26 10:47:16 - mmengine - INFO - Epoch(train) [3][200/293] lr: 5.000000e-04 eta: 9:44:17 time: 0.599687 data_time: 0.098860 memory: 9504 loss_kpt: 0.001048 acc_pose: 0.650950 loss: 0.001048 2022/09/26 10:47:46 - mmengine - INFO - Epoch(train) [3][250/293] lr: 5.000000e-04 eta: 9:45:20 time: 0.602450 data_time: 0.144420 memory: 9504 loss_kpt: 0.001010 acc_pose: 0.652439 loss: 0.001010 2022/09/26 10:48:10 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:48:41 - mmengine - INFO - Epoch(train) [4][50/293] lr: 5.000000e-04 eta: 9:19:22 time: 0.615133 data_time: 0.138033 memory: 9504 loss_kpt: 0.001000 acc_pose: 0.709758 loss: 0.001000 2022/09/26 10:49:11 - mmengine - INFO - Epoch(train) [4][100/293] lr: 5.000000e-04 eta: 9:21:10 time: 0.597812 data_time: 0.108258 memory: 9504 loss_kpt: 0.000996 acc_pose: 0.657222 loss: 0.000996 2022/09/26 10:49:25 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:49:43 - mmengine - INFO - Epoch(train) [4][150/293] lr: 5.000000e-04 eta: 9:24:48 time: 0.639635 data_time: 0.127874 memory: 9504 loss_kpt: 0.000997 acc_pose: 0.671809 loss: 0.000997 2022/09/26 10:50:13 - mmengine - INFO - Epoch(train) [4][200/293] lr: 5.000000e-04 eta: 9:26:27 time: 0.605631 data_time: 0.104352 memory: 9504 loss_kpt: 0.000981 acc_pose: 0.627848 loss: 0.000981 2022/09/26 10:50:44 - mmengine - INFO - Epoch(train) [4][250/293] lr: 5.000000e-04 eta: 9:28:00 time: 0.607192 data_time: 0.086585 memory: 9504 loss_kpt: 0.000989 acc_pose: 0.681678 loss: 0.000989 2022/09/26 10:51:07 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:51:38 - mmengine - INFO - Epoch(train) [5][50/293] lr: 5.000000e-04 eta: 9:09:31 time: 0.621421 data_time: 0.111016 memory: 9504 loss_kpt: 0.000983 acc_pose: 0.698740 loss: 0.000983 2022/09/26 10:52:07 - mmengine - INFO - Epoch(train) [5][100/293] lr: 5.000000e-04 eta: 9:10:33 time: 0.584616 data_time: 0.082400 memory: 9504 loss_kpt: 0.000987 acc_pose: 0.734003 loss: 0.000987 2022/09/26 10:52:37 - mmengine - INFO - Epoch(train) [5][150/293] lr: 5.000000e-04 eta: 9:11:58 time: 0.597640 data_time: 0.098044 memory: 9504 loss_kpt: 0.000967 acc_pose: 0.674404 loss: 0.000967 2022/09/26 10:53:09 - mmengine - INFO - Epoch(train) [5][200/293] lr: 5.000000e-04 eta: 9:14:36 time: 0.634363 data_time: 0.097368 memory: 9504 loss_kpt: 0.000939 acc_pose: 0.727356 loss: 0.000939 2022/09/26 10:53:40 - mmengine - INFO - Epoch(train) [5][250/293] lr: 5.000000e-04 eta: 9:16:16 time: 0.613882 data_time: 0.082367 memory: 9504 loss_kpt: 0.000943 acc_pose: 0.730783 loss: 0.000943 2022/09/26 10:54:06 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:54:37 - mmengine - INFO - Epoch(train) [6][50/293] lr: 5.000000e-04 eta: 9:01:52 time: 0.622439 data_time: 0.101166 memory: 9504 loss_kpt: 0.000943 acc_pose: 0.675593 loss: 0.000943 2022/09/26 10:55:07 - mmengine - INFO - Epoch(train) [6][100/293] lr: 5.000000e-04 eta: 9:02:50 time: 0.586146 data_time: 0.080632 memory: 9504 loss_kpt: 0.000940 acc_pose: 0.728407 loss: 0.000940 2022/09/26 10:55:35 - mmengine - INFO - Epoch(train) [6][150/293] lr: 5.000000e-04 eta: 9:02:54 time: 0.560119 data_time: 0.080034 memory: 9504 loss_kpt: 0.000920 acc_pose: 0.625462 loss: 0.000920 2022/09/26 10:56:12 - mmengine - INFO - Epoch(train) [6][200/293] lr: 5.000000e-04 eta: 9:08:15 time: 0.737226 data_time: 0.112909 memory: 9504 loss_kpt: 0.000944 acc_pose: 0.653447 loss: 0.000944 2022/09/26 10:56:41 - mmengine - INFO - Epoch(train) [6][250/293] lr: 5.000000e-04 eta: 9:08:50 time: 0.585499 data_time: 0.103261 memory: 9504 loss_kpt: 0.000926 acc_pose: 0.659181 loss: 0.000926 2022/09/26 10:57:06 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:57:38 - mmengine - INFO - Epoch(train) [7][50/293] lr: 5.000000e-04 eta: 8:57:31 time: 0.643827 data_time: 0.100927 memory: 9504 loss_kpt: 0.000918 acc_pose: 0.741009 loss: 0.000918 2022/09/26 10:58:09 - mmengine - INFO - Epoch(train) [7][100/293] lr: 5.000000e-04 eta: 8:58:55 time: 0.608875 data_time: 0.092243 memory: 9504 loss_kpt: 0.000899 acc_pose: 0.668739 loss: 0.000899 2022/09/26 10:58:39 - mmengine - INFO - Epoch(train) [7][150/293] lr: 5.000000e-04 eta: 9:00:18 time: 0.612366 data_time: 0.092202 memory: 9504 loss_kpt: 0.000901 acc_pose: 0.696791 loss: 0.000901 2022/09/26 10:59:11 - mmengine - INFO - Epoch(train) [7][200/293] lr: 5.000000e-04 eta: 9:01:55 time: 0.625652 data_time: 0.109362 memory: 9504 loss_kpt: 0.000911 acc_pose: 0.690304 loss: 0.000911 2022/09/26 10:59:36 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 10:59:41 - mmengine - INFO - Epoch(train) [7][250/293] lr: 5.000000e-04 eta: 9:02:50 time: 0.601189 data_time: 0.090360 memory: 9504 loss_kpt: 0.000918 acc_pose: 0.755804 loss: 0.000918 2022/09/26 11:00:07 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:00:37 - mmengine - INFO - Epoch(train) [8][50/293] lr: 5.000000e-04 eta: 8:52:03 time: 0.596011 data_time: 0.127913 memory: 9504 loss_kpt: 0.000910 acc_pose: 0.708446 loss: 0.000910 2022/09/26 11:01:09 - mmengine - INFO - Epoch(train) [8][100/293] lr: 5.000000e-04 eta: 8:53:57 time: 0.639308 data_time: 0.207838 memory: 9504 loss_kpt: 0.000891 acc_pose: 0.788282 loss: 0.000891 2022/09/26 11:01:39 - mmengine - INFO - Epoch(train) [8][150/293] lr: 5.000000e-04 eta: 8:54:46 time: 0.596192 data_time: 0.148833 memory: 9504 loss_kpt: 0.000873 acc_pose: 0.784226 loss: 0.000873 2022/09/26 11:02:09 - mmengine - INFO - Epoch(train) [8][200/293] lr: 5.000000e-04 eta: 8:55:39 time: 0.601262 data_time: 0.091639 memory: 9504 loss_kpt: 0.000889 acc_pose: 0.703854 loss: 0.000889 2022/09/26 11:02:39 - mmengine - INFO - Epoch(train) [8][250/293] lr: 5.000000e-04 eta: 8:56:37 time: 0.608551 data_time: 0.085811 memory: 9504 loss_kpt: 0.000883 acc_pose: 0.740652 loss: 0.000883 2022/09/26 11:03:04 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:05:11 - mmengine - INFO - Epoch(train) [9][50/293] lr: 5.000000e-04 eta: 9:26:57 time: 2.526006 data_time: 0.131984 memory: 9504 loss_kpt: 0.000877 acc_pose: 0.679222 loss: 0.000877 2022/09/26 11:05:41 - mmengine - INFO - Epoch(train) [9][100/293] lr: 5.000000e-04 eta: 9:27:18 time: 0.615875 data_time: 0.107895 memory: 9504 loss_kpt: 0.000867 acc_pose: 0.736911 loss: 0.000867 2022/09/26 11:06:11 - mmengine - INFO - Epoch(train) [9][150/293] lr: 5.000000e-04 eta: 9:27:21 time: 0.602760 data_time: 0.092983 memory: 9504 loss_kpt: 0.000871 acc_pose: 0.747567 loss: 0.000871 2022/09/26 11:06:42 - mmengine - INFO - Epoch(train) [9][200/293] lr: 5.000000e-04 eta: 9:27:24 time: 0.604730 data_time: 0.083039 memory: 9504 loss_kpt: 0.000881 acc_pose: 0.730052 loss: 0.000881 2022/09/26 11:07:13 - mmengine - INFO - Epoch(train) [9][250/293] lr: 5.000000e-04 eta: 9:27:40 time: 0.616471 data_time: 0.087850 memory: 9504 loss_kpt: 0.000868 acc_pose: 0.780428 loss: 0.000868 2022/09/26 11:07:37 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:08:09 - mmengine - INFO - Epoch(train) [10][50/293] lr: 5.000000e-04 eta: 9:18:47 time: 0.637437 data_time: 0.123639 memory: 9504 loss_kpt: 0.000863 acc_pose: 0.694831 loss: 0.000863 2022/09/26 11:08:38 - mmengine - INFO - Epoch(train) [10][100/293] lr: 5.000000e-04 eta: 9:18:27 time: 0.577793 data_time: 0.081794 memory: 9504 loss_kpt: 0.000861 acc_pose: 0.734203 loss: 0.000861 2022/09/26 11:09:08 - mmengine - INFO - Epoch(train) [10][150/293] lr: 5.000000e-04 eta: 9:18:21 time: 0.590598 data_time: 0.087886 memory: 9504 loss_kpt: 0.000838 acc_pose: 0.699389 loss: 0.000838 2022/09/26 11:09:38 - mmengine - INFO - Epoch(train) [10][200/293] lr: 5.000000e-04 eta: 9:18:34 time: 0.610698 data_time: 0.085985 memory: 9504 loss_kpt: 0.000858 acc_pose: 0.716318 loss: 0.000858 2022/09/26 11:10:07 - mmengine - INFO - Epoch(train) [10][250/293] lr: 5.000000e-04 eta: 9:18:11 time: 0.576093 data_time: 0.083589 memory: 9504 loss_kpt: 0.000872 acc_pose: 0.703379 loss: 0.000872 2022/09/26 11:10:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:10:33 - mmengine - INFO - Saving checkpoint at 10 epochs 2022/09/26 11:10:59 - mmengine - INFO - Epoch(val) [10][50/407] eta: 0:02:43 time: 0.459223 data_time: 0.185845 memory: 9504 2022/09/26 11:11:16 - mmengine - INFO - Epoch(val) [10][100/407] eta: 0:01:46 time: 0.346054 data_time: 0.055638 memory: 1378 2022/09/26 11:11:33 - mmengine - INFO - Epoch(val) [10][150/407] eta: 0:01:25 time: 0.332979 data_time: 0.060415 memory: 1378 2022/09/26 11:11:49 - mmengine - INFO - Epoch(val) [10][200/407] eta: 0:01:06 time: 0.322118 data_time: 0.027326 memory: 1378 2022/09/26 11:12:06 - mmengine - INFO - Epoch(val) [10][250/407] eta: 0:00:53 time: 0.341455 data_time: 0.064996 memory: 1378 2022/09/26 11:12:23 - mmengine - INFO - Epoch(val) [10][300/407] eta: 0:00:35 time: 0.331876 data_time: 0.054182 memory: 1378 2022/09/26 11:12:38 - mmengine - INFO - Epoch(val) [10][350/407] eta: 0:00:17 time: 0.313918 data_time: 0.028457 memory: 1378 2022/09/26 11:12:54 - mmengine - INFO - Epoch(val) [10][400/407] eta: 0:00:02 time: 0.316816 data_time: 0.034164 memory: 1378 2022/09/26 11:13:32 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 11:13:45 - mmengine - INFO - Epoch(val) [10][407/407] coco/AP: 0.620706 coco/AP .5: 0.862037 coco/AP .75: 0.688593 coco/AP (M): 0.587952 coco/AP (L): 0.681383 coco/AR: 0.685359 coco/AR .5: 0.907273 coco/AR .75: 0.751732 coco/AR (M): 0.645315 coco/AR (L): 0.742029 2022/09/26 11:13:48 - mmengine - INFO - The best checkpoint with 0.6207 coco/AP at 10 epoch is saved to best_coco/AP_epoch_10.pth. 2022/09/26 11:14:18 - mmengine - INFO - Epoch(train) [11][50/293] lr: 5.000000e-04 eta: 9:09:40 time: 0.596380 data_time: 0.182956 memory: 9504 loss_kpt: 0.000844 acc_pose: 0.734977 loss: 0.000844 2022/09/26 11:14:30 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:14:48 - mmengine - INFO - Epoch(train) [11][100/293] lr: 5.000000e-04 eta: 9:09:50 time: 0.602936 data_time: 0.079029 memory: 9504 loss_kpt: 0.000862 acc_pose: 0.768053 loss: 0.000862 2022/09/26 11:15:18 - mmengine - INFO - Epoch(train) [11][150/293] lr: 5.000000e-04 eta: 9:09:57 time: 0.600808 data_time: 0.087581 memory: 9504 loss_kpt: 0.000850 acc_pose: 0.708205 loss: 0.000850 2022/09/26 11:15:48 - mmengine - INFO - Epoch(train) [11][200/293] lr: 5.000000e-04 eta: 9:09:51 time: 0.589041 data_time: 0.078468 memory: 9504 loss_kpt: 0.000856 acc_pose: 0.689078 loss: 0.000856 2022/09/26 11:16:18 - mmengine - INFO - Epoch(train) [11][250/293] lr: 5.000000e-04 eta: 9:09:51 time: 0.595265 data_time: 0.089845 memory: 9504 loss_kpt: 0.000844 acc_pose: 0.770196 loss: 0.000844 2022/09/26 11:16:44 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:17:14 - mmengine - INFO - Epoch(train) [12][50/293] lr: 5.000000e-04 eta: 9:02:28 time: 0.613244 data_time: 0.102678 memory: 9504 loss_kpt: 0.000842 acc_pose: 0.773240 loss: 0.000842 2022/09/26 11:17:45 - mmengine - INFO - Epoch(train) [12][100/293] lr: 5.000000e-04 eta: 9:02:46 time: 0.610917 data_time: 0.086272 memory: 9504 loss_kpt: 0.000837 acc_pose: 0.690706 loss: 0.000837 2022/09/26 11:18:15 - mmengine - INFO - Epoch(train) [12][150/293] lr: 5.000000e-04 eta: 9:02:58 time: 0.605435 data_time: 0.092532 memory: 9504 loss_kpt: 0.000843 acc_pose: 0.789119 loss: 0.000843 2022/09/26 11:18:45 - mmengine - INFO - Epoch(train) [12][200/293] lr: 5.000000e-04 eta: 9:02:52 time: 0.586181 data_time: 0.089647 memory: 9504 loss_kpt: 0.000828 acc_pose: 0.738625 loss: 0.000828 2022/09/26 11:19:15 - mmengine - INFO - Epoch(train) [12][250/293] lr: 5.000000e-04 eta: 9:03:00 time: 0.604464 data_time: 0.083746 memory: 9504 loss_kpt: 0.000843 acc_pose: 0.715946 loss: 0.000843 2022/09/26 11:19:41 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:20:11 - mmengine - INFO - Epoch(train) [13][50/293] lr: 5.000000e-04 eta: 8:56:17 time: 0.611947 data_time: 0.124422 memory: 9504 loss_kpt: 0.000818 acc_pose: 0.744587 loss: 0.000818 2022/09/26 11:20:41 - mmengine - INFO - Epoch(train) [13][100/293] lr: 5.000000e-04 eta: 8:56:27 time: 0.602423 data_time: 0.089546 memory: 9504 loss_kpt: 0.000841 acc_pose: 0.704774 loss: 0.000841 2022/09/26 11:21:12 - mmengine - INFO - Epoch(train) [13][150/293] lr: 5.000000e-04 eta: 8:56:40 time: 0.607227 data_time: 0.086924 memory: 9504 loss_kpt: 0.000819 acc_pose: 0.750609 loss: 0.000819 2022/09/26 11:21:44 - mmengine - INFO - Epoch(train) [13][200/293] lr: 5.000000e-04 eta: 8:57:22 time: 0.646553 data_time: 0.083192 memory: 9504 loss_kpt: 0.000844 acc_pose: 0.755208 loss: 0.000844 2022/09/26 11:22:15 - mmengine - INFO - Epoch(train) [13][250/293] lr: 5.000000e-04 eta: 8:57:40 time: 0.617687 data_time: 0.085574 memory: 9504 loss_kpt: 0.000824 acc_pose: 0.735278 loss: 0.000824 2022/09/26 11:22:40 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:23:12 - mmengine - INFO - Epoch(train) [14][50/293] lr: 5.000000e-04 eta: 8:51:41 time: 0.626770 data_time: 0.156766 memory: 9504 loss_kpt: 0.000821 acc_pose: 0.743298 loss: 0.000821 2022/09/26 11:23:41 - mmengine - INFO - Epoch(train) [14][100/293] lr: 5.000000e-04 eta: 8:51:43 time: 0.594501 data_time: 0.096187 memory: 9504 loss_kpt: 0.000815 acc_pose: 0.691796 loss: 0.000815 2022/09/26 11:24:11 - mmengine - INFO - Epoch(train) [14][150/293] lr: 5.000000e-04 eta: 8:51:45 time: 0.594202 data_time: 0.099987 memory: 9504 loss_kpt: 0.000830 acc_pose: 0.752155 loss: 0.000830 2022/09/26 11:24:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:24:39 - mmengine - INFO - Epoch(train) [14][200/293] lr: 5.000000e-04 eta: 8:51:26 time: 0.566683 data_time: 0.111727 memory: 9504 loss_kpt: 0.000825 acc_pose: 0.753406 loss: 0.000825 2022/09/26 11:25:09 - mmengine - INFO - Epoch(train) [14][250/293] lr: 5.000000e-04 eta: 8:51:27 time: 0.594341 data_time: 0.089964 memory: 9504 loss_kpt: 0.000810 acc_pose: 0.772615 loss: 0.000810 2022/09/26 11:25:36 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:26:07 - mmengine - INFO - Epoch(train) [15][50/293] lr: 5.000000e-04 eta: 8:45:46 time: 0.613337 data_time: 0.097213 memory: 9504 loss_kpt: 0.000806 acc_pose: 0.777357 loss: 0.000806 2022/09/26 11:26:38 - mmengine - INFO - Epoch(train) [15][100/293] lr: 5.000000e-04 eta: 8:46:11 time: 0.626905 data_time: 0.083655 memory: 9504 loss_kpt: 0.000817 acc_pose: 0.761095 loss: 0.000817 2022/09/26 11:27:06 - mmengine - INFO - Epoch(train) [15][150/293] lr: 5.000000e-04 eta: 8:45:55 time: 0.567390 data_time: 0.080426 memory: 9504 loss_kpt: 0.000802 acc_pose: 0.704223 loss: 0.000802 2022/09/26 11:27:36 - mmengine - INFO - Epoch(train) [15][200/293] lr: 5.000000e-04 eta: 8:46:00 time: 0.600361 data_time: 0.091435 memory: 9504 loss_kpt: 0.000809 acc_pose: 0.733615 loss: 0.000809 2022/09/26 11:28:06 - mmengine - INFO - Epoch(train) [15][250/293] lr: 5.000000e-04 eta: 8:45:58 time: 0.589785 data_time: 0.097211 memory: 9504 loss_kpt: 0.000800 acc_pose: 0.719931 loss: 0.000800 2022/09/26 11:28:32 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:29:04 - mmengine - INFO - Epoch(train) [16][50/293] lr: 5.000000e-04 eta: 8:40:54 time: 0.633550 data_time: 0.137259 memory: 9504 loss_kpt: 0.000794 acc_pose: 0.766726 loss: 0.000794 2022/09/26 11:29:35 - mmengine - INFO - Epoch(train) [16][100/293] lr: 5.000000e-04 eta: 8:41:09 time: 0.614610 data_time: 0.078881 memory: 9504 loss_kpt: 0.000811 acc_pose: 0.758398 loss: 0.000811 2022/09/26 11:30:04 - mmengine - INFO - Epoch(train) [16][150/293] lr: 5.000000e-04 eta: 8:41:12 time: 0.596174 data_time: 0.089898 memory: 9504 loss_kpt: 0.000797 acc_pose: 0.745041 loss: 0.000797 2022/09/26 11:30:34 - mmengine - INFO - Epoch(train) [16][200/293] lr: 5.000000e-04 eta: 8:41:14 time: 0.596344 data_time: 0.087796 memory: 9504 loss_kpt: 0.000810 acc_pose: 0.757957 loss: 0.000810 2022/09/26 11:31:03 - mmengine - INFO - Epoch(train) [16][250/293] lr: 5.000000e-04 eta: 8:41:02 time: 0.574284 data_time: 0.080617 memory: 9504 loss_kpt: 0.000809 acc_pose: 0.723600 loss: 0.000809 2022/09/26 11:31:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:32:00 - mmengine - INFO - Epoch(train) [17][50/293] lr: 5.000000e-04 eta: 8:36:12 time: 0.623652 data_time: 0.139250 memory: 9504 loss_kpt: 0.000784 acc_pose: 0.735028 loss: 0.000784 2022/09/26 11:32:31 - mmengine - INFO - Epoch(train) [17][100/293] lr: 5.000000e-04 eta: 8:36:26 time: 0.614858 data_time: 0.088161 memory: 9504 loss_kpt: 0.000808 acc_pose: 0.752069 loss: 0.000808 2022/09/26 11:33:02 - mmengine - INFO - Epoch(train) [17][150/293] lr: 5.000000e-04 eta: 8:36:38 time: 0.614391 data_time: 0.089067 memory: 9504 loss_kpt: 0.000801 acc_pose: 0.707516 loss: 0.000801 2022/09/26 11:33:34 - mmengine - INFO - Epoch(train) [17][200/293] lr: 5.000000e-04 eta: 8:37:04 time: 0.638625 data_time: 0.102687 memory: 9504 loss_kpt: 0.000799 acc_pose: 0.738042 loss: 0.000799 2022/09/26 11:34:04 - mmengine - INFO - Epoch(train) [17][250/293] lr: 5.000000e-04 eta: 8:37:12 time: 0.608995 data_time: 0.085994 memory: 9504 loss_kpt: 0.000786 acc_pose: 0.756767 loss: 0.000786 2022/09/26 11:34:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:34:42 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:35:00 - mmengine - INFO - Epoch(train) [18][50/293] lr: 5.000000e-04 eta: 8:32:35 time: 0.617387 data_time: 0.123984 memory: 9504 loss_kpt: 0.000789 acc_pose: 0.761710 loss: 0.000789 2022/09/26 11:35:32 - mmengine - INFO - Epoch(train) [18][100/293] lr: 5.000000e-04 eta: 8:32:57 time: 0.632519 data_time: 0.150275 memory: 9504 loss_kpt: 0.000781 acc_pose: 0.761334 loss: 0.000781 2022/09/26 11:36:03 - mmengine - INFO - Epoch(train) [18][150/293] lr: 5.000000e-04 eta: 8:33:17 time: 0.631258 data_time: 0.107340 memory: 9504 loss_kpt: 0.000787 acc_pose: 0.754580 loss: 0.000787 2022/09/26 11:36:34 - mmengine - INFO - Epoch(train) [18][200/293] lr: 5.000000e-04 eta: 8:33:29 time: 0.619601 data_time: 0.085936 memory: 9504 loss_kpt: 0.000791 acc_pose: 0.759437 loss: 0.000791 2022/09/26 11:37:03 - mmengine - INFO - Epoch(train) [18][250/293] lr: 5.000000e-04 eta: 8:33:20 time: 0.579630 data_time: 0.090417 memory: 9504 loss_kpt: 0.000791 acc_pose: 0.764127 loss: 0.000791 2022/09/26 11:37:30 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:38:02 - mmengine - INFO - Epoch(train) [19][50/293] lr: 5.000000e-04 eta: 8:29:09 time: 0.638282 data_time: 0.165242 memory: 9504 loss_kpt: 0.000792 acc_pose: 0.742845 loss: 0.000792 2022/09/26 11:38:31 - mmengine - INFO - Epoch(train) [19][100/293] lr: 5.000000e-04 eta: 8:29:03 time: 0.583984 data_time: 0.128038 memory: 9504 loss_kpt: 0.000772 acc_pose: 0.791241 loss: 0.000772 2022/09/26 11:39:01 - mmengine - INFO - Epoch(train) [19][150/293] lr: 5.000000e-04 eta: 8:29:03 time: 0.596367 data_time: 0.120023 memory: 9504 loss_kpt: 0.000772 acc_pose: 0.759921 loss: 0.000772 2022/09/26 11:39:32 - mmengine - INFO - Epoch(train) [19][200/293] lr: 5.000000e-04 eta: 8:29:12 time: 0.615938 data_time: 0.084821 memory: 9504 loss_kpt: 0.000775 acc_pose: 0.735484 loss: 0.000775 2022/09/26 11:40:02 - mmengine - INFO - Epoch(train) [19][250/293] lr: 5.000000e-04 eta: 8:29:14 time: 0.602385 data_time: 0.088456 memory: 9504 loss_kpt: 0.000783 acc_pose: 0.703547 loss: 0.000783 2022/09/26 11:40:27 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:41:00 - mmengine - INFO - Epoch(train) [20][50/293] lr: 5.000000e-04 eta: 8:25:24 time: 0.654371 data_time: 0.113720 memory: 9504 loss_kpt: 0.000763 acc_pose: 0.746776 loss: 0.000763 2022/09/26 11:41:30 - mmengine - INFO - Epoch(train) [20][100/293] lr: 5.000000e-04 eta: 8:25:28 time: 0.606176 data_time: 0.086472 memory: 9504 loss_kpt: 0.000771 acc_pose: 0.762596 loss: 0.000771 2022/09/26 11:42:02 - mmengine - INFO - Epoch(train) [20][150/293] lr: 5.000000e-04 eta: 8:25:48 time: 0.638840 data_time: 0.104850 memory: 9504 loss_kpt: 0.000767 acc_pose: 0.768750 loss: 0.000767 2022/09/26 11:42:32 - mmengine - INFO - Epoch(train) [20][200/293] lr: 5.000000e-04 eta: 8:25:46 time: 0.596621 data_time: 0.084561 memory: 9504 loss_kpt: 0.000777 acc_pose: 0.675084 loss: 0.000777 2022/09/26 11:43:04 - mmengine - INFO - Epoch(train) [20][250/293] lr: 5.000000e-04 eta: 8:25:59 time: 0.628643 data_time: 0.101367 memory: 9504 loss_kpt: 0.000775 acc_pose: 0.783357 loss: 0.000775 2022/09/26 11:43:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:43:29 - mmengine - INFO - Saving checkpoint at 20 epochs 2022/09/26 11:43:56 - mmengine - INFO - Epoch(val) [20][50/407] eta: 0:02:04 time: 0.349096 data_time: 0.051866 memory: 9504 2022/09/26 11:44:11 - mmengine - INFO - Epoch(val) [20][100/407] eta: 0:01:36 time: 0.314737 data_time: 0.052748 memory: 1378 2022/09/26 11:44:27 - mmengine - INFO - Epoch(val) [20][150/407] eta: 0:01:22 time: 0.320281 data_time: 0.038212 memory: 1378 2022/09/26 11:44:45 - mmengine - INFO - Epoch(val) [20][200/407] eta: 0:01:11 time: 0.344698 data_time: 0.042873 memory: 1378 2022/09/26 11:45:01 - mmengine - INFO - Epoch(val) [20][250/407] eta: 0:00:49 time: 0.318055 data_time: 0.028707 memory: 1378 2022/09/26 11:45:17 - mmengine - INFO - Epoch(val) [20][300/407] eta: 0:00:35 time: 0.328039 data_time: 0.050210 memory: 1378 2022/09/26 11:45:34 - mmengine - INFO - Epoch(val) [20][350/407] eta: 0:00:19 time: 0.349379 data_time: 0.037078 memory: 1378 2022/09/26 11:45:46 - mmengine - INFO - Epoch(val) [20][400/407] eta: 0:00:01 time: 0.236355 data_time: 0.028830 memory: 1378 2022/09/26 11:46:19 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 11:46:32 - mmengine - INFO - Epoch(val) [20][407/407] coco/AP: 0.652362 coco/AP .5: 0.869337 coco/AP .75: 0.726760 coco/AP (M): 0.616641 coco/AP (L): 0.717881 coco/AR: 0.715790 coco/AR .5: 0.913885 coco/AR .75: 0.782746 coco/AR (M): 0.672139 coco/AR (L): 0.777852 2022/09/26 11:46:32 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_10.pth is removed 2022/09/26 11:46:35 - mmengine - INFO - The best checkpoint with 0.6524 coco/AP at 20 epoch is saved to best_coco/AP_epoch_20.pth. 2022/09/26 11:47:06 - mmengine - INFO - Epoch(train) [21][50/293] lr: 5.000000e-04 eta: 8:22:00 time: 0.612790 data_time: 0.246016 memory: 9504 loss_kpt: 0.000788 acc_pose: 0.750485 loss: 0.000788 2022/09/26 11:47:36 - mmengine - INFO - Epoch(train) [21][100/293] lr: 5.000000e-04 eta: 8:22:06 time: 0.612513 data_time: 0.091059 memory: 9504 loss_kpt: 0.000777 acc_pose: 0.793431 loss: 0.000777 2022/09/26 11:48:01 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:48:07 - mmengine - INFO - Epoch(train) [21][150/293] lr: 5.000000e-04 eta: 8:22:13 time: 0.614820 data_time: 0.100283 memory: 9504 loss_kpt: 0.000773 acc_pose: 0.754351 loss: 0.000773 2022/09/26 11:48:37 - mmengine - INFO - Epoch(train) [21][200/293] lr: 5.000000e-04 eta: 8:22:10 time: 0.597136 data_time: 0.089631 memory: 9504 loss_kpt: 0.000771 acc_pose: 0.797674 loss: 0.000771 2022/09/26 11:49:07 - mmengine - INFO - Epoch(train) [21][250/293] lr: 5.000000e-04 eta: 8:22:11 time: 0.604459 data_time: 0.084307 memory: 9504 loss_kpt: 0.000777 acc_pose: 0.768268 loss: 0.000777 2022/09/26 11:49:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:50:04 - mmengine - INFO - Epoch(train) [22][50/293] lr: 5.000000e-04 eta: 8:18:28 time: 0.624146 data_time: 0.144515 memory: 9504 loss_kpt: 0.000786 acc_pose: 0.790085 loss: 0.000786 2022/09/26 11:50:34 - mmengine - INFO - Epoch(train) [22][100/293] lr: 5.000000e-04 eta: 8:18:27 time: 0.599548 data_time: 0.091993 memory: 9504 loss_kpt: 0.000771 acc_pose: 0.795794 loss: 0.000771 2022/09/26 11:51:06 - mmengine - INFO - Epoch(train) [22][150/293] lr: 5.000000e-04 eta: 8:18:43 time: 0.640857 data_time: 0.086325 memory: 9504 loss_kpt: 0.000767 acc_pose: 0.805845 loss: 0.000767 2022/09/26 11:51:36 - mmengine - INFO - Epoch(train) [22][200/293] lr: 5.000000e-04 eta: 8:18:42 time: 0.599725 data_time: 0.089253 memory: 9504 loss_kpt: 0.000774 acc_pose: 0.748356 loss: 0.000774 2022/09/26 11:52:06 - mmengine - INFO - Epoch(train) [22][250/293] lr: 5.000000e-04 eta: 8:18:39 time: 0.599889 data_time: 0.077878 memory: 9504 loss_kpt: 0.000761 acc_pose: 0.759249 loss: 0.000761 2022/09/26 11:52:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:53:03 - mmengine - INFO - Epoch(train) [23][50/293] lr: 5.000000e-04 eta: 8:14:58 time: 0.605886 data_time: 0.096456 memory: 9504 loss_kpt: 0.000760 acc_pose: 0.804345 loss: 0.000760 2022/09/26 11:53:32 - mmengine - INFO - Epoch(train) [23][100/293] lr: 5.000000e-04 eta: 8:14:44 time: 0.572089 data_time: 0.089789 memory: 9504 loss_kpt: 0.000765 acc_pose: 0.785492 loss: 0.000765 2022/09/26 11:54:02 - mmengine - INFO - Epoch(train) [23][150/293] lr: 5.000000e-04 eta: 8:14:42 time: 0.597768 data_time: 0.078837 memory: 9504 loss_kpt: 0.000762 acc_pose: 0.805197 loss: 0.000762 2022/09/26 11:54:34 - mmengine - INFO - Epoch(train) [23][200/293] lr: 5.000000e-04 eta: 8:14:58 time: 0.645568 data_time: 0.090714 memory: 9504 loss_kpt: 0.000760 acc_pose: 0.768667 loss: 0.000760 2022/09/26 11:55:04 - mmengine - INFO - Epoch(train) [23][250/293] lr: 5.000000e-04 eta: 8:15:00 time: 0.611340 data_time: 0.092608 memory: 9504 loss_kpt: 0.000764 acc_pose: 0.809672 loss: 0.000764 2022/09/26 11:55:41 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:56:16 - mmengine - INFO - Epoch(train) [24][50/293] lr: 5.000000e-04 eta: 8:12:08 time: 0.707340 data_time: 0.168151 memory: 9504 loss_kpt: 0.000764 acc_pose: 0.743019 loss: 0.000764 2022/09/26 11:56:47 - mmengine - INFO - Epoch(train) [24][100/293] lr: 5.000000e-04 eta: 8:12:09 time: 0.607867 data_time: 0.110254 memory: 9504 loss_kpt: 0.000749 acc_pose: 0.784778 loss: 0.000749 2022/09/26 11:57:18 - mmengine - INFO - Epoch(train) [24][150/293] lr: 5.000000e-04 eta: 8:12:11 time: 0.613768 data_time: 0.094997 memory: 9504 loss_kpt: 0.000770 acc_pose: 0.737105 loss: 0.000770 2022/09/26 11:57:47 - mmengine - INFO - Epoch(train) [24][200/293] lr: 5.000000e-04 eta: 8:12:06 time: 0.595992 data_time: 0.103045 memory: 9504 loss_kpt: 0.000754 acc_pose: 0.802710 loss: 0.000754 2022/09/26 11:58:17 - mmengine - INFO - Epoch(train) [24][250/293] lr: 5.000000e-04 eta: 8:12:03 time: 0.601983 data_time: 0.092846 memory: 9504 loss_kpt: 0.000756 acc_pose: 0.716165 loss: 0.000756 2022/09/26 11:58:24 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:58:42 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 11:59:14 - mmengine - INFO - Epoch(train) [25][50/293] lr: 5.000000e-04 eta: 8:08:48 time: 0.630985 data_time: 0.114215 memory: 9504 loss_kpt: 0.000754 acc_pose: 0.759563 loss: 0.000754 2022/09/26 11:59:43 - mmengine - INFO - Epoch(train) [25][100/293] lr: 5.000000e-04 eta: 8:08:35 time: 0.576767 data_time: 0.088654 memory: 9504 loss_kpt: 0.000746 acc_pose: 0.754507 loss: 0.000746 2022/09/26 12:00:14 - mmengine - INFO - Epoch(train) [25][150/293] lr: 5.000000e-04 eta: 8:08:42 time: 0.627166 data_time: 0.100834 memory: 9504 loss_kpt: 0.000749 acc_pose: 0.768420 loss: 0.000749 2022/09/26 12:00:44 - mmengine - INFO - Epoch(train) [25][200/293] lr: 5.000000e-04 eta: 8:08:37 time: 0.597770 data_time: 0.089405 memory: 9504 loss_kpt: 0.000752 acc_pose: 0.782573 loss: 0.000752 2022/09/26 12:01:14 - mmengine - INFO - Epoch(train) [25][250/293] lr: 5.000000e-04 eta: 8:08:34 time: 0.605385 data_time: 0.142613 memory: 9504 loss_kpt: 0.000764 acc_pose: 0.818065 loss: 0.000764 2022/09/26 12:01:39 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:02:10 - mmengine - INFO - Epoch(train) [26][50/293] lr: 5.000000e-04 eta: 8:05:23 time: 0.620922 data_time: 0.148483 memory: 9504 loss_kpt: 0.000741 acc_pose: 0.780556 loss: 0.000741 2022/09/26 12:02:42 - mmengine - INFO - Epoch(train) [26][100/293] lr: 5.000000e-04 eta: 8:05:33 time: 0.639822 data_time: 0.098390 memory: 9504 loss_kpt: 0.000745 acc_pose: 0.779508 loss: 0.000745 2022/09/26 12:03:12 - mmengine - INFO - Epoch(train) [26][150/293] lr: 5.000000e-04 eta: 8:05:27 time: 0.595423 data_time: 0.113318 memory: 9504 loss_kpt: 0.000744 acc_pose: 0.683140 loss: 0.000744 2022/09/26 12:03:43 - mmengine - INFO - Epoch(train) [26][200/293] lr: 5.000000e-04 eta: 8:05:27 time: 0.615659 data_time: 0.101710 memory: 9504 loss_kpt: 0.000764 acc_pose: 0.807845 loss: 0.000764 2022/09/26 12:04:14 - mmengine - INFO - Epoch(train) [26][250/293] lr: 5.000000e-04 eta: 8:05:32 time: 0.628601 data_time: 0.106773 memory: 9504 loss_kpt: 0.000752 acc_pose: 0.766944 loss: 0.000752 2022/09/26 12:04:40 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:05:11 - mmengine - INFO - Epoch(train) [27][50/293] lr: 5.000000e-04 eta: 8:02:26 time: 0.616308 data_time: 0.177340 memory: 9504 loss_kpt: 0.000753 acc_pose: 0.766441 loss: 0.000753 2022/09/26 12:05:40 - mmengine - INFO - Epoch(train) [27][100/293] lr: 5.000000e-04 eta: 8:02:19 time: 0.594011 data_time: 0.095194 memory: 9504 loss_kpt: 0.000734 acc_pose: 0.758010 loss: 0.000734 2022/09/26 12:06:10 - mmengine - INFO - Epoch(train) [27][150/293] lr: 5.000000e-04 eta: 8:02:07 time: 0.581662 data_time: 0.095422 memory: 9504 loss_kpt: 0.000742 acc_pose: 0.766719 loss: 0.000742 2022/09/26 12:06:39 - mmengine - INFO - Epoch(train) [27][200/293] lr: 5.000000e-04 eta: 8:02:00 time: 0.595642 data_time: 0.091147 memory: 9504 loss_kpt: 0.000743 acc_pose: 0.792095 loss: 0.000743 2022/09/26 12:07:09 - mmengine - INFO - Epoch(train) [27][250/293] lr: 5.000000e-04 eta: 8:01:53 time: 0.597531 data_time: 0.084583 memory: 9504 loss_kpt: 0.000746 acc_pose: 0.778927 loss: 0.000746 2022/09/26 12:07:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:08:04 - mmengine - INFO - Epoch(train) [28][50/293] lr: 5.000000e-04 eta: 7:58:53 time: 0.617373 data_time: 0.110270 memory: 9504 loss_kpt: 0.000730 acc_pose: 0.786464 loss: 0.000730 2022/09/26 12:08:27 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:08:34 - mmengine - INFO - Epoch(train) [28][100/293] lr: 5.000000e-04 eta: 7:58:42 time: 0.581848 data_time: 0.089741 memory: 9504 loss_kpt: 0.000743 acc_pose: 0.806583 loss: 0.000743 2022/09/26 12:09:03 - mmengine - INFO - Epoch(train) [28][150/293] lr: 5.000000e-04 eta: 7:58:36 time: 0.599365 data_time: 0.092711 memory: 9504 loss_kpt: 0.000740 acc_pose: 0.758706 loss: 0.000740 2022/09/26 12:09:33 - mmengine - INFO - Epoch(train) [28][200/293] lr: 5.000000e-04 eta: 7:58:29 time: 0.598907 data_time: 0.078015 memory: 9504 loss_kpt: 0.000739 acc_pose: 0.763900 loss: 0.000739 2022/09/26 12:10:05 - mmengine - INFO - Epoch(train) [28][250/293] lr: 5.000000e-04 eta: 7:58:34 time: 0.634038 data_time: 0.121271 memory: 9504 loss_kpt: 0.000754 acc_pose: 0.808317 loss: 0.000754 2022/09/26 12:10:31 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:11:02 - mmengine - INFO - Epoch(train) [29][50/293] lr: 5.000000e-04 eta: 7:55:39 time: 0.613957 data_time: 0.101909 memory: 9504 loss_kpt: 0.000747 acc_pose: 0.762779 loss: 0.000747 2022/09/26 12:11:31 - mmengine - INFO - Epoch(train) [29][100/293] lr: 5.000000e-04 eta: 7:55:30 time: 0.590831 data_time: 0.085956 memory: 9504 loss_kpt: 0.000738 acc_pose: 0.775193 loss: 0.000738 2022/09/26 12:12:01 - mmengine - INFO - Epoch(train) [29][150/293] lr: 5.000000e-04 eta: 7:55:22 time: 0.596680 data_time: 0.095221 memory: 9504 loss_kpt: 0.000727 acc_pose: 0.764362 loss: 0.000727 2022/09/26 12:12:30 - mmengine - INFO - Epoch(train) [29][200/293] lr: 5.000000e-04 eta: 7:55:11 time: 0.585818 data_time: 0.070349 memory: 9504 loss_kpt: 0.000744 acc_pose: 0.744823 loss: 0.000744 2022/09/26 12:13:01 - mmengine - INFO - Epoch(train) [29][250/293] lr: 5.000000e-04 eta: 7:55:05 time: 0.601962 data_time: 0.079830 memory: 9504 loss_kpt: 0.000736 acc_pose: 0.800053 loss: 0.000736 2022/09/26 12:13:26 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:13:57 - mmengine - INFO - Epoch(train) [30][50/293] lr: 5.000000e-04 eta: 7:52:18 time: 0.622479 data_time: 0.091529 memory: 9504 loss_kpt: 0.000722 acc_pose: 0.764167 loss: 0.000722 2022/09/26 12:14:26 - mmengine - INFO - Epoch(train) [30][100/293] lr: 5.000000e-04 eta: 7:52:08 time: 0.587326 data_time: 0.085809 memory: 9504 loss_kpt: 0.000734 acc_pose: 0.798721 loss: 0.000734 2022/09/26 12:14:56 - mmengine - INFO - Epoch(train) [30][150/293] lr: 5.000000e-04 eta: 7:52:02 time: 0.603271 data_time: 0.098598 memory: 9504 loss_kpt: 0.000728 acc_pose: 0.822107 loss: 0.000728 2022/09/26 12:15:27 - mmengine - INFO - Epoch(train) [30][200/293] lr: 5.000000e-04 eta: 7:51:56 time: 0.605877 data_time: 0.083165 memory: 9504 loss_kpt: 0.000731 acc_pose: 0.737970 loss: 0.000731 2022/09/26 12:15:57 - mmengine - INFO - Epoch(train) [30][250/293] lr: 5.000000e-04 eta: 7:51:50 time: 0.604567 data_time: 0.084968 memory: 9504 loss_kpt: 0.000741 acc_pose: 0.793414 loss: 0.000741 2022/09/26 12:16:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:16:21 - mmengine - INFO - Saving checkpoint at 30 epochs 2022/09/26 12:16:51 - mmengine - INFO - Epoch(val) [30][50/407] eta: 0:02:04 time: 0.348330 data_time: 0.073183 memory: 9504 2022/09/26 12:17:07 - mmengine - INFO - Epoch(val) [30][100/407] eta: 0:01:41 time: 0.329513 data_time: 0.052106 memory: 1378 2022/09/26 12:17:23 - mmengine - INFO - Epoch(val) [30][150/407] eta: 0:01:21 time: 0.317255 data_time: 0.026837 memory: 1378 2022/09/26 12:17:39 - mmengine - INFO - Epoch(val) [30][200/407] eta: 0:01:05 time: 0.314539 data_time: 0.021858 memory: 1378 2022/09/26 12:17:56 - mmengine - INFO - Epoch(val) [30][250/407] eta: 0:00:52 time: 0.336456 data_time: 0.044212 memory: 1378 2022/09/26 12:18:12 - mmengine - INFO - Epoch(val) [30][300/407] eta: 0:00:34 time: 0.322863 data_time: 0.053785 memory: 1378 2022/09/26 12:18:30 - mmengine - INFO - Epoch(val) [30][350/407] eta: 0:00:20 time: 0.364245 data_time: 0.057590 memory: 1378 2022/09/26 12:18:39 - mmengine - INFO - Epoch(val) [30][400/407] eta: 0:00:01 time: 0.178955 data_time: 0.023408 memory: 1378 2022/09/26 12:19:13 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 12:19:26 - mmengine - INFO - Epoch(val) [30][407/407] coco/AP: 0.679430 coco/AP .5: 0.882906 coco/AP .75: 0.757246 coco/AP (M): 0.645679 coco/AP (L): 0.740848 coco/AR: 0.736304 coco/AR .5: 0.922072 coco/AR .75: 0.805730 coco/AR (M): 0.695548 coco/AR (L): 0.794946 2022/09/26 12:19:26 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_20.pth is removed 2022/09/26 12:19:29 - mmengine - INFO - The best checkpoint with 0.6794 coco/AP at 30 epoch is saved to best_coco/AP_epoch_30.pth. 2022/09/26 12:19:59 - mmengine - INFO - Epoch(train) [31][50/293] lr: 5.000000e-04 eta: 7:49:03 time: 0.604859 data_time: 0.139195 memory: 9504 loss_kpt: 0.000739 acc_pose: 0.801406 loss: 0.000739 2022/09/26 12:20:29 - mmengine - INFO - Epoch(train) [31][100/293] lr: 5.000000e-04 eta: 7:48:58 time: 0.604765 data_time: 0.088019 memory: 9504 loss_kpt: 0.000725 acc_pose: 0.751750 loss: 0.000725 2022/09/26 12:20:58 - mmengine - INFO - Epoch(train) [31][150/293] lr: 5.000000e-04 eta: 7:48:41 time: 0.568853 data_time: 0.086938 memory: 9504 loss_kpt: 0.000744 acc_pose: 0.782331 loss: 0.000744 2022/09/26 12:21:28 - mmengine - INFO - Epoch(train) [31][200/293] lr: 5.000000e-04 eta: 7:48:33 time: 0.600177 data_time: 0.120113 memory: 9504 loss_kpt: 0.000728 acc_pose: 0.788886 loss: 0.000728 2022/09/26 12:21:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:21:59 - mmengine - INFO - Epoch(train) [31][250/293] lr: 5.000000e-04 eta: 7:48:32 time: 0.622638 data_time: 0.175537 memory: 9504 loss_kpt: 0.000723 acc_pose: 0.797719 loss: 0.000723 2022/09/26 12:22:24 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:22:55 - mmengine - INFO - Epoch(train) [32][50/293] lr: 5.000000e-04 eta: 7:45:53 time: 0.617365 data_time: 0.105245 memory: 9504 loss_kpt: 0.000736 acc_pose: 0.819908 loss: 0.000736 2022/09/26 12:23:23 - mmengine - INFO - Epoch(train) [32][100/293] lr: 5.000000e-04 eta: 7:45:35 time: 0.563615 data_time: 0.091738 memory: 9504 loss_kpt: 0.000745 acc_pose: 0.710402 loss: 0.000745 2022/09/26 12:23:52 - mmengine - INFO - Epoch(train) [32][150/293] lr: 5.000000e-04 eta: 7:45:21 time: 0.577235 data_time: 0.132048 memory: 9504 loss_kpt: 0.000742 acc_pose: 0.771427 loss: 0.000742 2022/09/26 12:24:23 - mmengine - INFO - Epoch(train) [32][200/293] lr: 5.000000e-04 eta: 7:45:17 time: 0.613752 data_time: 0.094031 memory: 9504 loss_kpt: 0.000733 acc_pose: 0.799530 loss: 0.000733 2022/09/26 12:24:52 - mmengine - INFO - Epoch(train) [32][250/293] lr: 5.000000e-04 eta: 7:45:05 time: 0.586875 data_time: 0.088101 memory: 9504 loss_kpt: 0.000732 acc_pose: 0.785987 loss: 0.000732 2022/09/26 12:25:18 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:25:50 - mmengine - INFO - Epoch(train) [33][50/293] lr: 5.000000e-04 eta: 7:42:36 time: 0.637378 data_time: 0.103688 memory: 9504 loss_kpt: 0.000726 acc_pose: 0.859644 loss: 0.000726 2022/09/26 12:26:19 - mmengine - INFO - Epoch(train) [33][100/293] lr: 5.000000e-04 eta: 7:42:26 time: 0.590057 data_time: 0.123760 memory: 9504 loss_kpt: 0.000718 acc_pose: 0.832797 loss: 0.000718 2022/09/26 12:26:55 - mmengine - INFO - Epoch(train) [33][150/293] lr: 5.000000e-04 eta: 7:42:47 time: 0.710966 data_time: 0.111788 memory: 9504 loss_kpt: 0.000726 acc_pose: 0.816664 loss: 0.000726 2022/09/26 12:27:25 - mmengine - INFO - Epoch(train) [33][200/293] lr: 5.000000e-04 eta: 7:42:41 time: 0.610261 data_time: 0.112128 memory: 9504 loss_kpt: 0.000729 acc_pose: 0.783223 loss: 0.000729 2022/09/26 12:27:53 - mmengine - INFO - Epoch(train) [33][250/293] lr: 5.000000e-04 eta: 7:42:23 time: 0.564034 data_time: 0.133070 memory: 9504 loss_kpt: 0.000722 acc_pose: 0.803440 loss: 0.000722 2022/09/26 12:28:19 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:28:50 - mmengine - INFO - Epoch(train) [34][50/293] lr: 5.000000e-04 eta: 7:39:55 time: 0.627835 data_time: 0.108947 memory: 9504 loss_kpt: 0.000713 acc_pose: 0.784873 loss: 0.000713 2022/09/26 12:29:21 - mmengine - INFO - Epoch(train) [34][100/293] lr: 5.000000e-04 eta: 7:39:49 time: 0.607955 data_time: 0.091280 memory: 9504 loss_kpt: 0.000705 acc_pose: 0.789157 loss: 0.000705 2022/09/26 12:29:52 - mmengine - INFO - Epoch(train) [34][150/293] lr: 5.000000e-04 eta: 7:39:44 time: 0.614674 data_time: 0.110563 memory: 9504 loss_kpt: 0.000710 acc_pose: 0.795056 loss: 0.000710 2022/09/26 12:30:21 - mmengine - INFO - Epoch(train) [34][200/293] lr: 5.000000e-04 eta: 7:39:30 time: 0.584743 data_time: 0.102120 memory: 9504 loss_kpt: 0.000722 acc_pose: 0.774715 loss: 0.000722 2022/09/26 12:30:51 - mmengine - INFO - Epoch(train) [34][250/293] lr: 5.000000e-04 eta: 7:39:21 time: 0.599942 data_time: 0.097292 memory: 9504 loss_kpt: 0.000720 acc_pose: 0.795067 loss: 0.000720 2022/09/26 12:31:16 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:31:41 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:31:48 - mmengine - INFO - Epoch(train) [35][50/293] lr: 5.000000e-04 eta: 7:36:58 time: 0.631341 data_time: 0.106912 memory: 9504 loss_kpt: 0.000724 acc_pose: 0.800401 loss: 0.000724 2022/09/26 12:32:18 - mmengine - INFO - Epoch(train) [35][100/293] lr: 5.000000e-04 eta: 7:36:50 time: 0.604252 data_time: 0.092875 memory: 9504 loss_kpt: 0.000706 acc_pose: 0.781900 loss: 0.000706 2022/09/26 12:32:48 - mmengine - INFO - Epoch(train) [35][150/293] lr: 5.000000e-04 eta: 7:36:42 time: 0.604986 data_time: 0.085481 memory: 9504 loss_kpt: 0.000712 acc_pose: 0.827698 loss: 0.000712 2022/09/26 12:33:18 - mmengine - INFO - Epoch(train) [35][200/293] lr: 5.000000e-04 eta: 7:36:30 time: 0.592149 data_time: 0.104602 memory: 9504 loss_kpt: 0.000726 acc_pose: 0.807593 loss: 0.000726 2022/09/26 12:33:49 - mmengine - INFO - Epoch(train) [35][250/293] lr: 5.000000e-04 eta: 7:36:26 time: 0.622813 data_time: 0.093603 memory: 9504 loss_kpt: 0.000719 acc_pose: 0.734239 loss: 0.000719 2022/09/26 12:34:14 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:34:46 - mmengine - INFO - Epoch(train) [36][50/293] lr: 5.000000e-04 eta: 7:34:04 time: 0.621746 data_time: 0.130849 memory: 9504 loss_kpt: 0.000715 acc_pose: 0.781379 loss: 0.000715 2022/09/26 12:35:16 - mmengine - INFO - Epoch(train) [36][100/293] lr: 5.000000e-04 eta: 7:33:58 time: 0.613242 data_time: 0.089569 memory: 9504 loss_kpt: 0.000721 acc_pose: 0.713365 loss: 0.000721 2022/09/26 12:35:46 - mmengine - INFO - Epoch(train) [36][150/293] lr: 5.000000e-04 eta: 7:33:49 time: 0.604247 data_time: 0.094317 memory: 9504 loss_kpt: 0.000719 acc_pose: 0.792655 loss: 0.000719 2022/09/26 12:36:16 - mmengine - INFO - Epoch(train) [36][200/293] lr: 5.000000e-04 eta: 7:33:36 time: 0.591213 data_time: 0.122747 memory: 9504 loss_kpt: 0.000729 acc_pose: 0.758197 loss: 0.000729 2022/09/26 12:36:47 - mmengine - INFO - Epoch(train) [36][250/293] lr: 5.000000e-04 eta: 7:33:29 time: 0.612395 data_time: 0.135287 memory: 9504 loss_kpt: 0.000711 acc_pose: 0.778040 loss: 0.000711 2022/09/26 12:37:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:37:43 - mmengine - INFO - Epoch(train) [37][50/293] lr: 5.000000e-04 eta: 7:31:07 time: 0.608072 data_time: 0.131933 memory: 9504 loss_kpt: 0.000722 acc_pose: 0.770570 loss: 0.000722 2022/09/26 12:38:14 - mmengine - INFO - Epoch(train) [37][100/293] lr: 5.000000e-04 eta: 7:31:04 time: 0.631497 data_time: 0.096746 memory: 9504 loss_kpt: 0.000723 acc_pose: 0.789810 loss: 0.000723 2022/09/26 12:38:44 - mmengine - INFO - Epoch(train) [37][150/293] lr: 5.000000e-04 eta: 7:30:54 time: 0.599735 data_time: 0.082359 memory: 9504 loss_kpt: 0.000719 acc_pose: 0.832481 loss: 0.000719 2022/09/26 12:39:14 - mmengine - INFO - Epoch(train) [37][200/293] lr: 5.000000e-04 eta: 7:30:41 time: 0.588579 data_time: 0.093713 memory: 9504 loss_kpt: 0.000725 acc_pose: 0.777457 loss: 0.000725 2022/09/26 12:39:43 - mmengine - INFO - Epoch(train) [37][250/293] lr: 5.000000e-04 eta: 7:30:28 time: 0.593681 data_time: 0.088064 memory: 9504 loss_kpt: 0.000728 acc_pose: 0.780944 loss: 0.000728 2022/09/26 12:40:09 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:40:39 - mmengine - INFO - Epoch(train) [38][50/293] lr: 5.000000e-04 eta: 7:28:10 time: 0.609611 data_time: 0.126435 memory: 9504 loss_kpt: 0.000701 acc_pose: 0.787359 loss: 0.000701 2022/09/26 12:41:09 - mmengine - INFO - Epoch(train) [38][100/293] lr: 5.000000e-04 eta: 7:27:56 time: 0.586069 data_time: 0.089506 memory: 9504 loss_kpt: 0.000709 acc_pose: 0.789391 loss: 0.000709 2022/09/26 12:41:39 - mmengine - INFO - Epoch(train) [38][150/293] lr: 5.000000e-04 eta: 7:27:48 time: 0.609352 data_time: 0.118192 memory: 9504 loss_kpt: 0.000706 acc_pose: 0.825103 loss: 0.000706 2022/09/26 12:41:45 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:42:10 - mmengine - INFO - Epoch(train) [38][200/293] lr: 5.000000e-04 eta: 7:27:40 time: 0.614368 data_time: 0.098332 memory: 9504 loss_kpt: 0.000700 acc_pose: 0.781364 loss: 0.000700 2022/09/26 12:42:40 - mmengine - INFO - Epoch(train) [38][250/293] lr: 5.000000e-04 eta: 7:27:29 time: 0.601160 data_time: 0.095225 memory: 9504 loss_kpt: 0.000708 acc_pose: 0.795366 loss: 0.000708 2022/09/26 12:43:05 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:43:37 - mmengine - INFO - Epoch(train) [39][50/293] lr: 5.000000e-04 eta: 7:25:21 time: 0.638582 data_time: 0.099367 memory: 9504 loss_kpt: 0.000695 acc_pose: 0.780149 loss: 0.000695 2022/09/26 12:44:06 - mmengine - INFO - Epoch(train) [39][100/293] lr: 5.000000e-04 eta: 7:25:06 time: 0.582738 data_time: 0.092984 memory: 9504 loss_kpt: 0.000708 acc_pose: 0.819332 loss: 0.000708 2022/09/26 12:44:37 - mmengine - INFO - Epoch(train) [39][150/293] lr: 5.000000e-04 eta: 7:24:55 time: 0.601774 data_time: 0.145353 memory: 9504 loss_kpt: 0.000701 acc_pose: 0.833103 loss: 0.000701 2022/09/26 12:45:07 - mmengine - INFO - Epoch(train) [39][200/293] lr: 5.000000e-04 eta: 7:24:44 time: 0.601185 data_time: 0.086774 memory: 9504 loss_kpt: 0.000710 acc_pose: 0.816350 loss: 0.000710 2022/09/26 12:45:37 - mmengine - INFO - Epoch(train) [39][250/293] lr: 5.000000e-04 eta: 7:24:35 time: 0.611173 data_time: 0.100826 memory: 9504 loss_kpt: 0.000686 acc_pose: 0.789577 loss: 0.000686 2022/09/26 12:46:03 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:46:34 - mmengine - INFO - Epoch(train) [40][50/293] lr: 5.000000e-04 eta: 7:22:24 time: 0.618625 data_time: 0.119649 memory: 9504 loss_kpt: 0.000693 acc_pose: 0.782602 loss: 0.000693 2022/09/26 12:47:05 - mmengine - INFO - Epoch(train) [40][100/293] lr: 5.000000e-04 eta: 7:22:17 time: 0.617464 data_time: 0.085879 memory: 9504 loss_kpt: 0.000703 acc_pose: 0.785763 loss: 0.000703 2022/09/26 12:47:35 - mmengine - INFO - Epoch(train) [40][150/293] lr: 5.000000e-04 eta: 7:22:05 time: 0.598357 data_time: 0.093505 memory: 9504 loss_kpt: 0.000708 acc_pose: 0.767163 loss: 0.000708 2022/09/26 12:48:04 - mmengine - INFO - Epoch(train) [40][200/293] lr: 5.000000e-04 eta: 7:21:50 time: 0.585124 data_time: 0.096798 memory: 9504 loss_kpt: 0.000697 acc_pose: 0.782455 loss: 0.000697 2022/09/26 12:48:34 - mmengine - INFO - Epoch(train) [40][250/293] lr: 5.000000e-04 eta: 7:21:37 time: 0.594459 data_time: 0.081280 memory: 9504 loss_kpt: 0.000708 acc_pose: 0.814419 loss: 0.000708 2022/09/26 12:48:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:48:59 - mmengine - INFO - Saving checkpoint at 40 epochs 2022/09/26 12:49:23 - mmengine - INFO - Epoch(val) [40][50/407] eta: 0:02:01 time: 0.340346 data_time: 0.073196 memory: 9504 2022/09/26 12:49:39 - mmengine - INFO - Epoch(val) [40][100/407] eta: 0:01:37 time: 0.316126 data_time: 0.035022 memory: 1378 2022/09/26 12:49:55 - mmengine - INFO - Epoch(val) [40][150/407] eta: 0:01:23 time: 0.326132 data_time: 0.046051 memory: 1378 2022/09/26 12:50:11 - mmengine - INFO - Epoch(val) [40][200/407] eta: 0:01:05 time: 0.316304 data_time: 0.038328 memory: 1378 2022/09/26 12:50:27 - mmengine - INFO - Epoch(val) [40][250/407] eta: 0:00:51 time: 0.326246 data_time: 0.036942 memory: 1378 2022/09/26 12:50:43 - mmengine - INFO - Epoch(val) [40][300/407] eta: 0:00:33 time: 0.312018 data_time: 0.028634 memory: 1378 2022/09/26 12:50:59 - mmengine - INFO - Epoch(val) [40][350/407] eta: 0:00:18 time: 0.327026 data_time: 0.048755 memory: 1378 2022/09/26 12:51:14 - mmengine - INFO - Epoch(val) [40][400/407] eta: 0:00:02 time: 0.292100 data_time: 0.037701 memory: 1378 2022/09/26 12:51:48 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 12:52:01 - mmengine - INFO - Epoch(val) [40][407/407] coco/AP: 0.685480 coco/AP .5: 0.885002 coco/AP .75: 0.761811 coco/AP (M): 0.647508 coco/AP (L): 0.752557 coco/AR: 0.742727 coco/AR .5: 0.925063 coco/AR .75: 0.810611 coco/AR (M): 0.699208 coco/AR (L): 0.804422 2022/09/26 12:52:01 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_30.pth is removed 2022/09/26 12:52:03 - mmengine - INFO - The best checkpoint with 0.6855 coco/AP at 40 epoch is saved to best_coco/AP_epoch_40.pth. 2022/09/26 12:52:33 - mmengine - INFO - Epoch(train) [41][50/293] lr: 5.000000e-04 eta: 7:19:25 time: 0.599250 data_time: 0.152394 memory: 9504 loss_kpt: 0.000707 acc_pose: 0.722315 loss: 0.000707 2022/09/26 12:53:04 - mmengine - INFO - Epoch(train) [41][100/293] lr: 5.000000e-04 eta: 7:19:17 time: 0.615604 data_time: 0.100968 memory: 9504 loss_kpt: 0.000695 acc_pose: 0.784029 loss: 0.000695 2022/09/26 12:53:34 - mmengine - INFO - Epoch(train) [41][150/293] lr: 5.000000e-04 eta: 7:19:03 time: 0.589809 data_time: 0.074832 memory: 9504 loss_kpt: 0.000683 acc_pose: 0.777926 loss: 0.000683 2022/09/26 12:54:05 - mmengine - INFO - Epoch(train) [41][200/293] lr: 5.000000e-04 eta: 7:18:54 time: 0.616460 data_time: 0.179074 memory: 9504 loss_kpt: 0.000699 acc_pose: 0.801155 loss: 0.000699 2022/09/26 12:54:33 - mmengine - INFO - Epoch(train) [41][250/293] lr: 5.000000e-04 eta: 7:18:35 time: 0.568472 data_time: 0.129053 memory: 9504 loss_kpt: 0.000705 acc_pose: 0.740534 loss: 0.000705 2022/09/26 12:54:50 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:54:58 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:55:29 - mmengine - INFO - Epoch(train) [42][50/293] lr: 5.000000e-04 eta: 7:16:33 time: 0.633405 data_time: 0.092733 memory: 9504 loss_kpt: 0.000695 acc_pose: 0.772633 loss: 0.000695 2022/09/26 12:55:58 - mmengine - INFO - Epoch(train) [42][100/293] lr: 5.000000e-04 eta: 7:16:17 time: 0.578733 data_time: 0.107584 memory: 9504 loss_kpt: 0.000686 acc_pose: 0.801425 loss: 0.000686 2022/09/26 12:56:30 - mmengine - INFO - Epoch(train) [42][150/293] lr: 5.000000e-04 eta: 7:16:11 time: 0.632871 data_time: 0.170103 memory: 9504 loss_kpt: 0.000714 acc_pose: 0.781592 loss: 0.000714 2022/09/26 12:57:00 - mmengine - INFO - Epoch(train) [42][200/293] lr: 5.000000e-04 eta: 7:15:59 time: 0.599736 data_time: 0.095867 memory: 9504 loss_kpt: 0.000710 acc_pose: 0.804423 loss: 0.000710 2022/09/26 12:57:29 - mmengine - INFO - Epoch(train) [42][250/293] lr: 5.000000e-04 eta: 7:15:43 time: 0.584189 data_time: 0.080423 memory: 9504 loss_kpt: 0.000701 acc_pose: 0.786525 loss: 0.000701 2022/09/26 12:57:54 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 12:58:25 - mmengine - INFO - Epoch(train) [43][50/293] lr: 5.000000e-04 eta: 7:13:40 time: 0.618507 data_time: 0.113195 memory: 9504 loss_kpt: 0.000692 acc_pose: 0.775936 loss: 0.000692 2022/09/26 12:58:54 - mmengine - INFO - Epoch(train) [43][100/293] lr: 5.000000e-04 eta: 7:13:24 time: 0.579943 data_time: 0.069328 memory: 9504 loss_kpt: 0.000700 acc_pose: 0.746380 loss: 0.000700 2022/09/26 12:59:25 - mmengine - INFO - Epoch(train) [43][150/293] lr: 5.000000e-04 eta: 7:13:15 time: 0.618061 data_time: 0.074007 memory: 9504 loss_kpt: 0.000711 acc_pose: 0.791713 loss: 0.000711 2022/09/26 12:59:56 - mmengine - INFO - Epoch(train) [43][200/293] lr: 5.000000e-04 eta: 7:13:06 time: 0.620222 data_time: 0.071545 memory: 9504 loss_kpt: 0.000700 acc_pose: 0.816308 loss: 0.000700 2022/09/26 13:00:25 - mmengine - INFO - Epoch(train) [43][250/293] lr: 5.000000e-04 eta: 7:12:50 time: 0.580489 data_time: 0.079638 memory: 9504 loss_kpt: 0.000690 acc_pose: 0.786309 loss: 0.000690 2022/09/26 13:00:50 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:01:22 - mmengine - INFO - Epoch(train) [44][50/293] lr: 5.000000e-04 eta: 7:10:53 time: 0.636660 data_time: 0.144536 memory: 9504 loss_kpt: 0.000698 acc_pose: 0.830816 loss: 0.000698 2022/09/26 13:03:18 - mmengine - INFO - Epoch(train) [44][100/293] lr: 5.000000e-04 eta: 7:16:11 time: 2.321158 data_time: 0.085438 memory: 9504 loss_kpt: 0.000683 acc_pose: 0.817891 loss: 0.000683 2022/09/26 13:03:46 - mmengine - INFO - Epoch(train) [44][150/293] lr: 5.000000e-04 eta: 7:15:48 time: 0.558805 data_time: 0.093666 memory: 9504 loss_kpt: 0.000685 acc_pose: 0.792136 loss: 0.000685 2022/09/26 13:04:15 - mmengine - INFO - Epoch(train) [44][200/293] lr: 5.000000e-04 eta: 7:15:29 time: 0.572175 data_time: 0.110158 memory: 9504 loss_kpt: 0.000706 acc_pose: 0.775435 loss: 0.000706 2022/09/26 13:04:46 - mmengine - INFO - Epoch(train) [44][250/293] lr: 5.000000e-04 eta: 7:15:19 time: 0.626358 data_time: 0.123508 memory: 9504 loss_kpt: 0.000704 acc_pose: 0.811638 loss: 0.000704 2022/09/26 13:05:11 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:05:42 - mmengine - INFO - Epoch(train) [45][50/293] lr: 5.000000e-04 eta: 7:13:19 time: 0.624344 data_time: 0.088491 memory: 9504 loss_kpt: 0.000698 acc_pose: 0.801100 loss: 0.000698 2022/09/26 13:06:11 - mmengine - INFO - Epoch(train) [45][100/293] lr: 5.000000e-04 eta: 7:13:02 time: 0.590663 data_time: 0.099230 memory: 9504 loss_kpt: 0.000695 acc_pose: 0.809037 loss: 0.000695 2022/09/26 13:06:16 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:06:41 - mmengine - INFO - Epoch(train) [45][150/293] lr: 5.000000e-04 eta: 7:12:45 time: 0.586608 data_time: 0.080147 memory: 9504 loss_kpt: 0.000690 acc_pose: 0.814092 loss: 0.000690 2022/09/26 13:07:11 - mmengine - INFO - Epoch(train) [45][200/293] lr: 5.000000e-04 eta: 7:12:31 time: 0.605113 data_time: 0.139207 memory: 9504 loss_kpt: 0.000704 acc_pose: 0.799740 loss: 0.000704 2022/09/26 13:07:40 - mmengine - INFO - Epoch(train) [45][250/293] lr: 5.000000e-04 eta: 7:12:13 time: 0.583186 data_time: 0.105199 memory: 9504 loss_kpt: 0.000688 acc_pose: 0.750510 loss: 0.000688 2022/09/26 13:08:05 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:08:36 - mmengine - INFO - Epoch(train) [46][50/293] lr: 5.000000e-04 eta: 7:10:14 time: 0.617509 data_time: 0.109781 memory: 9504 loss_kpt: 0.000678 acc_pose: 0.816907 loss: 0.000678 2022/09/26 13:09:06 - mmengine - INFO - Epoch(train) [46][100/293] lr: 5.000000e-04 eta: 7:09:59 time: 0.600810 data_time: 0.117475 memory: 9504 loss_kpt: 0.000679 acc_pose: 0.789534 loss: 0.000679 2022/09/26 13:09:36 - mmengine - INFO - Epoch(train) [46][150/293] lr: 5.000000e-04 eta: 7:09:47 time: 0.612227 data_time: 0.180712 memory: 9504 loss_kpt: 0.000683 acc_pose: 0.801805 loss: 0.000683 2022/09/26 13:10:07 - mmengine - INFO - Epoch(train) [46][200/293] lr: 5.000000e-04 eta: 7:09:33 time: 0.609371 data_time: 0.106587 memory: 9504 loss_kpt: 0.000685 acc_pose: 0.760751 loss: 0.000685 2022/09/26 13:10:36 - mmengine - INFO - Epoch(train) [46][250/293] lr: 5.000000e-04 eta: 7:09:16 time: 0.589849 data_time: 0.098254 memory: 9504 loss_kpt: 0.000693 acc_pose: 0.819381 loss: 0.000693 2022/09/26 13:11:01 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:11:32 - mmengine - INFO - Epoch(train) [47][50/293] lr: 5.000000e-04 eta: 7:07:20 time: 0.621897 data_time: 0.100361 memory: 9504 loss_kpt: 0.000684 acc_pose: 0.838152 loss: 0.000684 2022/09/26 13:12:02 - mmengine - INFO - Epoch(train) [47][100/293] lr: 5.000000e-04 eta: 7:07:03 time: 0.589956 data_time: 0.171936 memory: 9504 loss_kpt: 0.000687 acc_pose: 0.784242 loss: 0.000687 2022/09/26 13:12:31 - mmengine - INFO - Epoch(train) [47][150/293] lr: 5.000000e-04 eta: 7:06:47 time: 0.594116 data_time: 0.089155 memory: 9504 loss_kpt: 0.000702 acc_pose: 0.752513 loss: 0.000702 2022/09/26 13:13:02 - mmengine - INFO - Epoch(train) [47][200/293] lr: 5.000000e-04 eta: 7:06:33 time: 0.609797 data_time: 0.160801 memory: 9504 loss_kpt: 0.000688 acc_pose: 0.797067 loss: 0.000688 2022/09/26 13:13:31 - mmengine - INFO - Epoch(train) [47][250/293] lr: 5.000000e-04 eta: 7:06:17 time: 0.591225 data_time: 0.073488 memory: 9504 loss_kpt: 0.000696 acc_pose: 0.811569 loss: 0.000696 2022/09/26 13:13:57 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:14:28 - mmengine - INFO - Epoch(train) [48][50/293] lr: 5.000000e-04 eta: 7:04:20 time: 0.610246 data_time: 0.096865 memory: 9504 loss_kpt: 0.000688 acc_pose: 0.767013 loss: 0.000688 2022/09/26 13:14:57 - mmengine - INFO - Epoch(train) [48][100/293] lr: 5.000000e-04 eta: 7:04:00 time: 0.572989 data_time: 0.082893 memory: 9504 loss_kpt: 0.000684 acc_pose: 0.778198 loss: 0.000684 2022/09/26 13:15:26 - mmengine - INFO - Epoch(train) [48][150/293] lr: 5.000000e-04 eta: 7:03:42 time: 0.582049 data_time: 0.088646 memory: 9504 loss_kpt: 0.000691 acc_pose: 0.781749 loss: 0.000691 2022/09/26 13:15:57 - mmengine - INFO - Epoch(train) [48][200/293] lr: 5.000000e-04 eta: 7:03:30 time: 0.618930 data_time: 0.102904 memory: 9504 loss_kpt: 0.000682 acc_pose: 0.818844 loss: 0.000682 2022/09/26 13:16:14 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:16:27 - mmengine - INFO - Epoch(train) [48][250/293] lr: 5.000000e-04 eta: 7:03:15 time: 0.603707 data_time: 0.084674 memory: 9504 loss_kpt: 0.000696 acc_pose: 0.794284 loss: 0.000696 2022/09/26 13:16:52 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:17:23 - mmengine - INFO - Epoch(train) [49][50/293] lr: 5.000000e-04 eta: 7:01:22 time: 0.620424 data_time: 0.106305 memory: 9504 loss_kpt: 0.000689 acc_pose: 0.764880 loss: 0.000689 2022/09/26 13:17:53 - mmengine - INFO - Epoch(train) [49][100/293] lr: 5.000000e-04 eta: 7:01:07 time: 0.598881 data_time: 0.089046 memory: 9504 loss_kpt: 0.000677 acc_pose: 0.774794 loss: 0.000677 2022/09/26 13:18:24 - mmengine - INFO - Epoch(train) [49][150/293] lr: 5.000000e-04 eta: 7:00:53 time: 0.609430 data_time: 0.078252 memory: 9504 loss_kpt: 0.000687 acc_pose: 0.844027 loss: 0.000687 2022/09/26 13:18:53 - mmengine - INFO - Epoch(train) [49][200/293] lr: 5.000000e-04 eta: 7:00:35 time: 0.588239 data_time: 0.074860 memory: 9504 loss_kpt: 0.000690 acc_pose: 0.817344 loss: 0.000690 2022/09/26 13:19:23 - mmengine - INFO - Epoch(train) [49][250/293] lr: 5.000000e-04 eta: 7:00:18 time: 0.595724 data_time: 0.102330 memory: 9504 loss_kpt: 0.000693 acc_pose: 0.789754 loss: 0.000693 2022/09/26 13:19:49 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:20:21 - mmengine - INFO - Epoch(train) [50][50/293] lr: 5.000000e-04 eta: 6:58:31 time: 0.641494 data_time: 0.109530 memory: 9504 loss_kpt: 0.000684 acc_pose: 0.791962 loss: 0.000684 2022/09/26 13:20:51 - mmengine - INFO - Epoch(train) [50][100/293] lr: 5.000000e-04 eta: 6:58:15 time: 0.600031 data_time: 0.080549 memory: 9504 loss_kpt: 0.000675 acc_pose: 0.847261 loss: 0.000675 2022/09/26 13:21:20 - mmengine - INFO - Epoch(train) [50][150/293] lr: 5.000000e-04 eta: 6:57:57 time: 0.585688 data_time: 0.124787 memory: 9504 loss_kpt: 0.000678 acc_pose: 0.804785 loss: 0.000678 2022/09/26 13:21:49 - mmengine - INFO - Epoch(train) [50][200/293] lr: 5.000000e-04 eta: 6:57:37 time: 0.573166 data_time: 0.114741 memory: 9504 loss_kpt: 0.000674 acc_pose: 0.844739 loss: 0.000674 2022/09/26 13:22:20 - mmengine - INFO - Epoch(train) [50][250/293] lr: 5.000000e-04 eta: 6:57:25 time: 0.621850 data_time: 0.078918 memory: 9504 loss_kpt: 0.000683 acc_pose: 0.787422 loss: 0.000683 2022/09/26 13:22:46 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:22:46 - mmengine - INFO - Saving checkpoint at 50 epochs 2022/09/26 13:23:09 - mmengine - INFO - Epoch(val) [50][50/407] eta: 0:02:02 time: 0.343124 data_time: 0.065745 memory: 9504 2022/09/26 13:23:26 - mmengine - INFO - Epoch(val) [50][100/407] eta: 0:01:40 time: 0.326223 data_time: 0.057839 memory: 1378 2022/09/26 13:23:42 - mmengine - INFO - Epoch(val) [50][150/407] eta: 0:01:26 time: 0.334846 data_time: 0.043630 memory: 1378 2022/09/26 13:23:58 - mmengine - INFO - Epoch(val) [50][200/407] eta: 0:01:06 time: 0.319390 data_time: 0.052881 memory: 1378 2022/09/26 13:24:15 - mmengine - INFO - Epoch(val) [50][250/407] eta: 0:00:51 time: 0.330824 data_time: 0.053570 memory: 1378 2022/09/26 13:24:31 - mmengine - INFO - Epoch(val) [50][300/407] eta: 0:00:34 time: 0.322544 data_time: 0.036241 memory: 1378 2022/09/26 13:24:48 - mmengine - INFO - Epoch(val) [50][350/407] eta: 0:00:19 time: 0.339819 data_time: 0.042601 memory: 1378 2022/09/26 13:25:01 - mmengine - INFO - Epoch(val) [50][400/407] eta: 0:00:01 time: 0.260846 data_time: 0.032216 memory: 1378 2022/09/26 13:25:35 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 13:25:47 - mmengine - INFO - Epoch(val) [50][407/407] coco/AP: 0.692133 coco/AP .5: 0.883855 coco/AP .75: 0.766972 coco/AP (M): 0.657050 coco/AP (L): 0.756549 coco/AR: 0.750000 coco/AR .5: 0.924433 coco/AR .75: 0.816908 coco/AR (M): 0.707675 coco/AR (L): 0.811074 2022/09/26 13:25:48 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_40.pth is removed 2022/09/26 13:25:50 - mmengine - INFO - The best checkpoint with 0.6921 coco/AP at 50 epoch is saved to best_coco/AP_epoch_50.pth. 2022/09/26 13:26:19 - mmengine - INFO - Epoch(train) [51][50/293] lr: 5.000000e-04 eta: 6:55:29 time: 0.580461 data_time: 0.157653 memory: 9504 loss_kpt: 0.000676 acc_pose: 0.780203 loss: 0.000676 2022/09/26 13:26:49 - mmengine - INFO - Epoch(train) [51][100/293] lr: 5.000000e-04 eta: 6:55:12 time: 0.590485 data_time: 0.091086 memory: 9504 loss_kpt: 0.000671 acc_pose: 0.738575 loss: 0.000671 2022/09/26 13:27:19 - mmengine - INFO - Epoch(train) [51][150/293] lr: 5.000000e-04 eta: 6:54:58 time: 0.610343 data_time: 0.085435 memory: 9504 loss_kpt: 0.000676 acc_pose: 0.825133 loss: 0.000676 2022/09/26 13:27:48 - mmengine - INFO - Epoch(train) [51][200/293] lr: 5.000000e-04 eta: 6:54:37 time: 0.573871 data_time: 0.079733 memory: 9504 loss_kpt: 0.000671 acc_pose: 0.774349 loss: 0.000671 2022/09/26 13:28:19 - mmengine - INFO - Epoch(train) [51][250/293] lr: 5.000000e-04 eta: 6:54:24 time: 0.619787 data_time: 0.098877 memory: 9504 loss_kpt: 0.000669 acc_pose: 0.745058 loss: 0.000669 2022/09/26 13:28:43 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:29:14 - mmengine - INFO - Epoch(train) [52][50/293] lr: 5.000000e-04 eta: 6:52:36 time: 0.615713 data_time: 0.098015 memory: 9504 loss_kpt: 0.000676 acc_pose: 0.826486 loss: 0.000676 2022/09/26 13:29:19 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:29:46 - mmengine - INFO - Epoch(train) [52][100/293] lr: 5.000000e-04 eta: 6:52:26 time: 0.634536 data_time: 0.096077 memory: 9504 loss_kpt: 0.000673 acc_pose: 0.796481 loss: 0.000673 2022/09/26 13:30:16 - mmengine - INFO - Epoch(train) [52][150/293] lr: 5.000000e-04 eta: 6:52:09 time: 0.597747 data_time: 0.083641 memory: 9504 loss_kpt: 0.000671 acc_pose: 0.832980 loss: 0.000671 2022/09/26 13:30:46 - mmengine - INFO - Epoch(train) [52][200/293] lr: 5.000000e-04 eta: 6:51:54 time: 0.607218 data_time: 0.089836 memory: 9504 loss_kpt: 0.000682 acc_pose: 0.833826 loss: 0.000682 2022/09/26 13:31:15 - mmengine - INFO - Epoch(train) [52][250/293] lr: 5.000000e-04 eta: 6:51:35 time: 0.581473 data_time: 0.134753 memory: 9504 loss_kpt: 0.000667 acc_pose: 0.808490 loss: 0.000667 2022/09/26 13:31:41 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:32:12 - mmengine - INFO - Epoch(train) [53][50/293] lr: 5.000000e-04 eta: 6:49:48 time: 0.611004 data_time: 0.092740 memory: 9504 loss_kpt: 0.000671 acc_pose: 0.812356 loss: 0.000671 2022/09/26 13:32:42 - mmengine - INFO - Epoch(train) [53][100/293] lr: 5.000000e-04 eta: 6:49:33 time: 0.614768 data_time: 0.084685 memory: 9504 loss_kpt: 0.000689 acc_pose: 0.820850 loss: 0.000689 2022/09/26 13:33:12 - mmengine - INFO - Epoch(train) [53][150/293] lr: 5.000000e-04 eta: 6:49:16 time: 0.590966 data_time: 0.078450 memory: 9504 loss_kpt: 0.000668 acc_pose: 0.829011 loss: 0.000668 2022/09/26 13:33:42 - mmengine - INFO - Epoch(train) [53][200/293] lr: 5.000000e-04 eta: 6:48:59 time: 0.596051 data_time: 0.087480 memory: 9504 loss_kpt: 0.000672 acc_pose: 0.757708 loss: 0.000672 2022/09/26 13:34:12 - mmengine - INFO - Epoch(train) [53][250/293] lr: 5.000000e-04 eta: 6:48:44 time: 0.610788 data_time: 0.078952 memory: 9504 loss_kpt: 0.000681 acc_pose: 0.779773 loss: 0.000681 2022/09/26 13:34:37 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:35:08 - mmengine - INFO - Epoch(train) [54][50/293] lr: 5.000000e-04 eta: 6:46:59 time: 0.619693 data_time: 0.104693 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.732481 loss: 0.000666 2022/09/26 13:35:39 - mmengine - INFO - Epoch(train) [54][100/293] lr: 5.000000e-04 eta: 6:46:44 time: 0.608074 data_time: 0.143624 memory: 9504 loss_kpt: 0.000681 acc_pose: 0.853260 loss: 0.000681 2022/09/26 13:36:08 - mmengine - INFO - Epoch(train) [54][150/293] lr: 5.000000e-04 eta: 6:46:26 time: 0.586296 data_time: 0.171667 memory: 9504 loss_kpt: 0.000680 acc_pose: 0.791212 loss: 0.000680 2022/09/26 13:36:38 - mmengine - INFO - Epoch(train) [54][200/293] lr: 5.000000e-04 eta: 6:46:10 time: 0.605748 data_time: 0.079772 memory: 9504 loss_kpt: 0.000665 acc_pose: 0.737219 loss: 0.000665 2022/09/26 13:37:10 - mmengine - INFO - Epoch(train) [54][250/293] lr: 5.000000e-04 eta: 6:45:56 time: 0.620623 data_time: 0.109000 memory: 9504 loss_kpt: 0.000673 acc_pose: 0.808527 loss: 0.000673 2022/09/26 13:37:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:38:04 - mmengine - INFO - Epoch(train) [55][50/293] lr: 5.000000e-04 eta: 6:44:12 time: 0.615085 data_time: 0.108798 memory: 9504 loss_kpt: 0.000678 acc_pose: 0.811425 loss: 0.000678 2022/09/26 13:38:35 - mmengine - INFO - Epoch(train) [55][100/293] lr: 5.000000e-04 eta: 6:43:56 time: 0.604585 data_time: 0.092790 memory: 9504 loss_kpt: 0.000667 acc_pose: 0.779937 loss: 0.000667 2022/09/26 13:39:05 - mmengine - INFO - Epoch(train) [55][150/293] lr: 5.000000e-04 eta: 6:43:40 time: 0.603175 data_time: 0.091806 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.785261 loss: 0.000666 2022/09/26 13:39:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:39:34 - mmengine - INFO - Epoch(train) [55][200/293] lr: 5.000000e-04 eta: 6:43:21 time: 0.585522 data_time: 0.077978 memory: 9504 loss_kpt: 0.000677 acc_pose: 0.810505 loss: 0.000677 2022/09/26 13:40:03 - mmengine - INFO - Epoch(train) [55][250/293] lr: 5.000000e-04 eta: 6:43:01 time: 0.575759 data_time: 0.077768 memory: 9504 loss_kpt: 0.000672 acc_pose: 0.801585 loss: 0.000672 2022/09/26 13:40:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:41:00 - mmengine - INFO - Epoch(train) [56][50/293] lr: 5.000000e-04 eta: 6:41:21 time: 0.634079 data_time: 0.093236 memory: 9504 loss_kpt: 0.000679 acc_pose: 0.767159 loss: 0.000679 2022/09/26 13:41:31 - mmengine - INFO - Epoch(train) [56][100/293] lr: 5.000000e-04 eta: 6:41:06 time: 0.607743 data_time: 0.078040 memory: 9504 loss_kpt: 0.000686 acc_pose: 0.787877 loss: 0.000686 2022/09/26 13:42:01 - mmengine - INFO - Epoch(train) [56][150/293] lr: 5.000000e-04 eta: 6:40:48 time: 0.597169 data_time: 0.098362 memory: 9504 loss_kpt: 0.000672 acc_pose: 0.798629 loss: 0.000672 2022/09/26 13:42:29 - mmengine - INFO - Epoch(train) [56][200/293] lr: 5.000000e-04 eta: 6:40:27 time: 0.572821 data_time: 0.128688 memory: 9504 loss_kpt: 0.000678 acc_pose: 0.780773 loss: 0.000678 2022/09/26 13:42:58 - mmengine - INFO - Epoch(train) [56][250/293] lr: 5.000000e-04 eta: 6:40:08 time: 0.584121 data_time: 0.082627 memory: 9504 loss_kpt: 0.000665 acc_pose: 0.868909 loss: 0.000665 2022/09/26 13:43:25 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:43:56 - mmengine - INFO - Epoch(train) [57][50/293] lr: 5.000000e-04 eta: 6:38:28 time: 0.617670 data_time: 0.146811 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.841180 loss: 0.000666 2022/09/26 13:44:25 - mmengine - INFO - Epoch(train) [57][100/293] lr: 5.000000e-04 eta: 6:38:10 time: 0.594529 data_time: 0.094580 memory: 9504 loss_kpt: 0.000660 acc_pose: 0.775869 loss: 0.000660 2022/09/26 13:44:56 - mmengine - INFO - Epoch(train) [57][150/293] lr: 5.000000e-04 eta: 6:37:56 time: 0.618339 data_time: 0.079723 memory: 9504 loss_kpt: 0.000670 acc_pose: 0.782355 loss: 0.000670 2022/09/26 13:45:27 - mmengine - INFO - Epoch(train) [57][200/293] lr: 5.000000e-04 eta: 6:37:40 time: 0.611268 data_time: 0.095026 memory: 9504 loss_kpt: 0.000676 acc_pose: 0.781189 loss: 0.000676 2022/09/26 13:45:57 - mmengine - INFO - Epoch(train) [57][250/293] lr: 5.000000e-04 eta: 6:37:23 time: 0.606042 data_time: 0.106257 memory: 9504 loss_kpt: 0.000684 acc_pose: 0.782572 loss: 0.000684 2022/09/26 13:46:23 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:46:53 - mmengine - INFO - Epoch(train) [58][50/293] lr: 5.000000e-04 eta: 6:35:43 time: 0.608903 data_time: 0.099491 memory: 9504 loss_kpt: 0.000667 acc_pose: 0.825528 loss: 0.000667 2022/09/26 13:47:24 - mmengine - INFO - Epoch(train) [58][100/293] lr: 5.000000e-04 eta: 6:35:27 time: 0.609178 data_time: 0.133144 memory: 9504 loss_kpt: 0.000670 acc_pose: 0.784477 loss: 0.000670 2022/09/26 13:47:53 - mmengine - INFO - Epoch(train) [58][150/293] lr: 5.000000e-04 eta: 6:35:07 time: 0.581473 data_time: 0.099145 memory: 9504 loss_kpt: 0.000673 acc_pose: 0.791813 loss: 0.000673 2022/09/26 13:48:22 - mmengine - INFO - Epoch(train) [58][200/293] lr: 5.000000e-04 eta: 6:34:48 time: 0.586218 data_time: 0.089721 memory: 9504 loss_kpt: 0.000678 acc_pose: 0.836369 loss: 0.000678 2022/09/26 13:48:53 - mmengine - INFO - Epoch(train) [58][250/293] lr: 5.000000e-04 eta: 6:34:32 time: 0.608573 data_time: 0.102637 memory: 9504 loss_kpt: 0.000675 acc_pose: 0.793860 loss: 0.000675 2022/09/26 13:49:19 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:49:23 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:49:50 - mmengine - INFO - Epoch(train) [59][50/293] lr: 5.000000e-04 eta: 6:32:54 time: 0.613426 data_time: 0.113598 memory: 9504 loss_kpt: 0.000667 acc_pose: 0.812940 loss: 0.000667 2022/09/26 13:50:19 - mmengine - INFO - Epoch(train) [59][100/293] lr: 5.000000e-04 eta: 6:32:36 time: 0.597416 data_time: 0.153371 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.848036 loss: 0.000666 2022/09/26 13:50:49 - mmengine - INFO - Epoch(train) [59][150/293] lr: 5.000000e-04 eta: 6:32:18 time: 0.595872 data_time: 0.126553 memory: 9504 loss_kpt: 0.000672 acc_pose: 0.816424 loss: 0.000672 2022/09/26 13:51:20 - mmengine - INFO - Epoch(train) [59][200/293] lr: 5.000000e-04 eta: 6:32:01 time: 0.607115 data_time: 0.089788 memory: 9504 loss_kpt: 0.000678 acc_pose: 0.799949 loss: 0.000678 2022/09/26 13:51:48 - mmengine - INFO - Epoch(train) [59][250/293] lr: 5.000000e-04 eta: 6:31:39 time: 0.565654 data_time: 0.081489 memory: 9504 loss_kpt: 0.000676 acc_pose: 0.740092 loss: 0.000676 2022/09/26 13:52:13 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:52:46 - mmengine - INFO - Epoch(train) [60][50/293] lr: 5.000000e-04 eta: 6:30:07 time: 0.646907 data_time: 0.101701 memory: 9504 loss_kpt: 0.000665 acc_pose: 0.786217 loss: 0.000665 2022/09/26 13:53:16 - mmengine - INFO - Epoch(train) [60][100/293] lr: 5.000000e-04 eta: 6:29:50 time: 0.608419 data_time: 0.082788 memory: 9504 loss_kpt: 0.000656 acc_pose: 0.791849 loss: 0.000656 2022/09/26 13:53:45 - mmengine - INFO - Epoch(train) [60][150/293] lr: 5.000000e-04 eta: 6:29:29 time: 0.569859 data_time: 0.081043 memory: 9504 loss_kpt: 0.000657 acc_pose: 0.799867 loss: 0.000657 2022/09/26 13:54:15 - mmengine - INFO - Epoch(train) [60][200/293] lr: 5.000000e-04 eta: 6:29:11 time: 0.602328 data_time: 0.122151 memory: 9504 loss_kpt: 0.000681 acc_pose: 0.811923 loss: 0.000681 2022/09/26 13:54:44 - mmengine - INFO - Epoch(train) [60][250/293] lr: 5.000000e-04 eta: 6:28:51 time: 0.579459 data_time: 0.101061 memory: 9504 loss_kpt: 0.000669 acc_pose: 0.803933 loss: 0.000669 2022/09/26 13:55:09 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 13:55:09 - mmengine - INFO - Saving checkpoint at 60 epochs 2022/09/26 13:55:32 - mmengine - INFO - Epoch(val) [60][50/407] eta: 0:02:06 time: 0.353088 data_time: 0.081373 memory: 9504 2022/09/26 13:55:49 - mmengine - INFO - Epoch(val) [60][100/407] eta: 0:01:42 time: 0.335299 data_time: 0.062796 memory: 1378 2022/09/26 13:56:05 - mmengine - INFO - Epoch(val) [60][150/407] eta: 0:01:23 time: 0.326386 data_time: 0.039859 memory: 1378 2022/09/26 13:56:22 - mmengine - INFO - Epoch(val) [60][200/407] eta: 0:01:08 time: 0.330513 data_time: 0.035587 memory: 1378 2022/09/26 13:56:37 - mmengine - INFO - Epoch(val) [60][250/407] eta: 0:00:47 time: 0.304767 data_time: 0.037946 memory: 1378 2022/09/26 13:56:54 - mmengine - INFO - Epoch(val) [60][300/407] eta: 0:00:35 time: 0.329929 data_time: 0.073778 memory: 1378 2022/09/26 13:57:10 - mmengine - INFO - Epoch(val) [60][350/407] eta: 0:00:18 time: 0.323979 data_time: 0.049330 memory: 1378 2022/09/26 13:57:24 - mmengine - INFO - Epoch(val) [60][400/407] eta: 0:00:01 time: 0.275261 data_time: 0.065512 memory: 1378 2022/09/26 13:57:56 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 13:58:09 - mmengine - INFO - Epoch(val) [60][407/407] coco/AP: 0.698995 coco/AP .5: 0.887745 coco/AP .75: 0.776998 coco/AP (M): 0.661195 coco/AP (L): 0.766925 coco/AR: 0.756722 coco/AR .5: 0.929314 coco/AR .75: 0.825252 coco/AR (M): 0.713166 coco/AR (L): 0.818989 2022/09/26 13:58:09 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_50.pth is removed 2022/09/26 13:58:13 - mmengine - INFO - The best checkpoint with 0.6990 coco/AP at 60 epoch is saved to best_coco/AP_epoch_60.pth. 2022/09/26 13:58:43 - mmengine - INFO - Epoch(train) [61][50/293] lr: 5.000000e-04 eta: 6:27:14 time: 0.604159 data_time: 0.142730 memory: 9504 loss_kpt: 0.000660 acc_pose: 0.785860 loss: 0.000660 2022/09/26 13:59:14 - mmengine - INFO - Epoch(train) [61][100/293] lr: 5.000000e-04 eta: 6:26:59 time: 0.617505 data_time: 0.086526 memory: 9504 loss_kpt: 0.000671 acc_pose: 0.847653 loss: 0.000671 2022/09/26 13:59:44 - mmengine - INFO - Epoch(train) [61][150/293] lr: 5.000000e-04 eta: 6:26:40 time: 0.595733 data_time: 0.100587 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.812245 loss: 0.000666 2022/09/26 14:00:13 - mmengine - INFO - Epoch(train) [61][200/293] lr: 5.000000e-04 eta: 6:26:22 time: 0.593520 data_time: 0.091424 memory: 9504 loss_kpt: 0.000663 acc_pose: 0.744647 loss: 0.000663 2022/09/26 14:00:43 - mmengine - INFO - Epoch(train) [61][250/293] lr: 5.000000e-04 eta: 6:26:02 time: 0.587592 data_time: 0.102947 memory: 9504 loss_kpt: 0.000673 acc_pose: 0.835222 loss: 0.000673 2022/09/26 14:01:09 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:01:40 - mmengine - INFO - Epoch(train) [62][50/293] lr: 5.000000e-04 eta: 6:24:28 time: 0.619761 data_time: 0.177632 memory: 9504 loss_kpt: 0.000652 acc_pose: 0.803575 loss: 0.000652 2022/09/26 14:02:10 - mmengine - INFO - Epoch(train) [62][100/293] lr: 5.000000e-04 eta: 6:24:10 time: 0.595484 data_time: 0.082721 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.826155 loss: 0.000644 2022/09/26 14:02:25 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:02:39 - mmengine - INFO - Epoch(train) [62][150/293] lr: 5.000000e-04 eta: 6:23:48 time: 0.568962 data_time: 0.081902 memory: 9504 loss_kpt: 0.000665 acc_pose: 0.856743 loss: 0.000665 2022/09/26 14:03:09 - mmengine - INFO - Epoch(train) [62][200/293] lr: 5.000000e-04 eta: 6:23:31 time: 0.608883 data_time: 0.078218 memory: 9504 loss_kpt: 0.000657 acc_pose: 0.834257 loss: 0.000657 2022/09/26 14:03:38 - mmengine - INFO - Epoch(train) [62][250/293] lr: 5.000000e-04 eta: 6:23:12 time: 0.587156 data_time: 0.089289 memory: 9504 loss_kpt: 0.000658 acc_pose: 0.782090 loss: 0.000658 2022/09/26 14:04:04 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:04:35 - mmengine - INFO - Epoch(train) [63][50/293] lr: 5.000000e-04 eta: 6:21:40 time: 0.625483 data_time: 0.110791 memory: 9504 loss_kpt: 0.000661 acc_pose: 0.839270 loss: 0.000661 2022/09/26 14:05:04 - mmengine - INFO - Epoch(train) [63][100/293] lr: 5.000000e-04 eta: 6:21:20 time: 0.585447 data_time: 0.116285 memory: 9504 loss_kpt: 0.000664 acc_pose: 0.813660 loss: 0.000664 2022/09/26 14:05:33 - mmengine - INFO - Epoch(train) [63][150/293] lr: 5.000000e-04 eta: 6:21:00 time: 0.581209 data_time: 0.121331 memory: 9504 loss_kpt: 0.000653 acc_pose: 0.804364 loss: 0.000653 2022/09/26 14:06:04 - mmengine - INFO - Epoch(train) [63][200/293] lr: 5.000000e-04 eta: 6:20:43 time: 0.609117 data_time: 0.088891 memory: 9504 loss_kpt: 0.000662 acc_pose: 0.759864 loss: 0.000662 2022/09/26 14:06:34 - mmengine - INFO - Epoch(train) [63][250/293] lr: 5.000000e-04 eta: 6:20:24 time: 0.593617 data_time: 0.093808 memory: 9504 loss_kpt: 0.000671 acc_pose: 0.836802 loss: 0.000671 2022/09/26 14:06:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:07:29 - mmengine - INFO - Epoch(train) [64][50/293] lr: 5.000000e-04 eta: 6:18:50 time: 0.600901 data_time: 0.094706 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.793915 loss: 0.000666 2022/09/26 14:07:59 - mmengine - INFO - Epoch(train) [64][100/293] lr: 5.000000e-04 eta: 6:18:32 time: 0.597547 data_time: 0.076732 memory: 9504 loss_kpt: 0.000654 acc_pose: 0.808371 loss: 0.000654 2022/09/26 14:08:27 - mmengine - INFO - Epoch(train) [64][150/293] lr: 5.000000e-04 eta: 6:18:10 time: 0.565545 data_time: 0.089632 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.821236 loss: 0.000644 2022/09/26 14:08:58 - mmengine - INFO - Epoch(train) [64][200/293] lr: 5.000000e-04 eta: 6:17:53 time: 0.612602 data_time: 0.079304 memory: 9504 loss_kpt: 0.000657 acc_pose: 0.790006 loss: 0.000657 2022/09/26 14:09:29 - mmengine - INFO - Epoch(train) [64][250/293] lr: 5.000000e-04 eta: 6:17:36 time: 0.616843 data_time: 0.088259 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.815828 loss: 0.000644 2022/09/26 14:09:55 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:10:27 - mmengine - INFO - Epoch(train) [65][50/293] lr: 5.000000e-04 eta: 6:16:07 time: 0.628673 data_time: 0.120256 memory: 9504 loss_kpt: 0.000661 acc_pose: 0.799869 loss: 0.000661 2022/09/26 14:10:55 - mmengine - INFO - Epoch(train) [65][100/293] lr: 5.000000e-04 eta: 6:15:45 time: 0.572896 data_time: 0.088590 memory: 9504 loss_kpt: 0.000663 acc_pose: 0.806624 loss: 0.000663 2022/09/26 14:11:25 - mmengine - INFO - Epoch(train) [65][150/293] lr: 5.000000e-04 eta: 6:15:26 time: 0.592446 data_time: 0.077075 memory: 9504 loss_kpt: 0.000658 acc_pose: 0.786534 loss: 0.000658 2022/09/26 14:11:56 - mmengine - INFO - Epoch(train) [65][200/293] lr: 5.000000e-04 eta: 6:15:10 time: 0.616213 data_time: 0.139886 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.775179 loss: 0.000644 2022/09/26 14:12:23 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:12:25 - mmengine - INFO - Epoch(train) [65][250/293] lr: 5.000000e-04 eta: 6:14:49 time: 0.579811 data_time: 0.089971 memory: 9504 loss_kpt: 0.000658 acc_pose: 0.838298 loss: 0.000658 2022/09/26 14:12:50 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:13:20 - mmengine - INFO - Epoch(train) [66][50/293] lr: 5.000000e-04 eta: 6:13:17 time: 0.598812 data_time: 0.115927 memory: 9504 loss_kpt: 0.000654 acc_pose: 0.815446 loss: 0.000654 2022/09/26 14:13:50 - mmengine - INFO - Epoch(train) [66][100/293] lr: 5.000000e-04 eta: 6:12:57 time: 0.587852 data_time: 0.102533 memory: 9504 loss_kpt: 0.000661 acc_pose: 0.818816 loss: 0.000661 2022/09/26 14:14:20 - mmengine - INFO - Epoch(train) [66][150/293] lr: 5.000000e-04 eta: 6:12:40 time: 0.609392 data_time: 0.069201 memory: 9504 loss_kpt: 0.000650 acc_pose: 0.816482 loss: 0.000650 2022/09/26 14:14:50 - mmengine - INFO - Epoch(train) [66][200/293] lr: 5.000000e-04 eta: 6:12:21 time: 0.596263 data_time: 0.094596 memory: 9504 loss_kpt: 0.000650 acc_pose: 0.777485 loss: 0.000650 2022/09/26 14:15:20 - mmengine - INFO - Epoch(train) [66][250/293] lr: 5.000000e-04 eta: 6:12:02 time: 0.597785 data_time: 0.080116 memory: 9504 loss_kpt: 0.000656 acc_pose: 0.815593 loss: 0.000656 2022/09/26 14:15:46 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:16:17 - mmengine - INFO - Epoch(train) [67][50/293] lr: 5.000000e-04 eta: 6:10:32 time: 0.607017 data_time: 0.105541 memory: 9504 loss_kpt: 0.000650 acc_pose: 0.826891 loss: 0.000650 2022/09/26 14:16:47 - mmengine - INFO - Epoch(train) [67][100/293] lr: 5.000000e-04 eta: 6:10:14 time: 0.605242 data_time: 0.097024 memory: 9504 loss_kpt: 0.000663 acc_pose: 0.813883 loss: 0.000663 2022/09/26 14:17:17 - mmengine - INFO - Epoch(train) [67][150/293] lr: 5.000000e-04 eta: 6:09:55 time: 0.591819 data_time: 0.077193 memory: 9504 loss_kpt: 0.000650 acc_pose: 0.814987 loss: 0.000650 2022/09/26 14:17:47 - mmengine - INFO - Epoch(train) [67][200/293] lr: 5.000000e-04 eta: 6:09:37 time: 0.606989 data_time: 0.089624 memory: 9504 loss_kpt: 0.000645 acc_pose: 0.832689 loss: 0.000645 2022/09/26 14:18:17 - mmengine - INFO - Epoch(train) [67][250/293] lr: 5.000000e-04 eta: 6:09:17 time: 0.591934 data_time: 0.079696 memory: 9504 loss_kpt: 0.000655 acc_pose: 0.782260 loss: 0.000655 2022/09/26 14:18:41 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:19:11 - mmengine - INFO - Epoch(train) [68][50/293] lr: 5.000000e-04 eta: 6:07:48 time: 0.609947 data_time: 0.097179 memory: 9504 loss_kpt: 0.000655 acc_pose: 0.817008 loss: 0.000655 2022/09/26 14:19:40 - mmengine - INFO - Epoch(train) [68][100/293] lr: 5.000000e-04 eta: 6:07:28 time: 0.581139 data_time: 0.078879 memory: 9504 loss_kpt: 0.000654 acc_pose: 0.810758 loss: 0.000654 2022/09/26 14:20:10 - mmengine - INFO - Epoch(train) [68][150/293] lr: 5.000000e-04 eta: 6:07:08 time: 0.587281 data_time: 0.091261 memory: 9504 loss_kpt: 0.000666 acc_pose: 0.853997 loss: 0.000666 2022/09/26 14:20:39 - mmengine - INFO - Epoch(train) [68][200/293] lr: 5.000000e-04 eta: 6:06:47 time: 0.583972 data_time: 0.099556 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.825831 loss: 0.000630 2022/09/26 14:21:08 - mmengine - INFO - Epoch(train) [68][250/293] lr: 5.000000e-04 eta: 6:06:26 time: 0.577177 data_time: 0.105632 memory: 9504 loss_kpt: 0.000665 acc_pose: 0.776642 loss: 0.000665 2022/09/26 14:21:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:22:05 - mmengine - INFO - Epoch(train) [69][50/293] lr: 5.000000e-04 eta: 6:05:01 time: 0.639798 data_time: 0.251547 memory: 9504 loss_kpt: 0.000655 acc_pose: 0.779252 loss: 0.000655 2022/09/26 14:22:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:22:35 - mmengine - INFO - Epoch(train) [69][100/293] lr: 5.000000e-04 eta: 6:04:43 time: 0.601781 data_time: 0.115337 memory: 9504 loss_kpt: 0.000629 acc_pose: 0.806680 loss: 0.000629 2022/09/26 14:23:04 - mmengine - INFO - Epoch(train) [69][150/293] lr: 5.000000e-04 eta: 6:04:22 time: 0.578340 data_time: 0.091166 memory: 9504 loss_kpt: 0.000645 acc_pose: 0.803144 loss: 0.000645 2022/09/26 14:23:34 - mmengine - INFO - Epoch(train) [69][200/293] lr: 5.000000e-04 eta: 6:04:02 time: 0.596887 data_time: 0.084293 memory: 9504 loss_kpt: 0.000640 acc_pose: 0.812374 loss: 0.000640 2022/09/26 14:24:03 - mmengine - INFO - Epoch(train) [69][250/293] lr: 5.000000e-04 eta: 6:03:41 time: 0.577708 data_time: 0.083314 memory: 9504 loss_kpt: 0.000650 acc_pose: 0.820109 loss: 0.000650 2022/09/26 14:24:27 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:24:58 - mmengine - INFO - Epoch(train) [70][50/293] lr: 5.000000e-04 eta: 6:02:15 time: 0.620629 data_time: 0.099526 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.850504 loss: 0.000644 2022/09/26 14:25:28 - mmengine - INFO - Epoch(train) [70][100/293] lr: 5.000000e-04 eta: 6:01:57 time: 0.600598 data_time: 0.076366 memory: 9504 loss_kpt: 0.000656 acc_pose: 0.809161 loss: 0.000656 2022/09/26 14:25:59 - mmengine - INFO - Epoch(train) [70][150/293] lr: 5.000000e-04 eta: 6:01:38 time: 0.604019 data_time: 0.103768 memory: 9504 loss_kpt: 0.000652 acc_pose: 0.873137 loss: 0.000652 2022/09/26 14:26:28 - mmengine - INFO - Epoch(train) [70][200/293] lr: 5.000000e-04 eta: 6:01:17 time: 0.581389 data_time: 0.075482 memory: 9504 loss_kpt: 0.000648 acc_pose: 0.785513 loss: 0.000648 2022/09/26 14:26:57 - mmengine - INFO - Epoch(train) [70][250/293] lr: 5.000000e-04 eta: 6:00:56 time: 0.575051 data_time: 0.087221 memory: 9504 loss_kpt: 0.000663 acc_pose: 0.795113 loss: 0.000663 2022/09/26 14:27:22 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:27:22 - mmengine - INFO - Saving checkpoint at 70 epochs 2022/09/26 14:27:45 - mmengine - INFO - Epoch(val) [70][50/407] eta: 0:01:58 time: 0.331139 data_time: 0.047420 memory: 9504 2022/09/26 14:28:02 - mmengine - INFO - Epoch(val) [70][100/407] eta: 0:01:43 time: 0.337186 data_time: 0.059161 memory: 1378 2022/09/26 14:28:19 - mmengine - INFO - Epoch(val) [70][150/407] eta: 0:01:25 time: 0.332855 data_time: 0.046753 memory: 1378 2022/09/26 14:28:35 - mmengine - INFO - Epoch(val) [70][200/407] eta: 0:01:05 time: 0.315700 data_time: 0.032584 memory: 1378 2022/09/26 14:28:50 - mmengine - INFO - Epoch(val) [70][250/407] eta: 0:00:49 time: 0.313409 data_time: 0.029247 memory: 1378 2022/09/26 14:29:06 - mmengine - INFO - Epoch(val) [70][300/407] eta: 0:00:33 time: 0.310197 data_time: 0.042624 memory: 1378 2022/09/26 14:29:23 - mmengine - INFO - Epoch(val) [70][350/407] eta: 0:00:19 time: 0.340704 data_time: 0.064896 memory: 1378 2022/09/26 14:29:36 - mmengine - INFO - Epoch(val) [70][400/407] eta: 0:00:01 time: 0.262107 data_time: 0.038098 memory: 1378 2022/09/26 14:30:08 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 14:30:21 - mmengine - INFO - Epoch(val) [70][407/407] coco/AP: 0.698083 coco/AP .5: 0.884697 coco/AP .75: 0.773491 coco/AP (M): 0.660409 coco/AP (L): 0.768592 coco/AR: 0.755557 coco/AR .5: 0.926795 coco/AR .75: 0.821631 coco/AR (M): 0.711527 coco/AR (L): 0.819101 2022/09/26 14:30:51 - mmengine - INFO - Epoch(train) [71][50/293] lr: 5.000000e-04 eta: 5:59:28 time: 0.597225 data_time: 0.093737 memory: 9504 loss_kpt: 0.000639 acc_pose: 0.859121 loss: 0.000639 2022/09/26 14:31:20 - mmengine - INFO - Epoch(train) [71][100/293] lr: 5.000000e-04 eta: 5:59:08 time: 0.587904 data_time: 0.110850 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.803906 loss: 0.000644 2022/09/26 14:31:51 - mmengine - INFO - Epoch(train) [71][150/293] lr: 5.000000e-04 eta: 5:58:49 time: 0.605392 data_time: 0.091112 memory: 9504 loss_kpt: 0.000640 acc_pose: 0.816420 loss: 0.000640 2022/09/26 14:32:21 - mmengine - INFO - Epoch(train) [71][200/293] lr: 5.000000e-04 eta: 5:58:30 time: 0.600612 data_time: 0.083328 memory: 9504 loss_kpt: 0.000650 acc_pose: 0.847982 loss: 0.000650 2022/09/26 14:32:49 - mmengine - INFO - Epoch(train) [71][250/293] lr: 5.000000e-04 eta: 5:58:09 time: 0.575165 data_time: 0.081483 memory: 9504 loss_kpt: 0.000651 acc_pose: 0.819167 loss: 0.000651 2022/09/26 14:33:14 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:33:44 - mmengine - INFO - Epoch(train) [72][50/293] lr: 5.000000e-04 eta: 5:56:43 time: 0.604783 data_time: 0.114012 memory: 9504 loss_kpt: 0.000654 acc_pose: 0.792605 loss: 0.000654 2022/09/26 14:34:13 - mmengine - INFO - Epoch(train) [72][100/293] lr: 5.000000e-04 eta: 5:56:22 time: 0.581364 data_time: 0.083927 memory: 9504 loss_kpt: 0.000646 acc_pose: 0.805476 loss: 0.000646 2022/09/26 14:34:43 - mmengine - INFO - Epoch(train) [72][150/293] lr: 5.000000e-04 eta: 5:56:03 time: 0.602311 data_time: 0.089985 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.850966 loss: 0.000641 2022/09/26 14:35:11 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:35:13 - mmengine - INFO - Epoch(train) [72][200/293] lr: 5.000000e-04 eta: 5:55:44 time: 0.603312 data_time: 0.084083 memory: 9504 loss_kpt: 0.000651 acc_pose: 0.839241 loss: 0.000651 2022/09/26 14:35:42 - mmengine - INFO - Epoch(train) [72][250/293] lr: 5.000000e-04 eta: 5:55:23 time: 0.578875 data_time: 0.110837 memory: 9504 loss_kpt: 0.000652 acc_pose: 0.814385 loss: 0.000652 2022/09/26 14:36:07 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:36:39 - mmengine - INFO - Epoch(train) [73][50/293] lr: 5.000000e-04 eta: 5:54:00 time: 0.623360 data_time: 0.140580 memory: 9504 loss_kpt: 0.000647 acc_pose: 0.815448 loss: 0.000647 2022/09/26 14:37:10 - mmengine - INFO - Epoch(train) [73][100/293] lr: 5.000000e-04 eta: 5:53:42 time: 0.618488 data_time: 0.099723 memory: 9504 loss_kpt: 0.000633 acc_pose: 0.826461 loss: 0.000633 2022/09/26 14:37:38 - mmengine - INFO - Epoch(train) [73][150/293] lr: 5.000000e-04 eta: 5:53:20 time: 0.570393 data_time: 0.083155 memory: 9504 loss_kpt: 0.000640 acc_pose: 0.840535 loss: 0.000640 2022/09/26 14:38:07 - mmengine - INFO - Epoch(train) [73][200/293] lr: 5.000000e-04 eta: 5:52:59 time: 0.581628 data_time: 0.077904 memory: 9504 loss_kpt: 0.000646 acc_pose: 0.797181 loss: 0.000646 2022/09/26 14:38:37 - mmengine - INFO - Epoch(train) [73][250/293] lr: 5.000000e-04 eta: 5:52:40 time: 0.601760 data_time: 0.085470 memory: 9504 loss_kpt: 0.000645 acc_pose: 0.815063 loss: 0.000645 2022/09/26 14:39:01 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:39:32 - mmengine - INFO - Epoch(train) [74][50/293] lr: 5.000000e-04 eta: 5:51:17 time: 0.617462 data_time: 0.118234 memory: 9504 loss_kpt: 0.000648 acc_pose: 0.791152 loss: 0.000648 2022/09/26 14:40:01 - mmengine - INFO - Epoch(train) [74][100/293] lr: 5.000000e-04 eta: 5:50:55 time: 0.572439 data_time: 0.071569 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.814072 loss: 0.000637 2022/09/26 14:40:31 - mmengine - INFO - Epoch(train) [74][150/293] lr: 5.000000e-04 eta: 5:50:36 time: 0.604065 data_time: 0.160228 memory: 9504 loss_kpt: 0.000640 acc_pose: 0.778185 loss: 0.000640 2022/09/26 14:40:59 - mmengine - INFO - Epoch(train) [74][200/293] lr: 5.000000e-04 eta: 5:50:12 time: 0.553288 data_time: 0.092274 memory: 9504 loss_kpt: 0.000649 acc_pose: 0.829639 loss: 0.000649 2022/09/26 14:41:28 - mmengine - INFO - Epoch(train) [74][250/293] lr: 5.000000e-04 eta: 5:49:52 time: 0.590568 data_time: 0.085706 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.830548 loss: 0.000634 2022/09/26 14:41:54 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:42:25 - mmengine - INFO - Epoch(train) [75][50/293] lr: 5.000000e-04 eta: 5:48:30 time: 0.616518 data_time: 0.094416 memory: 9504 loss_kpt: 0.000642 acc_pose: 0.752042 loss: 0.000642 2022/09/26 14:42:55 - mmengine - INFO - Epoch(train) [75][100/293] lr: 5.000000e-04 eta: 5:48:10 time: 0.595342 data_time: 0.077484 memory: 9504 loss_kpt: 0.000655 acc_pose: 0.747749 loss: 0.000655 2022/09/26 14:43:25 - mmengine - INFO - Epoch(train) [75][150/293] lr: 5.000000e-04 eta: 5:47:50 time: 0.593732 data_time: 0.072425 memory: 9504 loss_kpt: 0.000643 acc_pose: 0.805396 loss: 0.000643 2022/09/26 14:43:54 - mmengine - INFO - Epoch(train) [75][200/293] lr: 5.000000e-04 eta: 5:47:29 time: 0.584899 data_time: 0.095338 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.793934 loss: 0.000637 2022/09/26 14:44:23 - mmengine - INFO - Epoch(train) [75][250/293] lr: 5.000000e-04 eta: 5:47:07 time: 0.575640 data_time: 0.079151 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.818261 loss: 0.000641 2022/09/26 14:44:47 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:45:04 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:45:20 - mmengine - INFO - Epoch(train) [76][50/293] lr: 5.000000e-04 eta: 5:45:48 time: 0.640632 data_time: 0.083757 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.798767 loss: 0.000641 2022/09/26 14:45:48 - mmengine - INFO - Epoch(train) [76][100/293] lr: 5.000000e-04 eta: 5:45:26 time: 0.569067 data_time: 0.076261 memory: 9504 loss_kpt: 0.000638 acc_pose: 0.886190 loss: 0.000638 2022/09/26 14:46:18 - mmengine - INFO - Epoch(train) [76][150/293] lr: 5.000000e-04 eta: 5:45:06 time: 0.597782 data_time: 0.135567 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.789924 loss: 0.000634 2022/09/26 14:46:48 - mmengine - INFO - Epoch(train) [76][200/293] lr: 5.000000e-04 eta: 5:44:46 time: 0.596750 data_time: 0.103445 memory: 9504 loss_kpt: 0.000636 acc_pose: 0.847197 loss: 0.000636 2022/09/26 14:47:17 - mmengine - INFO - Epoch(train) [76][250/293] lr: 5.000000e-04 eta: 5:44:26 time: 0.591663 data_time: 0.071577 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.813111 loss: 0.000637 2022/09/26 14:47:42 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:48:13 - mmengine - INFO - Epoch(train) [77][50/293] lr: 5.000000e-04 eta: 5:43:04 time: 0.604004 data_time: 0.088949 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.803262 loss: 0.000634 2022/09/26 14:48:42 - mmengine - INFO - Epoch(train) [77][100/293] lr: 5.000000e-04 eta: 5:42:43 time: 0.590197 data_time: 0.091670 memory: 9504 loss_kpt: 0.000644 acc_pose: 0.800721 loss: 0.000644 2022/09/26 14:49:11 - mmengine - INFO - Epoch(train) [77][150/293] lr: 5.000000e-04 eta: 5:42:22 time: 0.582515 data_time: 0.073079 memory: 9504 loss_kpt: 0.000638 acc_pose: 0.817908 loss: 0.000638 2022/09/26 14:49:41 - mmengine - INFO - Epoch(train) [77][200/293] lr: 5.000000e-04 eta: 5:42:01 time: 0.590537 data_time: 0.154627 memory: 9504 loss_kpt: 0.000627 acc_pose: 0.848673 loss: 0.000627 2022/09/26 14:50:11 - mmengine - INFO - Epoch(train) [77][250/293] lr: 5.000000e-04 eta: 5:41:42 time: 0.606790 data_time: 0.098421 memory: 9504 loss_kpt: 0.000639 acc_pose: 0.836548 loss: 0.000639 2022/09/26 14:50:37 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:51:08 - mmengine - INFO - Epoch(train) [78][50/293] lr: 5.000000e-04 eta: 5:40:23 time: 0.625465 data_time: 0.141283 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.808252 loss: 0.000641 2022/09/26 14:51:37 - mmengine - INFO - Epoch(train) [78][100/293] lr: 5.000000e-04 eta: 5:40:02 time: 0.585704 data_time: 0.134704 memory: 9504 loss_kpt: 0.000638 acc_pose: 0.849326 loss: 0.000638 2022/09/26 14:52:07 - mmengine - INFO - Epoch(train) [78][150/293] lr: 5.000000e-04 eta: 5:39:42 time: 0.598639 data_time: 0.081916 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.831310 loss: 0.000641 2022/09/26 14:52:37 - mmengine - INFO - Epoch(train) [78][200/293] lr: 5.000000e-04 eta: 5:39:22 time: 0.594633 data_time: 0.120824 memory: 9504 loss_kpt: 0.000626 acc_pose: 0.810452 loss: 0.000626 2022/09/26 14:53:06 - mmengine - INFO - Epoch(train) [78][250/293] lr: 5.000000e-04 eta: 5:39:01 time: 0.586578 data_time: 0.139678 memory: 9504 loss_kpt: 0.000640 acc_pose: 0.745579 loss: 0.000640 2022/09/26 14:53:30 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:54:00 - mmengine - INFO - Epoch(train) [79][50/293] lr: 5.000000e-04 eta: 5:37:39 time: 0.591463 data_time: 0.112570 memory: 9504 loss_kpt: 0.000647 acc_pose: 0.825876 loss: 0.000647 2022/09/26 14:54:29 - mmengine - INFO - Epoch(train) [79][100/293] lr: 5.000000e-04 eta: 5:37:18 time: 0.588454 data_time: 0.157054 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.816169 loss: 0.000631 2022/09/26 14:54:57 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:55:00 - mmengine - INFO - Epoch(train) [79][150/293] lr: 5.000000e-04 eta: 5:36:59 time: 0.601144 data_time: 0.139583 memory: 9504 loss_kpt: 0.000636 acc_pose: 0.775975 loss: 0.000636 2022/09/26 14:55:28 - mmengine - INFO - Epoch(train) [79][200/293] lr: 5.000000e-04 eta: 5:36:36 time: 0.562788 data_time: 0.099413 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.820069 loss: 0.000634 2022/09/26 14:55:57 - mmengine - INFO - Epoch(train) [79][250/293] lr: 5.000000e-04 eta: 5:36:14 time: 0.580839 data_time: 0.093038 memory: 9504 loss_kpt: 0.000629 acc_pose: 0.799905 loss: 0.000629 2022/09/26 14:56:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:56:54 - mmengine - INFO - Epoch(train) [80][50/293] lr: 5.000000e-04 eta: 5:34:59 time: 0.658815 data_time: 0.107025 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.803665 loss: 0.000641 2022/09/26 14:57:24 - mmengine - INFO - Epoch(train) [80][100/293] lr: 5.000000e-04 eta: 5:34:39 time: 0.604583 data_time: 0.080789 memory: 9504 loss_kpt: 0.000638 acc_pose: 0.834456 loss: 0.000638 2022/09/26 14:57:53 - mmengine - INFO - Epoch(train) [80][150/293] lr: 5.000000e-04 eta: 5:34:18 time: 0.589239 data_time: 0.075623 memory: 9504 loss_kpt: 0.000642 acc_pose: 0.803175 loss: 0.000642 2022/09/26 14:58:22 - mmengine - INFO - Epoch(train) [80][200/293] lr: 5.000000e-04 eta: 5:33:56 time: 0.572319 data_time: 0.077103 memory: 9504 loss_kpt: 0.000638 acc_pose: 0.861398 loss: 0.000638 2022/09/26 14:58:51 - mmengine - INFO - Epoch(train) [80][250/293] lr: 5.000000e-04 eta: 5:33:35 time: 0.586809 data_time: 0.079693 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.834156 loss: 0.000637 2022/09/26 14:59:17 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 14:59:17 - mmengine - INFO - Saving checkpoint at 80 epochs 2022/09/26 14:59:41 - mmengine - INFO - Epoch(val) [80][50/407] eta: 0:02:06 time: 0.354099 data_time: 0.064918 memory: 9504 2022/09/26 14:59:57 - mmengine - INFO - Epoch(val) [80][100/407] eta: 0:01:36 time: 0.314286 data_time: 0.044488 memory: 1378 2022/09/26 15:00:13 - mmengine - INFO - Epoch(val) [80][150/407] eta: 0:01:24 time: 0.327953 data_time: 0.053065 memory: 1378 2022/09/26 15:00:30 - mmengine - INFO - Epoch(val) [80][200/407] eta: 0:01:10 time: 0.338575 data_time: 0.053543 memory: 1378 2022/09/26 15:00:46 - mmengine - INFO - Epoch(val) [80][250/407] eta: 0:00:51 time: 0.325961 data_time: 0.034891 memory: 1378 2022/09/26 15:01:03 - mmengine - INFO - Epoch(val) [80][300/407] eta: 0:00:36 time: 0.339358 data_time: 0.054622 memory: 1378 2022/09/26 15:01:20 - mmengine - INFO - Epoch(val) [80][350/407] eta: 0:00:19 time: 0.341400 data_time: 0.080023 memory: 1378 2022/09/26 15:01:31 - mmengine - INFO - Epoch(val) [80][400/407] eta: 0:00:01 time: 0.214172 data_time: 0.020379 memory: 1378 2022/09/26 15:02:04 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 15:02:17 - mmengine - INFO - Epoch(val) [80][407/407] coco/AP: 0.705590 coco/AP .5: 0.889621 coco/AP .75: 0.782075 coco/AP (M): 0.670586 coco/AP (L): 0.772015 coco/AR: 0.761980 coco/AR .5: 0.930101 coco/AR .75: 0.830919 coco/AR (M): 0.720568 coco/AR (L): 0.821628 2022/09/26 15:02:17 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_60.pth is removed 2022/09/26 15:02:19 - mmengine - INFO - The best checkpoint with 0.7056 coco/AP at 80 epoch is saved to best_coco/AP_epoch_80.pth. 2022/09/26 15:02:48 - mmengine - INFO - Epoch(train) [81][50/293] lr: 5.000000e-04 eta: 5:32:13 time: 0.572036 data_time: 0.158710 memory: 9504 loss_kpt: 0.000640 acc_pose: 0.829614 loss: 0.000640 2022/09/26 15:03:18 - mmengine - INFO - Epoch(train) [81][100/293] lr: 5.000000e-04 eta: 5:31:54 time: 0.603232 data_time: 0.083277 memory: 9504 loss_kpt: 0.000625 acc_pose: 0.809806 loss: 0.000625 2022/09/26 15:03:46 - mmengine - INFO - Epoch(train) [81][150/293] lr: 5.000000e-04 eta: 5:31:29 time: 0.551121 data_time: 0.130984 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.848491 loss: 0.000631 2022/09/26 15:04:15 - mmengine - INFO - Epoch(train) [81][200/293] lr: 5.000000e-04 eta: 5:31:08 time: 0.580311 data_time: 0.077212 memory: 9504 loss_kpt: 0.000620 acc_pose: 0.803023 loss: 0.000620 2022/09/26 15:04:44 - mmengine - INFO - Epoch(train) [81][250/293] lr: 5.000000e-04 eta: 5:30:46 time: 0.580404 data_time: 0.083412 memory: 9504 loss_kpt: 0.000632 acc_pose: 0.818975 loss: 0.000632 2022/09/26 15:05:08 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:05:38 - mmengine - INFO - Epoch(train) [82][50/293] lr: 5.000000e-04 eta: 5:29:28 time: 0.608120 data_time: 0.098689 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.811351 loss: 0.000637 2022/09/26 15:06:07 - mmengine - INFO - Epoch(train) [82][100/293] lr: 5.000000e-04 eta: 5:29:06 time: 0.575847 data_time: 0.124383 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.814687 loss: 0.000631 2022/09/26 15:06:36 - mmengine - INFO - Epoch(train) [82][150/293] lr: 5.000000e-04 eta: 5:28:44 time: 0.577497 data_time: 0.078824 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.867883 loss: 0.000630 2022/09/26 15:07:05 - mmengine - INFO - Epoch(train) [82][200/293] lr: 5.000000e-04 eta: 5:28:22 time: 0.576609 data_time: 0.076562 memory: 9504 loss_kpt: 0.000626 acc_pose: 0.802291 loss: 0.000626 2022/09/26 15:07:33 - mmengine - INFO - Epoch(train) [82][250/293] lr: 5.000000e-04 eta: 5:27:59 time: 0.564605 data_time: 0.078703 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.791324 loss: 0.000612 2022/09/26 15:07:43 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:07:58 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:08:28 - mmengine - INFO - Epoch(train) [83][50/293] lr: 5.000000e-04 eta: 5:26:41 time: 0.604784 data_time: 0.101108 memory: 9504 loss_kpt: 0.000629 acc_pose: 0.812076 loss: 0.000629 2022/09/26 15:08:58 - mmengine - INFO - Epoch(train) [83][100/293] lr: 5.000000e-04 eta: 5:26:20 time: 0.588695 data_time: 0.081306 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.834045 loss: 0.000637 2022/09/26 15:09:26 - mmengine - INFO - Epoch(train) [83][150/293] lr: 5.000000e-04 eta: 5:25:57 time: 0.562328 data_time: 0.075207 memory: 9504 loss_kpt: 0.000633 acc_pose: 0.833878 loss: 0.000633 2022/09/26 15:09:55 - mmengine - INFO - Epoch(train) [83][200/293] lr: 5.000000e-04 eta: 5:25:36 time: 0.584670 data_time: 0.078758 memory: 9504 loss_kpt: 0.000635 acc_pose: 0.837602 loss: 0.000635 2022/09/26 15:10:25 - mmengine - INFO - Epoch(train) [83][250/293] lr: 5.000000e-04 eta: 5:25:15 time: 0.597818 data_time: 0.093865 memory: 9504 loss_kpt: 0.000636 acc_pose: 0.758136 loss: 0.000636 2022/09/26 15:10:50 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:11:20 - mmengine - INFO - Epoch(train) [84][50/293] lr: 5.000000e-04 eta: 5:23:57 time: 0.589869 data_time: 0.098663 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.819424 loss: 0.000630 2022/09/26 15:11:51 - mmengine - INFO - Epoch(train) [84][100/293] lr: 5.000000e-04 eta: 5:23:38 time: 0.614415 data_time: 0.109310 memory: 9504 loss_kpt: 0.000627 acc_pose: 0.814605 loss: 0.000627 2022/09/26 15:12:20 - mmengine - INFO - Epoch(train) [84][150/293] lr: 5.000000e-04 eta: 5:23:16 time: 0.580845 data_time: 0.093036 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.835082 loss: 0.000630 2022/09/26 15:12:49 - mmengine - INFO - Epoch(train) [84][200/293] lr: 5.000000e-04 eta: 5:22:54 time: 0.582309 data_time: 0.090160 memory: 9504 loss_kpt: 0.000625 acc_pose: 0.859814 loss: 0.000625 2022/09/26 15:13:19 - mmengine - INFO - Epoch(train) [84][250/293] lr: 5.000000e-04 eta: 5:22:34 time: 0.605970 data_time: 0.077342 memory: 9504 loss_kpt: 0.000638 acc_pose: 0.791775 loss: 0.000638 2022/09/26 15:13:44 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:14:14 - mmengine - INFO - Epoch(train) [85][50/293] lr: 5.000000e-04 eta: 5:21:18 time: 0.605754 data_time: 0.109360 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.816001 loss: 0.000631 2022/09/26 15:14:43 - mmengine - INFO - Epoch(train) [85][100/293] lr: 5.000000e-04 eta: 5:20:56 time: 0.573418 data_time: 0.095116 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.781831 loss: 0.000634 2022/09/26 15:15:12 - mmengine - INFO - Epoch(train) [85][150/293] lr: 5.000000e-04 eta: 5:20:35 time: 0.592393 data_time: 0.087648 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.805594 loss: 0.000619 2022/09/26 15:15:41 - mmengine - INFO - Epoch(train) [85][200/293] lr: 5.000000e-04 eta: 5:20:13 time: 0.583020 data_time: 0.081710 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.796300 loss: 0.000630 2022/09/26 15:16:11 - mmengine - INFO - Epoch(train) [85][250/293] lr: 5.000000e-04 eta: 5:19:52 time: 0.588420 data_time: 0.095032 memory: 9504 loss_kpt: 0.000641 acc_pose: 0.841710 loss: 0.000641 2022/09/26 15:16:36 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:17:05 - mmengine - INFO - Epoch(train) [86][50/293] lr: 5.000000e-04 eta: 5:18:35 time: 0.592103 data_time: 0.095426 memory: 9504 loss_kpt: 0.000636 acc_pose: 0.777845 loss: 0.000636 2022/09/26 15:17:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:17:37 - mmengine - INFO - Epoch(train) [86][100/293] lr: 5.000000e-04 eta: 5:18:16 time: 0.622796 data_time: 0.082983 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.828001 loss: 0.000631 2022/09/26 15:18:06 - mmengine - INFO - Epoch(train) [86][150/293] lr: 5.000000e-04 eta: 5:17:55 time: 0.595093 data_time: 0.081071 memory: 9504 loss_kpt: 0.000627 acc_pose: 0.769120 loss: 0.000627 2022/09/26 15:18:36 - mmengine - INFO - Epoch(train) [86][200/293] lr: 5.000000e-04 eta: 5:17:34 time: 0.587371 data_time: 0.071298 memory: 9504 loss_kpt: 0.000611 acc_pose: 0.783058 loss: 0.000611 2022/09/26 15:19:06 - mmengine - INFO - Epoch(train) [86][250/293] lr: 5.000000e-04 eta: 5:17:13 time: 0.598857 data_time: 0.089317 memory: 9504 loss_kpt: 0.000629 acc_pose: 0.823415 loss: 0.000629 2022/09/26 15:19:31 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:20:01 - mmengine - INFO - Epoch(train) [87][50/293] lr: 5.000000e-04 eta: 5:15:57 time: 0.595527 data_time: 0.104171 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.830027 loss: 0.000619 2022/09/26 15:20:30 - mmengine - INFO - Epoch(train) [87][100/293] lr: 5.000000e-04 eta: 5:15:36 time: 0.587461 data_time: 0.085835 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.776561 loss: 0.000631 2022/09/26 15:21:02 - mmengine - INFO - Epoch(train) [87][150/293] lr: 5.000000e-04 eta: 5:15:18 time: 0.630309 data_time: 0.083730 memory: 9504 loss_kpt: 0.000622 acc_pose: 0.842251 loss: 0.000622 2022/09/26 15:21:32 - mmengine - INFO - Epoch(train) [87][200/293] lr: 5.000000e-04 eta: 5:14:57 time: 0.605610 data_time: 0.123114 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.838651 loss: 0.000634 2022/09/26 15:22:03 - mmengine - INFO - Epoch(train) [87][250/293] lr: 5.000000e-04 eta: 5:14:37 time: 0.611565 data_time: 0.071559 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.812106 loss: 0.000621 2022/09/26 15:22:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:22:59 - mmengine - INFO - Epoch(train) [88][50/293] lr: 5.000000e-04 eta: 5:13:22 time: 0.597950 data_time: 0.109496 memory: 9504 loss_kpt: 0.000622 acc_pose: 0.791919 loss: 0.000622 2022/09/26 15:23:28 - mmengine - INFO - Epoch(train) [88][100/293] lr: 5.000000e-04 eta: 5:13:01 time: 0.592274 data_time: 0.080415 memory: 9504 loss_kpt: 0.000623 acc_pose: 0.825245 loss: 0.000623 2022/09/26 15:23:57 - mmengine - INFO - Epoch(train) [88][150/293] lr: 5.000000e-04 eta: 5:12:39 time: 0.578640 data_time: 0.131895 memory: 9504 loss_kpt: 0.000611 acc_pose: 0.773848 loss: 0.000611 2022/09/26 15:24:27 - mmengine - INFO - Epoch(train) [88][200/293] lr: 5.000000e-04 eta: 5:12:17 time: 0.582773 data_time: 0.074076 memory: 9504 loss_kpt: 0.000620 acc_pose: 0.858248 loss: 0.000620 2022/09/26 15:24:56 - mmengine - INFO - Epoch(train) [88][250/293] lr: 5.000000e-04 eta: 5:11:55 time: 0.581516 data_time: 0.074539 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.819178 loss: 0.000621 2022/09/26 15:25:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:25:53 - mmengine - INFO - Epoch(train) [89][50/293] lr: 5.000000e-04 eta: 5:10:42 time: 0.628911 data_time: 0.193929 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.827451 loss: 0.000634 2022/09/26 15:26:22 - mmengine - INFO - Epoch(train) [89][100/293] lr: 5.000000e-04 eta: 5:10:21 time: 0.586969 data_time: 0.089465 memory: 9504 loss_kpt: 0.000633 acc_pose: 0.815086 loss: 0.000633 2022/09/26 15:26:52 - mmengine - INFO - Epoch(train) [89][150/293] lr: 5.000000e-04 eta: 5:10:00 time: 0.596526 data_time: 0.086979 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.858554 loss: 0.000631 2022/09/26 15:27:21 - mmengine - INFO - Epoch(train) [89][200/293] lr: 5.000000e-04 eta: 5:09:38 time: 0.584686 data_time: 0.079832 memory: 9504 loss_kpt: 0.000628 acc_pose: 0.826366 loss: 0.000628 2022/09/26 15:27:31 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:27:51 - mmengine - INFO - Epoch(train) [89][250/293] lr: 5.000000e-04 eta: 5:09:17 time: 0.597461 data_time: 0.083232 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.830603 loss: 0.000615 2022/09/26 15:28:16 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:28:47 - mmengine - INFO - Epoch(train) [90][50/293] lr: 5.000000e-04 eta: 5:08:04 time: 0.615780 data_time: 0.139014 memory: 9504 loss_kpt: 0.000631 acc_pose: 0.836668 loss: 0.000631 2022/09/26 15:29:16 - mmengine - INFO - Epoch(train) [90][100/293] lr: 5.000000e-04 eta: 5:07:42 time: 0.577839 data_time: 0.082550 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.804434 loss: 0.000637 2022/09/26 15:29:46 - mmengine - INFO - Epoch(train) [90][150/293] lr: 5.000000e-04 eta: 5:07:21 time: 0.600662 data_time: 0.080362 memory: 9504 loss_kpt: 0.000624 acc_pose: 0.843190 loss: 0.000624 2022/09/26 15:30:15 - mmengine - INFO - Epoch(train) [90][200/293] lr: 5.000000e-04 eta: 5:06:59 time: 0.586258 data_time: 0.079793 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.849519 loss: 0.000617 2022/09/26 15:30:44 - mmengine - INFO - Epoch(train) [90][250/293] lr: 5.000000e-04 eta: 5:06:37 time: 0.578069 data_time: 0.071997 memory: 9504 loss_kpt: 0.000627 acc_pose: 0.794901 loss: 0.000627 2022/09/26 15:31:08 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:31:08 - mmengine - INFO - Saving checkpoint at 90 epochs 2022/09/26 15:31:32 - mmengine - INFO - Epoch(val) [90][50/407] eta: 0:02:01 time: 0.339518 data_time: 0.074938 memory: 9504 2022/09/26 15:31:49 - mmengine - INFO - Epoch(val) [90][100/407] eta: 0:01:44 time: 0.340184 data_time: 0.067332 memory: 1378 2022/09/26 15:32:05 - mmengine - INFO - Epoch(val) [90][150/407] eta: 0:01:22 time: 0.319548 data_time: 0.047942 memory: 1378 2022/09/26 15:32:21 - mmengine - INFO - Epoch(val) [90][200/407] eta: 0:01:05 time: 0.318156 data_time: 0.061894 memory: 1378 2022/09/26 15:32:37 - mmengine - INFO - Epoch(val) [90][250/407] eta: 0:00:50 time: 0.324655 data_time: 0.040632 memory: 1378 2022/09/26 15:32:53 - mmengine - INFO - Epoch(val) [90][300/407] eta: 0:00:33 time: 0.314086 data_time: 0.040973 memory: 1378 2022/09/26 15:33:11 - mmengine - INFO - Epoch(val) [90][350/407] eta: 0:00:20 time: 0.359290 data_time: 0.093506 memory: 1378 2022/09/26 15:33:24 - mmengine - INFO - Epoch(val) [90][400/407] eta: 0:00:01 time: 0.257844 data_time: 0.046045 memory: 1378 2022/09/26 15:33:57 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 15:34:10 - mmengine - INFO - Epoch(val) [90][407/407] coco/AP: 0.702739 coco/AP .5: 0.885621 coco/AP .75: 0.778588 coco/AP (M): 0.669089 coco/AP (L): 0.768966 coco/AR: 0.761996 coco/AR .5: 0.928684 coco/AR .75: 0.829503 coco/AR (M): 0.720486 coco/AR (L): 0.821739 2022/09/26 15:34:41 - mmengine - INFO - Epoch(train) [91][50/293] lr: 5.000000e-04 eta: 5:05:24 time: 0.611735 data_time: 0.148674 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.778563 loss: 0.000637 2022/09/26 15:35:11 - mmengine - INFO - Epoch(train) [91][100/293] lr: 5.000000e-04 eta: 5:05:04 time: 0.606786 data_time: 0.094964 memory: 9504 loss_kpt: 0.000625 acc_pose: 0.813149 loss: 0.000625 2022/09/26 15:35:41 - mmengine - INFO - Epoch(train) [91][150/293] lr: 5.000000e-04 eta: 5:04:42 time: 0.595979 data_time: 0.089744 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.832353 loss: 0.000621 2022/09/26 15:36:10 - mmengine - INFO - Epoch(train) [91][200/293] lr: 5.000000e-04 eta: 5:04:20 time: 0.585110 data_time: 0.105724 memory: 9504 loss_kpt: 0.000628 acc_pose: 0.800565 loss: 0.000628 2022/09/26 15:36:39 - mmengine - INFO - Epoch(train) [91][250/293] lr: 5.000000e-04 eta: 5:03:58 time: 0.571829 data_time: 0.075387 memory: 9504 loss_kpt: 0.000632 acc_pose: 0.849169 loss: 0.000632 2022/09/26 15:37:03 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:37:34 - mmengine - INFO - Epoch(train) [92][50/293] lr: 5.000000e-04 eta: 5:02:46 time: 0.620993 data_time: 0.095410 memory: 9504 loss_kpt: 0.000625 acc_pose: 0.854329 loss: 0.000625 2022/09/26 15:38:02 - mmengine - INFO - Epoch(train) [92][100/293] lr: 5.000000e-04 eta: 5:02:23 time: 0.565220 data_time: 0.091031 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.788207 loss: 0.000634 2022/09/26 15:38:31 - mmengine - INFO - Epoch(train) [92][150/293] lr: 5.000000e-04 eta: 5:02:00 time: 0.566960 data_time: 0.107013 memory: 9504 loss_kpt: 0.000623 acc_pose: 0.813929 loss: 0.000623 2022/09/26 15:38:59 - mmengine - INFO - Epoch(train) [92][200/293] lr: 5.000000e-04 eta: 5:01:37 time: 0.571055 data_time: 0.071480 memory: 9504 loss_kpt: 0.000628 acc_pose: 0.872671 loss: 0.000628 2022/09/26 15:39:28 - mmengine - INFO - Epoch(train) [92][250/293] lr: 5.000000e-04 eta: 5:01:14 time: 0.573933 data_time: 0.093960 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.851197 loss: 0.000630 2022/09/26 15:39:53 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:40:20 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:40:24 - mmengine - INFO - Epoch(train) [93][50/293] lr: 5.000000e-04 eta: 5:00:03 time: 0.615929 data_time: 0.107241 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.812486 loss: 0.000621 2022/09/26 15:40:53 - mmengine - INFO - Epoch(train) [93][100/293] lr: 5.000000e-04 eta: 4:59:41 time: 0.586092 data_time: 0.072664 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.824487 loss: 0.000615 2022/09/26 15:41:22 - mmengine - INFO - Epoch(train) [93][150/293] lr: 5.000000e-04 eta: 4:59:19 time: 0.584794 data_time: 0.101391 memory: 9504 loss_kpt: 0.000635 acc_pose: 0.808936 loss: 0.000635 2022/09/26 15:41:51 - mmengine - INFO - Epoch(train) [93][200/293] lr: 5.000000e-04 eta: 4:58:56 time: 0.572548 data_time: 0.073067 memory: 9504 loss_kpt: 0.000618 acc_pose: 0.757026 loss: 0.000618 2022/09/26 15:42:21 - mmengine - INFO - Epoch(train) [93][250/293] lr: 5.000000e-04 eta: 4:58:34 time: 0.589208 data_time: 0.082661 memory: 9504 loss_kpt: 0.000636 acc_pose: 0.834801 loss: 0.000636 2022/09/26 15:42:45 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:43:15 - mmengine - INFO - Epoch(train) [94][50/293] lr: 5.000000e-04 eta: 4:57:22 time: 0.590707 data_time: 0.103702 memory: 9504 loss_kpt: 0.000616 acc_pose: 0.850594 loss: 0.000616 2022/09/26 15:43:44 - mmengine - INFO - Epoch(train) [94][100/293] lr: 5.000000e-04 eta: 4:57:00 time: 0.587935 data_time: 0.088871 memory: 9504 loss_kpt: 0.000627 acc_pose: 0.805167 loss: 0.000627 2022/09/26 15:44:14 - mmengine - INFO - Epoch(train) [94][150/293] lr: 5.000000e-04 eta: 4:56:38 time: 0.588066 data_time: 0.092356 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.881805 loss: 0.000619 2022/09/26 15:44:43 - mmengine - INFO - Epoch(train) [94][200/293] lr: 5.000000e-04 eta: 4:56:15 time: 0.578496 data_time: 0.089494 memory: 9504 loss_kpt: 0.000624 acc_pose: 0.855569 loss: 0.000624 2022/09/26 15:45:11 - mmengine - INFO - Epoch(train) [94][250/293] lr: 5.000000e-04 eta: 4:55:52 time: 0.566853 data_time: 0.071355 memory: 9504 loss_kpt: 0.000613 acc_pose: 0.872778 loss: 0.000613 2022/09/26 15:45:36 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:46:05 - mmengine - INFO - Epoch(train) [95][50/293] lr: 5.000000e-04 eta: 4:54:40 time: 0.589885 data_time: 0.146077 memory: 9504 loss_kpt: 0.000626 acc_pose: 0.847275 loss: 0.000626 2022/09/26 15:46:34 - mmengine - INFO - Epoch(train) [95][100/293] lr: 5.000000e-04 eta: 4:54:17 time: 0.569621 data_time: 0.125331 memory: 9504 loss_kpt: 0.000611 acc_pose: 0.829966 loss: 0.000611 2022/09/26 15:47:03 - mmengine - INFO - Epoch(train) [95][150/293] lr: 5.000000e-04 eta: 4:53:54 time: 0.577226 data_time: 0.096663 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.823101 loss: 0.000637 2022/09/26 15:47:32 - mmengine - INFO - Epoch(train) [95][200/293] lr: 5.000000e-04 eta: 4:53:32 time: 0.581589 data_time: 0.085038 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.827235 loss: 0.000619 2022/09/26 15:48:02 - mmengine - INFO - Epoch(train) [95][250/293] lr: 5.000000e-04 eta: 4:53:11 time: 0.596795 data_time: 0.103122 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.801803 loss: 0.000610 2022/09/26 15:48:27 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:48:57 - mmengine - INFO - Epoch(train) [96][50/293] lr: 5.000000e-04 eta: 4:52:00 time: 0.609834 data_time: 0.099309 memory: 9504 loss_kpt: 0.000630 acc_pose: 0.825780 loss: 0.000630 2022/09/26 15:49:26 - mmengine - INFO - Epoch(train) [96][100/293] lr: 5.000000e-04 eta: 4:51:38 time: 0.579884 data_time: 0.079988 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.851255 loss: 0.000609 2022/09/26 15:49:55 - mmengine - INFO - Epoch(train) [96][150/293] lr: 5.000000e-04 eta: 4:51:16 time: 0.588584 data_time: 0.085567 memory: 9504 loss_kpt: 0.000628 acc_pose: 0.807944 loss: 0.000628 2022/09/26 15:50:04 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:50:25 - mmengine - INFO - Epoch(train) [96][200/293] lr: 5.000000e-04 eta: 4:50:54 time: 0.594086 data_time: 0.084533 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.822480 loss: 0.000617 2022/09/26 15:50:56 - mmengine - INFO - Epoch(train) [96][250/293] lr: 5.000000e-04 eta: 4:50:34 time: 0.612420 data_time: 0.085629 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.873452 loss: 0.000619 2022/09/26 15:51:20 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:51:49 - mmengine - INFO - Epoch(train) [97][50/293] lr: 5.000000e-04 eta: 4:49:22 time: 0.585335 data_time: 0.154767 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.834906 loss: 0.000609 2022/09/26 15:52:20 - mmengine - INFO - Epoch(train) [97][100/293] lr: 5.000000e-04 eta: 4:49:01 time: 0.603231 data_time: 0.115470 memory: 9504 loss_kpt: 0.000616 acc_pose: 0.843587 loss: 0.000616 2022/09/26 15:52:48 - mmengine - INFO - Epoch(train) [97][150/293] lr: 5.000000e-04 eta: 4:48:38 time: 0.576429 data_time: 0.078676 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.829324 loss: 0.000612 2022/09/26 15:53:18 - mmengine - INFO - Epoch(train) [97][200/293] lr: 5.000000e-04 eta: 4:48:16 time: 0.584485 data_time: 0.086677 memory: 9504 loss_kpt: 0.000614 acc_pose: 0.774605 loss: 0.000614 2022/09/26 15:53:47 - mmengine - INFO - Epoch(train) [97][250/293] lr: 5.000000e-04 eta: 4:47:54 time: 0.588221 data_time: 0.086279 memory: 9504 loss_kpt: 0.000613 acc_pose: 0.828138 loss: 0.000613 2022/09/26 15:54:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:54:41 - mmengine - INFO - Epoch(train) [98][50/293] lr: 5.000000e-04 eta: 4:46:44 time: 0.597046 data_time: 0.090901 memory: 9504 loss_kpt: 0.000614 acc_pose: 0.826436 loss: 0.000614 2022/09/26 15:55:10 - mmengine - INFO - Epoch(train) [98][100/293] lr: 5.000000e-04 eta: 4:46:21 time: 0.577584 data_time: 0.083546 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.821391 loss: 0.000619 2022/09/26 15:55:39 - mmengine - INFO - Epoch(train) [98][150/293] lr: 5.000000e-04 eta: 4:45:58 time: 0.566550 data_time: 0.080996 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.858217 loss: 0.000617 2022/09/26 15:56:08 - mmengine - INFO - Epoch(train) [98][200/293] lr: 5.000000e-04 eta: 4:45:36 time: 0.588379 data_time: 0.137017 memory: 9504 loss_kpt: 0.000619 acc_pose: 0.835581 loss: 0.000619 2022/09/26 15:56:36 - mmengine - INFO - Epoch(train) [98][250/293] lr: 5.000000e-04 eta: 4:45:12 time: 0.557848 data_time: 0.120759 memory: 9504 loss_kpt: 0.000618 acc_pose: 0.851815 loss: 0.000618 2022/09/26 15:57:00 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:57:31 - mmengine - INFO - Epoch(train) [99][50/293] lr: 5.000000e-04 eta: 4:44:04 time: 0.625246 data_time: 0.098160 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.798207 loss: 0.000617 2022/09/26 15:58:00 - mmengine - INFO - Epoch(train) [99][100/293] lr: 5.000000e-04 eta: 4:43:41 time: 0.569246 data_time: 0.083483 memory: 9504 loss_kpt: 0.000616 acc_pose: 0.806522 loss: 0.000616 2022/09/26 15:58:28 - mmengine - INFO - Epoch(train) [99][150/293] lr: 5.000000e-04 eta: 4:43:18 time: 0.573759 data_time: 0.079983 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.834542 loss: 0.000615 2022/09/26 15:58:58 - mmengine - INFO - Epoch(train) [99][200/293] lr: 5.000000e-04 eta: 4:42:55 time: 0.588313 data_time: 0.090584 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.839582 loss: 0.000607 2022/09/26 15:59:27 - mmengine - INFO - Epoch(train) [99][250/293] lr: 5.000000e-04 eta: 4:42:33 time: 0.581779 data_time: 0.101248 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.791152 loss: 0.000605 2022/09/26 15:59:47 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 15:59:51 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:00:20 - mmengine - INFO - Epoch(train) [100][50/293] lr: 5.000000e-04 eta: 4:41:23 time: 0.584693 data_time: 0.105765 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.842253 loss: 0.000605 2022/09/26 16:00:50 - mmengine - INFO - Epoch(train) [100][100/293] lr: 5.000000e-04 eta: 4:41:01 time: 0.587537 data_time: 0.097035 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.791020 loss: 0.000617 2022/09/26 16:01:18 - mmengine - INFO - Epoch(train) [100][150/293] lr: 5.000000e-04 eta: 4:40:38 time: 0.573842 data_time: 0.086233 memory: 9504 loss_kpt: 0.000614 acc_pose: 0.808766 loss: 0.000614 2022/09/26 16:01:46 - mmengine - INFO - Epoch(train) [100][200/293] lr: 5.000000e-04 eta: 4:40:13 time: 0.548226 data_time: 0.089556 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.841010 loss: 0.000615 2022/09/26 16:02:16 - mmengine - INFO - Epoch(train) [100][250/293] lr: 5.000000e-04 eta: 4:39:52 time: 0.599142 data_time: 0.086519 memory: 9504 loss_kpt: 0.000620 acc_pose: 0.830457 loss: 0.000620 2022/09/26 16:02:41 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:02:41 - mmengine - INFO - Saving checkpoint at 100 epochs 2022/09/26 16:03:04 - mmengine - INFO - Epoch(val) [100][50/407] eta: 0:02:06 time: 0.354310 data_time: 0.077672 memory: 9504 2022/09/26 16:03:20 - mmengine - INFO - Epoch(val) [100][100/407] eta: 0:01:37 time: 0.318308 data_time: 0.050049 memory: 1378 2022/09/26 16:03:36 - mmengine - INFO - Epoch(val) [100][150/407] eta: 0:01:20 time: 0.311600 data_time: 0.043656 memory: 1378 2022/09/26 16:03:53 - mmengine - INFO - Epoch(val) [100][200/407] eta: 0:01:09 time: 0.334306 data_time: 0.055250 memory: 1378 2022/09/26 16:04:09 - mmengine - INFO - Epoch(val) [100][250/407] eta: 0:00:51 time: 0.331168 data_time: 0.076911 memory: 1378 2022/09/26 16:04:26 - mmengine - INFO - Epoch(val) [100][300/407] eta: 0:00:36 time: 0.338251 data_time: 0.067868 memory: 1378 2022/09/26 16:04:43 - mmengine - INFO - Epoch(val) [100][350/407] eta: 0:00:19 time: 0.338281 data_time: 0.086620 memory: 1378 2022/09/26 16:04:59 - mmengine - INFO - Epoch(val) [100][400/407] eta: 0:00:02 time: 0.311876 data_time: 0.127146 memory: 1378 2022/09/26 16:05:33 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 16:05:46 - mmengine - INFO - Epoch(val) [100][407/407] coco/AP: 0.713538 coco/AP .5: 0.895050 coco/AP .75: 0.791095 coco/AP (M): 0.675623 coco/AP (L): 0.780797 coco/AR: 0.768750 coco/AR .5: 0.932935 coco/AR .75: 0.838004 coco/AR (M): 0.725184 coco/AR (L): 0.831773 2022/09/26 16:05:46 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_80.pth is removed 2022/09/26 16:05:48 - mmengine - INFO - The best checkpoint with 0.7135 coco/AP at 100 epoch is saved to best_coco/AP_epoch_100.pth. 2022/09/26 16:08:27 - mmengine - INFO - Epoch(train) [101][50/293] lr: 5.000000e-04 eta: 4:41:04 time: 3.170442 data_time: 0.215756 memory: 9504 loss_kpt: 0.000613 acc_pose: 0.834729 loss: 0.000613 2022/09/26 16:09:07 - mmengine - INFO - Epoch(train) [101][100/293] lr: 5.000000e-04 eta: 4:40:53 time: 0.809160 data_time: 0.093751 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.836765 loss: 0.000606 2022/09/26 16:09:49 - mmengine - INFO - Epoch(train) [101][150/293] lr: 5.000000e-04 eta: 4:40:44 time: 0.827472 data_time: 0.086081 memory: 9504 loss_kpt: 0.000613 acc_pose: 0.877982 loss: 0.000613 2022/09/26 16:10:23 - mmengine - INFO - Epoch(train) [101][200/293] lr: 5.000000e-04 eta: 4:40:26 time: 0.687016 data_time: 0.085421 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.794828 loss: 0.000617 2022/09/26 16:10:53 - mmengine - INFO - Epoch(train) [101][250/293] lr: 5.000000e-04 eta: 4:40:04 time: 0.604526 data_time: 0.077482 memory: 9504 loss_kpt: 0.000608 acc_pose: 0.801852 loss: 0.000608 2022/09/26 16:11:19 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:11:49 - mmengine - INFO - Epoch(train) [102][50/293] lr: 5.000000e-04 eta: 4:38:54 time: 0.588691 data_time: 0.086116 memory: 9504 loss_kpt: 0.000626 acc_pose: 0.838984 loss: 0.000626 2022/09/26 16:12:18 - mmengine - INFO - Epoch(train) [102][100/293] lr: 5.000000e-04 eta: 4:38:31 time: 0.576547 data_time: 0.193287 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.760249 loss: 0.000621 2022/09/26 16:12:46 - mmengine - INFO - Epoch(train) [102][150/293] lr: 5.000000e-04 eta: 4:38:08 time: 0.576470 data_time: 0.099373 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.835096 loss: 0.000607 2022/09/26 16:13:16 - mmengine - INFO - Epoch(train) [102][200/293] lr: 5.000000e-04 eta: 4:37:45 time: 0.592495 data_time: 0.092809 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.833019 loss: 0.000612 2022/09/26 16:13:45 - mmengine - INFO - Epoch(train) [102][250/293] lr: 5.000000e-04 eta: 4:37:22 time: 0.588194 data_time: 0.106766 memory: 9504 loss_kpt: 0.000629 acc_pose: 0.802438 loss: 0.000629 2022/09/26 16:14:10 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:14:39 - mmengine - INFO - Epoch(train) [103][50/293] lr: 5.000000e-04 eta: 4:36:12 time: 0.585784 data_time: 0.087298 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.842454 loss: 0.000610 2022/09/26 16:15:08 - mmengine - INFO - Epoch(train) [103][100/293] lr: 5.000000e-04 eta: 4:35:48 time: 0.567163 data_time: 0.077716 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.813232 loss: 0.000610 2022/09/26 16:15:16 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:15:37 - mmengine - INFO - Epoch(train) [103][150/293] lr: 5.000000e-04 eta: 4:35:26 time: 0.595704 data_time: 0.080242 memory: 9504 loss_kpt: 0.000616 acc_pose: 0.823383 loss: 0.000616 2022/09/26 16:16:07 - mmengine - INFO - Epoch(train) [103][200/293] lr: 5.000000e-04 eta: 4:35:03 time: 0.583621 data_time: 0.082456 memory: 9504 loss_kpt: 0.000616 acc_pose: 0.765164 loss: 0.000616 2022/09/26 16:16:37 - mmengine - INFO - Epoch(train) [103][250/293] lr: 5.000000e-04 eta: 4:34:41 time: 0.603944 data_time: 0.089189 memory: 9504 loss_kpt: 0.000637 acc_pose: 0.836017 loss: 0.000637 2022/09/26 16:17:00 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:17:30 - mmengine - INFO - Epoch(train) [104][50/293] lr: 5.000000e-04 eta: 4:33:32 time: 0.590394 data_time: 0.095580 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.832532 loss: 0.000607 2022/09/26 16:17:59 - mmengine - INFO - Epoch(train) [104][100/293] lr: 5.000000e-04 eta: 4:33:09 time: 0.587001 data_time: 0.094811 memory: 9504 loss_kpt: 0.000614 acc_pose: 0.834189 loss: 0.000614 2022/09/26 16:18:28 - mmengine - INFO - Epoch(train) [104][150/293] lr: 5.000000e-04 eta: 4:32:46 time: 0.584646 data_time: 0.071201 memory: 9504 loss_kpt: 0.000599 acc_pose: 0.844263 loss: 0.000599 2022/09/26 16:18:58 - mmengine - INFO - Epoch(train) [104][200/293] lr: 5.000000e-04 eta: 4:32:23 time: 0.596116 data_time: 0.084694 memory: 9504 loss_kpt: 0.000616 acc_pose: 0.849670 loss: 0.000616 2022/09/26 16:19:27 - mmengine - INFO - Epoch(train) [104][250/293] lr: 5.000000e-04 eta: 4:32:00 time: 0.575594 data_time: 0.092482 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.798961 loss: 0.000609 2022/09/26 16:19:52 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:20:22 - mmengine - INFO - Epoch(train) [105][50/293] lr: 5.000000e-04 eta: 4:30:52 time: 0.598689 data_time: 0.128759 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.823764 loss: 0.000610 2022/09/26 16:20:51 - mmengine - INFO - Epoch(train) [105][100/293] lr: 5.000000e-04 eta: 4:30:28 time: 0.576550 data_time: 0.092031 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.845884 loss: 0.000609 2022/09/26 16:21:20 - mmengine - INFO - Epoch(train) [105][150/293] lr: 5.000000e-04 eta: 4:30:05 time: 0.585262 data_time: 0.082233 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.855721 loss: 0.000609 2022/09/26 16:21:49 - mmengine - INFO - Epoch(train) [105][200/293] lr: 5.000000e-04 eta: 4:29:41 time: 0.579023 data_time: 0.085151 memory: 9504 loss_kpt: 0.000611 acc_pose: 0.841784 loss: 0.000611 2022/09/26 16:22:18 - mmengine - INFO - Epoch(train) [105][250/293] lr: 5.000000e-04 eta: 4:29:18 time: 0.574382 data_time: 0.094016 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.807157 loss: 0.000610 2022/09/26 16:22:43 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:23:12 - mmengine - INFO - Epoch(train) [106][50/293] lr: 5.000000e-04 eta: 4:28:10 time: 0.596814 data_time: 0.097195 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.840973 loss: 0.000606 2022/09/26 16:23:42 - mmengine - INFO - Epoch(train) [106][100/293] lr: 5.000000e-04 eta: 4:27:47 time: 0.585558 data_time: 0.087154 memory: 9504 loss_kpt: 0.000618 acc_pose: 0.835237 loss: 0.000618 2022/09/26 16:24:11 - mmengine - INFO - Epoch(train) [106][150/293] lr: 5.000000e-04 eta: 4:27:24 time: 0.592536 data_time: 0.079967 memory: 9504 loss_kpt: 0.000634 acc_pose: 0.843656 loss: 0.000634 2022/09/26 16:24:39 - mmengine - INFO - Epoch(train) [106][200/293] lr: 5.000000e-04 eta: 4:27:00 time: 0.560682 data_time: 0.081848 memory: 9504 loss_kpt: 0.000620 acc_pose: 0.823796 loss: 0.000620 2022/09/26 16:24:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:25:08 - mmengine - INFO - Epoch(train) [106][250/293] lr: 5.000000e-04 eta: 4:26:36 time: 0.572055 data_time: 0.074914 memory: 9504 loss_kpt: 0.000593 acc_pose: 0.800612 loss: 0.000593 2022/09/26 16:25:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:26:04 - mmengine - INFO - Epoch(train) [107][50/293] lr: 5.000000e-04 eta: 4:25:29 time: 0.612610 data_time: 0.088201 memory: 9504 loss_kpt: 0.000599 acc_pose: 0.825669 loss: 0.000599 2022/09/26 16:26:33 - mmengine - INFO - Epoch(train) [107][100/293] lr: 5.000000e-04 eta: 4:25:06 time: 0.588251 data_time: 0.092926 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.818723 loss: 0.000615 2022/09/26 16:27:02 - mmengine - INFO - Epoch(train) [107][150/293] lr: 5.000000e-04 eta: 4:24:43 time: 0.573031 data_time: 0.085441 memory: 9504 loss_kpt: 0.000603 acc_pose: 0.833439 loss: 0.000603 2022/09/26 16:27:31 - mmengine - INFO - Epoch(train) [107][200/293] lr: 5.000000e-04 eta: 4:24:19 time: 0.578898 data_time: 0.074959 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.849326 loss: 0.000615 2022/09/26 16:27:59 - mmengine - INFO - Epoch(train) [107][250/293] lr: 5.000000e-04 eta: 4:23:55 time: 0.566881 data_time: 0.076993 memory: 9504 loss_kpt: 0.000602 acc_pose: 0.820560 loss: 0.000602 2022/09/26 16:28:24 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:28:54 - mmengine - INFO - Epoch(train) [108][50/293] lr: 5.000000e-04 eta: 4:22:49 time: 0.611956 data_time: 0.101570 memory: 9504 loss_kpt: 0.000597 acc_pose: 0.820842 loss: 0.000597 2022/09/26 16:29:24 - mmengine - INFO - Epoch(train) [108][100/293] lr: 5.000000e-04 eta: 4:22:26 time: 0.584813 data_time: 0.100038 memory: 9504 loss_kpt: 0.000599 acc_pose: 0.861639 loss: 0.000599 2022/09/26 16:29:53 - mmengine - INFO - Epoch(train) [108][150/293] lr: 5.000000e-04 eta: 4:22:02 time: 0.586222 data_time: 0.103067 memory: 9504 loss_kpt: 0.000623 acc_pose: 0.819618 loss: 0.000623 2022/09/26 16:30:22 - mmengine - INFO - Epoch(train) [108][200/293] lr: 5.000000e-04 eta: 4:21:39 time: 0.586512 data_time: 0.093083 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.811545 loss: 0.000605 2022/09/26 16:30:51 - mmengine - INFO - Epoch(train) [108][250/293] lr: 5.000000e-04 eta: 4:21:15 time: 0.572393 data_time: 0.092543 memory: 9504 loss_kpt: 0.000603 acc_pose: 0.795735 loss: 0.000603 2022/09/26 16:31:15 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:31:45 - mmengine - INFO - Epoch(train) [109][50/293] lr: 5.000000e-04 eta: 4:20:08 time: 0.585840 data_time: 0.123392 memory: 9504 loss_kpt: 0.000596 acc_pose: 0.806638 loss: 0.000596 2022/09/26 16:32:14 - mmengine - INFO - Epoch(train) [109][100/293] lr: 5.000000e-04 eta: 4:19:45 time: 0.586521 data_time: 0.070618 memory: 9504 loss_kpt: 0.000608 acc_pose: 0.769199 loss: 0.000608 2022/09/26 16:32:43 - mmengine - INFO - Epoch(train) [109][150/293] lr: 5.000000e-04 eta: 4:19:22 time: 0.579185 data_time: 0.071812 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.855109 loss: 0.000609 2022/09/26 16:33:13 - mmengine - INFO - Epoch(train) [109][200/293] lr: 5.000000e-04 eta: 4:18:59 time: 0.593489 data_time: 0.079396 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.830845 loss: 0.000606 2022/09/26 16:33:41 - mmengine - INFO - Epoch(train) [109][250/293] lr: 5.000000e-04 eta: 4:18:34 time: 0.563954 data_time: 0.077158 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.839527 loss: 0.000615 2022/09/26 16:34:05 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:34:35 - mmengine - INFO - Epoch(train) [110][50/293] lr: 5.000000e-04 eta: 4:17:29 time: 0.601921 data_time: 0.096283 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.809370 loss: 0.000598 2022/09/26 16:34:43 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:35:03 - mmengine - INFO - Epoch(train) [110][100/293] lr: 5.000000e-04 eta: 4:17:04 time: 0.565101 data_time: 0.095840 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.838544 loss: 0.000592 2022/09/26 16:35:32 - mmengine - INFO - Epoch(train) [110][150/293] lr: 5.000000e-04 eta: 4:16:40 time: 0.569752 data_time: 0.068055 memory: 9504 loss_kpt: 0.000596 acc_pose: 0.865316 loss: 0.000596 2022/09/26 16:36:00 - mmengine - INFO - Epoch(train) [110][200/293] lr: 5.000000e-04 eta: 4:16:16 time: 0.570326 data_time: 0.084409 memory: 9504 loss_kpt: 0.000601 acc_pose: 0.798167 loss: 0.000601 2022/09/26 16:36:28 - mmengine - INFO - Epoch(train) [110][250/293] lr: 5.000000e-04 eta: 4:15:52 time: 0.551517 data_time: 0.083378 memory: 9504 loss_kpt: 0.000618 acc_pose: 0.769027 loss: 0.000618 2022/09/26 16:36:52 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:36:52 - mmengine - INFO - Saving checkpoint at 110 epochs 2022/09/26 16:37:16 - mmengine - INFO - Epoch(val) [110][50/407] eta: 0:01:55 time: 0.323116 data_time: 0.046359 memory: 9504 2022/09/26 16:37:32 - mmengine - INFO - Epoch(val) [110][100/407] eta: 0:01:39 time: 0.324881 data_time: 0.049083 memory: 1378 2022/09/26 16:37:48 - mmengine - INFO - Epoch(val) [110][150/407] eta: 0:01:21 time: 0.316458 data_time: 0.021098 memory: 1378 2022/09/26 16:38:04 - mmengine - INFO - Epoch(val) [110][200/407] eta: 0:01:04 time: 0.312103 data_time: 0.046331 memory: 1378 2022/09/26 16:38:20 - mmengine - INFO - Epoch(val) [110][250/407] eta: 0:00:51 time: 0.327036 data_time: 0.040098 memory: 1378 2022/09/26 16:38:35 - mmengine - INFO - Epoch(val) [110][300/407] eta: 0:00:32 time: 0.303772 data_time: 0.047105 memory: 1378 2022/09/26 16:38:51 - mmengine - INFO - Epoch(val) [110][350/407] eta: 0:00:17 time: 0.311816 data_time: 0.029253 memory: 1378 2022/09/26 16:39:04 - mmengine - INFO - Epoch(val) [110][400/407] eta: 0:00:01 time: 0.261077 data_time: 0.027876 memory: 1378 2022/09/26 16:39:37 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 16:39:50 - mmengine - INFO - Epoch(val) [110][407/407] coco/AP: 0.715583 coco/AP .5: 0.893625 coco/AP .75: 0.796251 coco/AP (M): 0.679785 coco/AP (L): 0.781417 coco/AR: 0.770797 coco/AR .5: 0.933092 coco/AR .75: 0.840680 coco/AR (M): 0.728735 coco/AR (L): 0.831067 2022/09/26 16:39:50 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_100.pth is removed 2022/09/26 16:39:52 - mmengine - INFO - The best checkpoint with 0.7156 coco/AP at 110 epoch is saved to best_coco/AP_epoch_110.pth. 2022/09/26 16:40:22 - mmengine - INFO - Epoch(train) [111][50/293] lr: 5.000000e-04 eta: 4:14:45 time: 0.585928 data_time: 0.164635 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.818362 loss: 0.000605 2022/09/26 16:40:50 - mmengine - INFO - Epoch(train) [111][100/293] lr: 5.000000e-04 eta: 4:14:21 time: 0.570189 data_time: 0.075585 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.818163 loss: 0.000606 2022/09/26 16:41:19 - mmengine - INFO - Epoch(train) [111][150/293] lr: 5.000000e-04 eta: 4:13:58 time: 0.583671 data_time: 0.086067 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.836338 loss: 0.000607 2022/09/26 16:41:47 - mmengine - INFO - Epoch(train) [111][200/293] lr: 5.000000e-04 eta: 4:13:33 time: 0.550629 data_time: 0.074731 memory: 9504 loss_kpt: 0.000608 acc_pose: 0.824281 loss: 0.000608 2022/09/26 16:42:15 - mmengine - INFO - Epoch(train) [111][250/293] lr: 5.000000e-04 eta: 4:13:09 time: 0.563327 data_time: 0.077995 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.824847 loss: 0.000621 2022/09/26 16:42:39 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:43:10 - mmengine - INFO - Epoch(train) [112][50/293] lr: 5.000000e-04 eta: 4:12:04 time: 0.609751 data_time: 0.103594 memory: 9504 loss_kpt: 0.000603 acc_pose: 0.791162 loss: 0.000603 2022/09/26 16:43:38 - mmengine - INFO - Epoch(train) [112][100/293] lr: 5.000000e-04 eta: 4:11:40 time: 0.567414 data_time: 0.078479 memory: 9504 loss_kpt: 0.000613 acc_pose: 0.797214 loss: 0.000613 2022/09/26 16:44:08 - mmengine - INFO - Epoch(train) [112][150/293] lr: 5.000000e-04 eta: 4:11:17 time: 0.596072 data_time: 0.091687 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.823789 loss: 0.000606 2022/09/26 16:44:37 - mmengine - INFO - Epoch(train) [112][200/293] lr: 5.000000e-04 eta: 4:10:53 time: 0.576482 data_time: 0.083354 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.834418 loss: 0.000612 2022/09/26 16:45:06 - mmengine - INFO - Epoch(train) [112][250/293] lr: 5.000000e-04 eta: 4:10:30 time: 0.591934 data_time: 0.115415 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.849840 loss: 0.000598 2022/09/26 16:45:30 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:46:00 - mmengine - INFO - Epoch(train) [113][50/293] lr: 5.000000e-04 eta: 4:09:25 time: 0.587919 data_time: 0.093598 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.780462 loss: 0.000607 2022/09/26 16:46:29 - mmengine - INFO - Epoch(train) [113][100/293] lr: 5.000000e-04 eta: 4:09:01 time: 0.577033 data_time: 0.082516 memory: 9504 loss_kpt: 0.000599 acc_pose: 0.833477 loss: 0.000599 2022/09/26 16:46:57 - mmengine - INFO - Epoch(train) [113][150/293] lr: 5.000000e-04 eta: 4:08:37 time: 0.569848 data_time: 0.094577 memory: 9504 loss_kpt: 0.000602 acc_pose: 0.835038 loss: 0.000602 2022/09/26 16:47:16 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:47:27 - mmengine - INFO - Epoch(train) [113][200/293] lr: 5.000000e-04 eta: 4:08:14 time: 0.593567 data_time: 0.077727 memory: 9504 loss_kpt: 0.000617 acc_pose: 0.832429 loss: 0.000617 2022/09/26 16:47:56 - mmengine - INFO - Epoch(train) [113][250/293] lr: 5.000000e-04 eta: 4:07:50 time: 0.581362 data_time: 0.205181 memory: 9504 loss_kpt: 0.000587 acc_pose: 0.852677 loss: 0.000587 2022/09/26 16:48:20 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:48:51 - mmengine - INFO - Epoch(train) [114][50/293] lr: 5.000000e-04 eta: 4:06:47 time: 0.617423 data_time: 0.099589 memory: 9504 loss_kpt: 0.000602 acc_pose: 0.851085 loss: 0.000602 2022/09/26 16:49:19 - mmengine - INFO - Epoch(train) [114][100/293] lr: 5.000000e-04 eta: 4:06:22 time: 0.556556 data_time: 0.115117 memory: 9504 loss_kpt: 0.000594 acc_pose: 0.818110 loss: 0.000594 2022/09/26 16:49:48 - mmengine - INFO - Epoch(train) [114][150/293] lr: 5.000000e-04 eta: 4:05:58 time: 0.573176 data_time: 0.114846 memory: 9504 loss_kpt: 0.000615 acc_pose: 0.876461 loss: 0.000615 2022/09/26 16:50:17 - mmengine - INFO - Epoch(train) [114][200/293] lr: 5.000000e-04 eta: 4:05:35 time: 0.585116 data_time: 0.080758 memory: 9504 loss_kpt: 0.000613 acc_pose: 0.834492 loss: 0.000613 2022/09/26 16:50:46 - mmengine - INFO - Epoch(train) [114][250/293] lr: 5.000000e-04 eta: 4:05:11 time: 0.587111 data_time: 0.083037 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.843322 loss: 0.000612 2022/09/26 16:51:10 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:51:40 - mmengine - INFO - Epoch(train) [115][50/293] lr: 5.000000e-04 eta: 4:04:08 time: 0.615610 data_time: 0.110613 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.793748 loss: 0.000598 2022/09/26 16:52:09 - mmengine - INFO - Epoch(train) [115][100/293] lr: 5.000000e-04 eta: 4:03:44 time: 0.563765 data_time: 0.078926 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.863783 loss: 0.000607 2022/09/26 16:52:38 - mmengine - INFO - Epoch(train) [115][150/293] lr: 5.000000e-04 eta: 4:03:20 time: 0.590271 data_time: 0.094740 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.826152 loss: 0.000606 2022/09/26 16:53:06 - mmengine - INFO - Epoch(train) [115][200/293] lr: 5.000000e-04 eta: 4:02:56 time: 0.555066 data_time: 0.071434 memory: 9504 loss_kpt: 0.000618 acc_pose: 0.808215 loss: 0.000618 2022/09/26 16:53:35 - mmengine - INFO - Epoch(train) [115][250/293] lr: 5.000000e-04 eta: 4:02:32 time: 0.579023 data_time: 0.081428 memory: 9504 loss_kpt: 0.000589 acc_pose: 0.827505 loss: 0.000589 2022/09/26 16:53:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:54:29 - mmengine - INFO - Epoch(train) [116][50/293] lr: 5.000000e-04 eta: 4:01:28 time: 0.600862 data_time: 0.089919 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.842936 loss: 0.000610 2022/09/26 16:54:59 - mmengine - INFO - Epoch(train) [116][100/293] lr: 5.000000e-04 eta: 4:01:05 time: 0.589393 data_time: 0.079844 memory: 9504 loss_kpt: 0.000606 acc_pose: 0.790268 loss: 0.000606 2022/09/26 16:55:28 - mmengine - INFO - Epoch(train) [116][150/293] lr: 5.000000e-04 eta: 4:00:41 time: 0.587476 data_time: 0.084430 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.818364 loss: 0.000591 2022/09/26 16:55:56 - mmengine - INFO - Epoch(train) [116][200/293] lr: 5.000000e-04 eta: 4:00:17 time: 0.568229 data_time: 0.099989 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.820901 loss: 0.000612 2022/09/26 16:56:25 - mmengine - INFO - Epoch(train) [116][250/293] lr: 5.000000e-04 eta: 3:59:53 time: 0.565241 data_time: 0.093204 memory: 9504 loss_kpt: 0.000621 acc_pose: 0.827176 loss: 0.000621 2022/09/26 16:56:49 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:56:57 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 16:57:19 - mmengine - INFO - Epoch(train) [117][50/293] lr: 5.000000e-04 eta: 3:58:50 time: 0.603609 data_time: 0.178948 memory: 9504 loss_kpt: 0.000590 acc_pose: 0.842275 loss: 0.000590 2022/09/26 16:57:47 - mmengine - INFO - Epoch(train) [117][100/293] lr: 5.000000e-04 eta: 3:58:25 time: 0.557212 data_time: 0.091896 memory: 9504 loss_kpt: 0.000599 acc_pose: 0.866699 loss: 0.000599 2022/09/26 16:58:17 - mmengine - INFO - Epoch(train) [117][150/293] lr: 5.000000e-04 eta: 3:58:02 time: 0.590156 data_time: 0.089257 memory: 9504 loss_kpt: 0.000596 acc_pose: 0.840276 loss: 0.000596 2022/09/26 16:58:46 - mmengine - INFO - Epoch(train) [117][200/293] lr: 5.000000e-04 eta: 3:57:38 time: 0.580429 data_time: 0.085200 memory: 9504 loss_kpt: 0.000601 acc_pose: 0.835566 loss: 0.000601 2022/09/26 16:59:13 - mmengine - INFO - Epoch(train) [117][250/293] lr: 5.000000e-04 eta: 3:57:13 time: 0.551476 data_time: 0.074750 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.856282 loss: 0.000605 2022/09/26 16:59:38 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:00:08 - mmengine - INFO - Epoch(train) [118][50/293] lr: 5.000000e-04 eta: 3:56:10 time: 0.608590 data_time: 0.154397 memory: 9504 loss_kpt: 0.000594 acc_pose: 0.810686 loss: 0.000594 2022/09/26 17:00:38 - mmengine - INFO - Epoch(train) [118][100/293] lr: 5.000000e-04 eta: 3:55:47 time: 0.596108 data_time: 0.125767 memory: 9504 loss_kpt: 0.000596 acc_pose: 0.871315 loss: 0.000596 2022/09/26 17:01:07 - mmengine - INFO - Epoch(train) [118][150/293] lr: 5.000000e-04 eta: 3:55:23 time: 0.578873 data_time: 0.092225 memory: 9504 loss_kpt: 0.000609 acc_pose: 0.852174 loss: 0.000609 2022/09/26 17:01:36 - mmengine - INFO - Epoch(train) [118][200/293] lr: 5.000000e-04 eta: 3:55:00 time: 0.586141 data_time: 0.083482 memory: 9504 loss_kpt: 0.000601 acc_pose: 0.804071 loss: 0.000601 2022/09/26 17:02:05 - mmengine - INFO - Epoch(train) [118][250/293] lr: 5.000000e-04 eta: 3:54:35 time: 0.563637 data_time: 0.106096 memory: 9504 loss_kpt: 0.000600 acc_pose: 0.872921 loss: 0.000600 2022/09/26 17:02:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:02:59 - mmengine - INFO - Epoch(train) [119][50/293] lr: 5.000000e-04 eta: 3:53:33 time: 0.599188 data_time: 0.133822 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.838940 loss: 0.000591 2022/09/26 17:03:27 - mmengine - INFO - Epoch(train) [119][100/293] lr: 5.000000e-04 eta: 3:53:09 time: 0.573354 data_time: 0.079642 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.801412 loss: 0.000583 2022/09/26 17:03:55 - mmengine - INFO - Epoch(train) [119][150/293] lr: 5.000000e-04 eta: 3:52:44 time: 0.561846 data_time: 0.089889 memory: 9504 loss_kpt: 0.000610 acc_pose: 0.792993 loss: 0.000610 2022/09/26 17:04:24 - mmengine - INFO - Epoch(train) [119][200/293] lr: 5.000000e-04 eta: 3:52:20 time: 0.570336 data_time: 0.080479 memory: 9504 loss_kpt: 0.000595 acc_pose: 0.884502 loss: 0.000595 2022/09/26 17:04:52 - mmengine - INFO - Epoch(train) [119][250/293] lr: 5.000000e-04 eta: 3:51:56 time: 0.570729 data_time: 0.075453 memory: 9504 loss_kpt: 0.000600 acc_pose: 0.815400 loss: 0.000600 2022/09/26 17:05:17 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:05:46 - mmengine - INFO - Epoch(train) [120][50/293] lr: 5.000000e-04 eta: 3:50:53 time: 0.587838 data_time: 0.102056 memory: 9504 loss_kpt: 0.000579 acc_pose: 0.827026 loss: 0.000579 2022/09/26 17:06:15 - mmengine - INFO - Epoch(train) [120][100/293] lr: 5.000000e-04 eta: 3:50:28 time: 0.564920 data_time: 0.075649 memory: 9504 loss_kpt: 0.000597 acc_pose: 0.832103 loss: 0.000597 2022/09/26 17:06:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:06:44 - mmengine - INFO - Epoch(train) [120][150/293] lr: 5.000000e-04 eta: 3:50:05 time: 0.585446 data_time: 0.086020 memory: 9504 loss_kpt: 0.000604 acc_pose: 0.818237 loss: 0.000604 2022/09/26 17:07:12 - mmengine - INFO - Epoch(train) [120][200/293] lr: 5.000000e-04 eta: 3:49:41 time: 0.569863 data_time: 0.079405 memory: 9504 loss_kpt: 0.000612 acc_pose: 0.782790 loss: 0.000612 2022/09/26 17:07:42 - mmengine - INFO - Epoch(train) [120][250/293] lr: 5.000000e-04 eta: 3:49:18 time: 0.597021 data_time: 0.085044 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.865448 loss: 0.000598 2022/09/26 17:08:06 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:08:06 - mmengine - INFO - Saving checkpoint at 120 epochs 2022/09/26 17:08:29 - mmengine - INFO - Epoch(val) [120][50/407] eta: 0:01:58 time: 0.330779 data_time: 0.045646 memory: 9504 2022/09/26 17:08:45 - mmengine - INFO - Epoch(val) [120][100/407] eta: 0:01:40 time: 0.327167 data_time: 0.079419 memory: 1378 2022/09/26 17:09:02 - mmengine - INFO - Epoch(val) [120][150/407] eta: 0:01:26 time: 0.338461 data_time: 0.063022 memory: 1378 2022/09/26 17:09:17 - mmengine - INFO - Epoch(val) [120][200/407] eta: 0:01:03 time: 0.305644 data_time: 0.022820 memory: 1378 2022/09/26 17:09:33 - mmengine - INFO - Epoch(val) [120][250/407] eta: 0:00:48 time: 0.311173 data_time: 0.034359 memory: 1378 2022/09/26 17:09:49 - mmengine - INFO - Epoch(val) [120][300/407] eta: 0:00:34 time: 0.322596 data_time: 0.055486 memory: 1378 2022/09/26 17:10:05 - mmengine - INFO - Epoch(val) [120][350/407] eta: 0:00:18 time: 0.320575 data_time: 0.049093 memory: 1378 2022/09/26 17:10:18 - mmengine - INFO - Epoch(val) [120][400/407] eta: 0:00:01 time: 0.255987 data_time: 0.033058 memory: 1378 2022/09/26 17:10:51 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 17:11:04 - mmengine - INFO - Epoch(val) [120][407/407] coco/AP: 0.714974 coco/AP .5: 0.893722 coco/AP .75: 0.793237 coco/AP (M): 0.678055 coco/AP (L): 0.781036 coco/AR: 0.770435 coco/AR .5: 0.932777 coco/AR .75: 0.840208 coco/AR (M): 0.728326 coco/AR (L): 0.830509 2022/09/26 17:11:33 - mmengine - INFO - Epoch(train) [121][50/293] lr: 5.000000e-04 eta: 3:48:15 time: 0.587716 data_time: 0.085728 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.834334 loss: 0.000575 2022/09/26 17:12:03 - mmengine - INFO - Epoch(train) [121][100/293] lr: 5.000000e-04 eta: 3:47:51 time: 0.586777 data_time: 0.085429 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.826564 loss: 0.000592 2022/09/26 17:12:32 - mmengine - INFO - Epoch(train) [121][150/293] lr: 5.000000e-04 eta: 3:47:28 time: 0.579989 data_time: 0.076868 memory: 9504 loss_kpt: 0.000595 acc_pose: 0.841122 loss: 0.000595 2022/09/26 17:13:01 - mmengine - INFO - Epoch(train) [121][200/293] lr: 5.000000e-04 eta: 3:47:04 time: 0.581242 data_time: 0.127628 memory: 9504 loss_kpt: 0.000601 acc_pose: 0.823132 loss: 0.000601 2022/09/26 17:13:30 - mmengine - INFO - Epoch(train) [121][250/293] lr: 5.000000e-04 eta: 3:46:40 time: 0.579185 data_time: 0.082550 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.849368 loss: 0.000598 2022/09/26 17:13:54 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:14:24 - mmengine - INFO - Epoch(train) [122][50/293] lr: 5.000000e-04 eta: 3:45:38 time: 0.601547 data_time: 0.098586 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.862567 loss: 0.000588 2022/09/26 17:14:53 - mmengine - INFO - Epoch(train) [122][100/293] lr: 5.000000e-04 eta: 3:45:14 time: 0.567852 data_time: 0.071119 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.848680 loss: 0.000605 2022/09/26 17:15:22 - mmengine - INFO - Epoch(train) [122][150/293] lr: 5.000000e-04 eta: 3:44:50 time: 0.579454 data_time: 0.083932 memory: 9504 loss_kpt: 0.000603 acc_pose: 0.782389 loss: 0.000603 2022/09/26 17:15:51 - mmengine - INFO - Epoch(train) [122][200/293] lr: 5.000000e-04 eta: 3:44:26 time: 0.581985 data_time: 0.080995 memory: 9504 loss_kpt: 0.000603 acc_pose: 0.850920 loss: 0.000603 2022/09/26 17:16:19 - mmengine - INFO - Epoch(train) [122][250/293] lr: 5.000000e-04 eta: 3:44:02 time: 0.568891 data_time: 0.077749 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.820757 loss: 0.000580 2022/09/26 17:16:45 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:17:15 - mmengine - INFO - Epoch(train) [123][50/293] lr: 5.000000e-04 eta: 3:43:00 time: 0.604123 data_time: 0.092345 memory: 9504 loss_kpt: 0.000597 acc_pose: 0.836574 loss: 0.000597 2022/09/26 17:17:44 - mmengine - INFO - Epoch(train) [123][100/293] lr: 5.000000e-04 eta: 3:42:37 time: 0.583107 data_time: 0.088611 memory: 9504 loss_kpt: 0.000605 acc_pose: 0.859000 loss: 0.000605 2022/09/26 17:18:14 - mmengine - INFO - Epoch(train) [123][150/293] lr: 5.000000e-04 eta: 3:42:13 time: 0.586427 data_time: 0.104559 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.794567 loss: 0.000592 2022/09/26 17:18:42 - mmengine - INFO - Epoch(train) [123][200/293] lr: 5.000000e-04 eta: 3:41:49 time: 0.568588 data_time: 0.101783 memory: 9504 loss_kpt: 0.000597 acc_pose: 0.860182 loss: 0.000597 2022/09/26 17:19:11 - mmengine - INFO - Epoch(train) [123][250/293] lr: 5.000000e-04 eta: 3:41:24 time: 0.570011 data_time: 0.085973 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.839996 loss: 0.000592 2022/09/26 17:19:13 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:19:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:20:06 - mmengine - INFO - Epoch(train) [124][50/293] lr: 5.000000e-04 eta: 3:40:24 time: 0.621748 data_time: 0.113596 memory: 9504 loss_kpt: 0.000607 acc_pose: 0.804700 loss: 0.000607 2022/09/26 17:20:35 - mmengine - INFO - Epoch(train) [124][100/293] lr: 5.000000e-04 eta: 3:40:00 time: 0.582102 data_time: 0.090590 memory: 9504 loss_kpt: 0.000576 acc_pose: 0.846655 loss: 0.000576 2022/09/26 17:21:04 - mmengine - INFO - Epoch(train) [124][150/293] lr: 5.000000e-04 eta: 3:39:36 time: 0.581014 data_time: 0.086709 memory: 9504 loss_kpt: 0.000594 acc_pose: 0.871522 loss: 0.000594 2022/09/26 17:21:34 - mmengine - INFO - Epoch(train) [124][200/293] lr: 5.000000e-04 eta: 3:39:13 time: 0.605526 data_time: 0.088600 memory: 9504 loss_kpt: 0.000597 acc_pose: 0.851326 loss: 0.000597 2022/09/26 17:22:03 - mmengine - INFO - Epoch(train) [124][250/293] lr: 5.000000e-04 eta: 3:38:49 time: 0.580641 data_time: 0.075876 memory: 9504 loss_kpt: 0.000590 acc_pose: 0.869141 loss: 0.000590 2022/09/26 17:22:26 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:22:55 - mmengine - INFO - Epoch(train) [125][50/293] lr: 5.000000e-04 eta: 3:37:48 time: 0.587279 data_time: 0.168197 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.822175 loss: 0.000584 2022/09/26 17:23:24 - mmengine - INFO - Epoch(train) [125][100/293] lr: 5.000000e-04 eta: 3:37:24 time: 0.579563 data_time: 0.125624 memory: 9504 loss_kpt: 0.000593 acc_pose: 0.822651 loss: 0.000593 2022/09/26 17:23:54 - mmengine - INFO - Epoch(train) [125][150/293] lr: 5.000000e-04 eta: 3:37:00 time: 0.590998 data_time: 0.115154 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.842510 loss: 0.000591 2022/09/26 17:24:23 - mmengine - INFO - Epoch(train) [125][200/293] lr: 5.000000e-04 eta: 3:36:36 time: 0.574591 data_time: 0.073139 memory: 9504 loss_kpt: 0.000593 acc_pose: 0.830835 loss: 0.000593 2022/09/26 17:24:53 - mmengine - INFO - Epoch(train) [125][250/293] lr: 5.000000e-04 eta: 3:36:13 time: 0.611061 data_time: 0.088457 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.849422 loss: 0.000598 2022/09/26 17:25:18 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:25:48 - mmengine - INFO - Epoch(train) [126][50/293] lr: 5.000000e-04 eta: 3:35:13 time: 0.600332 data_time: 0.107761 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.880468 loss: 0.000580 2022/09/26 17:26:17 - mmengine - INFO - Epoch(train) [126][100/293] lr: 5.000000e-04 eta: 3:34:49 time: 0.579362 data_time: 0.084722 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.814745 loss: 0.000591 2022/09/26 17:26:46 - mmengine - INFO - Epoch(train) [126][150/293] lr: 5.000000e-04 eta: 3:34:25 time: 0.587342 data_time: 0.090203 memory: 9504 loss_kpt: 0.000596 acc_pose: 0.843169 loss: 0.000596 2022/09/26 17:27:15 - mmengine - INFO - Epoch(train) [126][200/293] lr: 5.000000e-04 eta: 3:34:01 time: 0.578028 data_time: 0.084470 memory: 9504 loss_kpt: 0.000589 acc_pose: 0.779524 loss: 0.000589 2022/09/26 17:27:43 - mmengine - INFO - Epoch(train) [126][250/293] lr: 5.000000e-04 eta: 3:33:37 time: 0.573617 data_time: 0.094179 memory: 9504 loss_kpt: 0.000603 acc_pose: 0.838972 loss: 0.000603 2022/09/26 17:28:07 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:28:38 - mmengine - INFO - Epoch(train) [127][50/293] lr: 5.000000e-04 eta: 3:32:37 time: 0.614671 data_time: 0.086965 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.894273 loss: 0.000574 2022/09/26 17:28:56 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:29:06 - mmengine - INFO - Epoch(train) [127][100/293] lr: 5.000000e-04 eta: 3:32:12 time: 0.569564 data_time: 0.068958 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.890442 loss: 0.000592 2022/09/26 17:29:35 - mmengine - INFO - Epoch(train) [127][150/293] lr: 5.000000e-04 eta: 3:31:48 time: 0.575212 data_time: 0.069297 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.835187 loss: 0.000598 2022/09/26 17:30:03 - mmengine - INFO - Epoch(train) [127][200/293] lr: 5.000000e-04 eta: 3:31:24 time: 0.560646 data_time: 0.076592 memory: 9504 loss_kpt: 0.000586 acc_pose: 0.825378 loss: 0.000586 2022/09/26 17:30:31 - mmengine - INFO - Epoch(train) [127][250/293] lr: 5.000000e-04 eta: 3:30:59 time: 0.554614 data_time: 0.076658 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.837072 loss: 0.000580 2022/09/26 17:30:55 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:31:24 - mmengine - INFO - Epoch(train) [128][50/293] lr: 5.000000e-04 eta: 3:29:58 time: 0.581646 data_time: 0.087699 memory: 9504 loss_kpt: 0.000589 acc_pose: 0.853766 loss: 0.000589 2022/09/26 17:31:53 - mmengine - INFO - Epoch(train) [128][100/293] lr: 5.000000e-04 eta: 3:29:33 time: 0.564330 data_time: 0.087688 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.803127 loss: 0.000591 2022/09/26 17:32:22 - mmengine - INFO - Epoch(train) [128][150/293] lr: 5.000000e-04 eta: 3:29:10 time: 0.594304 data_time: 0.079933 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.854843 loss: 0.000588 2022/09/26 17:32:51 - mmengine - INFO - Epoch(train) [128][200/293] lr: 5.000000e-04 eta: 3:28:45 time: 0.569667 data_time: 0.085243 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.871721 loss: 0.000591 2022/09/26 17:33:20 - mmengine - INFO - Epoch(train) [128][250/293] lr: 5.000000e-04 eta: 3:28:22 time: 0.581715 data_time: 0.069638 memory: 9504 loss_kpt: 0.000587 acc_pose: 0.815235 loss: 0.000587 2022/09/26 17:33:44 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:34:14 - mmengine - INFO - Epoch(train) [129][50/293] lr: 5.000000e-04 eta: 3:27:21 time: 0.598719 data_time: 0.093650 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.795393 loss: 0.000584 2022/09/26 17:34:42 - mmengine - INFO - Epoch(train) [129][100/293] lr: 5.000000e-04 eta: 3:26:57 time: 0.563158 data_time: 0.071238 memory: 9504 loss_kpt: 0.000586 acc_pose: 0.793440 loss: 0.000586 2022/09/26 17:35:10 - mmengine - INFO - Epoch(train) [129][150/293] lr: 5.000000e-04 eta: 3:26:32 time: 0.556621 data_time: 0.078709 memory: 9504 loss_kpt: 0.000600 acc_pose: 0.861753 loss: 0.000600 2022/09/26 17:35:40 - mmengine - INFO - Epoch(train) [129][200/293] lr: 5.000000e-04 eta: 3:26:09 time: 0.598902 data_time: 0.078948 memory: 9504 loss_kpt: 0.000604 acc_pose: 0.821656 loss: 0.000604 2022/09/26 17:36:08 - mmengine - INFO - Epoch(train) [129][250/293] lr: 5.000000e-04 eta: 3:25:44 time: 0.563300 data_time: 0.082491 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.859709 loss: 0.000583 2022/09/26 17:36:32 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:37:02 - mmengine - INFO - Epoch(train) [130][50/293] lr: 5.000000e-04 eta: 3:24:44 time: 0.601097 data_time: 0.125411 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.822770 loss: 0.000592 2022/09/26 17:37:31 - mmengine - INFO - Epoch(train) [130][100/293] lr: 5.000000e-04 eta: 3:24:20 time: 0.580972 data_time: 0.078269 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.861921 loss: 0.000574 2022/09/26 17:38:00 - mmengine - INFO - Epoch(train) [130][150/293] lr: 5.000000e-04 eta: 3:23:56 time: 0.580098 data_time: 0.079791 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.851298 loss: 0.000580 2022/09/26 17:38:29 - mmengine - INFO - Epoch(train) [130][200/293] lr: 5.000000e-04 eta: 3:23:33 time: 0.584764 data_time: 0.084697 memory: 9504 loss_kpt: 0.000581 acc_pose: 0.848401 loss: 0.000581 2022/09/26 17:38:31 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:38:58 - mmengine - INFO - Epoch(train) [130][250/293] lr: 5.000000e-04 eta: 3:23:09 time: 0.584648 data_time: 0.077633 memory: 9504 loss_kpt: 0.000589 acc_pose: 0.833035 loss: 0.000589 2022/09/26 17:39:23 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:39:23 - mmengine - INFO - Saving checkpoint at 130 epochs 2022/09/26 17:39:48 - mmengine - INFO - Epoch(val) [130][50/407] eta: 0:02:06 time: 0.355706 data_time: 0.075201 memory: 9504 2022/09/26 17:40:03 - mmengine - INFO - Epoch(val) [130][100/407] eta: 0:01:34 time: 0.309379 data_time: 0.032003 memory: 1378 2022/09/26 17:40:19 - mmengine - INFO - Epoch(val) [130][150/407] eta: 0:01:22 time: 0.320745 data_time: 0.060366 memory: 1378 2022/09/26 17:40:35 - mmengine - INFO - Epoch(val) [130][200/407] eta: 0:01:05 time: 0.314510 data_time: 0.027216 memory: 1378 2022/09/26 17:40:51 - mmengine - INFO - Epoch(val) [130][250/407] eta: 0:00:50 time: 0.322239 data_time: 0.058494 memory: 1378 2022/09/26 17:41:08 - mmengine - INFO - Epoch(val) [130][300/407] eta: 0:00:35 time: 0.335492 data_time: 0.053949 memory: 1378 2022/09/26 17:41:24 - mmengine - INFO - Epoch(val) [130][350/407] eta: 0:00:18 time: 0.327739 data_time: 0.067075 memory: 1378 2022/09/26 17:41:36 - mmengine - INFO - Epoch(val) [130][400/407] eta: 0:00:01 time: 0.241912 data_time: 0.047827 memory: 1378 2022/09/26 17:42:09 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 17:42:22 - mmengine - INFO - Epoch(val) [130][407/407] coco/AP: 0.713631 coco/AP .5: 0.892904 coco/AP .75: 0.788985 coco/AP (M): 0.678477 coco/AP (L): 0.779419 coco/AR: 0.769238 coco/AR .5: 0.932462 coco/AR .75: 0.836587 coco/AR (M): 0.727452 coco/AR (L): 0.830137 2022/09/26 17:42:50 - mmengine - INFO - Epoch(train) [131][50/293] lr: 5.000000e-04 eta: 3:22:08 time: 0.570472 data_time: 0.093477 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.852209 loss: 0.000580 2022/09/26 17:43:18 - mmengine - INFO - Epoch(train) [131][100/293] lr: 5.000000e-04 eta: 3:21:44 time: 0.562383 data_time: 0.074447 memory: 9504 loss_kpt: 0.000594 acc_pose: 0.863546 loss: 0.000594 2022/09/26 17:43:47 - mmengine - INFO - Epoch(train) [131][150/293] lr: 5.000000e-04 eta: 3:21:19 time: 0.575555 data_time: 0.146089 memory: 9504 loss_kpt: 0.000578 acc_pose: 0.854426 loss: 0.000578 2022/09/26 17:44:17 - mmengine - INFO - Epoch(train) [131][200/293] lr: 5.000000e-04 eta: 3:20:56 time: 0.596107 data_time: 0.081805 memory: 9504 loss_kpt: 0.000587 acc_pose: 0.837925 loss: 0.000587 2022/09/26 17:44:46 - mmengine - INFO - Epoch(train) [131][250/293] lr: 5.000000e-04 eta: 3:20:32 time: 0.577666 data_time: 0.094783 memory: 9504 loss_kpt: 0.000594 acc_pose: 0.838491 loss: 0.000594 2022/09/26 17:45:09 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:45:41 - mmengine - INFO - Epoch(train) [132][50/293] lr: 5.000000e-04 eta: 3:19:33 time: 0.627076 data_time: 0.116853 memory: 9504 loss_kpt: 0.000585 acc_pose: 0.824411 loss: 0.000585 2022/09/26 17:46:09 - mmengine - INFO - Epoch(train) [132][100/293] lr: 5.000000e-04 eta: 3:19:09 time: 0.570654 data_time: 0.085583 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.869887 loss: 0.000591 2022/09/26 17:46:38 - mmengine - INFO - Epoch(train) [132][150/293] lr: 5.000000e-04 eta: 3:18:45 time: 0.573825 data_time: 0.082890 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.805157 loss: 0.000582 2022/09/26 17:47:08 - mmengine - INFO - Epoch(train) [132][200/293] lr: 5.000000e-04 eta: 3:18:21 time: 0.604893 data_time: 0.088379 memory: 9504 loss_kpt: 0.000595 acc_pose: 0.853399 loss: 0.000595 2022/09/26 17:47:36 - mmengine - INFO - Epoch(train) [132][250/293] lr: 5.000000e-04 eta: 3:17:56 time: 0.553450 data_time: 0.134498 memory: 9504 loss_kpt: 0.000593 acc_pose: 0.859265 loss: 0.000593 2022/09/26 17:47:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:48:30 - mmengine - INFO - Epoch(train) [133][50/293] lr: 5.000000e-04 eta: 3:16:58 time: 0.622139 data_time: 0.096607 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.865845 loss: 0.000583 2022/09/26 17:49:01 - mmengine - INFO - Epoch(train) [133][100/293] lr: 5.000000e-04 eta: 3:16:35 time: 0.604428 data_time: 0.089499 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.811916 loss: 0.000582 2022/09/26 17:49:30 - mmengine - INFO - Epoch(train) [133][150/293] lr: 5.000000e-04 eta: 3:16:11 time: 0.589393 data_time: 0.091574 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.838289 loss: 0.000591 2022/09/26 17:49:58 - mmengine - INFO - Epoch(train) [133][200/293] lr: 5.000000e-04 eta: 3:15:46 time: 0.560714 data_time: 0.107889 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.843765 loss: 0.000588 2022/09/26 17:50:27 - mmengine - INFO - Epoch(train) [133][250/293] lr: 5.000000e-04 eta: 3:15:22 time: 0.571384 data_time: 0.085176 memory: 9504 loss_kpt: 0.000586 acc_pose: 0.847281 loss: 0.000586 2022/09/26 17:50:51 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:51:10 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:51:22 - mmengine - INFO - Epoch(train) [134][50/293] lr: 5.000000e-04 eta: 3:14:24 time: 0.625033 data_time: 0.129402 memory: 9504 loss_kpt: 0.000587 acc_pose: 0.881347 loss: 0.000587 2022/09/26 17:51:50 - mmengine - INFO - Epoch(train) [134][100/293] lr: 5.000000e-04 eta: 3:13:59 time: 0.567860 data_time: 0.073141 memory: 9504 loss_kpt: 0.000590 acc_pose: 0.813243 loss: 0.000590 2022/09/26 17:52:20 - mmengine - INFO - Epoch(train) [134][150/293] lr: 5.000000e-04 eta: 3:13:35 time: 0.586426 data_time: 0.124119 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.847438 loss: 0.000582 2022/09/26 17:52:48 - mmengine - INFO - Epoch(train) [134][200/293] lr: 5.000000e-04 eta: 3:13:11 time: 0.578611 data_time: 0.071987 memory: 9504 loss_kpt: 0.000585 acc_pose: 0.860276 loss: 0.000585 2022/09/26 17:53:18 - mmengine - INFO - Epoch(train) [134][250/293] lr: 5.000000e-04 eta: 3:12:47 time: 0.596877 data_time: 0.082494 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.865022 loss: 0.000588 2022/09/26 17:53:42 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:54:12 - mmengine - INFO - Epoch(train) [135][50/293] lr: 5.000000e-04 eta: 3:11:49 time: 0.596401 data_time: 0.196031 memory: 9504 loss_kpt: 0.000581 acc_pose: 0.839151 loss: 0.000581 2022/09/26 17:54:41 - mmengine - INFO - Epoch(train) [135][100/293] lr: 5.000000e-04 eta: 3:11:24 time: 0.573318 data_time: 0.080006 memory: 9504 loss_kpt: 0.000593 acc_pose: 0.817613 loss: 0.000593 2022/09/26 17:55:10 - mmengine - INFO - Epoch(train) [135][150/293] lr: 5.000000e-04 eta: 3:11:00 time: 0.579778 data_time: 0.106455 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.870433 loss: 0.000588 2022/09/26 17:55:39 - mmengine - INFO - Epoch(train) [135][200/293] lr: 5.000000e-04 eta: 3:10:36 time: 0.583735 data_time: 0.080394 memory: 9504 loss_kpt: 0.000590 acc_pose: 0.815880 loss: 0.000590 2022/09/26 17:56:08 - mmengine - INFO - Epoch(train) [135][250/293] lr: 5.000000e-04 eta: 3:10:12 time: 0.586606 data_time: 0.086458 memory: 9504 loss_kpt: 0.000596 acc_pose: 0.822520 loss: 0.000596 2022/09/26 17:56:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:57:03 - mmengine - INFO - Epoch(train) [136][50/293] lr: 5.000000e-04 eta: 3:09:14 time: 0.597867 data_time: 0.096211 memory: 9504 loss_kpt: 0.000579 acc_pose: 0.841207 loss: 0.000579 2022/09/26 17:57:34 - mmengine - INFO - Epoch(train) [136][100/293] lr: 5.000000e-04 eta: 3:08:51 time: 0.625055 data_time: 0.098243 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.856778 loss: 0.000575 2022/09/26 17:58:04 - mmengine - INFO - Epoch(train) [136][150/293] lr: 5.000000e-04 eta: 3:08:27 time: 0.584053 data_time: 0.078809 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.861185 loss: 0.000574 2022/09/26 17:58:31 - mmengine - INFO - Epoch(train) [136][200/293] lr: 5.000000e-04 eta: 3:08:02 time: 0.557254 data_time: 0.081457 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.838839 loss: 0.000584 2022/09/26 17:59:01 - mmengine - INFO - Epoch(train) [136][250/293] lr: 5.000000e-04 eta: 3:07:38 time: 0.589857 data_time: 0.085825 memory: 9504 loss_kpt: 0.000586 acc_pose: 0.836206 loss: 0.000586 2022/09/26 17:59:26 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 17:59:57 - mmengine - INFO - Epoch(train) [137][50/293] lr: 5.000000e-04 eta: 3:06:40 time: 0.613140 data_time: 0.099358 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.840649 loss: 0.000598 2022/09/26 18:00:27 - mmengine - INFO - Epoch(train) [137][100/293] lr: 5.000000e-04 eta: 3:06:17 time: 0.606350 data_time: 0.078514 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.831917 loss: 0.000583 2022/09/26 18:00:56 - mmengine - INFO - Epoch(train) [137][150/293] lr: 5.000000e-04 eta: 3:05:53 time: 0.588168 data_time: 0.085816 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.864069 loss: 0.000570 2022/09/26 18:00:58 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:01:26 - mmengine - INFO - Epoch(train) [137][200/293] lr: 5.000000e-04 eta: 3:05:29 time: 0.598129 data_time: 0.113119 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.873465 loss: 0.000584 2022/09/26 18:01:55 - mmengine - INFO - Epoch(train) [137][250/293] lr: 5.000000e-04 eta: 3:05:05 time: 0.575578 data_time: 0.104103 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.779128 loss: 0.000583 2022/09/26 18:02:20 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:02:50 - mmengine - INFO - Epoch(train) [138][50/293] lr: 5.000000e-04 eta: 3:04:07 time: 0.595258 data_time: 0.187672 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.847311 loss: 0.000580 2022/09/26 18:03:19 - mmengine - INFO - Epoch(train) [138][100/293] lr: 5.000000e-04 eta: 3:03:43 time: 0.579759 data_time: 0.154971 memory: 9504 loss_kpt: 0.000593 acc_pose: 0.829814 loss: 0.000593 2022/09/26 18:03:48 - mmengine - INFO - Epoch(train) [138][150/293] lr: 5.000000e-04 eta: 3:03:19 time: 0.595979 data_time: 0.083727 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.848761 loss: 0.000574 2022/09/26 18:04:17 - mmengine - INFO - Epoch(train) [138][200/293] lr: 5.000000e-04 eta: 3:02:54 time: 0.576691 data_time: 0.080932 memory: 9504 loss_kpt: 0.000585 acc_pose: 0.855186 loss: 0.000585 2022/09/26 18:04:46 - mmengine - INFO - Epoch(train) [138][250/293] lr: 5.000000e-04 eta: 3:02:30 time: 0.570911 data_time: 0.079480 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.884812 loss: 0.000575 2022/09/26 18:05:11 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:05:40 - mmengine - INFO - Epoch(train) [139][50/293] lr: 5.000000e-04 eta: 3:01:32 time: 0.579367 data_time: 0.103134 memory: 9504 loss_kpt: 0.000579 acc_pose: 0.828123 loss: 0.000579 2022/09/26 18:06:09 - mmengine - INFO - Epoch(train) [139][100/293] lr: 5.000000e-04 eta: 3:01:07 time: 0.578442 data_time: 0.100096 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.869655 loss: 0.000588 2022/09/26 18:06:38 - mmengine - INFO - Epoch(train) [139][150/293] lr: 5.000000e-04 eta: 3:00:43 time: 0.584291 data_time: 0.080912 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.848974 loss: 0.000575 2022/09/26 18:07:07 - mmengine - INFO - Epoch(train) [139][200/293] lr: 5.000000e-04 eta: 3:00:19 time: 0.567738 data_time: 0.089338 memory: 9504 loss_kpt: 0.000594 acc_pose: 0.856076 loss: 0.000594 2022/09/26 18:07:36 - mmengine - INFO - Epoch(train) [139][250/293] lr: 5.000000e-04 eta: 2:59:54 time: 0.580182 data_time: 0.074038 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.853418 loss: 0.000588 2022/09/26 18:07:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:08:29 - mmengine - INFO - Epoch(train) [140][50/293] lr: 5.000000e-04 eta: 2:58:57 time: 0.589084 data_time: 0.103410 memory: 9504 loss_kpt: 0.000595 acc_pose: 0.825708 loss: 0.000595 2022/09/26 18:08:59 - mmengine - INFO - Epoch(train) [140][100/293] lr: 5.000000e-04 eta: 2:58:33 time: 0.603211 data_time: 0.079298 memory: 9504 loss_kpt: 0.000565 acc_pose: 0.853078 loss: 0.000565 2022/09/26 18:09:26 - mmengine - INFO - Epoch(train) [140][150/293] lr: 5.000000e-04 eta: 2:58:08 time: 0.549019 data_time: 0.080211 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.804892 loss: 0.000592 2022/09/26 18:09:55 - mmengine - INFO - Epoch(train) [140][200/293] lr: 5.000000e-04 eta: 2:57:44 time: 0.579202 data_time: 0.078229 memory: 9504 loss_kpt: 0.000587 acc_pose: 0.797740 loss: 0.000587 2022/09/26 18:10:24 - mmengine - INFO - Epoch(train) [140][250/293] lr: 5.000000e-04 eta: 2:57:19 time: 0.564938 data_time: 0.087221 memory: 9504 loss_kpt: 0.000598 acc_pose: 0.819475 loss: 0.000598 2022/09/26 18:10:37 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:10:49 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:10:49 - mmengine - INFO - Saving checkpoint at 140 epochs 2022/09/26 18:11:14 - mmengine - INFO - Epoch(val) [140][50/407] eta: 0:02:08 time: 0.359734 data_time: 0.083049 memory: 9504 2022/09/26 18:11:31 - mmengine - INFO - Epoch(val) [140][100/407] eta: 0:01:42 time: 0.332864 data_time: 0.062373 memory: 1378 2022/09/26 18:11:47 - mmengine - INFO - Epoch(val) [140][150/407] eta: 0:01:23 time: 0.325632 data_time: 0.049856 memory: 1378 2022/09/26 18:12:02 - mmengine - INFO - Epoch(val) [140][200/407] eta: 0:01:03 time: 0.305750 data_time: 0.047283 memory: 1378 2022/09/26 18:12:19 - mmengine - INFO - Epoch(val) [140][250/407] eta: 0:00:52 time: 0.333654 data_time: 0.060732 memory: 1378 2022/09/26 18:12:36 - mmengine - INFO - Epoch(val) [140][300/407] eta: 0:00:36 time: 0.336643 data_time: 0.064184 memory: 1378 2022/09/26 18:12:53 - mmengine - INFO - Epoch(val) [140][350/407] eta: 0:00:19 time: 0.336024 data_time: 0.079828 memory: 1378 2022/09/26 18:13:04 - mmengine - INFO - Epoch(val) [140][400/407] eta: 0:00:01 time: 0.227400 data_time: 0.038980 memory: 1378 2022/09/26 18:13:37 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 18:13:50 - mmengine - INFO - Epoch(val) [140][407/407] coco/AP: 0.718800 coco/AP .5: 0.895368 coco/AP .75: 0.795430 coco/AP (M): 0.684074 coco/AP (L): 0.784120 coco/AR: 0.773363 coco/AR .5: 0.933564 coco/AR .75: 0.841152 coco/AR (M): 0.732587 coco/AR (L): 0.832627 2022/09/26 18:13:50 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_110.pth is removed 2022/09/26 18:13:52 - mmengine - INFO - The best checkpoint with 0.7188 coco/AP at 140 epoch is saved to best_coco/AP_epoch_140.pth. 2022/09/26 18:14:21 - mmengine - INFO - Epoch(train) [141][50/293] lr: 5.000000e-04 eta: 2:56:21 time: 0.570383 data_time: 0.168309 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.872330 loss: 0.000575 2022/09/26 18:14:50 - mmengine - INFO - Epoch(train) [141][100/293] lr: 5.000000e-04 eta: 2:55:57 time: 0.582306 data_time: 0.080151 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.821095 loss: 0.000580 2022/09/26 18:15:19 - mmengine - INFO - Epoch(train) [141][150/293] lr: 5.000000e-04 eta: 2:55:32 time: 0.577293 data_time: 0.078816 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.858537 loss: 0.000580 2022/09/26 18:15:47 - mmengine - INFO - Epoch(train) [141][200/293] lr: 5.000000e-04 eta: 2:55:08 time: 0.565001 data_time: 0.086305 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.867580 loss: 0.000584 2022/09/26 18:16:17 - mmengine - INFO - Epoch(train) [141][250/293] lr: 5.000000e-04 eta: 2:54:44 time: 0.606347 data_time: 0.083808 memory: 9504 loss_kpt: 0.000590 acc_pose: 0.804883 loss: 0.000590 2022/09/26 18:16:42 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:17:12 - mmengine - INFO - Epoch(train) [142][50/293] lr: 5.000000e-04 eta: 2:53:47 time: 0.608897 data_time: 0.095894 memory: 9504 loss_kpt: 0.000587 acc_pose: 0.833520 loss: 0.000587 2022/09/26 18:17:41 - mmengine - INFO - Epoch(train) [142][100/293] lr: 5.000000e-04 eta: 2:53:23 time: 0.583158 data_time: 0.074413 memory: 9504 loss_kpt: 0.000585 acc_pose: 0.878052 loss: 0.000585 2022/09/26 18:18:11 - mmengine - INFO - Epoch(train) [142][150/293] lr: 5.000000e-04 eta: 2:52:59 time: 0.590682 data_time: 0.083979 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.842719 loss: 0.000588 2022/09/26 18:18:39 - mmengine - INFO - Epoch(train) [142][200/293] lr: 5.000000e-04 eta: 2:52:34 time: 0.569527 data_time: 0.086131 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.861874 loss: 0.000570 2022/09/26 18:19:08 - mmengine - INFO - Epoch(train) [142][250/293] lr: 5.000000e-04 eta: 2:52:10 time: 0.576552 data_time: 0.083551 memory: 9504 loss_kpt: 0.000568 acc_pose: 0.838930 loss: 0.000568 2022/09/26 18:19:32 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:20:02 - mmengine - INFO - Epoch(train) [143][50/293] lr: 5.000000e-04 eta: 2:51:13 time: 0.594331 data_time: 0.098388 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.830396 loss: 0.000566 2022/09/26 18:20:31 - mmengine - INFO - Epoch(train) [143][100/293] lr: 5.000000e-04 eta: 2:50:49 time: 0.579857 data_time: 0.087184 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.825492 loss: 0.000583 2022/09/26 18:21:00 - mmengine - INFO - Epoch(train) [143][150/293] lr: 5.000000e-04 eta: 2:50:24 time: 0.580932 data_time: 0.082161 memory: 9504 loss_kpt: 0.000577 acc_pose: 0.780931 loss: 0.000577 2022/09/26 18:21:28 - mmengine - INFO - Epoch(train) [143][200/293] lr: 5.000000e-04 eta: 2:50:00 time: 0.571434 data_time: 0.084348 memory: 9504 loss_kpt: 0.000567 acc_pose: 0.837878 loss: 0.000567 2022/09/26 18:21:57 - mmengine - INFO - Epoch(train) [143][250/293] lr: 5.000000e-04 eta: 2:49:35 time: 0.569684 data_time: 0.075339 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.804726 loss: 0.000584 2022/09/26 18:22:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:22:52 - mmengine - INFO - Epoch(train) [144][50/293] lr: 5.000000e-04 eta: 2:48:39 time: 0.609235 data_time: 0.140271 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.807692 loss: 0.000580 2022/09/26 18:23:21 - mmengine - INFO - Epoch(train) [144][100/293] lr: 5.000000e-04 eta: 2:48:14 time: 0.572748 data_time: 0.098677 memory: 9504 loss_kpt: 0.000591 acc_pose: 0.885232 loss: 0.000591 2022/09/26 18:23:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:23:49 - mmengine - INFO - Epoch(train) [144][150/293] lr: 5.000000e-04 eta: 2:47:50 time: 0.576174 data_time: 0.086235 memory: 9504 loss_kpt: 0.000579 acc_pose: 0.853548 loss: 0.000579 2022/09/26 18:24:18 - mmengine - INFO - Epoch(train) [144][200/293] lr: 5.000000e-04 eta: 2:47:25 time: 0.578458 data_time: 0.133751 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.856597 loss: 0.000574 2022/09/26 18:24:48 - mmengine - INFO - Epoch(train) [144][250/293] lr: 5.000000e-04 eta: 2:47:01 time: 0.585886 data_time: 0.090058 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.806970 loss: 0.000588 2022/09/26 18:25:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:25:42 - mmengine - INFO - Epoch(train) [145][50/293] lr: 5.000000e-04 eta: 2:46:05 time: 0.602770 data_time: 0.106405 memory: 9504 loss_kpt: 0.000573 acc_pose: 0.849735 loss: 0.000573 2022/09/26 18:26:10 - mmengine - INFO - Epoch(train) [145][100/293] lr: 5.000000e-04 eta: 2:45:40 time: 0.560945 data_time: 0.078020 memory: 9504 loss_kpt: 0.000578 acc_pose: 0.835240 loss: 0.000578 2022/09/26 18:26:40 - mmengine - INFO - Epoch(train) [145][150/293] lr: 5.000000e-04 eta: 2:45:16 time: 0.589977 data_time: 0.099504 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.855097 loss: 0.000584 2022/09/26 18:27:08 - mmengine - INFO - Epoch(train) [145][200/293] lr: 5.000000e-04 eta: 2:44:51 time: 0.558873 data_time: 0.071120 memory: 9504 loss_kpt: 0.000577 acc_pose: 0.845711 loss: 0.000577 2022/09/26 18:27:37 - mmengine - INFO - Epoch(train) [145][250/293] lr: 5.000000e-04 eta: 2:44:27 time: 0.592495 data_time: 0.088982 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.839665 loss: 0.000584 2022/09/26 18:28:01 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:28:32 - mmengine - INFO - Epoch(train) [146][50/293] lr: 5.000000e-04 eta: 2:43:31 time: 0.611514 data_time: 0.109877 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.842569 loss: 0.000572 2022/09/26 18:29:01 - mmengine - INFO - Epoch(train) [146][100/293] lr: 5.000000e-04 eta: 2:43:07 time: 0.585048 data_time: 0.075284 memory: 9504 loss_kpt: 0.000581 acc_pose: 0.849827 loss: 0.000581 2022/09/26 18:29:29 - mmengine - INFO - Epoch(train) [146][150/293] lr: 5.000000e-04 eta: 2:42:42 time: 0.566584 data_time: 0.082296 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.838051 loss: 0.000566 2022/09/26 18:29:58 - mmengine - INFO - Epoch(train) [146][200/293] lr: 5.000000e-04 eta: 2:42:17 time: 0.583576 data_time: 0.086910 memory: 9504 loss_kpt: 0.000578 acc_pose: 0.838688 loss: 0.000578 2022/09/26 18:30:28 - mmengine - INFO - Epoch(train) [146][250/293] lr: 5.000000e-04 eta: 2:41:53 time: 0.600368 data_time: 0.086907 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.849638 loss: 0.000582 2022/09/26 18:30:53 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:31:23 - mmengine - INFO - Epoch(train) [147][50/293] lr: 5.000000e-04 eta: 2:40:57 time: 0.598531 data_time: 0.113339 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.831444 loss: 0.000580 2022/09/26 18:31:52 - mmengine - INFO - Epoch(train) [147][100/293] lr: 5.000000e-04 eta: 2:40:33 time: 0.587301 data_time: 0.083515 memory: 9504 loss_kpt: 0.000571 acc_pose: 0.853624 loss: 0.000571 2022/09/26 18:32:22 - mmengine - INFO - Epoch(train) [147][150/293] lr: 5.000000e-04 eta: 2:40:09 time: 0.592961 data_time: 0.082526 memory: 9504 loss_kpt: 0.000576 acc_pose: 0.811386 loss: 0.000576 2022/09/26 18:32:50 - mmengine - INFO - Epoch(train) [147][200/293] lr: 5.000000e-04 eta: 2:39:44 time: 0.573574 data_time: 0.089554 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.876272 loss: 0.000574 2022/09/26 18:33:03 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:33:19 - mmengine - INFO - Epoch(train) [147][250/293] lr: 5.000000e-04 eta: 2:39:20 time: 0.570787 data_time: 0.068112 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.834062 loss: 0.000572 2022/09/26 18:33:44 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:34:14 - mmengine - INFO - Epoch(train) [148][50/293] lr: 5.000000e-04 eta: 2:38:24 time: 0.611299 data_time: 0.102288 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.843602 loss: 0.000583 2022/09/26 18:34:42 - mmengine - INFO - Epoch(train) [148][100/293] lr: 5.000000e-04 eta: 2:37:59 time: 0.562460 data_time: 0.095427 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.862594 loss: 0.000584 2022/09/26 18:35:12 - mmengine - INFO - Epoch(train) [148][150/293] lr: 5.000000e-04 eta: 2:37:35 time: 0.602166 data_time: 0.091859 memory: 9504 loss_kpt: 0.000579 acc_pose: 0.858102 loss: 0.000579 2022/09/26 18:35:40 - mmengine - INFO - Epoch(train) [148][200/293] lr: 5.000000e-04 eta: 2:37:11 time: 0.560904 data_time: 0.090488 memory: 9504 loss_kpt: 0.000589 acc_pose: 0.824512 loss: 0.000589 2022/09/26 18:36:10 - mmengine - INFO - Epoch(train) [148][250/293] lr: 5.000000e-04 eta: 2:36:46 time: 0.583306 data_time: 0.077570 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.856226 loss: 0.000570 2022/09/26 18:36:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:37:03 - mmengine - INFO - Epoch(train) [149][50/293] lr: 5.000000e-04 eta: 2:35:50 time: 0.586877 data_time: 0.091745 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.829949 loss: 0.000575 2022/09/26 18:37:32 - mmengine - INFO - Epoch(train) [149][100/293] lr: 5.000000e-04 eta: 2:35:26 time: 0.570926 data_time: 0.071300 memory: 9504 loss_kpt: 0.000577 acc_pose: 0.855969 loss: 0.000577 2022/09/26 18:38:01 - mmengine - INFO - Epoch(train) [149][150/293] lr: 5.000000e-04 eta: 2:35:01 time: 0.585064 data_time: 0.113458 memory: 9504 loss_kpt: 0.000571 acc_pose: 0.878021 loss: 0.000571 2022/09/26 18:38:29 - mmengine - INFO - Epoch(train) [149][200/293] lr: 5.000000e-04 eta: 2:34:37 time: 0.572967 data_time: 0.125213 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.861334 loss: 0.000575 2022/09/26 18:38:59 - mmengine - INFO - Epoch(train) [149][250/293] lr: 5.000000e-04 eta: 2:34:12 time: 0.588040 data_time: 0.091399 memory: 9504 loss_kpt: 0.000568 acc_pose: 0.836453 loss: 0.000568 2022/09/26 18:39:23 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:39:53 - mmengine - INFO - Epoch(train) [150][50/293] lr: 5.000000e-04 eta: 2:33:17 time: 0.613754 data_time: 0.118707 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.801318 loss: 0.000572 2022/09/26 18:40:23 - mmengine - INFO - Epoch(train) [150][100/293] lr: 5.000000e-04 eta: 2:32:53 time: 0.584880 data_time: 0.094331 memory: 9504 loss_kpt: 0.000569 acc_pose: 0.814319 loss: 0.000569 2022/09/26 18:40:51 - mmengine - INFO - Epoch(train) [150][150/293] lr: 5.000000e-04 eta: 2:32:28 time: 0.560734 data_time: 0.072502 memory: 9504 loss_kpt: 0.000568 acc_pose: 0.862537 loss: 0.000568 2022/09/26 18:41:19 - mmengine - INFO - Epoch(train) [150][200/293] lr: 5.000000e-04 eta: 2:32:03 time: 0.567487 data_time: 0.071331 memory: 9504 loss_kpt: 0.000585 acc_pose: 0.842256 loss: 0.000585 2022/09/26 18:41:48 - mmengine - INFO - Epoch(train) [150][250/293] lr: 5.000000e-04 eta: 2:31:38 time: 0.571507 data_time: 0.112354 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.901582 loss: 0.000572 2022/09/26 18:42:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:42:12 - mmengine - INFO - Saving checkpoint at 150 epochs 2022/09/26 18:42:37 - mmengine - INFO - Epoch(val) [150][50/407] eta: 0:02:02 time: 0.344456 data_time: 0.051551 memory: 9504 2022/09/26 18:42:52 - mmengine - INFO - Epoch(val) [150][100/407] eta: 0:01:36 time: 0.315133 data_time: 0.048001 memory: 1378 2022/09/26 18:43:09 - mmengine - INFO - Epoch(val) [150][150/407] eta: 0:01:25 time: 0.334505 data_time: 0.031900 memory: 1378 2022/09/26 18:43:25 - mmengine - INFO - Epoch(val) [150][200/407] eta: 0:01:05 time: 0.316696 data_time: 0.031730 memory: 1378 2022/09/26 18:43:42 - mmengine - INFO - Epoch(val) [150][250/407] eta: 0:00:52 time: 0.332572 data_time: 0.059125 memory: 1378 2022/09/26 18:43:57 - mmengine - INFO - Epoch(val) [150][300/407] eta: 0:00:33 time: 0.314342 data_time: 0.031411 memory: 1378 2022/09/26 18:44:14 - mmengine - INFO - Epoch(val) [150][350/407] eta: 0:00:19 time: 0.341386 data_time: 0.058111 memory: 1378 2022/09/26 18:44:27 - mmengine - INFO - Epoch(val) [150][400/407] eta: 0:00:01 time: 0.247860 data_time: 0.028909 memory: 1378 2022/09/26 18:44:59 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 18:45:13 - mmengine - INFO - Epoch(val) [150][407/407] coco/AP: 0.719084 coco/AP .5: 0.896896 coco/AP .75: 0.792594 coco/AP (M): 0.682211 coco/AP (L): 0.785970 coco/AR: 0.772276 coco/AR .5: 0.933722 coco/AR .75: 0.837689 coco/AR (M): 0.730483 coco/AR (L): 0.832962 2022/09/26 18:45:13 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_140.pth is removed 2022/09/26 18:45:15 - mmengine - INFO - The best checkpoint with 0.7191 coco/AP at 150 epoch is saved to best_coco/AP_epoch_150.pth. 2022/09/26 18:45:44 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:45:44 - mmengine - INFO - Epoch(train) [151][50/293] lr: 5.000000e-04 eta: 2:30:43 time: 0.584322 data_time: 0.098030 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.813928 loss: 0.000570 2022/09/26 18:46:13 - mmengine - INFO - Epoch(train) [151][100/293] lr: 5.000000e-04 eta: 2:30:18 time: 0.581130 data_time: 0.104351 memory: 9504 loss_kpt: 0.000562 acc_pose: 0.868760 loss: 0.000562 2022/09/26 18:46:43 - mmengine - INFO - Epoch(train) [151][150/293] lr: 5.000000e-04 eta: 2:29:54 time: 0.592168 data_time: 0.086856 memory: 9504 loss_kpt: 0.000564 acc_pose: 0.827864 loss: 0.000564 2022/09/26 18:47:12 - mmengine - INFO - Epoch(train) [151][200/293] lr: 5.000000e-04 eta: 2:29:29 time: 0.569886 data_time: 0.182983 memory: 9504 loss_kpt: 0.000571 acc_pose: 0.857257 loss: 0.000571 2022/09/26 18:47:39 - mmengine - INFO - Epoch(train) [151][250/293] lr: 5.000000e-04 eta: 2:29:04 time: 0.553029 data_time: 0.088676 memory: 9504 loss_kpt: 0.000592 acc_pose: 0.883928 loss: 0.000592 2022/09/26 18:48:04 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:48:33 - mmengine - INFO - Epoch(train) [152][50/293] lr: 5.000000e-04 eta: 2:28:09 time: 0.597381 data_time: 0.096284 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.860777 loss: 0.000588 2022/09/26 18:49:02 - mmengine - INFO - Epoch(train) [152][100/293] lr: 5.000000e-04 eta: 2:27:44 time: 0.571943 data_time: 0.091227 memory: 9504 loss_kpt: 0.000581 acc_pose: 0.821421 loss: 0.000581 2022/09/26 18:49:31 - mmengine - INFO - Epoch(train) [152][150/293] lr: 5.000000e-04 eta: 2:27:20 time: 0.578796 data_time: 0.091499 memory: 9504 loss_kpt: 0.000567 acc_pose: 0.843705 loss: 0.000567 2022/09/26 18:50:00 - mmengine - INFO - Epoch(train) [152][200/293] lr: 5.000000e-04 eta: 2:26:55 time: 0.585589 data_time: 0.083833 memory: 9504 loss_kpt: 0.000562 acc_pose: 0.848508 loss: 0.000562 2022/09/26 18:50:28 - mmengine - INFO - Epoch(train) [152][250/293] lr: 5.000000e-04 eta: 2:26:31 time: 0.564044 data_time: 0.092040 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.888346 loss: 0.000582 2022/09/26 18:50:53 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:51:23 - mmengine - INFO - Epoch(train) [153][50/293] lr: 5.000000e-04 eta: 2:25:36 time: 0.601906 data_time: 0.099096 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.842786 loss: 0.000574 2022/09/26 18:51:53 - mmengine - INFO - Epoch(train) [153][100/293] lr: 5.000000e-04 eta: 2:25:11 time: 0.590859 data_time: 0.105478 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.858486 loss: 0.000580 2022/09/26 18:52:22 - mmengine - INFO - Epoch(train) [153][150/293] lr: 5.000000e-04 eta: 2:24:47 time: 0.575768 data_time: 0.079987 memory: 9504 loss_kpt: 0.000561 acc_pose: 0.816941 loss: 0.000561 2022/09/26 18:52:50 - mmengine - INFO - Epoch(train) [153][200/293] lr: 5.000000e-04 eta: 2:24:22 time: 0.564461 data_time: 0.101338 memory: 9504 loss_kpt: 0.000576 acc_pose: 0.837383 loss: 0.000576 2022/09/26 18:53:18 - mmengine - INFO - Epoch(train) [153][250/293] lr: 5.000000e-04 eta: 2:23:57 time: 0.566524 data_time: 0.080029 memory: 9504 loss_kpt: 0.000588 acc_pose: 0.843121 loss: 0.000588 2022/09/26 18:53:42 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:54:13 - mmengine - INFO - Epoch(train) [154][50/293] lr: 5.000000e-04 eta: 2:23:03 time: 0.609839 data_time: 0.119059 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.848595 loss: 0.000572 2022/09/26 18:54:41 - mmengine - INFO - Epoch(train) [154][100/293] lr: 5.000000e-04 eta: 2:22:38 time: 0.560916 data_time: 0.084969 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.812544 loss: 0.000574 2022/09/26 18:55:10 - mmengine - INFO - Epoch(train) [154][150/293] lr: 5.000000e-04 eta: 2:22:13 time: 0.586891 data_time: 0.161965 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.814075 loss: 0.000582 2022/09/26 18:55:22 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:55:40 - mmengine - INFO - Epoch(train) [154][200/293] lr: 5.000000e-04 eta: 2:21:49 time: 0.595035 data_time: 0.149855 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.845558 loss: 0.000583 2022/09/26 18:56:08 - mmengine - INFO - Epoch(train) [154][250/293] lr: 5.000000e-04 eta: 2:21:24 time: 0.554021 data_time: 0.080441 memory: 9504 loss_kpt: 0.000582 acc_pose: 0.864884 loss: 0.000582 2022/09/26 18:56:32 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:57:02 - mmengine - INFO - Epoch(train) [155][50/293] lr: 5.000000e-04 eta: 2:20:29 time: 0.604359 data_time: 0.107167 memory: 9504 loss_kpt: 0.000567 acc_pose: 0.855718 loss: 0.000567 2022/09/26 18:57:32 - mmengine - INFO - Epoch(train) [155][100/293] lr: 5.000000e-04 eta: 2:20:05 time: 0.590734 data_time: 0.091186 memory: 9504 loss_kpt: 0.000597 acc_pose: 0.834320 loss: 0.000597 2022/09/26 18:58:00 - mmengine - INFO - Epoch(train) [155][150/293] lr: 5.000000e-04 eta: 2:19:40 time: 0.570719 data_time: 0.079726 memory: 9504 loss_kpt: 0.000560 acc_pose: 0.836661 loss: 0.000560 2022/09/26 18:58:30 - mmengine - INFO - Epoch(train) [155][200/293] lr: 5.000000e-04 eta: 2:19:16 time: 0.594539 data_time: 0.090699 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.842768 loss: 0.000575 2022/09/26 18:58:59 - mmengine - INFO - Epoch(train) [155][250/293] lr: 5.000000e-04 eta: 2:18:51 time: 0.585902 data_time: 0.084786 memory: 9504 loss_kpt: 0.000583 acc_pose: 0.830406 loss: 0.000583 2022/09/26 18:59:24 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 18:59:54 - mmengine - INFO - Epoch(train) [156][50/293] lr: 5.000000e-04 eta: 2:17:57 time: 0.605922 data_time: 0.108063 memory: 9504 loss_kpt: 0.000571 acc_pose: 0.834500 loss: 0.000571 2022/09/26 19:00:22 - mmengine - INFO - Epoch(train) [156][100/293] lr: 5.000000e-04 eta: 2:17:32 time: 0.570951 data_time: 0.087252 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.863773 loss: 0.000572 2022/09/26 19:00:53 - mmengine - INFO - Epoch(train) [156][150/293] lr: 5.000000e-04 eta: 2:17:08 time: 0.603715 data_time: 0.087610 memory: 9504 loss_kpt: 0.000578 acc_pose: 0.845289 loss: 0.000578 2022/09/26 19:01:22 - mmengine - INFO - Epoch(train) [156][200/293] lr: 5.000000e-04 eta: 2:16:44 time: 0.584139 data_time: 0.092676 memory: 9504 loss_kpt: 0.000567 acc_pose: 0.791575 loss: 0.000567 2022/09/26 19:01:51 - mmengine - INFO - Epoch(train) [156][250/293] lr: 5.000000e-04 eta: 2:16:19 time: 0.577899 data_time: 0.095876 memory: 9504 loss_kpt: 0.000584 acc_pose: 0.826303 loss: 0.000584 2022/09/26 19:02:15 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:02:45 - mmengine - INFO - Epoch(train) [157][50/293] lr: 5.000000e-04 eta: 2:15:25 time: 0.593269 data_time: 0.097252 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.859099 loss: 0.000575 2022/09/26 19:03:15 - mmengine - INFO - Epoch(train) [157][100/293] lr: 5.000000e-04 eta: 2:15:00 time: 0.596243 data_time: 0.086197 memory: 9504 loss_kpt: 0.000559 acc_pose: 0.853175 loss: 0.000559 2022/09/26 19:03:44 - mmengine - INFO - Epoch(train) [157][150/293] lr: 5.000000e-04 eta: 2:14:36 time: 0.579521 data_time: 0.122584 memory: 9504 loss_kpt: 0.000567 acc_pose: 0.852791 loss: 0.000567 2022/09/26 19:04:13 - mmengine - INFO - Epoch(train) [157][200/293] lr: 5.000000e-04 eta: 2:14:11 time: 0.591554 data_time: 0.091033 memory: 9504 loss_kpt: 0.000576 acc_pose: 0.862098 loss: 0.000576 2022/09/26 19:04:42 - mmengine - INFO - Epoch(train) [157][250/293] lr: 5.000000e-04 eta: 2:13:46 time: 0.579231 data_time: 0.094677 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.843769 loss: 0.000572 2022/09/26 19:05:07 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:05:07 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:05:37 - mmengine - INFO - Epoch(train) [158][50/293] lr: 5.000000e-04 eta: 2:12:52 time: 0.598185 data_time: 0.096823 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.842938 loss: 0.000557 2022/09/26 19:06:06 - mmengine - INFO - Epoch(train) [158][100/293] lr: 5.000000e-04 eta: 2:12:28 time: 0.580692 data_time: 0.079499 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.848162 loss: 0.000575 2022/09/26 19:06:35 - mmengine - INFO - Epoch(train) [158][150/293] lr: 5.000000e-04 eta: 2:12:03 time: 0.585803 data_time: 0.086763 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.819451 loss: 0.000572 2022/09/26 19:07:05 - mmengine - INFO - Epoch(train) [158][200/293] lr: 5.000000e-04 eta: 2:11:39 time: 0.588776 data_time: 0.089758 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.897511 loss: 0.000572 2022/09/26 19:07:34 - mmengine - INFO - Epoch(train) [158][250/293] lr: 5.000000e-04 eta: 2:11:14 time: 0.582900 data_time: 0.083413 memory: 9504 loss_kpt: 0.000565 acc_pose: 0.865006 loss: 0.000565 2022/09/26 19:07:58 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:08:28 - mmengine - INFO - Epoch(train) [159][50/293] lr: 5.000000e-04 eta: 2:10:20 time: 0.603203 data_time: 0.115722 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.845446 loss: 0.000566 2022/09/26 19:08:57 - mmengine - INFO - Epoch(train) [159][100/293] lr: 5.000000e-04 eta: 2:09:56 time: 0.587262 data_time: 0.093534 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.843555 loss: 0.000570 2022/09/26 19:09:26 - mmengine - INFO - Epoch(train) [159][150/293] lr: 5.000000e-04 eta: 2:09:31 time: 0.573792 data_time: 0.088428 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.840374 loss: 0.000566 2022/09/26 19:09:55 - mmengine - INFO - Epoch(train) [159][200/293] lr: 5.000000e-04 eta: 2:09:06 time: 0.583015 data_time: 0.078022 memory: 9504 loss_kpt: 0.000569 acc_pose: 0.841413 loss: 0.000569 2022/09/26 19:10:23 - mmengine - INFO - Epoch(train) [159][250/293] lr: 5.000000e-04 eta: 2:08:41 time: 0.559021 data_time: 0.087555 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.845246 loss: 0.000557 2022/09/26 19:10:48 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:11:18 - mmengine - INFO - Epoch(train) [160][50/293] lr: 5.000000e-04 eta: 2:07:48 time: 0.604230 data_time: 0.119852 memory: 9504 loss_kpt: 0.000564 acc_pose: 0.880816 loss: 0.000564 2022/09/26 19:11:47 - mmengine - INFO - Epoch(train) [160][100/293] lr: 5.000000e-04 eta: 2:07:23 time: 0.578813 data_time: 0.082113 memory: 9504 loss_kpt: 0.000572 acc_pose: 0.859494 loss: 0.000572 2022/09/26 19:12:16 - mmengine - INFO - Epoch(train) [160][150/293] lr: 5.000000e-04 eta: 2:06:58 time: 0.590261 data_time: 0.095660 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.845844 loss: 0.000563 2022/09/26 19:12:43 - mmengine - INFO - Epoch(train) [160][200/293] lr: 5.000000e-04 eta: 2:06:33 time: 0.543053 data_time: 0.080250 memory: 9504 loss_kpt: 0.000576 acc_pose: 0.856860 loss: 0.000576 2022/09/26 19:13:12 - mmengine - INFO - Epoch(train) [160][250/293] lr: 5.000000e-04 eta: 2:06:08 time: 0.564488 data_time: 0.072841 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.873378 loss: 0.000563 2022/09/26 19:13:37 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:13:37 - mmengine - INFO - Saving checkpoint at 160 epochs 2022/09/26 19:14:01 - mmengine - INFO - Epoch(val) [160][50/407] eta: 0:01:58 time: 0.332335 data_time: 0.051829 memory: 9504 2022/09/26 19:14:17 - mmengine - INFO - Epoch(val) [160][100/407] eta: 0:01:40 time: 0.328927 data_time: 0.049338 memory: 1378 2022/09/26 19:14:34 - mmengine - INFO - Epoch(val) [160][150/407] eta: 0:01:25 time: 0.334357 data_time: 0.042506 memory: 1378 2022/09/26 19:14:50 - mmengine - INFO - Epoch(val) [160][200/407] eta: 0:01:07 time: 0.328239 data_time: 0.051370 memory: 1378 2022/09/26 19:15:07 - mmengine - INFO - Epoch(val) [160][250/407] eta: 0:00:53 time: 0.337871 data_time: 0.060691 memory: 1378 2022/09/26 19:15:24 - mmengine - INFO - Epoch(val) [160][300/407] eta: 0:00:36 time: 0.343711 data_time: 0.077352 memory: 1378 2022/09/26 19:15:42 - mmengine - INFO - Epoch(val) [160][350/407] eta: 0:00:19 time: 0.349304 data_time: 0.057505 memory: 1378 2022/09/26 19:15:52 - mmengine - INFO - Epoch(val) [160][400/407] eta: 0:00:01 time: 0.201042 data_time: 0.051266 memory: 1378 2022/09/26 19:16:25 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 19:16:38 - mmengine - INFO - Epoch(val) [160][407/407] coco/AP: 0.719445 coco/AP .5: 0.891877 coco/AP .75: 0.794815 coco/AP (M): 0.680025 coco/AP (L): 0.789427 coco/AR: 0.774418 coco/AR .5: 0.930888 coco/AR .75: 0.840995 coco/AR (M): 0.729008 coco/AR (L): 0.839316 2022/09/26 19:16:38 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_150.pth is removed 2022/09/26 19:16:41 - mmengine - INFO - The best checkpoint with 0.7194 coco/AP at 160 epoch is saved to best_coco/AP_epoch_160.pth. 2022/09/26 19:17:10 - mmengine - INFO - Epoch(train) [161][50/293] lr: 5.000000e-04 eta: 2:05:14 time: 0.568151 data_time: 0.137528 memory: 9504 loss_kpt: 0.000568 acc_pose: 0.864828 loss: 0.000568 2022/09/26 19:17:38 - mmengine - INFO - Epoch(train) [161][100/293] lr: 5.000000e-04 eta: 2:04:49 time: 0.557974 data_time: 0.086288 memory: 9504 loss_kpt: 0.000558 acc_pose: 0.846508 loss: 0.000558 2022/09/26 19:17:48 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:18:06 - mmengine - INFO - Epoch(train) [161][150/293] lr: 5.000000e-04 eta: 2:04:24 time: 0.557320 data_time: 0.083670 memory: 9504 loss_kpt: 0.000576 acc_pose: 0.841162 loss: 0.000576 2022/09/26 19:18:35 - mmengine - INFO - Epoch(train) [161][200/293] lr: 5.000000e-04 eta: 2:03:59 time: 0.591338 data_time: 0.085787 memory: 9504 loss_kpt: 0.000558 acc_pose: 0.845920 loss: 0.000558 2022/09/26 19:19:04 - mmengine - INFO - Epoch(train) [161][250/293] lr: 5.000000e-04 eta: 2:03:35 time: 0.579408 data_time: 0.084169 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.866033 loss: 0.000580 2022/09/26 19:19:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:19:59 - mmengine - INFO - Epoch(train) [162][50/293] lr: 5.000000e-04 eta: 2:02:41 time: 0.615328 data_time: 0.100620 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.891488 loss: 0.000566 2022/09/26 19:20:28 - mmengine - INFO - Epoch(train) [162][100/293] lr: 5.000000e-04 eta: 2:02:17 time: 0.567118 data_time: 0.081294 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.859157 loss: 0.000563 2022/09/26 19:20:57 - mmengine - INFO - Epoch(train) [162][150/293] lr: 5.000000e-04 eta: 2:01:52 time: 0.592335 data_time: 0.080834 memory: 9504 loss_kpt: 0.000573 acc_pose: 0.866878 loss: 0.000573 2022/09/26 19:21:26 - mmengine - INFO - Epoch(train) [162][200/293] lr: 5.000000e-04 eta: 2:01:27 time: 0.579927 data_time: 0.084596 memory: 9504 loss_kpt: 0.000556 acc_pose: 0.851927 loss: 0.000556 2022/09/26 19:21:55 - mmengine - INFO - Epoch(train) [162][250/293] lr: 5.000000e-04 eta: 2:01:02 time: 0.582004 data_time: 0.089885 memory: 9504 loss_kpt: 0.000571 acc_pose: 0.814619 loss: 0.000571 2022/09/26 19:22:20 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:22:50 - mmengine - INFO - Epoch(train) [163][50/293] lr: 5.000000e-04 eta: 2:00:09 time: 0.597974 data_time: 0.110784 memory: 9504 loss_kpt: 0.000574 acc_pose: 0.881655 loss: 0.000574 2022/09/26 19:23:19 - mmengine - INFO - Epoch(train) [163][100/293] lr: 5.000000e-04 eta: 1:59:44 time: 0.577494 data_time: 0.109724 memory: 9504 loss_kpt: 0.000559 acc_pose: 0.858248 loss: 0.000559 2022/09/26 19:23:48 - mmengine - INFO - Epoch(train) [163][150/293] lr: 5.000000e-04 eta: 1:59:20 time: 0.577229 data_time: 0.086031 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.873380 loss: 0.000566 2022/09/26 19:24:17 - mmengine - INFO - Epoch(train) [163][200/293] lr: 5.000000e-04 eta: 1:58:55 time: 0.581650 data_time: 0.081892 memory: 9504 loss_kpt: 0.000564 acc_pose: 0.856962 loss: 0.000564 2022/09/26 19:24:44 - mmengine - INFO - Epoch(train) [163][250/293] lr: 5.000000e-04 eta: 1:58:30 time: 0.547586 data_time: 0.070518 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.862552 loss: 0.000566 2022/09/26 19:25:08 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:25:39 - mmengine - INFO - Epoch(train) [164][50/293] lr: 5.000000e-04 eta: 1:57:37 time: 0.618365 data_time: 0.167691 memory: 9504 loss_kpt: 0.000560 acc_pose: 0.843626 loss: 0.000560 2022/09/26 19:26:08 - mmengine - INFO - Epoch(train) [164][100/293] lr: 5.000000e-04 eta: 1:57:12 time: 0.594409 data_time: 0.082844 memory: 9504 loss_kpt: 0.000562 acc_pose: 0.839901 loss: 0.000562 2022/09/26 19:26:37 - mmengine - INFO - Epoch(train) [164][150/293] lr: 5.000000e-04 eta: 1:56:48 time: 0.580947 data_time: 0.089449 memory: 9504 loss_kpt: 0.000575 acc_pose: 0.840281 loss: 0.000575 2022/09/26 19:27:07 - mmengine - INFO - Epoch(train) [164][200/293] lr: 5.000000e-04 eta: 1:56:23 time: 0.594523 data_time: 0.145151 memory: 9504 loss_kpt: 0.000578 acc_pose: 0.866804 loss: 0.000578 2022/09/26 19:27:30 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:27:34 - mmengine - INFO - Epoch(train) [164][250/293] lr: 5.000000e-04 eta: 1:55:58 time: 0.545105 data_time: 0.109774 memory: 9504 loss_kpt: 0.000558 acc_pose: 0.856761 loss: 0.000558 2022/09/26 19:28:00 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:28:31 - mmengine - INFO - Epoch(train) [165][50/293] lr: 5.000000e-04 eta: 1:55:05 time: 0.622072 data_time: 0.121743 memory: 9504 loss_kpt: 0.000577 acc_pose: 0.886693 loss: 0.000577 2022/09/26 19:28:59 - mmengine - INFO - Epoch(train) [165][100/293] lr: 5.000000e-04 eta: 1:54:40 time: 0.562530 data_time: 0.087522 memory: 9504 loss_kpt: 0.000564 acc_pose: 0.850043 loss: 0.000564 2022/09/26 19:29:27 - mmengine - INFO - Epoch(train) [165][150/293] lr: 5.000000e-04 eta: 1:54:15 time: 0.559368 data_time: 0.083841 memory: 9504 loss_kpt: 0.000565 acc_pose: 0.810273 loss: 0.000565 2022/09/26 19:29:55 - mmengine - INFO - Epoch(train) [165][200/293] lr: 5.000000e-04 eta: 1:53:50 time: 0.559341 data_time: 0.082413 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.830337 loss: 0.000563 2022/09/26 19:30:24 - mmengine - INFO - Epoch(train) [165][250/293] lr: 5.000000e-04 eta: 1:53:25 time: 0.578512 data_time: 0.125028 memory: 9504 loss_kpt: 0.000564 acc_pose: 0.843433 loss: 0.000564 2022/09/26 19:30:48 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:31:18 - mmengine - INFO - Epoch(train) [166][50/293] lr: 5.000000e-04 eta: 1:52:33 time: 0.616286 data_time: 0.110420 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.886421 loss: 0.000563 2022/09/26 19:31:47 - mmengine - INFO - Epoch(train) [166][100/293] lr: 5.000000e-04 eta: 1:52:08 time: 0.575424 data_time: 0.083874 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.879074 loss: 0.000563 2022/09/26 19:32:17 - mmengine - INFO - Epoch(train) [166][150/293] lr: 5.000000e-04 eta: 1:51:43 time: 0.590052 data_time: 0.082533 memory: 9504 loss_kpt: 0.000553 acc_pose: 0.852276 loss: 0.000553 2022/09/26 19:32:45 - mmengine - INFO - Epoch(train) [166][200/293] lr: 5.000000e-04 eta: 1:51:18 time: 0.559718 data_time: 0.089905 memory: 9504 loss_kpt: 0.000562 acc_pose: 0.824899 loss: 0.000562 2022/09/26 19:33:14 - mmengine - INFO - Epoch(train) [166][250/293] lr: 5.000000e-04 eta: 1:50:53 time: 0.577458 data_time: 0.088649 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.857916 loss: 0.000570 2022/09/26 19:33:38 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:34:08 - mmengine - INFO - Epoch(train) [167][50/293] lr: 5.000000e-04 eta: 1:50:01 time: 0.604937 data_time: 0.129518 memory: 9504 loss_kpt: 0.000558 acc_pose: 0.848698 loss: 0.000558 2022/09/26 19:34:37 - mmengine - INFO - Epoch(train) [167][100/293] lr: 5.000000e-04 eta: 1:49:36 time: 0.580110 data_time: 0.085199 memory: 9504 loss_kpt: 0.000567 acc_pose: 0.890384 loss: 0.000567 2022/09/26 19:35:06 - mmengine - INFO - Epoch(train) [167][150/293] lr: 5.000000e-04 eta: 1:49:11 time: 0.582364 data_time: 0.081895 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.774859 loss: 0.000570 2022/09/26 19:35:36 - mmengine - INFO - Epoch(train) [167][200/293] lr: 5.000000e-04 eta: 1:48:46 time: 0.589523 data_time: 0.116710 memory: 9504 loss_kpt: 0.000577 acc_pose: 0.870283 loss: 0.000577 2022/09/26 19:36:04 - mmengine - INFO - Epoch(train) [167][250/293] lr: 5.000000e-04 eta: 1:48:21 time: 0.567943 data_time: 0.092201 memory: 9504 loss_kpt: 0.000559 acc_pose: 0.801060 loss: 0.000559 2022/09/26 19:36:29 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:37:00 - mmengine - INFO - Epoch(train) [168][50/293] lr: 5.000000e-04 eta: 1:47:29 time: 0.611928 data_time: 0.097672 memory: 9504 loss_kpt: 0.000562 acc_pose: 0.854317 loss: 0.000562 2022/09/26 19:37:11 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:37:29 - mmengine - INFO - Epoch(train) [168][100/293] lr: 5.000000e-04 eta: 1:47:04 time: 0.577749 data_time: 0.089894 memory: 9504 loss_kpt: 0.000566 acc_pose: 0.835151 loss: 0.000566 2022/09/26 19:37:58 - mmengine - INFO - Epoch(train) [168][150/293] lr: 5.000000e-04 eta: 1:46:39 time: 0.591601 data_time: 0.088417 memory: 9504 loss_kpt: 0.000570 acc_pose: 0.834351 loss: 0.000570 2022/09/26 19:38:27 - mmengine - INFO - Epoch(train) [168][200/293] lr: 5.000000e-04 eta: 1:46:14 time: 0.580333 data_time: 0.079883 memory: 9504 loss_kpt: 0.000580 acc_pose: 0.842397 loss: 0.000580 2022/09/26 19:38:56 - mmengine - INFO - Epoch(train) [168][250/293] lr: 5.000000e-04 eta: 1:45:50 time: 0.575356 data_time: 0.081015 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.822286 loss: 0.000557 2022/09/26 19:39:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:39:52 - mmengine - INFO - Epoch(train) [169][50/293] lr: 5.000000e-04 eta: 1:44:58 time: 0.627786 data_time: 0.255634 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.865461 loss: 0.000557 2022/09/26 19:40:21 - mmengine - INFO - Epoch(train) [169][100/293] lr: 5.000000e-04 eta: 1:44:33 time: 0.581683 data_time: 0.104029 memory: 9504 loss_kpt: 0.000561 acc_pose: 0.837876 loss: 0.000561 2022/09/26 19:40:49 - mmengine - INFO - Epoch(train) [169][150/293] lr: 5.000000e-04 eta: 1:44:08 time: 0.557977 data_time: 0.081030 memory: 9504 loss_kpt: 0.000565 acc_pose: 0.845552 loss: 0.000565 2022/09/26 19:41:17 - mmengine - INFO - Epoch(train) [169][200/293] lr: 5.000000e-04 eta: 1:43:43 time: 0.567018 data_time: 0.082068 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.842653 loss: 0.000557 2022/09/26 19:41:47 - mmengine - INFO - Epoch(train) [169][250/293] lr: 5.000000e-04 eta: 1:43:18 time: 0.593406 data_time: 0.089089 memory: 9504 loss_kpt: 0.000561 acc_pose: 0.821400 loss: 0.000561 2022/09/26 19:42:11 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:42:42 - mmengine - INFO - Epoch(train) [170][50/293] lr: 5.000000e-04 eta: 1:42:26 time: 0.608549 data_time: 0.113411 memory: 9504 loss_kpt: 0.000563 acc_pose: 0.835876 loss: 0.000563 2022/09/26 19:43:10 - mmengine - INFO - Epoch(train) [170][100/293] lr: 5.000000e-04 eta: 1:42:01 time: 0.564138 data_time: 0.073420 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.823797 loss: 0.000557 2022/09/26 19:43:39 - mmengine - INFO - Epoch(train) [170][150/293] lr: 5.000000e-04 eta: 1:41:36 time: 0.587286 data_time: 0.094334 memory: 9504 loss_kpt: 0.000553 acc_pose: 0.836666 loss: 0.000553 2022/09/26 19:44:07 - mmengine - INFO - Epoch(train) [170][200/293] lr: 5.000000e-04 eta: 1:41:11 time: 0.562212 data_time: 0.079522 memory: 9504 loss_kpt: 0.000557 acc_pose: 0.851435 loss: 0.000557 2022/09/26 19:44:37 - mmengine - INFO - Epoch(train) [170][250/293] lr: 5.000000e-04 eta: 1:40:46 time: 0.591321 data_time: 0.091022 memory: 9504 loss_kpt: 0.000561 acc_pose: 0.858770 loss: 0.000561 2022/09/26 19:45:01 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:45:01 - mmengine - INFO - Saving checkpoint at 170 epochs 2022/09/26 19:45:26 - mmengine - INFO - Epoch(val) [170][50/407] eta: 0:02:04 time: 0.349061 data_time: 0.074430 memory: 9504 2022/09/26 19:45:42 - mmengine - INFO - Epoch(val) [170][100/407] eta: 0:01:40 time: 0.328963 data_time: 0.077619 memory: 1378 2022/09/26 19:45:58 - mmengine - INFO - Epoch(val) [170][150/407] eta: 0:01:22 time: 0.319908 data_time: 0.036437 memory: 1378 2022/09/26 19:46:14 - mmengine - INFO - Epoch(val) [170][200/407] eta: 0:01:07 time: 0.327362 data_time: 0.041520 memory: 1378 2022/09/26 19:46:31 - mmengine - INFO - Epoch(val) [170][250/407] eta: 0:00:51 time: 0.328688 data_time: 0.058502 memory: 1378 2022/09/26 19:46:47 - mmengine - INFO - Epoch(val) [170][300/407] eta: 0:00:34 time: 0.320785 data_time: 0.024878 memory: 1378 2022/09/26 19:47:03 - mmengine - INFO - Epoch(val) [170][350/407] eta: 0:00:18 time: 0.329838 data_time: 0.059459 memory: 1378 2022/09/26 19:47:15 - mmengine - INFO - Epoch(val) [170][400/407] eta: 0:00:01 time: 0.222462 data_time: 0.049317 memory: 1378 2022/09/26 19:47:47 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 19:48:00 - mmengine - INFO - Epoch(val) [170][407/407] coco/AP: 0.720311 coco/AP .5: 0.896382 coco/AP .75: 0.795094 coco/AP (M): 0.682764 coco/AP (L): 0.787741 coco/AR: 0.774213 coco/AR .5: 0.935453 coco/AR .75: 0.839893 coco/AR (M): 0.731084 coco/AR (L): 0.836343 2022/09/26 19:48:01 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_160.pth is removed 2022/09/26 19:48:03 - mmengine - INFO - The best checkpoint with 0.7203 coco/AP at 170 epoch is saved to best_coco/AP_epoch_170.pth. 2022/09/26 19:48:32 - mmengine - INFO - Epoch(train) [171][50/293] lr: 5.000000e-05 eta: 1:39:54 time: 0.570433 data_time: 0.193154 memory: 9504 loss_kpt: 0.000550 acc_pose: 0.867302 loss: 0.000550 2022/09/26 19:49:00 - mmengine - INFO - Epoch(train) [171][100/293] lr: 5.000000e-05 eta: 1:39:29 time: 0.560394 data_time: 0.088267 memory: 9504 loss_kpt: 0.000546 acc_pose: 0.843435 loss: 0.000546 2022/09/26 19:49:28 - mmengine - INFO - Epoch(train) [171][150/293] lr: 5.000000e-05 eta: 1:39:03 time: 0.558376 data_time: 0.089637 memory: 9504 loss_kpt: 0.000541 acc_pose: 0.896519 loss: 0.000541 2022/09/26 19:49:51 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:49:57 - mmengine - INFO - Epoch(train) [171][200/293] lr: 5.000000e-05 eta: 1:38:39 time: 0.585055 data_time: 0.091761 memory: 9504 loss_kpt: 0.000539 acc_pose: 0.853972 loss: 0.000539 2022/09/26 19:50:26 - mmengine - INFO - Epoch(train) [171][250/293] lr: 5.000000e-05 eta: 1:38:14 time: 0.578461 data_time: 0.102867 memory: 9504 loss_kpt: 0.000544 acc_pose: 0.867720 loss: 0.000544 2022/09/26 19:50:51 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:51:20 - mmengine - INFO - Epoch(train) [172][50/293] lr: 5.000000e-05 eta: 1:37:22 time: 0.596692 data_time: 0.141785 memory: 9504 loss_kpt: 0.000532 acc_pose: 0.876552 loss: 0.000532 2022/09/26 19:51:49 - mmengine - INFO - Epoch(train) [172][100/293] lr: 5.000000e-05 eta: 1:36:57 time: 0.572148 data_time: 0.081736 memory: 9504 loss_kpt: 0.000530 acc_pose: 0.820626 loss: 0.000530 2022/09/26 19:52:17 - mmengine - INFO - Epoch(train) [172][150/293] lr: 5.000000e-05 eta: 1:36:32 time: 0.566390 data_time: 0.081216 memory: 9504 loss_kpt: 0.000542 acc_pose: 0.892870 loss: 0.000542 2022/09/26 19:52:46 - mmengine - INFO - Epoch(train) [172][200/293] lr: 5.000000e-05 eta: 1:36:07 time: 0.567362 data_time: 0.088725 memory: 9504 loss_kpt: 0.000539 acc_pose: 0.851041 loss: 0.000539 2022/09/26 19:53:15 - mmengine - INFO - Epoch(train) [172][250/293] lr: 5.000000e-05 eta: 1:35:42 time: 0.580973 data_time: 0.084089 memory: 9504 loss_kpt: 0.000535 acc_pose: 0.863296 loss: 0.000535 2022/09/26 19:53:39 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:54:10 - mmengine - INFO - Epoch(train) [173][50/293] lr: 5.000000e-05 eta: 1:34:50 time: 0.607851 data_time: 0.120297 memory: 9504 loss_kpt: 0.000535 acc_pose: 0.854513 loss: 0.000535 2022/09/26 19:54:39 - mmengine - INFO - Epoch(train) [173][100/293] lr: 5.000000e-05 eta: 1:34:25 time: 0.585582 data_time: 0.090305 memory: 9504 loss_kpt: 0.000542 acc_pose: 0.815219 loss: 0.000542 2022/09/26 19:55:08 - mmengine - INFO - Epoch(train) [173][150/293] lr: 5.000000e-05 eta: 1:34:00 time: 0.593484 data_time: 0.101794 memory: 9504 loss_kpt: 0.000533 acc_pose: 0.867191 loss: 0.000533 2022/09/26 19:55:39 - mmengine - INFO - Epoch(train) [173][200/293] lr: 5.000000e-05 eta: 1:33:36 time: 0.607485 data_time: 0.085631 memory: 9504 loss_kpt: 0.000534 acc_pose: 0.854692 loss: 0.000534 2022/09/26 19:56:07 - mmengine - INFO - Epoch(train) [173][250/293] lr: 5.000000e-05 eta: 1:33:10 time: 0.560044 data_time: 0.084527 memory: 9504 loss_kpt: 0.000525 acc_pose: 0.863905 loss: 0.000525 2022/09/26 19:56:32 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:57:01 - mmengine - INFO - Epoch(train) [174][50/293] lr: 5.000000e-05 eta: 1:32:19 time: 0.588410 data_time: 0.093692 memory: 9504 loss_kpt: 0.000521 acc_pose: 0.858902 loss: 0.000521 2022/09/26 19:57:31 - mmengine - INFO - Epoch(train) [174][100/293] lr: 5.000000e-05 eta: 1:31:54 time: 0.594264 data_time: 0.085611 memory: 9504 loss_kpt: 0.000532 acc_pose: 0.869872 loss: 0.000532 2022/09/26 19:58:00 - mmengine - INFO - Epoch(train) [174][150/293] lr: 5.000000e-05 eta: 1:31:29 time: 0.575422 data_time: 0.081203 memory: 9504 loss_kpt: 0.000521 acc_pose: 0.835973 loss: 0.000521 2022/09/26 19:58:29 - mmengine - INFO - Epoch(train) [174][200/293] lr: 5.000000e-05 eta: 1:31:04 time: 0.585981 data_time: 0.087886 memory: 9504 loss_kpt: 0.000521 acc_pose: 0.839570 loss: 0.000521 2022/09/26 19:58:56 - mmengine - INFO - Epoch(train) [174][250/293] lr: 5.000000e-05 eta: 1:30:39 time: 0.548620 data_time: 0.077126 memory: 9504 loss_kpt: 0.000539 acc_pose: 0.865886 loss: 0.000539 2022/09/26 19:59:21 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:59:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 19:59:51 - mmengine - INFO - Epoch(train) [175][50/293] lr: 5.000000e-05 eta: 1:29:47 time: 0.596817 data_time: 0.114023 memory: 9504 loss_kpt: 0.000515 acc_pose: 0.887296 loss: 0.000515 2022/09/26 20:00:19 - mmengine - INFO - Epoch(train) [175][100/293] lr: 5.000000e-05 eta: 1:29:22 time: 0.568020 data_time: 0.086297 memory: 9504 loss_kpt: 0.000535 acc_pose: 0.879915 loss: 0.000535 2022/09/26 20:00:48 - mmengine - INFO - Epoch(train) [175][150/293] lr: 5.000000e-05 eta: 1:28:57 time: 0.567887 data_time: 0.076441 memory: 9504 loss_kpt: 0.000526 acc_pose: 0.864044 loss: 0.000526 2022/09/26 20:01:16 - mmengine - INFO - Epoch(train) [175][200/293] lr: 5.000000e-05 eta: 1:28:32 time: 0.560402 data_time: 0.086756 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.891882 loss: 0.000527 2022/09/26 20:01:46 - mmengine - INFO - Epoch(train) [175][250/293] lr: 5.000000e-05 eta: 1:28:07 time: 0.593510 data_time: 0.166602 memory: 9504 loss_kpt: 0.000530 acc_pose: 0.859896 loss: 0.000530 2022/09/26 20:02:10 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:02:41 - mmengine - INFO - Epoch(train) [176][50/293] lr: 5.000000e-05 eta: 1:27:16 time: 0.617013 data_time: 0.095047 memory: 9504 loss_kpt: 0.000524 acc_pose: 0.851680 loss: 0.000524 2022/09/26 20:03:10 - mmengine - INFO - Epoch(train) [176][100/293] lr: 5.000000e-05 eta: 1:26:51 time: 0.573085 data_time: 0.087538 memory: 9504 loss_kpt: 0.000534 acc_pose: 0.907063 loss: 0.000534 2022/09/26 20:03:39 - mmengine - INFO - Epoch(train) [176][150/293] lr: 5.000000e-05 eta: 1:26:26 time: 0.584016 data_time: 0.154496 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.876052 loss: 0.000527 2022/09/26 20:04:08 - mmengine - INFO - Epoch(train) [176][200/293] lr: 5.000000e-05 eta: 1:26:01 time: 0.575050 data_time: 0.200284 memory: 9504 loss_kpt: 0.000526 acc_pose: 0.863482 loss: 0.000526 2022/09/26 20:04:37 - mmengine - INFO - Epoch(train) [176][250/293] lr: 5.000000e-05 eta: 1:25:36 time: 0.584714 data_time: 0.098004 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.862924 loss: 0.000518 2022/09/26 20:05:02 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:05:32 - mmengine - INFO - Epoch(train) [177][50/293] lr: 5.000000e-05 eta: 1:24:45 time: 0.609069 data_time: 0.105506 memory: 9504 loss_kpt: 0.000525 acc_pose: 0.830003 loss: 0.000525 2022/09/26 20:06:01 - mmengine - INFO - Epoch(train) [177][100/293] lr: 5.000000e-05 eta: 1:24:20 time: 0.582968 data_time: 0.087725 memory: 9504 loss_kpt: 0.000534 acc_pose: 0.855919 loss: 0.000534 2022/09/26 20:06:30 - mmengine - INFO - Epoch(train) [177][150/293] lr: 5.000000e-05 eta: 1:23:55 time: 0.569314 data_time: 0.085530 memory: 9504 loss_kpt: 0.000535 acc_pose: 0.837884 loss: 0.000535 2022/09/26 20:06:59 - mmengine - INFO - Epoch(train) [177][200/293] lr: 5.000000e-05 eta: 1:23:30 time: 0.576821 data_time: 0.086749 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.803900 loss: 0.000519 2022/09/26 20:07:28 - mmengine - INFO - Epoch(train) [177][250/293] lr: 5.000000e-05 eta: 1:23:05 time: 0.596670 data_time: 0.089916 memory: 9504 loss_kpt: 0.000523 acc_pose: 0.847658 loss: 0.000523 2022/09/26 20:07:53 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:08:23 - mmengine - INFO - Epoch(train) [178][50/293] lr: 5.000000e-05 eta: 1:22:14 time: 0.603480 data_time: 0.123559 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.862553 loss: 0.000518 2022/09/26 20:08:52 - mmengine - INFO - Epoch(train) [178][100/293] lr: 5.000000e-05 eta: 1:21:49 time: 0.580241 data_time: 0.085729 memory: 9504 loss_kpt: 0.000520 acc_pose: 0.854749 loss: 0.000520 2022/09/26 20:09:15 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:09:22 - mmengine - INFO - Epoch(train) [178][150/293] lr: 5.000000e-05 eta: 1:21:24 time: 0.596338 data_time: 0.071598 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.911160 loss: 0.000512 2022/09/26 20:09:51 - mmengine - INFO - Epoch(train) [178][200/293] lr: 5.000000e-05 eta: 1:20:59 time: 0.579887 data_time: 0.079018 memory: 9504 loss_kpt: 0.000531 acc_pose: 0.894561 loss: 0.000531 2022/09/26 20:10:20 - mmengine - INFO - Epoch(train) [178][250/293] lr: 5.000000e-05 eta: 1:20:34 time: 0.581494 data_time: 0.081134 memory: 9504 loss_kpt: 0.000524 acc_pose: 0.866528 loss: 0.000524 2022/09/26 20:10:45 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:11:15 - mmengine - INFO - Epoch(train) [179][50/293] lr: 5.000000e-05 eta: 1:19:43 time: 0.593802 data_time: 0.124835 memory: 9504 loss_kpt: 0.000522 acc_pose: 0.876356 loss: 0.000522 2022/09/26 20:11:43 - mmengine - INFO - Epoch(train) [179][100/293] lr: 5.000000e-05 eta: 1:19:18 time: 0.571484 data_time: 0.085096 memory: 9504 loss_kpt: 0.000525 acc_pose: 0.878674 loss: 0.000525 2022/09/26 20:12:13 - mmengine - INFO - Epoch(train) [179][150/293] lr: 5.000000e-05 eta: 1:18:53 time: 0.582732 data_time: 0.074335 memory: 9504 loss_kpt: 0.000530 acc_pose: 0.871725 loss: 0.000530 2022/09/26 20:12:41 - mmengine - INFO - Epoch(train) [179][200/293] lr: 5.000000e-05 eta: 1:18:27 time: 0.570829 data_time: 0.081219 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.899720 loss: 0.000519 2022/09/26 20:13:11 - mmengine - INFO - Epoch(train) [179][250/293] lr: 5.000000e-05 eta: 1:18:02 time: 0.586689 data_time: 0.087916 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.855238 loss: 0.000516 2022/09/26 20:13:35 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:14:06 - mmengine - INFO - Epoch(train) [180][50/293] lr: 5.000000e-05 eta: 1:17:12 time: 0.616132 data_time: 0.101435 memory: 9504 loss_kpt: 0.000525 acc_pose: 0.908870 loss: 0.000525 2022/09/26 20:14:35 - mmengine - INFO - Epoch(train) [180][100/293] lr: 5.000000e-05 eta: 1:16:47 time: 0.568226 data_time: 0.080920 memory: 9504 loss_kpt: 0.000520 acc_pose: 0.850673 loss: 0.000520 2022/09/26 20:15:04 - mmengine - INFO - Epoch(train) [180][150/293] lr: 5.000000e-05 eta: 1:16:22 time: 0.591401 data_time: 0.088365 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.893737 loss: 0.000518 2022/09/26 20:15:33 - mmengine - INFO - Epoch(train) [180][200/293] lr: 5.000000e-05 eta: 1:15:57 time: 0.573881 data_time: 0.103735 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.878663 loss: 0.000510 2022/09/26 20:16:03 - mmengine - INFO - Epoch(train) [180][250/293] lr: 5.000000e-05 eta: 1:15:32 time: 0.598061 data_time: 0.093236 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.856740 loss: 0.000511 2022/09/26 20:16:28 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:16:28 - mmengine - INFO - Saving checkpoint at 180 epochs 2022/09/26 20:16:53 - mmengine - INFO - Epoch(val) [180][50/407] eta: 0:02:06 time: 0.353059 data_time: 0.073762 memory: 9504 2022/09/26 20:17:09 - mmengine - INFO - Epoch(val) [180][100/407] eta: 0:01:39 time: 0.324790 data_time: 0.049532 memory: 1378 2022/09/26 20:17:26 - mmengine - INFO - Epoch(val) [180][150/407] eta: 0:01:27 time: 0.339502 data_time: 0.049403 memory: 1378 2022/09/26 20:17:42 - mmengine - INFO - Epoch(val) [180][200/407] eta: 0:01:07 time: 0.326130 data_time: 0.052933 memory: 1378 2022/09/26 20:17:59 - mmengine - INFO - Epoch(val) [180][250/407] eta: 0:00:52 time: 0.331415 data_time: 0.058691 memory: 1378 2022/09/26 20:18:14 - mmengine - INFO - Epoch(val) [180][300/407] eta: 0:00:33 time: 0.313426 data_time: 0.042321 memory: 1378 2022/09/26 20:18:31 - mmengine - INFO - Epoch(val) [180][350/407] eta: 0:00:19 time: 0.339573 data_time: 0.071834 memory: 1378 2022/09/26 20:18:42 - mmengine - INFO - Epoch(val) [180][400/407] eta: 0:00:01 time: 0.201199 data_time: 0.019122 memory: 1378 2022/09/26 20:19:15 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 20:19:28 - mmengine - INFO - Epoch(val) [180][407/407] coco/AP: 0.732818 coco/AP .5: 0.900589 coco/AP .75: 0.810656 coco/AP (M): 0.695253 coco/AP (L): 0.800015 coco/AR: 0.785658 coco/AR .5: 0.938759 coco/AR .75: 0.852960 coco/AR (M): 0.742775 coco/AR (L): 0.847789 2022/09/26 20:19:28 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_170.pth is removed 2022/09/26 20:19:30 - mmengine - INFO - The best checkpoint with 0.7328 coco/AP at 180 epoch is saved to best_coco/AP_epoch_180.pth. 2022/09/26 20:19:58 - mmengine - INFO - Epoch(train) [181][50/293] lr: 5.000000e-05 eta: 1:14:41 time: 0.559827 data_time: 0.210527 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.896881 loss: 0.000516 2022/09/26 20:20:26 - mmengine - INFO - Epoch(train) [181][100/293] lr: 5.000000e-05 eta: 1:14:15 time: 0.560803 data_time: 0.128562 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.887778 loss: 0.000516 2022/09/26 20:20:55 - mmengine - INFO - Epoch(train) [181][150/293] lr: 5.000000e-05 eta: 1:13:50 time: 0.577244 data_time: 0.077643 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.893811 loss: 0.000516 2022/09/26 20:21:25 - mmengine - INFO - Epoch(train) [181][200/293] lr: 5.000000e-05 eta: 1:13:25 time: 0.592759 data_time: 0.085236 memory: 9504 loss_kpt: 0.000526 acc_pose: 0.859355 loss: 0.000526 2022/09/26 20:21:54 - mmengine - INFO - Epoch(train) [181][250/293] lr: 5.000000e-05 eta: 1:13:00 time: 0.588119 data_time: 0.068102 memory: 9504 loss_kpt: 0.000523 acc_pose: 0.828976 loss: 0.000523 2022/09/26 20:22:00 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:22:19 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:22:49 - mmengine - INFO - Epoch(train) [182][50/293] lr: 5.000000e-05 eta: 1:12:10 time: 0.606665 data_time: 0.093694 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.835252 loss: 0.000519 2022/09/26 20:23:18 - mmengine - INFO - Epoch(train) [182][100/293] lr: 5.000000e-05 eta: 1:11:45 time: 0.569130 data_time: 0.077318 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.898163 loss: 0.000512 2022/09/26 20:23:47 - mmengine - INFO - Epoch(train) [182][150/293] lr: 5.000000e-05 eta: 1:11:20 time: 0.585823 data_time: 0.089885 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.859802 loss: 0.000511 2022/09/26 20:24:16 - mmengine - INFO - Epoch(train) [182][200/293] lr: 5.000000e-05 eta: 1:10:54 time: 0.573776 data_time: 0.074905 memory: 9504 loss_kpt: 0.000520 acc_pose: 0.853128 loss: 0.000520 2022/09/26 20:24:46 - mmengine - INFO - Epoch(train) [182][250/293] lr: 5.000000e-05 eta: 1:10:29 time: 0.601634 data_time: 0.079670 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.915200 loss: 0.000518 2022/09/26 20:25:11 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:25:41 - mmengine - INFO - Epoch(train) [183][50/293] lr: 5.000000e-05 eta: 1:09:39 time: 0.595240 data_time: 0.109004 memory: 9504 loss_kpt: 0.000508 acc_pose: 0.837434 loss: 0.000508 2022/09/26 20:26:11 - mmengine - INFO - Epoch(train) [183][100/293] lr: 5.000000e-05 eta: 1:09:14 time: 0.602783 data_time: 0.094363 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.880357 loss: 0.000517 2022/09/26 20:26:40 - mmengine - INFO - Epoch(train) [183][150/293] lr: 5.000000e-05 eta: 1:08:49 time: 0.589272 data_time: 0.091639 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.839755 loss: 0.000527 2022/09/26 20:27:09 - mmengine - INFO - Epoch(train) [183][200/293] lr: 5.000000e-05 eta: 1:08:24 time: 0.573539 data_time: 0.111683 memory: 9504 loss_kpt: 0.000520 acc_pose: 0.900622 loss: 0.000520 2022/09/26 20:27:37 - mmengine - INFO - Epoch(train) [183][250/293] lr: 5.000000e-05 eta: 1:07:59 time: 0.562156 data_time: 0.082621 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.874108 loss: 0.000527 2022/09/26 20:28:02 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:28:33 - mmengine - INFO - Epoch(train) [184][50/293] lr: 5.000000e-05 eta: 1:07:08 time: 0.621805 data_time: 0.110400 memory: 9504 loss_kpt: 0.000522 acc_pose: 0.876220 loss: 0.000522 2022/09/26 20:29:03 - mmengine - INFO - Epoch(train) [184][100/293] lr: 5.000000e-05 eta: 1:06:43 time: 0.599068 data_time: 0.075328 memory: 9504 loss_kpt: 0.000513 acc_pose: 0.841182 loss: 0.000513 2022/09/26 20:29:32 - mmengine - INFO - Epoch(train) [184][150/293] lr: 5.000000e-05 eta: 1:06:18 time: 0.592662 data_time: 0.079939 memory: 9504 loss_kpt: 0.000528 acc_pose: 0.875871 loss: 0.000528 2022/09/26 20:30:01 - mmengine - INFO - Epoch(train) [184][200/293] lr: 5.000000e-05 eta: 1:05:53 time: 0.572416 data_time: 0.092392 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.893522 loss: 0.000516 2022/09/26 20:30:30 - mmengine - INFO - Epoch(train) [184][250/293] lr: 5.000000e-05 eta: 1:05:28 time: 0.585032 data_time: 0.072394 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.890106 loss: 0.000507 2022/09/26 20:30:55 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:31:25 - mmengine - INFO - Epoch(train) [185][50/293] lr: 5.000000e-05 eta: 1:04:38 time: 0.606041 data_time: 0.096262 memory: 9504 loss_kpt: 0.000533 acc_pose: 0.829231 loss: 0.000533 2022/09/26 20:31:47 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:31:54 - mmengine - INFO - Epoch(train) [185][100/293] lr: 5.000000e-05 eta: 1:04:13 time: 0.579566 data_time: 0.070767 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.881698 loss: 0.000514 2022/09/26 20:32:24 - mmengine - INFO - Epoch(train) [185][150/293] lr: 5.000000e-05 eta: 1:03:48 time: 0.606669 data_time: 0.073258 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.885828 loss: 0.000527 2022/09/26 20:32:54 - mmengine - INFO - Epoch(train) [185][200/293] lr: 5.000000e-05 eta: 1:03:23 time: 0.581653 data_time: 0.079159 memory: 9504 loss_kpt: 0.000522 acc_pose: 0.854113 loss: 0.000522 2022/09/26 20:33:22 - mmengine - INFO - Epoch(train) [185][250/293] lr: 5.000000e-05 eta: 1:02:57 time: 0.562865 data_time: 0.063571 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.882738 loss: 0.000527 2022/09/26 20:33:48 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:34:18 - mmengine - INFO - Epoch(train) [186][50/293] lr: 5.000000e-05 eta: 1:02:07 time: 0.607443 data_time: 0.197865 memory: 9504 loss_kpt: 0.000521 acc_pose: 0.872903 loss: 0.000521 2022/09/26 20:34:49 - mmengine - INFO - Epoch(train) [186][100/293] lr: 5.000000e-05 eta: 1:01:42 time: 0.604496 data_time: 0.100937 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.819094 loss: 0.000519 2022/09/26 20:35:17 - mmengine - INFO - Epoch(train) [186][150/293] lr: 5.000000e-05 eta: 1:01:17 time: 0.572874 data_time: 0.111995 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.913823 loss: 0.000509 2022/09/26 20:35:47 - mmengine - INFO - Epoch(train) [186][200/293] lr: 5.000000e-05 eta: 1:00:52 time: 0.589740 data_time: 0.205923 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.881429 loss: 0.000518 2022/09/26 20:36:15 - mmengine - INFO - Epoch(train) [186][250/293] lr: 5.000000e-05 eta: 1:00:27 time: 0.560101 data_time: 0.075419 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.845521 loss: 0.000510 2022/09/26 20:36:40 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:37:10 - mmengine - INFO - Epoch(train) [187][50/293] lr: 5.000000e-05 eta: 0:59:37 time: 0.599757 data_time: 0.098198 memory: 9504 loss_kpt: 0.000504 acc_pose: 0.863505 loss: 0.000504 2022/09/26 20:37:38 - mmengine - INFO - Epoch(train) [187][100/293] lr: 5.000000e-05 eta: 0:59:11 time: 0.561900 data_time: 0.088270 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.838495 loss: 0.000511 2022/09/26 20:38:08 - mmengine - INFO - Epoch(train) [187][150/293] lr: 5.000000e-05 eta: 0:58:46 time: 0.599798 data_time: 0.075584 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.884675 loss: 0.000516 2022/09/26 20:38:38 - mmengine - INFO - Epoch(train) [187][200/293] lr: 5.000000e-05 eta: 0:58:21 time: 0.595532 data_time: 0.093753 memory: 9504 loss_kpt: 0.000524 acc_pose: 0.866108 loss: 0.000524 2022/09/26 20:39:08 - mmengine - INFO - Epoch(train) [187][250/293] lr: 5.000000e-05 eta: 0:57:56 time: 0.600362 data_time: 0.084467 memory: 9504 loss_kpt: 0.000527 acc_pose: 0.820943 loss: 0.000527 2022/09/26 20:39:33 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:40:04 - mmengine - INFO - Epoch(train) [188][50/293] lr: 5.000000e-05 eta: 0:57:06 time: 0.607402 data_time: 0.088165 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.894850 loss: 0.000510 2022/09/26 20:40:34 - mmengine - INFO - Epoch(train) [188][100/293] lr: 5.000000e-05 eta: 0:56:41 time: 0.596906 data_time: 0.082643 memory: 9504 loss_kpt: 0.000515 acc_pose: 0.867158 loss: 0.000515 2022/09/26 20:41:02 - mmengine - INFO - Epoch(train) [188][150/293] lr: 5.000000e-05 eta: 0:56:16 time: 0.570474 data_time: 0.078972 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.872459 loss: 0.000514 2022/09/26 20:41:31 - mmengine - INFO - Epoch(train) [188][200/293] lr: 5.000000e-05 eta: 0:55:51 time: 0.575059 data_time: 0.077330 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.891405 loss: 0.000518 2022/09/26 20:41:36 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:42:01 - mmengine - INFO - Epoch(train) [188][250/293] lr: 5.000000e-05 eta: 0:55:26 time: 0.592176 data_time: 0.083205 memory: 9504 loss_kpt: 0.000499 acc_pose: 0.890590 loss: 0.000499 2022/09/26 20:42:25 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:42:55 - mmengine - INFO - Epoch(train) [189][50/293] lr: 5.000000e-05 eta: 0:54:36 time: 0.601451 data_time: 0.094235 memory: 9504 loss_kpt: 0.000513 acc_pose: 0.861090 loss: 0.000513 2022/09/26 20:43:24 - mmengine - INFO - Epoch(train) [189][100/293] lr: 5.000000e-05 eta: 0:54:11 time: 0.576967 data_time: 0.089119 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.795812 loss: 0.000518 2022/09/26 20:43:53 - mmengine - INFO - Epoch(train) [189][150/293] lr: 5.000000e-05 eta: 0:53:45 time: 0.587727 data_time: 0.078443 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.861548 loss: 0.000512 2022/09/26 20:44:22 - mmengine - INFO - Epoch(train) [189][200/293] lr: 5.000000e-05 eta: 0:53:20 time: 0.584031 data_time: 0.085048 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.861417 loss: 0.000507 2022/09/26 20:44:51 - mmengine - INFO - Epoch(train) [189][250/293] lr: 5.000000e-05 eta: 0:52:55 time: 0.576574 data_time: 0.077188 memory: 9504 loss_kpt: 0.000521 acc_pose: 0.909575 loss: 0.000521 2022/09/26 20:45:17 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:45:48 - mmengine - INFO - Epoch(train) [190][50/293] lr: 5.000000e-05 eta: 0:52:05 time: 0.623671 data_time: 0.090160 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.871912 loss: 0.000512 2022/09/26 20:46:18 - mmengine - INFO - Epoch(train) [190][100/293] lr: 5.000000e-05 eta: 0:51:40 time: 0.589834 data_time: 0.081784 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.825093 loss: 0.000516 2022/09/26 20:46:47 - mmengine - INFO - Epoch(train) [190][150/293] lr: 5.000000e-05 eta: 0:51:15 time: 0.586850 data_time: 0.093531 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.838137 loss: 0.000514 2022/09/26 20:47:17 - mmengine - INFO - Epoch(train) [190][200/293] lr: 5.000000e-05 eta: 0:50:50 time: 0.584999 data_time: 0.080283 memory: 9504 loss_kpt: 0.000515 acc_pose: 0.887073 loss: 0.000515 2022/09/26 20:47:47 - mmengine - INFO - Epoch(train) [190][250/293] lr: 5.000000e-05 eta: 0:50:25 time: 0.600732 data_time: 0.177148 memory: 9504 loss_kpt: 0.000525 acc_pose: 0.858491 loss: 0.000525 2022/09/26 20:48:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:48:12 - mmengine - INFO - Saving checkpoint at 190 epochs 2022/09/26 20:48:38 - mmengine - INFO - Epoch(val) [190][50/407] eta: 0:02:01 time: 0.340238 data_time: 0.089715 memory: 9504 2022/09/26 20:48:54 - mmengine - INFO - Epoch(val) [190][100/407] eta: 0:01:37 time: 0.318903 data_time: 0.050048 memory: 1378 2022/09/26 20:49:10 - mmengine - INFO - Epoch(val) [190][150/407] eta: 0:01:23 time: 0.326362 data_time: 0.071558 memory: 1378 2022/09/26 20:49:26 - mmengine - INFO - Epoch(val) [190][200/407] eta: 0:01:03 time: 0.309059 data_time: 0.031671 memory: 1378 2022/09/26 20:49:42 - mmengine - INFO - Epoch(val) [190][250/407] eta: 0:00:52 time: 0.332711 data_time: 0.062699 memory: 1378 2022/09/26 20:49:58 - mmengine - INFO - Epoch(val) [190][300/407] eta: 0:00:34 time: 0.325770 data_time: 0.042777 memory: 1378 2022/09/26 20:50:15 - mmengine - INFO - Epoch(val) [190][350/407] eta: 0:00:18 time: 0.329460 data_time: 0.073427 memory: 1378 2022/09/26 20:50:26 - mmengine - INFO - Epoch(val) [190][400/407] eta: 0:00:01 time: 0.215246 data_time: 0.023552 memory: 1378 2022/09/26 20:50:59 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 20:51:11 - mmengine - INFO - Epoch(val) [190][407/407] coco/AP: 0.732019 coco/AP .5: 0.899936 coco/AP .75: 0.808937 coco/AP (M): 0.694803 coco/AP (L): 0.799112 coco/AR: 0.785186 coco/AR .5: 0.939074 coco/AR .75: 0.851385 coco/AR (M): 0.742884 coco/AR (L): 0.846414 2022/09/26 20:51:43 - mmengine - INFO - Epoch(train) [191][50/293] lr: 5.000000e-05 eta: 0:49:35 time: 0.627068 data_time: 0.106068 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.817682 loss: 0.000511 2022/09/26 20:52:12 - mmengine - INFO - Epoch(train) [191][100/293] lr: 5.000000e-05 eta: 0:49:10 time: 0.586097 data_time: 0.087496 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.855319 loss: 0.000518 2022/09/26 20:52:41 - mmengine - INFO - Epoch(train) [191][150/293] lr: 5.000000e-05 eta: 0:48:45 time: 0.582349 data_time: 0.097854 memory: 9504 loss_kpt: 0.000528 acc_pose: 0.875713 loss: 0.000528 2022/09/26 20:53:09 - mmengine - INFO - Epoch(train) [191][200/293] lr: 5.000000e-05 eta: 0:48:19 time: 0.562322 data_time: 0.157392 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.899614 loss: 0.000517 2022/09/26 20:53:39 - mmengine - INFO - Epoch(train) [191][250/293] lr: 5.000000e-05 eta: 0:47:54 time: 0.589391 data_time: 0.090039 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.906087 loss: 0.000518 2022/09/26 20:54:03 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:54:26 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:54:34 - mmengine - INFO - Epoch(train) [192][50/293] lr: 5.000000e-05 eta: 0:47:05 time: 0.608920 data_time: 0.087732 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.879188 loss: 0.000514 2022/09/26 20:55:03 - mmengine - INFO - Epoch(train) [192][100/293] lr: 5.000000e-05 eta: 0:46:40 time: 0.575967 data_time: 0.078853 memory: 9504 loss_kpt: 0.000501 acc_pose: 0.871153 loss: 0.000501 2022/09/26 20:55:33 - mmengine - INFO - Epoch(train) [192][150/293] lr: 5.000000e-05 eta: 0:46:14 time: 0.602509 data_time: 0.091806 memory: 9504 loss_kpt: 0.000505 acc_pose: 0.849272 loss: 0.000505 2022/09/26 20:56:02 - mmengine - INFO - Epoch(train) [192][200/293] lr: 5.000000e-05 eta: 0:45:49 time: 0.590994 data_time: 0.075409 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.874230 loss: 0.000510 2022/09/26 20:56:31 - mmengine - INFO - Epoch(train) [192][250/293] lr: 5.000000e-05 eta: 0:45:24 time: 0.563340 data_time: 0.085637 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.902635 loss: 0.000516 2022/09/26 20:56:56 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 20:57:27 - mmengine - INFO - Epoch(train) [193][50/293] lr: 5.000000e-05 eta: 0:44:35 time: 0.609248 data_time: 0.093626 memory: 9504 loss_kpt: 0.000523 acc_pose: 0.868698 loss: 0.000523 2022/09/26 20:57:56 - mmengine - INFO - Epoch(train) [193][100/293] lr: 5.000000e-05 eta: 0:44:09 time: 0.595664 data_time: 0.090198 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.867505 loss: 0.000514 2022/09/26 20:58:25 - mmengine - INFO - Epoch(train) [193][150/293] lr: 5.000000e-05 eta: 0:43:44 time: 0.578253 data_time: 0.089800 memory: 9504 loss_kpt: 0.000524 acc_pose: 0.868470 loss: 0.000524 2022/09/26 20:58:54 - mmengine - INFO - Epoch(train) [193][200/293] lr: 5.000000e-05 eta: 0:43:19 time: 0.581643 data_time: 0.075673 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.872970 loss: 0.000518 2022/09/26 20:59:26 - mmengine - INFO - Epoch(train) [193][250/293] lr: 5.000000e-05 eta: 0:42:54 time: 0.626634 data_time: 0.105706 memory: 9504 loss_kpt: 0.000525 acc_pose: 0.865653 loss: 0.000525 2022/09/26 20:59:49 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:00:18 - mmengine - INFO - Epoch(train) [194][50/293] lr: 5.000000e-05 eta: 0:42:04 time: 0.589512 data_time: 0.117481 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.842881 loss: 0.000509 2022/09/26 21:00:48 - mmengine - INFO - Epoch(train) [194][100/293] lr: 5.000000e-05 eta: 0:41:39 time: 0.585191 data_time: 0.074463 memory: 9504 loss_kpt: 0.000513 acc_pose: 0.872905 loss: 0.000513 2022/09/26 21:01:17 - mmengine - INFO - Epoch(train) [194][150/293] lr: 5.000000e-05 eta: 0:41:14 time: 0.578816 data_time: 0.082139 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.829719 loss: 0.000516 2022/09/26 21:01:45 - mmengine - INFO - Epoch(train) [194][200/293] lr: 5.000000e-05 eta: 0:40:48 time: 0.568098 data_time: 0.082373 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.844020 loss: 0.000512 2022/09/26 21:02:14 - mmengine - INFO - Epoch(train) [194][250/293] lr: 5.000000e-05 eta: 0:40:23 time: 0.574366 data_time: 0.076913 memory: 9504 loss_kpt: 0.000506 acc_pose: 0.843901 loss: 0.000506 2022/09/26 21:02:39 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:03:10 - mmengine - INFO - Epoch(train) [195][50/293] lr: 5.000000e-05 eta: 0:39:34 time: 0.629139 data_time: 0.090344 memory: 9504 loss_kpt: 0.000502 acc_pose: 0.902841 loss: 0.000502 2022/09/26 21:03:39 - mmengine - INFO - Epoch(train) [195][100/293] lr: 5.000000e-05 eta: 0:39:09 time: 0.580359 data_time: 0.076667 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.770071 loss: 0.000511 2022/09/26 21:04:08 - mmengine - INFO - Epoch(train) [195][150/293] lr: 5.000000e-05 eta: 0:38:43 time: 0.570895 data_time: 0.083683 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.866213 loss: 0.000519 2022/09/26 21:04:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:04:38 - mmengine - INFO - Epoch(train) [195][200/293] lr: 5.000000e-05 eta: 0:38:18 time: 0.597795 data_time: 0.077412 memory: 9504 loss_kpt: 0.000505 acc_pose: 0.898290 loss: 0.000505 2022/09/26 21:05:07 - mmengine - INFO - Epoch(train) [195][250/293] lr: 5.000000e-05 eta: 0:37:53 time: 0.584344 data_time: 0.085860 memory: 9504 loss_kpt: 0.000513 acc_pose: 0.847164 loss: 0.000513 2022/09/26 21:05:32 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:06:03 - mmengine - INFO - Epoch(train) [196][50/293] lr: 5.000000e-05 eta: 0:37:04 time: 0.623722 data_time: 0.102006 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.873422 loss: 0.000514 2022/09/26 21:06:32 - mmengine - INFO - Epoch(train) [196][100/293] lr: 5.000000e-05 eta: 0:36:39 time: 0.570789 data_time: 0.109722 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.876435 loss: 0.000514 2022/09/26 21:07:00 - mmengine - INFO - Epoch(train) [196][150/293] lr: 5.000000e-05 eta: 0:36:13 time: 0.568110 data_time: 0.102266 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.854846 loss: 0.000509 2022/09/26 21:07:28 - mmengine - INFO - Epoch(train) [196][200/293] lr: 5.000000e-05 eta: 0:35:48 time: 0.562658 data_time: 0.085136 memory: 9504 loss_kpt: 0.000503 acc_pose: 0.865682 loss: 0.000503 2022/09/26 21:07:57 - mmengine - INFO - Epoch(train) [196][250/293] lr: 5.000000e-05 eta: 0:35:22 time: 0.587256 data_time: 0.077442 memory: 9504 loss_kpt: 0.000499 acc_pose: 0.863610 loss: 0.000499 2022/09/26 21:08:22 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:08:53 - mmengine - INFO - Epoch(train) [197][50/293] lr: 5.000000e-05 eta: 0:34:34 time: 0.624424 data_time: 0.118331 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.901933 loss: 0.000509 2022/09/26 21:09:22 - mmengine - INFO - Epoch(train) [197][100/293] lr: 5.000000e-05 eta: 0:34:08 time: 0.577383 data_time: 0.075396 memory: 9504 loss_kpt: 0.000502 acc_pose: 0.876867 loss: 0.000502 2022/09/26 21:09:51 - mmengine - INFO - Epoch(train) [197][150/293] lr: 5.000000e-05 eta: 0:33:43 time: 0.593961 data_time: 0.090590 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.865612 loss: 0.000507 2022/09/26 21:10:21 - mmengine - INFO - Epoch(train) [197][200/293] lr: 5.000000e-05 eta: 0:33:18 time: 0.594906 data_time: 0.101433 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.866965 loss: 0.000509 2022/09/26 21:10:52 - mmengine - INFO - Epoch(train) [197][250/293] lr: 5.000000e-05 eta: 0:32:52 time: 0.608305 data_time: 0.088797 memory: 9504 loss_kpt: 0.000508 acc_pose: 0.855466 loss: 0.000508 2022/09/26 21:11:16 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:11:46 - mmengine - INFO - Epoch(train) [198][50/293] lr: 5.000000e-05 eta: 0:32:04 time: 0.596916 data_time: 0.118154 memory: 9504 loss_kpt: 0.000514 acc_pose: 0.866426 loss: 0.000514 2022/09/26 21:12:14 - mmengine - INFO - Epoch(train) [198][100/293] lr: 5.000000e-05 eta: 0:31:38 time: 0.572702 data_time: 0.079732 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.873558 loss: 0.000517 2022/09/26 21:12:44 - mmengine - INFO - Epoch(train) [198][150/293] lr: 5.000000e-05 eta: 0:31:13 time: 0.591024 data_time: 0.088613 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.851810 loss: 0.000509 2022/09/26 21:13:13 - mmengine - INFO - Epoch(train) [198][200/293] lr: 5.000000e-05 eta: 0:30:47 time: 0.584936 data_time: 0.082949 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.873541 loss: 0.000518 2022/09/26 21:13:42 - mmengine - INFO - Epoch(train) [198][250/293] lr: 5.000000e-05 eta: 0:30:22 time: 0.572301 data_time: 0.072489 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.832684 loss: 0.000517 2022/09/26 21:13:58 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:14:06 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:14:36 - mmengine - INFO - Epoch(train) [199][50/293] lr: 5.000000e-05 eta: 0:29:33 time: 0.609730 data_time: 0.091557 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.873581 loss: 0.000517 2022/09/26 21:15:06 - mmengine - INFO - Epoch(train) [199][100/293] lr: 5.000000e-05 eta: 0:29:08 time: 0.595807 data_time: 0.090212 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.825427 loss: 0.000507 2022/09/26 21:15:36 - mmengine - INFO - Epoch(train) [199][150/293] lr: 5.000000e-05 eta: 0:28:43 time: 0.591459 data_time: 0.096991 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.872982 loss: 0.000507 2022/09/26 21:16:04 - mmengine - INFO - Epoch(train) [199][200/293] lr: 5.000000e-05 eta: 0:28:17 time: 0.570408 data_time: 0.082775 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.900159 loss: 0.000518 2022/09/26 21:16:34 - mmengine - INFO - Epoch(train) [199][250/293] lr: 5.000000e-05 eta: 0:27:52 time: 0.585895 data_time: 0.087810 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.867686 loss: 0.000519 2022/09/26 21:16:59 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:17:30 - mmengine - INFO - Epoch(train) [200][50/293] lr: 5.000000e-05 eta: 0:27:03 time: 0.623137 data_time: 0.104342 memory: 9504 loss_kpt: 0.000519 acc_pose: 0.883600 loss: 0.000519 2022/09/26 21:18:00 - mmengine - INFO - Epoch(train) [200][100/293] lr: 5.000000e-05 eta: 0:26:38 time: 0.587855 data_time: 0.091304 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.861765 loss: 0.000507 2022/09/26 21:18:29 - mmengine - INFO - Epoch(train) [200][150/293] lr: 5.000000e-05 eta: 0:26:13 time: 0.589259 data_time: 0.084411 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.861490 loss: 0.000511 2022/09/26 21:18:58 - mmengine - INFO - Epoch(train) [200][200/293] lr: 5.000000e-05 eta: 0:25:47 time: 0.571637 data_time: 0.108184 memory: 9504 loss_kpt: 0.000504 acc_pose: 0.847585 loss: 0.000504 2022/09/26 21:19:28 - mmengine - INFO - Epoch(train) [200][250/293] lr: 5.000000e-05 eta: 0:25:22 time: 0.596030 data_time: 0.089286 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.875165 loss: 0.000517 2022/09/26 21:19:54 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:19:54 - mmengine - INFO - Saving checkpoint at 200 epochs 2022/09/26 21:20:18 - mmengine - INFO - Epoch(val) [200][50/407] eta: 0:02:01 time: 0.341493 data_time: 0.069009 memory: 9504 2022/09/26 21:20:34 - mmengine - INFO - Epoch(val) [200][100/407] eta: 0:01:38 time: 0.321043 data_time: 0.048498 memory: 1378 2022/09/26 21:20:50 - mmengine - INFO - Epoch(val) [200][150/407] eta: 0:01:22 time: 0.319446 data_time: 0.044647 memory: 1378 2022/09/26 21:21:05 - mmengine - INFO - Epoch(val) [200][200/407] eta: 0:01:03 time: 0.305444 data_time: 0.054240 memory: 1378 2022/09/26 21:21:22 - mmengine - INFO - Epoch(val) [200][250/407] eta: 0:00:51 time: 0.328500 data_time: 0.066083 memory: 1378 2022/09/26 21:21:38 - mmengine - INFO - Epoch(val) [200][300/407] eta: 0:00:33 time: 0.315163 data_time: 0.047166 memory: 1378 2022/09/26 21:21:54 - mmengine - INFO - Epoch(val) [200][350/407] eta: 0:00:18 time: 0.332124 data_time: 0.046973 memory: 1378 2022/09/26 21:22:07 - mmengine - INFO - Epoch(val) [200][400/407] eta: 0:00:01 time: 0.250670 data_time: 0.046213 memory: 1378 2022/09/26 21:22:39 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 21:22:52 - mmengine - INFO - Epoch(val) [200][407/407] coco/AP: 0.732005 coco/AP .5: 0.899801 coco/AP .75: 0.809002 coco/AP (M): 0.694087 coco/AP (L): 0.801088 coco/AR: 0.785107 coco/AR .5: 0.938130 coco/AR .75: 0.853275 coco/AR (M): 0.741437 coco/AR (L): 0.848235 2022/09/26 21:23:22 - mmengine - INFO - Epoch(train) [201][50/293] lr: 5.000000e-06 eta: 0:24:33 time: 0.598748 data_time: 0.138621 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.862526 loss: 0.000510 2022/09/26 21:23:52 - mmengine - INFO - Epoch(train) [201][100/293] lr: 5.000000e-06 eta: 0:24:08 time: 0.599319 data_time: 0.074396 memory: 9504 loss_kpt: 0.000513 acc_pose: 0.862256 loss: 0.000513 2022/09/26 21:24:21 - mmengine - INFO - Epoch(train) [201][150/293] lr: 5.000000e-06 eta: 0:23:42 time: 0.580303 data_time: 0.082432 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.866824 loss: 0.000517 2022/09/26 21:24:51 - mmengine - INFO - Epoch(train) [201][200/293] lr: 5.000000e-06 eta: 0:23:17 time: 0.583359 data_time: 0.079307 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.889673 loss: 0.000510 2022/09/26 21:25:19 - mmengine - INFO - Epoch(train) [201][250/293] lr: 5.000000e-06 eta: 0:22:52 time: 0.574315 data_time: 0.085414 memory: 9504 loss_kpt: 0.000506 acc_pose: 0.894414 loss: 0.000506 2022/09/26 21:25:43 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:26:14 - mmengine - INFO - Epoch(train) [202][50/293] lr: 5.000000e-06 eta: 0:22:03 time: 0.617992 data_time: 0.116736 memory: 9504 loss_kpt: 0.000506 acc_pose: 0.851637 loss: 0.000506 2022/09/26 21:26:45 - mmengine - INFO - Epoch(train) [202][100/293] lr: 5.000000e-06 eta: 0:21:38 time: 0.613556 data_time: 0.243989 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.881909 loss: 0.000507 2022/09/26 21:26:49 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:27:13 - mmengine - INFO - Epoch(train) [202][150/293] lr: 5.000000e-06 eta: 0:21:12 time: 0.569970 data_time: 0.123623 memory: 9504 loss_kpt: 0.000491 acc_pose: 0.849664 loss: 0.000491 2022/09/26 21:27:42 - mmengine - INFO - Epoch(train) [202][200/293] lr: 5.000000e-06 eta: 0:20:47 time: 0.579063 data_time: 0.068735 memory: 9504 loss_kpt: 0.000508 acc_pose: 0.857891 loss: 0.000508 2022/09/26 21:28:11 - mmengine - INFO - Epoch(train) [202][250/293] lr: 5.000000e-06 eta: 0:20:21 time: 0.574697 data_time: 0.111739 memory: 9504 loss_kpt: 0.000520 acc_pose: 0.856703 loss: 0.000520 2022/09/26 21:28:36 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:29:07 - mmengine - INFO - Epoch(train) [203][50/293] lr: 5.000000e-06 eta: 0:19:33 time: 0.618998 data_time: 0.089893 memory: 9504 loss_kpt: 0.000508 acc_pose: 0.858443 loss: 0.000508 2022/09/26 21:29:37 - mmengine - INFO - Epoch(train) [203][100/293] lr: 5.000000e-06 eta: 0:19:08 time: 0.600049 data_time: 0.098828 memory: 9504 loss_kpt: 0.000515 acc_pose: 0.880374 loss: 0.000515 2022/09/26 21:30:04 - mmengine - INFO - Epoch(train) [203][150/293] lr: 5.000000e-06 eta: 0:18:42 time: 0.558664 data_time: 0.085266 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.867798 loss: 0.000512 2022/09/26 21:30:34 - mmengine - INFO - Epoch(train) [203][200/293] lr: 5.000000e-06 eta: 0:18:17 time: 0.587418 data_time: 0.094541 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.838614 loss: 0.000511 2022/09/26 21:31:04 - mmengine - INFO - Epoch(train) [203][250/293] lr: 5.000000e-06 eta: 0:17:51 time: 0.597443 data_time: 0.096042 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.867953 loss: 0.000507 2022/09/26 21:31:28 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:31:58 - mmengine - INFO - Epoch(train) [204][50/293] lr: 5.000000e-06 eta: 0:17:03 time: 0.599835 data_time: 0.142174 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.835321 loss: 0.000512 2022/09/26 21:32:27 - mmengine - INFO - Epoch(train) [204][100/293] lr: 5.000000e-06 eta: 0:16:38 time: 0.583372 data_time: 0.092635 memory: 9504 loss_kpt: 0.000513 acc_pose: 0.835754 loss: 0.000513 2022/09/26 21:32:56 - mmengine - INFO - Epoch(train) [204][150/293] lr: 5.000000e-06 eta: 0:16:12 time: 0.571308 data_time: 0.103081 memory: 9504 loss_kpt: 0.000502 acc_pose: 0.886903 loss: 0.000502 2022/09/26 21:33:25 - mmengine - INFO - Epoch(train) [204][200/293] lr: 5.000000e-06 eta: 0:15:47 time: 0.573168 data_time: 0.072255 memory: 9504 loss_kpt: 0.000494 acc_pose: 0.878488 loss: 0.000494 2022/09/26 21:33:53 - mmengine - INFO - Epoch(train) [204][250/293] lr: 5.000000e-06 eta: 0:15:21 time: 0.569166 data_time: 0.091291 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.869458 loss: 0.000516 2022/09/26 21:34:17 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:34:48 - mmengine - INFO - Epoch(train) [205][50/293] lr: 5.000000e-06 eta: 0:14:33 time: 0.615533 data_time: 0.215253 memory: 9504 loss_kpt: 0.000510 acc_pose: 0.866436 loss: 0.000510 2022/09/26 21:35:18 - mmengine - INFO - Epoch(train) [205][100/293] lr: 5.000000e-06 eta: 0:14:08 time: 0.587552 data_time: 0.078389 memory: 9504 loss_kpt: 0.000504 acc_pose: 0.899692 loss: 0.000504 2022/09/26 21:35:47 - mmengine - INFO - Epoch(train) [205][150/293] lr: 5.000000e-06 eta: 0:13:42 time: 0.579944 data_time: 0.083918 memory: 9504 loss_kpt: 0.000503 acc_pose: 0.862982 loss: 0.000503 2022/09/26 21:36:17 - mmengine - INFO - Epoch(train) [205][200/293] lr: 5.000000e-06 eta: 0:13:17 time: 0.609787 data_time: 0.094260 memory: 9504 loss_kpt: 0.000507 acc_pose: 0.865491 loss: 0.000507 2022/09/26 21:36:34 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:36:47 - mmengine - INFO - Epoch(train) [205][250/293] lr: 5.000000e-06 eta: 0:12:51 time: 0.596629 data_time: 0.135098 memory: 9504 loss_kpt: 0.000500 acc_pose: 0.887042 loss: 0.000500 2022/09/26 21:37:12 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:37:42 - mmengine - INFO - Epoch(train) [206][50/293] lr: 5.000000e-06 eta: 0:12:03 time: 0.612090 data_time: 0.094864 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.889873 loss: 0.000511 2022/09/26 21:38:12 - mmengine - INFO - Epoch(train) [206][100/293] lr: 5.000000e-06 eta: 0:11:38 time: 0.600835 data_time: 0.126793 memory: 9504 loss_kpt: 0.000495 acc_pose: 0.878443 loss: 0.000495 2022/09/26 21:38:41 - mmengine - INFO - Epoch(train) [206][150/293] lr: 5.000000e-06 eta: 0:11:12 time: 0.579514 data_time: 0.077114 memory: 9504 loss_kpt: 0.000509 acc_pose: 0.868105 loss: 0.000509 2022/09/26 21:39:11 - mmengine - INFO - Epoch(train) [206][200/293] lr: 5.000000e-06 eta: 0:10:47 time: 0.590257 data_time: 0.083972 memory: 9504 loss_kpt: 0.000504 acc_pose: 0.859345 loss: 0.000504 2022/09/26 21:39:40 - mmengine - INFO - Epoch(train) [206][250/293] lr: 5.000000e-06 eta: 0:10:21 time: 0.581064 data_time: 0.074087 memory: 9504 loss_kpt: 0.000498 acc_pose: 0.857129 loss: 0.000498 2022/09/26 21:40:04 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:40:34 - mmengine - INFO - Epoch(train) [207][50/293] lr: 5.000000e-06 eta: 0:09:33 time: 0.597947 data_time: 0.105142 memory: 9504 loss_kpt: 0.000508 acc_pose: 0.861410 loss: 0.000508 2022/09/26 21:41:05 - mmengine - INFO - Epoch(train) [207][100/293] lr: 5.000000e-06 eta: 0:09:08 time: 0.608961 data_time: 0.089897 memory: 9504 loss_kpt: 0.000503 acc_pose: 0.884968 loss: 0.000503 2022/09/26 21:41:34 - mmengine - INFO - Epoch(train) [207][150/293] lr: 5.000000e-06 eta: 0:08:42 time: 0.587669 data_time: 0.084664 memory: 9504 loss_kpt: 0.000502 acc_pose: 0.919002 loss: 0.000502 2022/09/26 21:42:03 - mmengine - INFO - Epoch(train) [207][200/293] lr: 5.000000e-06 eta: 0:08:17 time: 0.576223 data_time: 0.091386 memory: 9504 loss_kpt: 0.000518 acc_pose: 0.884655 loss: 0.000518 2022/09/26 21:42:32 - mmengine - INFO - Epoch(train) [207][250/293] lr: 5.000000e-06 eta: 0:07:51 time: 0.588040 data_time: 0.081535 memory: 9504 loss_kpt: 0.000517 acc_pose: 0.886316 loss: 0.000517 2022/09/26 21:42:58 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:43:29 - mmengine - INFO - Epoch(train) [208][50/293] lr: 5.000000e-06 eta: 0:07:04 time: 0.626000 data_time: 0.093609 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.900121 loss: 0.000516 2022/09/26 21:43:58 - mmengine - INFO - Epoch(train) [208][100/293] lr: 5.000000e-06 eta: 0:06:38 time: 0.578248 data_time: 0.084689 memory: 9504 loss_kpt: 0.000506 acc_pose: 0.882951 loss: 0.000506 2022/09/26 21:44:28 - mmengine - INFO - Epoch(train) [208][150/293] lr: 5.000000e-06 eta: 0:06:12 time: 0.606924 data_time: 0.119926 memory: 9504 loss_kpt: 0.000520 acc_pose: 0.874944 loss: 0.000520 2022/09/26 21:44:58 - mmengine - INFO - Epoch(train) [208][200/293] lr: 5.000000e-06 eta: 0:05:47 time: 0.592377 data_time: 0.074597 memory: 9504 loss_kpt: 0.000505 acc_pose: 0.867803 loss: 0.000505 2022/09/26 21:45:28 - mmengine - INFO - Epoch(train) [208][250/293] lr: 5.000000e-06 eta: 0:05:21 time: 0.602272 data_time: 0.080052 memory: 9504 loss_kpt: 0.000503 acc_pose: 0.902807 loss: 0.000503 2022/09/26 21:45:53 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:46:24 - mmengine - INFO - Epoch(train) [209][50/293] lr: 5.000000e-06 eta: 0:04:34 time: 0.612817 data_time: 0.105801 memory: 9504 loss_kpt: 0.000501 acc_pose: 0.847589 loss: 0.000501 2022/09/26 21:46:27 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:46:53 - mmengine - INFO - Epoch(train) [209][100/293] lr: 5.000000e-06 eta: 0:04:08 time: 0.586580 data_time: 0.085713 memory: 9504 loss_kpt: 0.000505 acc_pose: 0.899048 loss: 0.000505 2022/09/26 21:47:21 - mmengine - INFO - Epoch(train) [209][150/293] lr: 5.000000e-06 eta: 0:03:43 time: 0.552026 data_time: 0.088802 memory: 9504 loss_kpt: 0.000498 acc_pose: 0.850862 loss: 0.000498 2022/09/26 21:47:50 - mmengine - INFO - Epoch(train) [209][200/293] lr: 5.000000e-06 eta: 0:03:17 time: 0.581184 data_time: 0.088431 memory: 9504 loss_kpt: 0.000496 acc_pose: 0.886877 loss: 0.000496 2022/09/26 21:48:19 - mmengine - INFO - Epoch(train) [209][250/293] lr: 5.000000e-06 eta: 0:02:51 time: 0.591452 data_time: 0.085068 memory: 9504 loss_kpt: 0.000516 acc_pose: 0.834989 loss: 0.000516 2022/09/26 21:48:45 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:49:15 - mmengine - INFO - Epoch(train) [210][50/293] lr: 5.000000e-06 eta: 0:02:04 time: 0.600305 data_time: 0.115692 memory: 9504 loss_kpt: 0.000502 acc_pose: 0.830064 loss: 0.000502 2022/09/26 21:49:44 - mmengine - INFO - Epoch(train) [210][100/293] lr: 5.000000e-06 eta: 0:01:38 time: 0.587453 data_time: 0.132458 memory: 9504 loss_kpt: 0.000511 acc_pose: 0.855677 loss: 0.000511 2022/09/26 21:50:14 - mmengine - INFO - Epoch(train) [210][150/293] lr: 5.000000e-06 eta: 0:01:13 time: 0.590248 data_time: 0.084068 memory: 9504 loss_kpt: 0.000505 acc_pose: 0.872825 loss: 0.000505 2022/09/26 21:50:43 - mmengine - INFO - Epoch(train) [210][200/293] lr: 5.000000e-06 eta: 0:00:47 time: 0.596479 data_time: 0.088266 memory: 9504 loss_kpt: 0.000503 acc_pose: 0.908415 loss: 0.000503 2022/09/26 21:51:13 - mmengine - INFO - Epoch(train) [210][250/293] lr: 5.000000e-06 eta: 0:00:22 time: 0.590985 data_time: 0.087166 memory: 9504 loss_kpt: 0.000512 acc_pose: 0.859731 loss: 0.000512 2022/09/26 21:51:38 - mmengine - INFO - Exp name: td-hm_res101_dark-8xb64-210e_coco-256x192_20220926_103758 2022/09/26 21:51:38 - mmengine - INFO - Saving checkpoint at 210 epochs 2022/09/26 21:52:02 - mmengine - INFO - Epoch(val) [210][50/407] eta: 0:02:08 time: 0.360704 data_time: 0.100298 memory: 9504 2022/09/26 21:52:18 - mmengine - INFO - Epoch(val) [210][100/407] eta: 0:01:38 time: 0.321171 data_time: 0.075384 memory: 1378 2022/09/26 21:52:36 - mmengine - INFO - Epoch(val) [210][150/407] eta: 0:01:30 time: 0.350500 data_time: 0.099174 memory: 1378 2022/09/26 21:52:52 - mmengine - INFO - Epoch(val) [210][200/407] eta: 0:01:06 time: 0.319924 data_time: 0.057239 memory: 1378 2022/09/26 21:53:08 - mmengine - INFO - Epoch(val) [210][250/407] eta: 0:00:51 time: 0.325838 data_time: 0.066297 memory: 1378 2022/09/26 21:53:24 - mmengine - INFO - Epoch(val) [210][300/407] eta: 0:00:34 time: 0.318244 data_time: 0.048218 memory: 1378 2022/09/26 21:53:41 - mmengine - INFO - Epoch(val) [210][350/407] eta: 0:00:19 time: 0.334042 data_time: 0.088160 memory: 1378 2022/09/26 21:53:54 - mmengine - INFO - Epoch(val) [210][400/407] eta: 0:00:01 time: 0.260479 data_time: 0.055077 memory: 1378 2022/09/26 21:54:26 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 21:54:39 - mmengine - INFO - Epoch(val) [210][407/407] coco/AP: 0.733493 coco/AP .5: 0.900247 coco/AP .75: 0.810136 coco/AP (M): 0.695229 coco/AP (L): 0.802446 coco/AR: 0.786241 coco/AR .5: 0.937815 coco/AR .75: 0.853432 coco/AR (M): 0.742338 coco/AR (L): 0.849796 2022/09/26 21:54:39 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220926/res101_dark_256/best_coco/AP_epoch_180.pth is removed 2022/09/26 21:54:42 - mmengine - INFO - The best checkpoint with 0.7335 coco/AP at 210 epoch is saved to best_coco/AP_epoch_210.pth.