2022/09/23 12:18:50 - mmengine - INFO - ------------------------------------------------------------ System environment: sys.platform: linux Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0] CUDA available: True numpy_random_seed: 880373564 GPU 0,1,2,3,4,5,6,7: NVIDIA A100-SXM4-80GB CUDA_HOME: /mnt/lustre/share/cuda-11.3 NVCC: Cuda compilation tools, release 11.3, V11.3.109 GCC: gcc (GCC) 5.4.0 PyTorch: 1.12.1 PyTorch compiling details: PyTorch built with: - GCC 9.3 - C++ Version: 201402 - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815) - OpenMP 201511 (a.k.a. OpenMP 4.5) - LAPACK is enabled (usually provided by MKL) - NNPACK is enabled - CPU capability usage: AVX2 - CUDA Runtime 11.3 - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37 - CuDNN 8.3.2 (built against CUDA 11.5) - Magma 2.5.2 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, TorchVision: 0.13.1 OpenCV: 4.6.0 MMEngine: 0.1.0 Runtime environment: cudnn_benchmark: False mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: None Distributed launcher: slurm Distributed training: True GPU number: 8 ------------------------------------------------------------ 2022/09/23 12:18:51 - mmengine - INFO - Config: default_scope = 'mmpose' default_hooks = dict( timer=dict(type='IterTimerHook'), logger=dict(type='LoggerHook', interval=50), param_scheduler=dict(type='ParamSchedulerHook'), checkpoint=dict( type='CheckpointHook', interval=10, max_keep_ckpts=1, save_best='coco/AP', rule='greater'), sampler_seed=dict(type='DistSamplerSeedHook'), visualization=dict(type='PoseVisualizationHook', enable=False)) custom_hooks = [dict(type='SyncBuffersHook')] env_cfg = dict( cudnn_benchmark=False, mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0), dist_cfg=dict(backend='nccl')) vis_backends = [dict(type='LocalVisBackend')] visualizer = dict( type='PoseLocalVisualizer', vis_backends=[dict(type='LocalVisBackend')], name='visualizer') log_processor = dict( type='LogProcessor', window_size=50, by_epoch=True, num_digits=6) log_level = 'INFO' load_from = None resume = False file_client_args = dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' })) train_cfg = dict(by_epoch=True, max_epochs=210, val_interval=10) val_cfg = dict() test_cfg = dict() optim_wrapper = dict(optimizer=dict(type='Adam', lr=0.0005)) param_scheduler = [ dict( type='LinearLR', begin=0, end=500, start_factor=0.001, by_epoch=False), dict( type='MultiStepLR', begin=0, end=210, milestones=[170, 200], gamma=0.1, by_epoch=True) ] auto_scale_lr = dict(base_batch_size=512) codec = dict( type='MSRAHeatmap', input_size=(288, 384), heatmap_size=(72, 96), sigma=3, unbiased=True) model = dict( type='TopdownPoseEstimator', data_preprocessor=dict( type='PoseDataPreprocessor', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], bgr_to_rgb=True), backbone=dict( type='ResNet', depth=50, init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')), head=dict( type='HeatmapHead', in_channels=2048, out_channels=17, loss=dict(type='KeypointMSELoss', use_target_weight=True), decoder=dict( type='MSRAHeatmap', input_size=(288, 384), heatmap_size=(72, 96), sigma=3, unbiased=True)), test_cfg=dict(flip_test=True, flip_mode='heatmap', shift_heatmap=True)) dataset_type = 'CocoDataset' data_mode = 'topdown' data_root = 'data/coco/' train_pipeline = [ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='RandomFlip', direction='horizontal'), dict(type='RandomHalfBody'), dict(type='RandomBBoxTransform'), dict(type='TopdownAffine', input_size=(288, 384)), dict( type='GenerateTarget', target_type='heatmap', encoder=dict( type='MSRAHeatmap', input_size=(288, 384), heatmap_size=(72, 96), sigma=3, unbiased=True)), dict(type='PackPoseInputs') ] val_pipeline = [ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(288, 384)), dict(type='PackPoseInputs') ] train_dataloader = dict( batch_size=64, num_workers=2, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=True), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_train2017.json', data_prefix=dict(img='train2017/'), pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='RandomFlip', direction='horizontal'), dict(type='RandomHalfBody'), dict(type='RandomBBoxTransform'), dict(type='TopdownAffine', input_size=(288, 384)), dict( type='GenerateTarget', target_type='heatmap', encoder=dict( type='MSRAHeatmap', input_size=(288, 384), heatmap_size=(72, 96), sigma=3, unbiased=True)), dict(type='PackPoseInputs') ])) val_dataloader = dict( batch_size=32, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_val2017.json', bbox_file= 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json', data_prefix=dict(img='val2017/'), test_mode=True, pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(288, 384)), dict(type='PackPoseInputs') ])) test_dataloader = dict( batch_size=32, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_val2017.json', bbox_file= 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json', data_prefix=dict(img='val2017/'), test_mode=True, pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(288, 384)), dict(type='PackPoseInputs') ])) val_evaluator = dict( type='CocoMetric', ann_file='data/coco/annotations/person_keypoints_val2017.json') test_evaluator = dict( type='CocoMetric', ann_file='data/coco/annotations/person_keypoints_val2017.json') launcher = 'slurm' work_dir = '/mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/' 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "data sampler" registry tree. As a workaround, the current "data sampler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optimizer wrapper constructor" registry tree. As a workaround, the current "optimizer wrapper constructor" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optimizer" registry tree. As a workaround, the current "optimizer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optim_wrapper" registry tree. As a workaround, the current "optim_wrapper" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:32 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:36 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "data sampler" registry tree. As a workaround, the current "data sampler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:37 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:39 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/23 12:19:39 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. Name of parameter - Initialization information backbone.conv1.weight - torch.Size([64, 3, 7, 7]): PretrainedInit: load from torchvision://resnet50 backbone.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.conv1.weight - torch.Size([64, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.downsample.0.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.downsample.1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.downsample.1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.conv1.weight - torch.Size([128, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.downsample.1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.downsample.1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.conv1.weight - torch.Size([256, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.downsample.0.weight - torch.Size([1024, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.downsample.1.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.downsample.1.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.conv1.weight - torch.Size([512, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.downsample.0.weight - torch.Size([2048, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.downsample.1.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.downsample.1.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 head.deconv_layers.0.weight - torch.Size([2048, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.1.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.1.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.3.weight - torch.Size([256, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.4.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.4.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.6.weight - torch.Size([256, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.7.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.7.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.final_layer.weight - torch.Size([17, 256, 1, 1]): NormalInit: mean=0, std=0.001, bias=0 head.final_layer.bias - torch.Size([17]): NormalInit: mean=0, std=0.001, bias=0 2022/09/23 12:19:39 - mmengine - INFO - Checkpoints will be saved to /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384 by HardDiskBackend. 2022/09/23 12:20:30 - mmengine - INFO - Epoch(train) [1][50/293] lr: 4.954910e-05 eta: 17:31:16 time: 1.025973 data_time: 0.419884 memory: 14267 loss_kpt: 0.002151 acc_pose: 0.198693 loss: 0.002151 2022/09/23 12:21:01 - mmengine - INFO - Epoch(train) [1][100/293] lr: 9.959920e-05 eta: 14:01:19 time: 0.617506 data_time: 0.091000 memory: 14267 loss_kpt: 0.001766 acc_pose: 0.427167 loss: 0.001766 2022/09/23 12:21:31 - mmengine - INFO - Epoch(train) [1][150/293] lr: 1.496493e-04 eta: 12:45:23 time: 0.601060 data_time: 0.091578 memory: 14267 loss_kpt: 0.001487 acc_pose: 0.540481 loss: 0.001487 2022/09/23 12:22:03 - mmengine - INFO - Epoch(train) [1][200/293] lr: 1.996994e-04 eta: 12:14:11 time: 0.628567 data_time: 0.121747 memory: 14267 loss_kpt: 0.001360 acc_pose: 0.572568 loss: 0.001360 2022/09/23 12:22:33 - mmengine - INFO - Epoch(train) [1][250/293] lr: 2.497495e-04 eta: 11:51:34 time: 0.610441 data_time: 0.083334 memory: 14267 loss_kpt: 0.001273 acc_pose: 0.575356 loss: 0.001273 2022/09/23 12:22:59 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:23:32 - mmengine - INFO - Epoch(train) [2][50/293] lr: 3.428427e-04 eta: 10:13:54 time: 0.646196 data_time: 0.114581 memory: 14267 loss_kpt: 0.001211 acc_pose: 0.605309 loss: 0.001211 2022/09/23 12:24:02 - mmengine - INFO - Epoch(train) [2][100/293] lr: 3.928928e-04 eta: 10:14:17 time: 0.608797 data_time: 0.096169 memory: 14267 loss_kpt: 0.001170 acc_pose: 0.610457 loss: 0.001170 2022/09/23 12:24:32 - mmengine - INFO - Epoch(train) [2][150/293] lr: 4.429429e-04 eta: 10:13:55 time: 0.604097 data_time: 0.096148 memory: 14267 loss_kpt: 0.001176 acc_pose: 0.551761 loss: 0.001176 2022/09/23 12:25:03 - mmengine - INFO - Epoch(train) [2][200/293] lr: 4.929930e-04 eta: 10:13:41 time: 0.605606 data_time: 0.087540 memory: 14267 loss_kpt: 0.001125 acc_pose: 0.582579 loss: 0.001125 2022/09/23 12:25:32 - mmengine - INFO - Epoch(train) [2][250/293] lr: 5.000000e-04 eta: 10:12:12 time: 0.592748 data_time: 0.081821 memory: 14267 loss_kpt: 0.001119 acc_pose: 0.636513 loss: 0.001119 2022/09/23 12:25:58 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:26:29 - mmengine - INFO - Epoch(train) [3][50/293] lr: 5.000000e-04 eta: 9:32:26 time: 0.633470 data_time: 0.141199 memory: 14267 loss_kpt: 0.001071 acc_pose: 0.635758 loss: 0.001071 2022/09/23 12:27:00 - mmengine - INFO - Epoch(train) [3][100/293] lr: 5.000000e-04 eta: 9:35:05 time: 0.606306 data_time: 0.107775 memory: 14267 loss_kpt: 0.001061 acc_pose: 0.652931 loss: 0.001061 2022/09/23 12:27:30 - mmengine - INFO - Epoch(train) [3][150/293] lr: 5.000000e-04 eta: 9:36:47 time: 0.598625 data_time: 0.106506 memory: 14267 loss_kpt: 0.001051 acc_pose: 0.654079 loss: 0.001051 2022/09/23 12:27:59 - mmengine - INFO - Epoch(train) [3][200/293] lr: 5.000000e-04 eta: 9:37:07 time: 0.581863 data_time: 0.096965 memory: 14267 loss_kpt: 0.001052 acc_pose: 0.698905 loss: 0.001052 2022/09/23 12:28:29 - mmengine - INFO - Epoch(train) [3][250/293] lr: 5.000000e-04 eta: 9:38:37 time: 0.602638 data_time: 0.094488 memory: 14267 loss_kpt: 0.001022 acc_pose: 0.668435 loss: 0.001022 2022/09/23 12:28:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:29:26 - mmengine - INFO - Epoch(train) [4][50/293] lr: 5.000000e-04 eta: 9:14:16 time: 0.632267 data_time: 0.107712 memory: 14267 loss_kpt: 0.001006 acc_pose: 0.645337 loss: 0.001006 2022/09/23 12:29:56 - mmengine - INFO - Epoch(train) [4][100/293] lr: 5.000000e-04 eta: 9:16:39 time: 0.604017 data_time: 0.080546 memory: 14267 loss_kpt: 0.001024 acc_pose: 0.649195 loss: 0.001024 2022/09/23 12:30:09 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:30:26 - mmengine - INFO - Epoch(train) [4][150/293] lr: 5.000000e-04 eta: 9:18:30 time: 0.598808 data_time: 0.098460 memory: 14267 loss_kpt: 0.000992 acc_pose: 0.726508 loss: 0.000992 2022/09/23 12:30:57 - mmengine - INFO - Epoch(train) [4][200/293] lr: 5.000000e-04 eta: 9:20:33 time: 0.607488 data_time: 0.091752 memory: 14267 loss_kpt: 0.000996 acc_pose: 0.638750 loss: 0.000996 2022/09/23 12:31:27 - mmengine - INFO - Epoch(train) [4][250/293] lr: 5.000000e-04 eta: 9:21:49 time: 0.595497 data_time: 0.106631 memory: 14267 loss_kpt: 0.000998 acc_pose: 0.633582 loss: 0.000998 2022/09/23 12:31:53 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:32:25 - mmengine - INFO - Epoch(train) [5][50/293] lr: 5.000000e-04 eta: 9:04:31 time: 0.638172 data_time: 0.119414 memory: 14267 loss_kpt: 0.000957 acc_pose: 0.752058 loss: 0.000957 2022/09/23 12:32:55 - mmengine - INFO - Epoch(train) [5][100/293] lr: 5.000000e-04 eta: 9:06:44 time: 0.609543 data_time: 0.103245 memory: 14267 loss_kpt: 0.000970 acc_pose: 0.661444 loss: 0.000970 2022/09/23 12:33:24 - mmengine - INFO - Epoch(train) [5][150/293] lr: 5.000000e-04 eta: 9:07:40 time: 0.580667 data_time: 0.087178 memory: 14267 loss_kpt: 0.000950 acc_pose: 0.686531 loss: 0.000950 2022/09/23 12:33:54 - mmengine - INFO - Epoch(train) [5][200/293] lr: 5.000000e-04 eta: 9:08:42 time: 0.586749 data_time: 0.094687 memory: 14267 loss_kpt: 0.000974 acc_pose: 0.583561 loss: 0.000974 2022/09/23 12:34:24 - mmengine - INFO - Epoch(train) [5][250/293] lr: 5.000000e-04 eta: 9:10:21 time: 0.606929 data_time: 0.084284 memory: 14267 loss_kpt: 0.000948 acc_pose: 0.733491 loss: 0.000948 2022/09/23 12:34:48 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:35:20 - mmengine - INFO - Epoch(train) [6][50/293] lr: 5.000000e-04 eta: 8:56:18 time: 0.622088 data_time: 0.099709 memory: 14267 loss_kpt: 0.000930 acc_pose: 0.686866 loss: 0.000930 2022/09/23 12:35:52 - mmengine - INFO - Epoch(train) [6][100/293] lr: 5.000000e-04 eta: 8:59:18 time: 0.644073 data_time: 0.088282 memory: 14267 loss_kpt: 0.000936 acc_pose: 0.703576 loss: 0.000936 2022/09/23 12:36:21 - mmengine - INFO - Epoch(train) [6][150/293] lr: 5.000000e-04 eta: 9:00:05 time: 0.579509 data_time: 0.087546 memory: 14267 loss_kpt: 0.000944 acc_pose: 0.708756 loss: 0.000944 2022/09/23 12:36:51 - mmengine - INFO - Epoch(train) [6][200/293] lr: 5.000000e-04 eta: 9:01:46 time: 0.611869 data_time: 0.091958 memory: 14267 loss_kpt: 0.000919 acc_pose: 0.713113 loss: 0.000919 2022/09/23 12:37:21 - mmengine - INFO - Epoch(train) [6][250/293] lr: 5.000000e-04 eta: 9:02:31 time: 0.584454 data_time: 0.102508 memory: 14267 loss_kpt: 0.000903 acc_pose: 0.700534 loss: 0.000903 2022/09/23 12:37:45 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:38:16 - mmengine - INFO - Epoch(train) [7][50/293] lr: 5.000000e-04 eta: 8:50:57 time: 0.622766 data_time: 0.096738 memory: 14267 loss_kpt: 0.000910 acc_pose: 0.749063 loss: 0.000910 2022/09/23 12:38:46 - mmengine - INFO - Epoch(train) [7][100/293] lr: 5.000000e-04 eta: 8:52:00 time: 0.589167 data_time: 0.104638 memory: 14267 loss_kpt: 0.000895 acc_pose: 0.708313 loss: 0.000895 2022/09/23 12:39:16 - mmengine - INFO - Epoch(train) [7][150/293] lr: 5.000000e-04 eta: 8:53:41 time: 0.616856 data_time: 0.095541 memory: 14267 loss_kpt: 0.000900 acc_pose: 0.695687 loss: 0.000900 2022/09/23 12:39:46 - mmengine - INFO - Epoch(train) [7][200/293] lr: 5.000000e-04 eta: 8:54:30 time: 0.586715 data_time: 0.102135 memory: 14267 loss_kpt: 0.000897 acc_pose: 0.713514 loss: 0.000897 2022/09/23 12:40:10 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:40:15 - mmengine - INFO - Epoch(train) [7][250/293] lr: 5.000000e-04 eta: 8:55:18 time: 0.589183 data_time: 0.092473 memory: 14267 loss_kpt: 0.000901 acc_pose: 0.676597 loss: 0.000901 2022/09/23 12:40:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:41:12 - mmengine - INFO - Epoch(train) [8][50/293] lr: 5.000000e-04 eta: 8:45:33 time: 0.625203 data_time: 0.106937 memory: 14267 loss_kpt: 0.000903 acc_pose: 0.684966 loss: 0.000903 2022/09/23 12:41:41 - mmengine - INFO - Epoch(train) [8][100/293] lr: 5.000000e-04 eta: 8:46:26 time: 0.588388 data_time: 0.085592 memory: 14267 loss_kpt: 0.000878 acc_pose: 0.747010 loss: 0.000878 2022/09/23 12:42:11 - mmengine - INFO - Epoch(train) [8][150/293] lr: 5.000000e-04 eta: 8:47:22 time: 0.593479 data_time: 0.087844 memory: 14267 loss_kpt: 0.000887 acc_pose: 0.683892 loss: 0.000887 2022/09/23 12:42:41 - mmengine - INFO - Epoch(train) [8][200/293] lr: 5.000000e-04 eta: 8:48:19 time: 0.596813 data_time: 0.080737 memory: 14267 loss_kpt: 0.000916 acc_pose: 0.743465 loss: 0.000916 2022/09/23 12:43:12 - mmengine - INFO - Epoch(train) [8][250/293] lr: 5.000000e-04 eta: 8:49:42 time: 0.619956 data_time: 0.094865 memory: 14267 loss_kpt: 0.000869 acc_pose: 0.706144 loss: 0.000869 2022/09/23 12:43:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:44:09 - mmengine - INFO - Epoch(train) [9][50/293] lr: 5.000000e-04 eta: 8:41:29 time: 0.639223 data_time: 0.097418 memory: 14267 loss_kpt: 0.000862 acc_pose: 0.708363 loss: 0.000862 2022/09/23 12:44:39 - mmengine - INFO - Epoch(train) [9][100/293] lr: 5.000000e-04 eta: 8:42:35 time: 0.605782 data_time: 0.094780 memory: 14267 loss_kpt: 0.000863 acc_pose: 0.696166 loss: 0.000863 2022/09/23 12:45:09 - mmengine - INFO - Epoch(train) [9][150/293] lr: 5.000000e-04 eta: 8:43:20 time: 0.591061 data_time: 0.093806 memory: 14267 loss_kpt: 0.000868 acc_pose: 0.697690 loss: 0.000868 2022/09/23 12:45:38 - mmengine - INFO - Epoch(train) [9][200/293] lr: 5.000000e-04 eta: 8:43:55 time: 0.584588 data_time: 0.079348 memory: 14267 loss_kpt: 0.000866 acc_pose: 0.678080 loss: 0.000866 2022/09/23 12:46:08 - mmengine - INFO - Epoch(train) [9][250/293] lr: 5.000000e-04 eta: 8:44:34 time: 0.591255 data_time: 0.078595 memory: 14267 loss_kpt: 0.000868 acc_pose: 0.757222 loss: 0.000868 2022/09/23 12:46:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:47:04 - mmengine - INFO - Epoch(train) [10][50/293] lr: 5.000000e-04 eta: 8:37:03 time: 0.626392 data_time: 0.115035 memory: 14267 loss_kpt: 0.000852 acc_pose: 0.747982 loss: 0.000852 2022/09/23 12:47:33 - mmengine - INFO - Epoch(train) [10][100/293] lr: 5.000000e-04 eta: 8:37:34 time: 0.580794 data_time: 0.084227 memory: 14267 loss_kpt: 0.000846 acc_pose: 0.714033 loss: 0.000846 2022/09/23 12:48:04 - mmengine - INFO - Epoch(train) [10][150/293] lr: 5.000000e-04 eta: 8:38:45 time: 0.620706 data_time: 0.089789 memory: 14267 loss_kpt: 0.000871 acc_pose: 0.681750 loss: 0.000871 2022/09/23 12:48:35 - mmengine - INFO - Epoch(train) [10][200/293] lr: 5.000000e-04 eta: 8:39:52 time: 0.619791 data_time: 0.092916 memory: 14267 loss_kpt: 0.000838 acc_pose: 0.714474 loss: 0.000838 2022/09/23 12:49:04 - mmengine - INFO - Epoch(train) [10][250/293] lr: 5.000000e-04 eta: 8:40:16 time: 0.581627 data_time: 0.095751 memory: 14267 loss_kpt: 0.000841 acc_pose: 0.675362 loss: 0.000841 2022/09/23 12:49:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:49:30 - mmengine - INFO - Saving checkpoint at 10 epochs 2022/09/23 12:50:00 - mmengine - INFO - Epoch(val) [10][50/407] eta: 0:03:20 time: 0.560402 data_time: 0.261918 memory: 14267 2022/09/23 12:50:18 - mmengine - INFO - Epoch(val) [10][100/407] eta: 0:01:47 time: 0.350265 data_time: 0.049352 memory: 1464 2022/09/23 12:50:35 - mmengine - INFO - Epoch(val) [10][150/407] eta: 0:01:26 time: 0.336855 data_time: 0.035917 memory: 1464 2022/09/23 12:50:52 - mmengine - INFO - Epoch(val) [10][200/407] eta: 0:01:12 time: 0.348268 data_time: 0.051094 memory: 1464 2022/09/23 12:51:09 - mmengine - INFO - Epoch(val) [10][250/407] eta: 0:00:52 time: 0.335585 data_time: 0.054234 memory: 1464 2022/09/23 12:51:26 - mmengine - INFO - Epoch(val) [10][300/407] eta: 0:00:36 time: 0.345223 data_time: 0.037925 memory: 1464 2022/09/23 12:51:44 - mmengine - INFO - Epoch(val) [10][350/407] eta: 0:00:20 time: 0.356036 data_time: 0.072217 memory: 1464 2022/09/23 12:52:01 - mmengine - INFO - Epoch(val) [10][400/407] eta: 0:00:02 time: 0.341571 data_time: 0.028227 memory: 1464 2022/09/23 12:52:37 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 12:52:50 - mmengine - INFO - Epoch(val) [10][407/407] coco/AP: 0.620982 coco/AP .5: 0.854200 coco/AP .75: 0.680327 coco/AP (M): 0.578668 coco/AP (L): 0.695851 coco/AR: 0.680290 coco/AR .5: 0.897827 coco/AR .75: 0.737563 coco/AR (M): 0.628462 coco/AR (L): 0.753214 2022/09/23 12:52:53 - mmengine - INFO - The best checkpoint with 0.6210 coco/AP at 10 epoch is saved to best_coco/AP_epoch_10.pth. 2022/09/23 12:53:22 - mmengine - INFO - Epoch(train) [11][50/293] lr: 5.000000e-04 eta: 8:32:49 time: 0.585851 data_time: 0.089977 memory: 14267 loss_kpt: 0.000852 acc_pose: 0.717697 loss: 0.000852 2022/09/23 12:53:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:53:52 - mmengine - INFO - Epoch(train) [11][100/293] lr: 5.000000e-04 eta: 8:33:40 time: 0.605212 data_time: 0.093421 memory: 14267 loss_kpt: 0.000835 acc_pose: 0.762287 loss: 0.000835 2022/09/23 12:54:21 - mmengine - INFO - Epoch(train) [11][150/293] lr: 5.000000e-04 eta: 8:33:56 time: 0.571971 data_time: 0.091049 memory: 14267 loss_kpt: 0.000840 acc_pose: 0.693945 loss: 0.000840 2022/09/23 12:54:51 - mmengine - INFO - Epoch(train) [11][200/293] lr: 5.000000e-04 eta: 8:34:41 time: 0.603653 data_time: 0.106717 memory: 14267 loss_kpt: 0.000837 acc_pose: 0.705922 loss: 0.000837 2022/09/23 12:55:22 - mmengine - INFO - Epoch(train) [11][250/293] lr: 5.000000e-04 eta: 8:35:33 time: 0.614562 data_time: 0.080866 memory: 14267 loss_kpt: 0.000851 acc_pose: 0.726865 loss: 0.000851 2022/09/23 12:55:46 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:56:18 - mmengine - INFO - Epoch(train) [12][50/293] lr: 5.000000e-04 eta: 8:29:24 time: 0.626593 data_time: 0.102367 memory: 14267 loss_kpt: 0.000829 acc_pose: 0.707736 loss: 0.000829 2022/09/23 12:56:47 - mmengine - INFO - Epoch(train) [12][100/293] lr: 5.000000e-04 eta: 8:30:01 time: 0.597154 data_time: 0.126745 memory: 14267 loss_kpt: 0.000834 acc_pose: 0.721062 loss: 0.000834 2022/09/23 12:57:17 - mmengine - INFO - Epoch(train) [12][150/293] lr: 5.000000e-04 eta: 8:30:31 time: 0.590467 data_time: 0.081826 memory: 14267 loss_kpt: 0.000833 acc_pose: 0.731911 loss: 0.000833 2022/09/23 12:57:47 - mmengine - INFO - Epoch(train) [12][200/293] lr: 5.000000e-04 eta: 8:30:59 time: 0.590435 data_time: 0.097841 memory: 14267 loss_kpt: 0.000831 acc_pose: 0.719795 loss: 0.000831 2022/09/23 12:58:16 - mmengine - INFO - Epoch(train) [12][250/293] lr: 5.000000e-04 eta: 8:31:28 time: 0.594059 data_time: 0.089214 memory: 14267 loss_kpt: 0.000843 acc_pose: 0.760355 loss: 0.000843 2022/09/23 12:58:42 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 12:59:13 - mmengine - INFO - Epoch(train) [13][50/293] lr: 5.000000e-04 eta: 8:25:40 time: 0.615971 data_time: 0.091601 memory: 14267 loss_kpt: 0.000822 acc_pose: 0.792526 loss: 0.000822 2022/09/23 12:59:43 - mmengine - INFO - Epoch(train) [13][100/293] lr: 5.000000e-04 eta: 8:26:21 time: 0.606943 data_time: 0.092973 memory: 14267 loss_kpt: 0.000818 acc_pose: 0.737651 loss: 0.000818 2022/09/23 13:00:13 - mmengine - INFO - Epoch(train) [13][150/293] lr: 5.000000e-04 eta: 8:26:57 time: 0.603459 data_time: 0.089735 memory: 14267 loss_kpt: 0.000811 acc_pose: 0.736422 loss: 0.000811 2022/09/23 13:00:43 - mmengine - INFO - Epoch(train) [13][200/293] lr: 5.000000e-04 eta: 8:27:27 time: 0.598808 data_time: 0.103972 memory: 14267 loss_kpt: 0.000830 acc_pose: 0.709050 loss: 0.000830 2022/09/23 13:01:13 - mmengine - INFO - Epoch(train) [13][250/293] lr: 5.000000e-04 eta: 8:28:01 time: 0.605304 data_time: 0.085825 memory: 14267 loss_kpt: 0.000803 acc_pose: 0.699455 loss: 0.000803 2022/09/23 13:01:39 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:02:09 - mmengine - INFO - Epoch(train) [14][50/293] lr: 5.000000e-04 eta: 8:22:27 time: 0.599890 data_time: 0.119521 memory: 14267 loss_kpt: 0.000820 acc_pose: 0.758669 loss: 0.000820 2022/09/23 13:02:38 - mmengine - INFO - Epoch(train) [14][100/293] lr: 5.000000e-04 eta: 8:22:53 time: 0.593707 data_time: 0.105970 memory: 14267 loss_kpt: 0.000822 acc_pose: 0.689653 loss: 0.000822 2022/09/23 13:03:08 - mmengine - INFO - Epoch(train) [14][150/293] lr: 5.000000e-04 eta: 8:23:22 time: 0.599689 data_time: 0.076403 memory: 14267 loss_kpt: 0.000810 acc_pose: 0.766151 loss: 0.000810 2022/09/23 13:03:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:03:38 - mmengine - INFO - Epoch(train) [14][200/293] lr: 5.000000e-04 eta: 8:23:47 time: 0.594936 data_time: 0.089300 memory: 14267 loss_kpt: 0.000811 acc_pose: 0.754741 loss: 0.000811 2022/09/23 13:04:09 - mmengine - INFO - Epoch(train) [14][250/293] lr: 5.000000e-04 eta: 8:24:25 time: 0.617373 data_time: 0.100975 memory: 14267 loss_kpt: 0.000807 acc_pose: 0.777760 loss: 0.000807 2022/09/23 13:04:35 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:05:07 - mmengine - INFO - Epoch(train) [15][50/293] lr: 5.000000e-04 eta: 8:19:39 time: 0.636048 data_time: 0.101152 memory: 14267 loss_kpt: 0.000797 acc_pose: 0.735709 loss: 0.000797 2022/09/23 13:05:37 - mmengine - INFO - Epoch(train) [15][100/293] lr: 5.000000e-04 eta: 8:20:08 time: 0.603640 data_time: 0.081739 memory: 14267 loss_kpt: 0.000807 acc_pose: 0.743425 loss: 0.000807 2022/09/23 13:06:06 - mmengine - INFO - Epoch(train) [15][150/293] lr: 5.000000e-04 eta: 8:20:21 time: 0.581332 data_time: 0.075514 memory: 14267 loss_kpt: 0.000804 acc_pose: 0.733649 loss: 0.000804 2022/09/23 13:06:36 - mmengine - INFO - Epoch(train) [15][200/293] lr: 5.000000e-04 eta: 8:20:49 time: 0.605669 data_time: 0.122851 memory: 14267 loss_kpt: 0.000819 acc_pose: 0.772214 loss: 0.000819 2022/09/23 13:07:06 - mmengine - INFO - Epoch(train) [15][250/293] lr: 5.000000e-04 eta: 8:21:14 time: 0.603063 data_time: 0.107238 memory: 14267 loss_kpt: 0.000813 acc_pose: 0.686030 loss: 0.000813 2022/09/23 13:07:32 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:08:03 - mmengine - INFO - Epoch(train) [16][50/293] lr: 5.000000e-04 eta: 8:16:44 time: 0.633583 data_time: 0.099600 memory: 14267 loss_kpt: 0.000798 acc_pose: 0.743829 loss: 0.000798 2022/09/23 13:08:34 - mmengine - INFO - Epoch(train) [16][100/293] lr: 5.000000e-04 eta: 8:17:12 time: 0.607051 data_time: 0.095605 memory: 14267 loss_kpt: 0.000798 acc_pose: 0.761884 loss: 0.000798 2022/09/23 13:09:04 - mmengine - INFO - Epoch(train) [16][150/293] lr: 5.000000e-04 eta: 8:17:44 time: 0.617098 data_time: 0.137884 memory: 14267 loss_kpt: 0.000822 acc_pose: 0.793012 loss: 0.000822 2022/09/23 13:09:35 - mmengine - INFO - Epoch(train) [16][200/293] lr: 5.000000e-04 eta: 8:18:15 time: 0.615549 data_time: 0.105702 memory: 14267 loss_kpt: 0.000800 acc_pose: 0.775239 loss: 0.000800 2022/09/23 13:10:05 - mmengine - INFO - Epoch(train) [16][250/293] lr: 5.000000e-04 eta: 8:18:38 time: 0.604846 data_time: 0.092024 memory: 14267 loss_kpt: 0.000800 acc_pose: 0.773977 loss: 0.000800 2022/09/23 13:10:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:11:01 - mmengine - INFO - Epoch(train) [17][50/293] lr: 5.000000e-04 eta: 8:14:08 time: 0.610056 data_time: 0.097897 memory: 14267 loss_kpt: 0.000810 acc_pose: 0.756383 loss: 0.000810 2022/09/23 13:11:31 - mmengine - INFO - Epoch(train) [17][100/293] lr: 5.000000e-04 eta: 8:14:33 time: 0.608533 data_time: 0.127583 memory: 14267 loss_kpt: 0.000796 acc_pose: 0.714402 loss: 0.000796 2022/09/23 13:12:01 - mmengine - INFO - Epoch(train) [17][150/293] lr: 5.000000e-04 eta: 8:14:42 time: 0.583354 data_time: 0.090961 memory: 14267 loss_kpt: 0.000782 acc_pose: 0.735046 loss: 0.000782 2022/09/23 13:12:30 - mmengine - INFO - Epoch(train) [17][200/293] lr: 5.000000e-04 eta: 8:14:57 time: 0.593458 data_time: 0.083728 memory: 14267 loss_kpt: 0.000799 acc_pose: 0.724612 loss: 0.000799 2022/09/23 13:13:00 - mmengine - INFO - Epoch(train) [17][250/293] lr: 5.000000e-04 eta: 8:15:08 time: 0.590638 data_time: 0.093207 memory: 14267 loss_kpt: 0.000801 acc_pose: 0.734116 loss: 0.000801 2022/09/23 13:13:25 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:13:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:13:57 - mmengine - INFO - Epoch(train) [18][50/293] lr: 5.000000e-04 eta: 8:11:07 time: 0.632723 data_time: 0.111190 memory: 14267 loss_kpt: 0.000805 acc_pose: 0.721008 loss: 0.000805 2022/09/23 13:14:26 - mmengine - INFO - Epoch(train) [18][100/293] lr: 5.000000e-04 eta: 8:11:19 time: 0.591291 data_time: 0.089561 memory: 14267 loss_kpt: 0.000787 acc_pose: 0.750341 loss: 0.000787 2022/09/23 13:14:55 - mmengine - INFO - Epoch(train) [18][150/293] lr: 5.000000e-04 eta: 8:11:28 time: 0.585413 data_time: 0.093317 memory: 14267 loss_kpt: 0.000797 acc_pose: 0.748776 loss: 0.000797 2022/09/23 13:15:25 - mmengine - INFO - Epoch(train) [18][200/293] lr: 5.000000e-04 eta: 8:11:33 time: 0.580722 data_time: 0.097025 memory: 14267 loss_kpt: 0.000790 acc_pose: 0.763768 loss: 0.000790 2022/09/23 13:15:54 - mmengine - INFO - Epoch(train) [18][250/293] lr: 5.000000e-04 eta: 8:11:44 time: 0.591704 data_time: 0.081330 memory: 14267 loss_kpt: 0.000795 acc_pose: 0.746430 loss: 0.000795 2022/09/23 13:16:20 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:16:51 - mmengine - INFO - Epoch(train) [19][50/293] lr: 5.000000e-04 eta: 8:07:55 time: 0.634147 data_time: 0.114815 memory: 14267 loss_kpt: 0.000786 acc_pose: 0.756962 loss: 0.000786 2022/09/23 13:17:21 - mmengine - INFO - Epoch(train) [19][100/293] lr: 5.000000e-04 eta: 8:08:08 time: 0.594209 data_time: 0.097710 memory: 14267 loss_kpt: 0.000789 acc_pose: 0.788596 loss: 0.000789 2022/09/23 13:17:50 - mmengine - INFO - Epoch(train) [19][150/293] lr: 5.000000e-04 eta: 8:08:12 time: 0.581344 data_time: 0.089575 memory: 14267 loss_kpt: 0.000783 acc_pose: 0.735241 loss: 0.000783 2022/09/23 13:18:19 - mmengine - INFO - Epoch(train) [19][200/293] lr: 5.000000e-04 eta: 8:08:16 time: 0.579891 data_time: 0.081680 memory: 14267 loss_kpt: 0.000772 acc_pose: 0.742085 loss: 0.000772 2022/09/23 13:18:49 - mmengine - INFO - Epoch(train) [19][250/293] lr: 5.000000e-04 eta: 8:08:29 time: 0.599202 data_time: 0.098010 memory: 14267 loss_kpt: 0.000792 acc_pose: 0.742013 loss: 0.000792 2022/09/23 13:19:15 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:19:46 - mmengine - INFO - Epoch(train) [20][50/293] lr: 5.000000e-04 eta: 8:04:44 time: 0.619487 data_time: 0.105102 memory: 14267 loss_kpt: 0.000774 acc_pose: 0.797083 loss: 0.000774 2022/09/23 13:20:15 - mmengine - INFO - Epoch(train) [20][100/293] lr: 5.000000e-04 eta: 8:04:48 time: 0.580829 data_time: 0.077826 memory: 14267 loss_kpt: 0.000766 acc_pose: 0.799968 loss: 0.000766 2022/09/23 13:20:45 - mmengine - INFO - Epoch(train) [20][150/293] lr: 5.000000e-04 eta: 8:05:03 time: 0.604926 data_time: 0.100782 memory: 14267 loss_kpt: 0.000789 acc_pose: 0.764288 loss: 0.000789 2022/09/23 13:21:15 - mmengine - INFO - Epoch(train) [20][200/293] lr: 5.000000e-04 eta: 8:05:15 time: 0.600612 data_time: 0.086574 memory: 14267 loss_kpt: 0.000778 acc_pose: 0.761597 loss: 0.000778 2022/09/23 13:21:45 - mmengine - INFO - Epoch(train) [20][250/293] lr: 5.000000e-04 eta: 8:05:24 time: 0.595015 data_time: 0.094935 memory: 14267 loss_kpt: 0.000795 acc_pose: 0.782239 loss: 0.000795 2022/09/23 13:22:10 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:22:10 - mmengine - INFO - Saving checkpoint at 20 epochs 2022/09/23 13:22:34 - mmengine - INFO - Epoch(val) [20][50/407] eta: 0:02:11 time: 0.368471 data_time: 0.059238 memory: 14267 2022/09/23 13:22:51 - mmengine - INFO - Epoch(val) [20][100/407] eta: 0:01:45 time: 0.344093 data_time: 0.033725 memory: 1464 2022/09/23 13:23:09 - mmengine - INFO - Epoch(val) [20][150/407] eta: 0:01:30 time: 0.352099 data_time: 0.039109 memory: 1464 2022/09/23 13:23:26 - mmengine - INFO - Epoch(val) [20][200/407] eta: 0:01:12 time: 0.348739 data_time: 0.034034 memory: 1464 2022/09/23 13:23:43 - mmengine - INFO - Epoch(val) [20][250/407] eta: 0:00:53 time: 0.340692 data_time: 0.039317 memory: 1464 2022/09/23 13:24:01 - mmengine - INFO - Epoch(val) [20][300/407] eta: 0:00:36 time: 0.345200 data_time: 0.029631 memory: 1464 2022/09/23 13:24:19 - mmengine - INFO - Epoch(val) [20][350/407] eta: 0:00:20 time: 0.361490 data_time: 0.049248 memory: 1464 2022/09/23 13:24:32 - mmengine - INFO - Epoch(val) [20][400/407] eta: 0:00:01 time: 0.267832 data_time: 0.022220 memory: 1464 2022/09/23 13:25:05 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 13:25:18 - mmengine - INFO - Epoch(val) [20][407/407] coco/AP: 0.656240 coco/AP .5: 0.867424 coco/AP .75: 0.721621 coco/AP (M): 0.610198 coco/AP (L): 0.734225 coco/AR: 0.712217 coco/AR .5: 0.909477 coco/AR .75: 0.773772 coco/AR (M): 0.660585 coco/AR (L): 0.785916 2022/09/23 13:25:18 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_10.pth is removed 2022/09/23 13:25:21 - mmengine - INFO - The best checkpoint with 0.6562 coco/AP at 20 epoch is saved to best_coco/AP_epoch_20.pth. 2022/09/23 13:25:51 - mmengine - INFO - Epoch(train) [21][50/293] lr: 5.000000e-04 eta: 8:01:40 time: 0.600521 data_time: 0.107847 memory: 14267 loss_kpt: 0.000804 acc_pose: 0.753315 loss: 0.000804 2022/09/23 13:26:21 - mmengine - INFO - Epoch(train) [21][100/293] lr: 5.000000e-04 eta: 8:01:55 time: 0.607446 data_time: 0.095321 memory: 14267 loss_kpt: 0.000757 acc_pose: 0.778126 loss: 0.000757 2022/09/23 13:26:46 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:26:52 - mmengine - INFO - Epoch(train) [21][150/293] lr: 5.000000e-04 eta: 8:02:13 time: 0.614855 data_time: 0.081436 memory: 14267 loss_kpt: 0.000770 acc_pose: 0.725981 loss: 0.000770 2022/09/23 13:27:21 - mmengine - INFO - Epoch(train) [21][200/293] lr: 5.000000e-04 eta: 8:02:10 time: 0.571733 data_time: 0.083889 memory: 14267 loss_kpt: 0.000784 acc_pose: 0.734575 loss: 0.000784 2022/09/23 13:27:50 - mmengine - INFO - Epoch(train) [21][250/293] lr: 5.000000e-04 eta: 8:02:13 time: 0.586823 data_time: 0.096088 memory: 14267 loss_kpt: 0.000775 acc_pose: 0.774297 loss: 0.000775 2022/09/23 13:28:15 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:28:47 - mmengine - INFO - Epoch(train) [22][50/293] lr: 5.000000e-04 eta: 7:58:52 time: 0.628454 data_time: 0.114591 memory: 14267 loss_kpt: 0.000769 acc_pose: 0.784803 loss: 0.000769 2022/09/23 13:29:16 - mmengine - INFO - Epoch(train) [22][100/293] lr: 5.000000e-04 eta: 7:58:57 time: 0.589179 data_time: 0.096569 memory: 14267 loss_kpt: 0.000754 acc_pose: 0.779570 loss: 0.000754 2022/09/23 13:29:46 - mmengine - INFO - Epoch(train) [22][150/293] lr: 5.000000e-04 eta: 7:59:04 time: 0.595620 data_time: 0.083272 memory: 14267 loss_kpt: 0.000755 acc_pose: 0.848777 loss: 0.000755 2022/09/23 13:30:15 - mmengine - INFO - Epoch(train) [22][200/293] lr: 5.000000e-04 eta: 7:59:00 time: 0.571923 data_time: 0.079673 memory: 14267 loss_kpt: 0.000770 acc_pose: 0.773130 loss: 0.000770 2022/09/23 13:30:46 - mmengine - INFO - Epoch(train) [22][250/293] lr: 5.000000e-04 eta: 7:59:17 time: 0.620055 data_time: 0.085500 memory: 14267 loss_kpt: 0.000769 acc_pose: 0.797451 loss: 0.000769 2022/09/23 13:31:11 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:31:43 - mmengine - INFO - Epoch(train) [23][50/293] lr: 5.000000e-04 eta: 7:56:05 time: 0.630591 data_time: 0.110381 memory: 14267 loss_kpt: 0.000773 acc_pose: 0.802261 loss: 0.000773 2022/09/23 13:32:14 - mmengine - INFO - Epoch(train) [23][100/293] lr: 5.000000e-04 eta: 7:56:25 time: 0.630203 data_time: 0.101349 memory: 14267 loss_kpt: 0.000776 acc_pose: 0.684689 loss: 0.000776 2022/09/23 13:32:43 - mmengine - INFO - Epoch(train) [23][150/293] lr: 5.000000e-04 eta: 7:56:26 time: 0.583796 data_time: 0.095650 memory: 14267 loss_kpt: 0.000772 acc_pose: 0.803144 loss: 0.000772 2022/09/23 13:33:12 - mmengine - INFO - Epoch(train) [23][200/293] lr: 5.000000e-04 eta: 7:56:22 time: 0.573600 data_time: 0.084729 memory: 14267 loss_kpt: 0.000762 acc_pose: 0.754867 loss: 0.000762 2022/09/23 13:33:41 - mmengine - INFO - Epoch(train) [23][250/293] lr: 5.000000e-04 eta: 7:56:23 time: 0.586654 data_time: 0.089718 memory: 14267 loss_kpt: 0.000758 acc_pose: 0.781093 loss: 0.000758 2022/09/23 13:34:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:34:39 - mmengine - INFO - Epoch(train) [24][50/293] lr: 5.000000e-04 eta: 7:53:17 time: 0.628858 data_time: 0.141629 memory: 14267 loss_kpt: 0.000768 acc_pose: 0.783499 loss: 0.000768 2022/09/23 13:35:08 - mmengine - INFO - Epoch(train) [24][100/293] lr: 5.000000e-04 eta: 7:53:15 time: 0.577602 data_time: 0.089790 memory: 14267 loss_kpt: 0.000765 acc_pose: 0.783767 loss: 0.000765 2022/09/23 13:35:38 - mmengine - INFO - Epoch(train) [24][150/293] lr: 5.000000e-04 eta: 7:53:19 time: 0.596258 data_time: 0.098936 memory: 14267 loss_kpt: 0.000754 acc_pose: 0.781333 loss: 0.000754 2022/09/23 13:36:08 - mmengine - INFO - Epoch(train) [24][200/293] lr: 5.000000e-04 eta: 7:53:27 time: 0.606204 data_time: 0.105519 memory: 14267 loss_kpt: 0.000752 acc_pose: 0.767486 loss: 0.000752 2022/09/23 13:36:37 - mmengine - INFO - Epoch(train) [24][250/293] lr: 5.000000e-04 eta: 7:53:25 time: 0.581454 data_time: 0.085155 memory: 14267 loss_kpt: 0.000769 acc_pose: 0.739335 loss: 0.000769 2022/09/23 13:36:44 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:37:03 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:37:36 - mmengine - INFO - Epoch(train) [25][50/293] lr: 5.000000e-04 eta: 7:50:33 time: 0.647369 data_time: 0.118385 memory: 14267 loss_kpt: 0.000769 acc_pose: 0.764709 loss: 0.000769 2022/09/23 13:38:05 - mmengine - INFO - Epoch(train) [25][100/293] lr: 5.000000e-04 eta: 7:50:37 time: 0.595561 data_time: 0.093803 memory: 14267 loss_kpt: 0.000754 acc_pose: 0.806352 loss: 0.000754 2022/09/23 13:38:36 - mmengine - INFO - Epoch(train) [25][150/293] lr: 5.000000e-04 eta: 7:50:48 time: 0.617806 data_time: 0.097193 memory: 14267 loss_kpt: 0.000772 acc_pose: 0.744754 loss: 0.000772 2022/09/23 13:39:05 - mmengine - INFO - Epoch(train) [25][200/293] lr: 5.000000e-04 eta: 7:50:42 time: 0.573675 data_time: 0.090943 memory: 14267 loss_kpt: 0.000760 acc_pose: 0.782113 loss: 0.000760 2022/09/23 13:39:35 - mmengine - INFO - Epoch(train) [25][250/293] lr: 5.000000e-04 eta: 7:50:49 time: 0.607521 data_time: 0.101745 memory: 14267 loss_kpt: 0.000751 acc_pose: 0.824581 loss: 0.000751 2022/09/23 13:40:00 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:40:32 - mmengine - INFO - Epoch(train) [26][50/293] lr: 5.000000e-04 eta: 7:48:00 time: 0.639988 data_time: 0.117471 memory: 14267 loss_kpt: 0.000742 acc_pose: 0.804428 loss: 0.000742 2022/09/23 13:41:02 - mmengine - INFO - Epoch(train) [26][100/293] lr: 5.000000e-04 eta: 7:48:00 time: 0.589454 data_time: 0.088594 memory: 14267 loss_kpt: 0.000740 acc_pose: 0.754280 loss: 0.000740 2022/09/23 13:41:33 - mmengine - INFO - Epoch(train) [26][150/293] lr: 5.000000e-04 eta: 7:48:10 time: 0.618726 data_time: 0.087498 memory: 14267 loss_kpt: 0.000747 acc_pose: 0.767375 loss: 0.000747 2022/09/23 13:42:02 - mmengine - INFO - Epoch(train) [26][200/293] lr: 5.000000e-04 eta: 7:48:08 time: 0.587416 data_time: 0.092987 memory: 14267 loss_kpt: 0.000745 acc_pose: 0.772977 loss: 0.000745 2022/09/23 13:42:32 - mmengine - INFO - Epoch(train) [26][250/293] lr: 5.000000e-04 eta: 7:48:12 time: 0.604952 data_time: 0.088893 memory: 14267 loss_kpt: 0.000748 acc_pose: 0.786844 loss: 0.000748 2022/09/23 13:42:57 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:43:29 - mmengine - INFO - Epoch(train) [27][50/293] lr: 5.000000e-04 eta: 7:45:25 time: 0.631246 data_time: 0.098187 memory: 14267 loss_kpt: 0.000770 acc_pose: 0.807168 loss: 0.000770 2022/09/23 13:43:57 - mmengine - INFO - Epoch(train) [27][100/293] lr: 5.000000e-04 eta: 7:45:15 time: 0.562554 data_time: 0.079596 memory: 14267 loss_kpt: 0.000756 acc_pose: 0.769977 loss: 0.000756 2022/09/23 13:44:27 - mmengine - INFO - Epoch(train) [27][150/293] lr: 5.000000e-04 eta: 7:45:18 time: 0.601647 data_time: 0.110923 memory: 14267 loss_kpt: 0.000741 acc_pose: 0.743811 loss: 0.000741 2022/09/23 13:44:57 - mmengine - INFO - Epoch(train) [27][200/293] lr: 5.000000e-04 eta: 7:45:23 time: 0.609217 data_time: 0.097044 memory: 14267 loss_kpt: 0.000740 acc_pose: 0.752690 loss: 0.000740 2022/09/23 13:45:27 - mmengine - INFO - Epoch(train) [27][250/293] lr: 5.000000e-04 eta: 7:45:22 time: 0.595302 data_time: 0.086640 memory: 14267 loss_kpt: 0.000742 acc_pose: 0.777535 loss: 0.000742 2022/09/23 13:45:52 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:46:24 - mmengine - INFO - Epoch(train) [28][50/293] lr: 5.000000e-04 eta: 7:42:40 time: 0.629817 data_time: 0.098455 memory: 14267 loss_kpt: 0.000755 acc_pose: 0.760175 loss: 0.000755 2022/09/23 13:46:46 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:46:53 - mmengine - INFO - Epoch(train) [28][100/293] lr: 5.000000e-04 eta: 7:42:36 time: 0.582678 data_time: 0.087607 memory: 14267 loss_kpt: 0.000735 acc_pose: 0.716525 loss: 0.000735 2022/09/23 13:47:21 - mmengine - INFO - Epoch(train) [28][150/293] lr: 5.000000e-04 eta: 7:42:29 time: 0.576038 data_time: 0.093720 memory: 14267 loss_kpt: 0.000740 acc_pose: 0.763631 loss: 0.000740 2022/09/23 13:47:52 - mmengine - INFO - Epoch(train) [28][200/293] lr: 5.000000e-04 eta: 7:42:32 time: 0.608252 data_time: 0.090925 memory: 14267 loss_kpt: 0.000748 acc_pose: 0.720107 loss: 0.000748 2022/09/23 13:48:22 - mmengine - INFO - Epoch(train) [28][250/293] lr: 5.000000e-04 eta: 7:42:33 time: 0.601840 data_time: 0.095814 memory: 14267 loss_kpt: 0.000741 acc_pose: 0.802287 loss: 0.000741 2022/09/23 13:48:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:49:19 - mmengine - INFO - Epoch(train) [29][50/293] lr: 5.000000e-04 eta: 7:39:57 time: 0.633675 data_time: 0.109551 memory: 14267 loss_kpt: 0.000735 acc_pose: 0.769534 loss: 0.000735 2022/09/23 13:49:48 - mmengine - INFO - Epoch(train) [29][100/293] lr: 5.000000e-04 eta: 7:39:53 time: 0.584912 data_time: 0.090055 memory: 14267 loss_kpt: 0.000741 acc_pose: 0.809682 loss: 0.000741 2022/09/23 13:50:16 - mmengine - INFO - Epoch(train) [29][150/293] lr: 5.000000e-04 eta: 7:39:41 time: 0.564329 data_time: 0.081698 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.784498 loss: 0.000722 2022/09/23 13:50:46 - mmengine - INFO - Epoch(train) [29][200/293] lr: 5.000000e-04 eta: 7:39:42 time: 0.602384 data_time: 0.092281 memory: 14267 loss_kpt: 0.000746 acc_pose: 0.848140 loss: 0.000746 2022/09/23 13:51:16 - mmengine - INFO - Epoch(train) [29][250/293] lr: 5.000000e-04 eta: 7:39:38 time: 0.590107 data_time: 0.095733 memory: 14267 loss_kpt: 0.000748 acc_pose: 0.806597 loss: 0.000748 2022/09/23 13:51:42 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:52:14 - mmengine - INFO - Epoch(train) [30][50/293] lr: 5.000000e-04 eta: 7:37:07 time: 0.632976 data_time: 0.106527 memory: 14267 loss_kpt: 0.000736 acc_pose: 0.820779 loss: 0.000736 2022/09/23 13:52:43 - mmengine - INFO - Epoch(train) [30][100/293] lr: 5.000000e-04 eta: 7:37:03 time: 0.589151 data_time: 0.081562 memory: 14267 loss_kpt: 0.000742 acc_pose: 0.757070 loss: 0.000742 2022/09/23 13:53:13 - mmengine - INFO - Epoch(train) [30][150/293] lr: 5.000000e-04 eta: 7:37:02 time: 0.601034 data_time: 0.113458 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.756331 loss: 0.000722 2022/09/23 13:53:42 - mmengine - INFO - Epoch(train) [30][200/293] lr: 5.000000e-04 eta: 7:36:55 time: 0.580680 data_time: 0.121009 memory: 14267 loss_kpt: 0.000732 acc_pose: 0.786507 loss: 0.000732 2022/09/23 13:54:12 - mmengine - INFO - Epoch(train) [30][250/293] lr: 5.000000e-04 eta: 7:36:55 time: 0.602880 data_time: 0.096107 memory: 14267 loss_kpt: 0.000748 acc_pose: 0.770528 loss: 0.000748 2022/09/23 13:54:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 13:54:37 - mmengine - INFO - Saving checkpoint at 30 epochs 2022/09/23 13:55:01 - mmengine - INFO - Epoch(val) [30][50/407] eta: 0:02:09 time: 0.361752 data_time: 0.065049 memory: 14267 2022/09/23 13:55:18 - mmengine - INFO - Epoch(val) [30][100/407] eta: 0:01:45 time: 0.343119 data_time: 0.031782 memory: 1464 2022/09/23 13:55:36 - mmengine - INFO - Epoch(val) [30][150/407] eta: 0:01:30 time: 0.351365 data_time: 0.040007 memory: 1464 2022/09/23 13:55:53 - mmengine - INFO - Epoch(val) [30][200/407] eta: 0:01:11 time: 0.345164 data_time: 0.037311 memory: 1464 2022/09/23 13:56:11 - mmengine - INFO - Epoch(val) [30][250/407] eta: 0:00:54 time: 0.348902 data_time: 0.034828 memory: 1464 2022/09/23 13:56:28 - mmengine - INFO - Epoch(val) [30][300/407] eta: 0:00:37 time: 0.348354 data_time: 0.047750 memory: 1464 2022/09/23 13:56:45 - mmengine - INFO - Epoch(val) [30][350/407] eta: 0:00:19 time: 0.341308 data_time: 0.038798 memory: 1464 2022/09/23 13:56:59 - mmengine - INFO - Epoch(val) [30][400/407] eta: 0:00:01 time: 0.270608 data_time: 0.029083 memory: 1464 2022/09/23 13:57:32 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 13:57:45 - mmengine - INFO - Epoch(val) [30][407/407] coco/AP: 0.675464 coco/AP .5: 0.877139 coco/AP .75: 0.737584 coco/AP (M): 0.631436 coco/AP (L): 0.749506 coco/AR: 0.730479 coco/AR .5: 0.916089 coco/AR .75: 0.789358 coco/AR (M): 0.680388 coco/AR (L): 0.801895 2022/09/23 13:57:45 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_20.pth is removed 2022/09/23 13:57:48 - mmengine - INFO - The best checkpoint with 0.6755 coco/AP at 30 epoch is saved to best_coco/AP_epoch_30.pth. 2022/09/23 13:58:17 - mmengine - INFO - Epoch(train) [31][50/293] lr: 5.000000e-04 eta: 7:34:17 time: 0.598978 data_time: 0.095513 memory: 14267 loss_kpt: 0.000745 acc_pose: 0.821480 loss: 0.000745 2022/09/23 13:58:47 - mmengine - INFO - Epoch(train) [31][100/293] lr: 5.000000e-04 eta: 7:34:13 time: 0.591135 data_time: 0.101246 memory: 14267 loss_kpt: 0.000751 acc_pose: 0.758355 loss: 0.000751 2022/09/23 13:59:16 - mmengine - INFO - Epoch(train) [31][150/293] lr: 5.000000e-04 eta: 7:34:05 time: 0.578571 data_time: 0.089376 memory: 14267 loss_kpt: 0.000743 acc_pose: 0.781068 loss: 0.000743 2022/09/23 13:59:46 - mmengine - INFO - Epoch(train) [31][200/293] lr: 5.000000e-04 eta: 7:34:01 time: 0.594876 data_time: 0.102201 memory: 14267 loss_kpt: 0.000728 acc_pose: 0.836339 loss: 0.000728 2022/09/23 13:59:52 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:00:15 - mmengine - INFO - Epoch(train) [31][250/293] lr: 5.000000e-04 eta: 7:33:57 time: 0.593214 data_time: 0.095603 memory: 14267 loss_kpt: 0.000749 acc_pose: 0.778105 loss: 0.000749 2022/09/23 14:00:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:01:13 - mmengine - INFO - Epoch(train) [32][50/293] lr: 5.000000e-04 eta: 7:31:34 time: 0.634491 data_time: 0.121732 memory: 14267 loss_kpt: 0.000729 acc_pose: 0.763772 loss: 0.000729 2022/09/23 14:01:43 - mmengine - INFO - Epoch(train) [32][100/293] lr: 5.000000e-04 eta: 7:31:36 time: 0.613733 data_time: 0.090838 memory: 14267 loss_kpt: 0.000727 acc_pose: 0.770079 loss: 0.000727 2022/09/23 14:02:13 - mmengine - INFO - Epoch(train) [32][150/293] lr: 5.000000e-04 eta: 7:31:31 time: 0.591518 data_time: 0.095152 memory: 14267 loss_kpt: 0.000736 acc_pose: 0.742508 loss: 0.000736 2022/09/23 14:02:42 - mmengine - INFO - Epoch(train) [32][200/293] lr: 5.000000e-04 eta: 7:31:24 time: 0.586251 data_time: 0.089308 memory: 14267 loss_kpt: 0.000738 acc_pose: 0.774890 loss: 0.000738 2022/09/23 14:03:11 - mmengine - INFO - Epoch(train) [32][250/293] lr: 5.000000e-04 eta: 7:31:15 time: 0.580190 data_time: 0.098941 memory: 14267 loss_kpt: 0.000724 acc_pose: 0.751642 loss: 0.000724 2022/09/23 14:03:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:04:08 - mmengine - INFO - Epoch(train) [33][50/293] lr: 5.000000e-04 eta: 7:28:52 time: 0.621240 data_time: 0.115013 memory: 14267 loss_kpt: 0.000740 acc_pose: 0.743830 loss: 0.000740 2022/09/23 14:04:38 - mmengine - INFO - Epoch(train) [33][100/293] lr: 5.000000e-04 eta: 7:28:53 time: 0.612841 data_time: 0.085793 memory: 14267 loss_kpt: 0.000733 acc_pose: 0.811560 loss: 0.000733 2022/09/23 14:05:10 - mmengine - INFO - Epoch(train) [33][150/293] lr: 5.000000e-04 eta: 7:28:56 time: 0.624803 data_time: 0.098670 memory: 14267 loss_kpt: 0.000731 acc_pose: 0.772204 loss: 0.000731 2022/09/23 14:05:40 - mmengine - INFO - Epoch(train) [33][200/293] lr: 5.000000e-04 eta: 7:28:54 time: 0.605674 data_time: 0.099650 memory: 14267 loss_kpt: 0.000728 acc_pose: 0.768414 loss: 0.000728 2022/09/23 14:06:10 - mmengine - INFO - Epoch(train) [33][250/293] lr: 5.000000e-04 eta: 7:28:52 time: 0.605229 data_time: 0.081542 memory: 14267 loss_kpt: 0.000715 acc_pose: 0.802306 loss: 0.000715 2022/09/23 14:06:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:07:10 - mmengine - INFO - Epoch(train) [34][50/293] lr: 5.000000e-04 eta: 7:26:44 time: 0.668819 data_time: 0.107554 memory: 14267 loss_kpt: 0.000728 acc_pose: 0.745066 loss: 0.000728 2022/09/23 14:07:41 - mmengine - INFO - Epoch(train) [34][100/293] lr: 5.000000e-04 eta: 7:26:44 time: 0.615783 data_time: 0.084257 memory: 14267 loss_kpt: 0.000733 acc_pose: 0.762905 loss: 0.000733 2022/09/23 14:08:10 - mmengine - INFO - Epoch(train) [34][150/293] lr: 5.000000e-04 eta: 7:26:38 time: 0.590297 data_time: 0.074341 memory: 14267 loss_kpt: 0.000726 acc_pose: 0.782515 loss: 0.000726 2022/09/23 14:08:40 - mmengine - INFO - Epoch(train) [34][200/293] lr: 5.000000e-04 eta: 7:26:33 time: 0.600893 data_time: 0.110210 memory: 14267 loss_kpt: 0.000716 acc_pose: 0.792514 loss: 0.000716 2022/09/23 14:09:10 - mmengine - INFO - Epoch(train) [34][250/293] lr: 5.000000e-04 eta: 7:26:27 time: 0.595003 data_time: 0.092826 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.798960 loss: 0.000722 2022/09/23 14:09:36 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:10:01 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:10:08 - mmengine - INFO - Epoch(train) [35][50/293] lr: 5.000000e-04 eta: 7:24:15 time: 0.638263 data_time: 0.104916 memory: 14267 loss_kpt: 0.000698 acc_pose: 0.775283 loss: 0.000698 2022/09/23 14:10:39 - mmengine - INFO - Epoch(train) [35][100/293] lr: 5.000000e-04 eta: 7:24:13 time: 0.609473 data_time: 0.091096 memory: 14267 loss_kpt: 0.000709 acc_pose: 0.758929 loss: 0.000709 2022/09/23 14:11:09 - mmengine - INFO - Epoch(train) [35][150/293] lr: 5.000000e-04 eta: 7:24:10 time: 0.608487 data_time: 0.085014 memory: 14267 loss_kpt: 0.000724 acc_pose: 0.780670 loss: 0.000724 2022/09/23 14:11:40 - mmengine - INFO - Epoch(train) [35][200/293] lr: 5.000000e-04 eta: 7:24:08 time: 0.615335 data_time: 0.087725 memory: 14267 loss_kpt: 0.000736 acc_pose: 0.741589 loss: 0.000736 2022/09/23 14:12:10 - mmengine - INFO - Epoch(train) [35][250/293] lr: 5.000000e-04 eta: 7:24:05 time: 0.607059 data_time: 0.091356 memory: 14267 loss_kpt: 0.000739 acc_pose: 0.822293 loss: 0.000739 2022/09/23 14:12:36 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:13:08 - mmengine - INFO - Epoch(train) [36][50/293] lr: 5.000000e-04 eta: 7:21:52 time: 0.628372 data_time: 0.105868 memory: 14267 loss_kpt: 0.000723 acc_pose: 0.742984 loss: 0.000723 2022/09/23 14:13:37 - mmengine - INFO - Epoch(train) [36][100/293] lr: 5.000000e-04 eta: 7:21:45 time: 0.592061 data_time: 0.091062 memory: 14267 loss_kpt: 0.000736 acc_pose: 0.774454 loss: 0.000736 2022/09/23 14:14:08 - mmengine - INFO - Epoch(train) [36][150/293] lr: 5.000000e-04 eta: 7:21:41 time: 0.607605 data_time: 0.088185 memory: 14267 loss_kpt: 0.000709 acc_pose: 0.759031 loss: 0.000709 2022/09/23 14:14:38 - mmengine - INFO - Epoch(train) [36][200/293] lr: 5.000000e-04 eta: 7:21:38 time: 0.610128 data_time: 0.077671 memory: 14267 loss_kpt: 0.000704 acc_pose: 0.743094 loss: 0.000704 2022/09/23 14:15:09 - mmengine - INFO - Epoch(train) [36][250/293] lr: 5.000000e-04 eta: 7:21:33 time: 0.607536 data_time: 0.110288 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.779426 loss: 0.000722 2022/09/23 14:15:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:16:04 - mmengine - INFO - Epoch(train) [37][50/293] lr: 5.000000e-04 eta: 7:19:22 time: 0.621526 data_time: 0.114629 memory: 14267 loss_kpt: 0.000719 acc_pose: 0.761306 loss: 0.000719 2022/09/23 14:16:34 - mmengine - INFO - Epoch(train) [37][100/293] lr: 5.000000e-04 eta: 7:19:13 time: 0.589108 data_time: 0.079416 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.770710 loss: 0.000722 2022/09/23 14:17:04 - mmengine - INFO - Epoch(train) [37][150/293] lr: 5.000000e-04 eta: 7:19:09 time: 0.610387 data_time: 0.086582 memory: 14267 loss_kpt: 0.000711 acc_pose: 0.764417 loss: 0.000711 2022/09/23 14:17:35 - mmengine - INFO - Epoch(train) [37][200/293] lr: 5.000000e-04 eta: 7:19:05 time: 0.607224 data_time: 0.079247 memory: 14267 loss_kpt: 0.000719 acc_pose: 0.829886 loss: 0.000719 2022/09/23 14:18:05 - mmengine - INFO - Epoch(train) [37][250/293] lr: 5.000000e-04 eta: 7:19:00 time: 0.610812 data_time: 0.089227 memory: 14267 loss_kpt: 0.000717 acc_pose: 0.768335 loss: 0.000717 2022/09/23 14:18:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:19:01 - mmengine - INFO - Epoch(train) [38][50/293] lr: 5.000000e-04 eta: 7:16:50 time: 0.611440 data_time: 0.092679 memory: 14267 loss_kpt: 0.000706 acc_pose: 0.805567 loss: 0.000706 2022/09/23 14:19:31 - mmengine - INFO - Epoch(train) [38][100/293] lr: 5.000000e-04 eta: 7:16:42 time: 0.598390 data_time: 0.095484 memory: 14267 loss_kpt: 0.000709 acc_pose: 0.747433 loss: 0.000709 2022/09/23 14:20:02 - mmengine - INFO - Epoch(train) [38][150/293] lr: 5.000000e-04 eta: 7:16:41 time: 0.626315 data_time: 0.103385 memory: 14267 loss_kpt: 0.000716 acc_pose: 0.765582 loss: 0.000716 2022/09/23 14:20:07 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:20:31 - mmengine - INFO - Epoch(train) [38][200/293] lr: 5.000000e-04 eta: 7:16:30 time: 0.581632 data_time: 0.077361 memory: 14267 loss_kpt: 0.000727 acc_pose: 0.722724 loss: 0.000727 2022/09/23 14:21:01 - mmengine - INFO - Epoch(train) [38][250/293] lr: 5.000000e-04 eta: 7:16:21 time: 0.593131 data_time: 0.086631 memory: 14267 loss_kpt: 0.000717 acc_pose: 0.787401 loss: 0.000717 2022/09/23 14:21:25 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:21:57 - mmengine - INFO - Epoch(train) [39][50/293] lr: 5.000000e-04 eta: 7:14:18 time: 0.632102 data_time: 0.103536 memory: 14267 loss_kpt: 0.000707 acc_pose: 0.761147 loss: 0.000707 2022/09/23 14:22:27 - mmengine - INFO - Epoch(train) [39][100/293] lr: 5.000000e-04 eta: 7:14:10 time: 0.598478 data_time: 0.093406 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.821764 loss: 0.000722 2022/09/23 14:22:57 - mmengine - INFO - Epoch(train) [39][150/293] lr: 5.000000e-04 eta: 7:14:03 time: 0.601765 data_time: 0.098840 memory: 14267 loss_kpt: 0.000720 acc_pose: 0.757739 loss: 0.000720 2022/09/23 14:23:27 - mmengine - INFO - Epoch(train) [39][200/293] lr: 5.000000e-04 eta: 7:13:56 time: 0.607541 data_time: 0.079171 memory: 14267 loss_kpt: 0.000713 acc_pose: 0.753356 loss: 0.000713 2022/09/23 14:23:58 - mmengine - INFO - Epoch(train) [39][250/293] lr: 5.000000e-04 eta: 7:13:50 time: 0.609648 data_time: 0.100662 memory: 14267 loss_kpt: 0.000722 acc_pose: 0.806454 loss: 0.000722 2022/09/23 14:24:24 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:24:55 - mmengine - INFO - Epoch(train) [40][50/293] lr: 5.000000e-04 eta: 7:11:48 time: 0.626275 data_time: 0.097536 memory: 14267 loss_kpt: 0.000702 acc_pose: 0.802797 loss: 0.000702 2022/09/23 14:25:25 - mmengine - INFO - Epoch(train) [40][100/293] lr: 5.000000e-04 eta: 7:11:40 time: 0.600534 data_time: 0.084649 memory: 14267 loss_kpt: 0.000708 acc_pose: 0.815142 loss: 0.000708 2022/09/23 14:25:56 - mmengine - INFO - Epoch(train) [40][150/293] lr: 5.000000e-04 eta: 7:11:36 time: 0.616258 data_time: 0.093558 memory: 14267 loss_kpt: 0.000705 acc_pose: 0.807153 loss: 0.000705 2022/09/23 14:26:25 - mmengine - INFO - Epoch(train) [40][200/293] lr: 5.000000e-04 eta: 7:11:25 time: 0.587981 data_time: 0.088867 memory: 14267 loss_kpt: 0.000701 acc_pose: 0.769603 loss: 0.000701 2022/09/23 14:26:55 - mmengine - INFO - Epoch(train) [40][250/293] lr: 5.000000e-04 eta: 7:11:16 time: 0.601877 data_time: 0.090558 memory: 14267 loss_kpt: 0.000700 acc_pose: 0.852043 loss: 0.000700 2022/09/23 14:27:20 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:27:20 - mmengine - INFO - Saving checkpoint at 40 epochs 2022/09/23 14:27:45 - mmengine - INFO - Epoch(val) [40][50/407] eta: 0:02:11 time: 0.369120 data_time: 0.056845 memory: 14267 2022/09/23 14:28:03 - mmengine - INFO - Epoch(val) [40][100/407] eta: 0:01:45 time: 0.344512 data_time: 0.026148 memory: 1464 2022/09/23 14:28:21 - mmengine - INFO - Epoch(val) [40][150/407] eta: 0:01:34 time: 0.368297 data_time: 0.041993 memory: 1464 2022/09/23 14:28:38 - mmengine - INFO - Epoch(val) [40][200/407] eta: 0:01:10 time: 0.338865 data_time: 0.031249 memory: 1464 2022/09/23 14:28:55 - mmengine - INFO - Epoch(val) [40][250/407] eta: 0:00:51 time: 0.330587 data_time: 0.026922 memory: 1464 2022/09/23 14:29:12 - mmengine - INFO - Epoch(val) [40][300/407] eta: 0:00:36 time: 0.344071 data_time: 0.031525 memory: 1464 2022/09/23 14:29:29 - mmengine - INFO - Epoch(val) [40][350/407] eta: 0:00:19 time: 0.349143 data_time: 0.046004 memory: 1464 2022/09/23 14:29:43 - mmengine - INFO - Epoch(val) [40][400/407] eta: 0:00:01 time: 0.285186 data_time: 0.024387 memory: 1464 2022/09/23 14:30:17 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 14:30:30 - mmengine - INFO - Epoch(val) [40][407/407] coco/AP: 0.692887 coco/AP .5: 0.885965 coco/AP .75: 0.756519 coco/AP (M): 0.647454 coco/AP (L): 0.769036 coco/AR: 0.747497 coco/AR .5: 0.923961 coco/AR .75: 0.806832 coco/AR (M): 0.697378 coco/AR (L): 0.818618 2022/09/23 14:30:30 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_30.pth is removed 2022/09/23 14:30:33 - mmengine - INFO - The best checkpoint with 0.6929 coco/AP at 40 epoch is saved to best_coco/AP_epoch_40.pth. 2022/09/23 14:31:04 - mmengine - INFO - Epoch(train) [41][50/293] lr: 5.000000e-04 eta: 7:09:15 time: 0.617403 data_time: 0.091772 memory: 14267 loss_kpt: 0.000699 acc_pose: 0.714395 loss: 0.000699 2022/09/23 14:31:36 - mmengine - INFO - Epoch(train) [41][100/293] lr: 5.000000e-04 eta: 7:09:14 time: 0.638639 data_time: 0.102053 memory: 14267 loss_kpt: 0.000717 acc_pose: 0.741968 loss: 0.000717 2022/09/23 14:32:05 - mmengine - INFO - Epoch(train) [41][150/293] lr: 5.000000e-04 eta: 7:09:04 time: 0.590823 data_time: 0.093442 memory: 14267 loss_kpt: 0.000711 acc_pose: 0.798522 loss: 0.000711 2022/09/23 14:32:36 - mmengine - INFO - Epoch(train) [41][200/293] lr: 5.000000e-04 eta: 7:08:56 time: 0.608916 data_time: 0.098724 memory: 14267 loss_kpt: 0.000707 acc_pose: 0.793703 loss: 0.000707 2022/09/23 14:33:05 - mmengine - INFO - Epoch(train) [41][250/293] lr: 5.000000e-04 eta: 7:08:45 time: 0.589859 data_time: 0.082112 memory: 14267 loss_kpt: 0.000708 acc_pose: 0.830821 loss: 0.000708 2022/09/23 14:33:23 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:33:31 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:34:03 - mmengine - INFO - Epoch(train) [42][50/293] lr: 5.000000e-04 eta: 7:06:51 time: 0.642420 data_time: 0.120509 memory: 14267 loss_kpt: 0.000702 acc_pose: 0.798660 loss: 0.000702 2022/09/23 14:34:34 - mmengine - INFO - Epoch(train) [42][100/293] lr: 5.000000e-04 eta: 7:06:46 time: 0.619835 data_time: 0.079925 memory: 14267 loss_kpt: 0.000696 acc_pose: 0.810643 loss: 0.000696 2022/09/23 14:35:04 - mmengine - INFO - Epoch(train) [42][150/293] lr: 5.000000e-04 eta: 7:06:36 time: 0.596780 data_time: 0.072718 memory: 14267 loss_kpt: 0.000711 acc_pose: 0.727812 loss: 0.000711 2022/09/23 14:35:36 - mmengine - INFO - Epoch(train) [42][200/293] lr: 5.000000e-04 eta: 7:06:32 time: 0.627827 data_time: 0.083336 memory: 14267 loss_kpt: 0.000724 acc_pose: 0.809465 loss: 0.000724 2022/09/23 14:36:05 - mmengine - INFO - Epoch(train) [42][250/293] lr: 5.000000e-04 eta: 7:06:21 time: 0.592020 data_time: 0.091418 memory: 14267 loss_kpt: 0.000705 acc_pose: 0.767222 loss: 0.000705 2022/09/23 14:36:32 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:37:05 - mmengine - INFO - Epoch(train) [43][50/293] lr: 5.000000e-04 eta: 7:04:31 time: 0.656785 data_time: 0.124220 memory: 14267 loss_kpt: 0.000690 acc_pose: 0.765827 loss: 0.000690 2022/09/23 14:37:34 - mmengine - INFO - Epoch(train) [43][100/293] lr: 5.000000e-04 eta: 7:04:18 time: 0.581823 data_time: 0.080519 memory: 14267 loss_kpt: 0.000687 acc_pose: 0.767042 loss: 0.000687 2022/09/23 14:38:03 - mmengine - INFO - Epoch(train) [43][150/293] lr: 5.000000e-04 eta: 7:04:04 time: 0.581789 data_time: 0.094263 memory: 14267 loss_kpt: 0.000701 acc_pose: 0.787862 loss: 0.000701 2022/09/23 14:38:34 - mmengine - INFO - Epoch(train) [43][200/293] lr: 5.000000e-04 eta: 7:03:58 time: 0.621424 data_time: 0.079829 memory: 14267 loss_kpt: 0.000697 acc_pose: 0.772343 loss: 0.000697 2022/09/23 14:39:04 - mmengine - INFO - Epoch(train) [43][250/293] lr: 5.000000e-04 eta: 7:03:49 time: 0.602167 data_time: 0.098021 memory: 14267 loss_kpt: 0.000707 acc_pose: 0.794408 loss: 0.000707 2022/09/23 14:39:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:39:59 - mmengine - INFO - Epoch(train) [44][50/293] lr: 5.000000e-04 eta: 7:01:53 time: 0.616201 data_time: 0.087722 memory: 14267 loss_kpt: 0.000713 acc_pose: 0.785488 loss: 0.000713 2022/09/23 14:40:30 - mmengine - INFO - Epoch(train) [44][100/293] lr: 5.000000e-04 eta: 7:01:44 time: 0.606052 data_time: 0.101267 memory: 14267 loss_kpt: 0.000698 acc_pose: 0.858617 loss: 0.000698 2022/09/23 14:41:00 - mmengine - INFO - Epoch(train) [44][150/293] lr: 5.000000e-04 eta: 7:01:34 time: 0.604182 data_time: 0.104516 memory: 14267 loss_kpt: 0.000697 acc_pose: 0.797051 loss: 0.000697 2022/09/23 14:41:30 - mmengine - INFO - Epoch(train) [44][200/293] lr: 5.000000e-04 eta: 7:01:24 time: 0.598135 data_time: 0.092113 memory: 14267 loss_kpt: 0.000704 acc_pose: 0.796565 loss: 0.000704 2022/09/23 14:41:58 - mmengine - INFO - Epoch(train) [44][250/293] lr: 5.000000e-04 eta: 7:01:07 time: 0.567983 data_time: 0.083098 memory: 14267 loss_kpt: 0.000703 acc_pose: 0.806498 loss: 0.000703 2022/09/23 14:42:24 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:42:55 - mmengine - INFO - Epoch(train) [45][50/293] lr: 5.000000e-04 eta: 6:59:15 time: 0.627891 data_time: 0.102608 memory: 14267 loss_kpt: 0.000700 acc_pose: 0.738780 loss: 0.000700 2022/09/23 14:43:25 - mmengine - INFO - Epoch(train) [45][100/293] lr: 5.000000e-04 eta: 6:59:04 time: 0.596579 data_time: 0.089685 memory: 14267 loss_kpt: 0.000701 acc_pose: 0.746879 loss: 0.000701 2022/09/23 14:43:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:43:53 - mmengine - INFO - Epoch(train) [45][150/293] lr: 5.000000e-04 eta: 6:58:47 time: 0.566425 data_time: 0.076449 memory: 14267 loss_kpt: 0.000693 acc_pose: 0.796044 loss: 0.000693 2022/09/23 14:44:23 - mmengine - INFO - Epoch(train) [45][200/293] lr: 5.000000e-04 eta: 6:58:36 time: 0.598118 data_time: 0.099773 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.754646 loss: 0.000688 2022/09/23 14:44:53 - mmengine - INFO - Epoch(train) [45][250/293] lr: 5.000000e-04 eta: 6:58:25 time: 0.601026 data_time: 0.095615 memory: 14267 loss_kpt: 0.000700 acc_pose: 0.797266 loss: 0.000700 2022/09/23 14:45:19 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:45:50 - mmengine - INFO - Epoch(train) [46][50/293] lr: 5.000000e-04 eta: 6:56:35 time: 0.624209 data_time: 0.123061 memory: 14267 loss_kpt: 0.000709 acc_pose: 0.715074 loss: 0.000709 2022/09/23 14:46:19 - mmengine - INFO - Epoch(train) [46][100/293] lr: 5.000000e-04 eta: 6:56:22 time: 0.589405 data_time: 0.085371 memory: 14267 loss_kpt: 0.000711 acc_pose: 0.761695 loss: 0.000711 2022/09/23 14:46:49 - mmengine - INFO - Epoch(train) [46][150/293] lr: 5.000000e-04 eta: 6:56:09 time: 0.590192 data_time: 0.081810 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.822358 loss: 0.000688 2022/09/23 14:47:20 - mmengine - INFO - Epoch(train) [46][200/293] lr: 5.000000e-04 eta: 6:56:00 time: 0.611888 data_time: 0.093397 memory: 14267 loss_kpt: 0.000703 acc_pose: 0.825322 loss: 0.000703 2022/09/23 14:47:49 - mmengine - INFO - Epoch(train) [46][250/293] lr: 5.000000e-04 eta: 6:55:47 time: 0.588358 data_time: 0.106032 memory: 14267 loss_kpt: 0.000700 acc_pose: 0.765651 loss: 0.000700 2022/09/23 14:48:15 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:48:46 - mmengine - INFO - Epoch(train) [47][50/293] lr: 5.000000e-04 eta: 6:53:59 time: 0.630497 data_time: 0.111314 memory: 14267 loss_kpt: 0.000694 acc_pose: 0.782856 loss: 0.000694 2022/09/23 14:49:17 - mmengine - INFO - Epoch(train) [47][100/293] lr: 5.000000e-04 eta: 6:53:50 time: 0.610629 data_time: 0.109841 memory: 14267 loss_kpt: 0.000697 acc_pose: 0.828458 loss: 0.000697 2022/09/23 14:49:48 - mmengine - INFO - Epoch(train) [47][150/293] lr: 5.000000e-04 eta: 6:53:42 time: 0.620369 data_time: 0.097894 memory: 14267 loss_kpt: 0.000686 acc_pose: 0.804286 loss: 0.000686 2022/09/23 14:50:17 - mmengine - INFO - Epoch(train) [47][200/293] lr: 5.000000e-04 eta: 6:53:29 time: 0.589751 data_time: 0.087814 memory: 14267 loss_kpt: 0.000697 acc_pose: 0.790258 loss: 0.000697 2022/09/23 14:50:47 - mmengine - INFO - Epoch(train) [47][250/293] lr: 5.000000e-04 eta: 6:53:16 time: 0.596273 data_time: 0.105394 memory: 14267 loss_kpt: 0.000716 acc_pose: 0.804758 loss: 0.000716 2022/09/23 14:51:12 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:51:44 - mmengine - INFO - Epoch(train) [48][50/293] lr: 5.000000e-04 eta: 6:51:32 time: 0.637793 data_time: 0.096224 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.776408 loss: 0.000688 2022/09/23 14:52:13 - mmengine - INFO - Epoch(train) [48][100/293] lr: 5.000000e-04 eta: 6:51:17 time: 0.581529 data_time: 0.084410 memory: 14267 loss_kpt: 0.000697 acc_pose: 0.791123 loss: 0.000697 2022/09/23 14:52:43 - mmengine - INFO - Epoch(train) [48][150/293] lr: 5.000000e-04 eta: 6:51:07 time: 0.612616 data_time: 0.084067 memory: 14267 loss_kpt: 0.000691 acc_pose: 0.777735 loss: 0.000691 2022/09/23 14:53:13 - mmengine - INFO - Epoch(train) [48][200/293] lr: 5.000000e-04 eta: 6:50:54 time: 0.593898 data_time: 0.091718 memory: 14267 loss_kpt: 0.000687 acc_pose: 0.806442 loss: 0.000687 2022/09/23 14:53:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:53:43 - mmengine - INFO - Epoch(train) [48][250/293] lr: 5.000000e-04 eta: 6:50:41 time: 0.597084 data_time: 0.083457 memory: 14267 loss_kpt: 0.000699 acc_pose: 0.760081 loss: 0.000699 2022/09/23 14:54:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:54:40 - mmengine - INFO - Epoch(train) [49][50/293] lr: 5.000000e-04 eta: 6:48:57 time: 0.629368 data_time: 0.119067 memory: 14267 loss_kpt: 0.000687 acc_pose: 0.780935 loss: 0.000687 2022/09/23 14:55:10 - mmengine - INFO - Epoch(train) [49][100/293] lr: 5.000000e-04 eta: 6:48:46 time: 0.609450 data_time: 0.084423 memory: 14267 loss_kpt: 0.000684 acc_pose: 0.836551 loss: 0.000684 2022/09/23 14:55:40 - mmengine - INFO - Epoch(train) [49][150/293] lr: 5.000000e-04 eta: 6:48:33 time: 0.593060 data_time: 0.109892 memory: 14267 loss_kpt: 0.000696 acc_pose: 0.787292 loss: 0.000696 2022/09/23 14:56:10 - mmengine - INFO - Epoch(train) [49][200/293] lr: 5.000000e-04 eta: 6:48:22 time: 0.607403 data_time: 0.107265 memory: 14267 loss_kpt: 0.000691 acc_pose: 0.819518 loss: 0.000691 2022/09/23 14:56:39 - mmengine - INFO - Epoch(train) [49][250/293] lr: 5.000000e-04 eta: 6:48:06 time: 0.581912 data_time: 0.086407 memory: 14267 loss_kpt: 0.000693 acc_pose: 0.770879 loss: 0.000693 2022/09/23 14:57:06 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 14:57:37 - mmengine - INFO - Epoch(train) [50][50/293] lr: 5.000000e-04 eta: 6:46:21 time: 0.616566 data_time: 0.093531 memory: 14267 loss_kpt: 0.000684 acc_pose: 0.786371 loss: 0.000684 2022/09/23 14:58:07 - mmengine - INFO - Epoch(train) [50][100/293] lr: 5.000000e-04 eta: 6:46:11 time: 0.612740 data_time: 0.091375 memory: 14267 loss_kpt: 0.000686 acc_pose: 0.797625 loss: 0.000686 2022/09/23 14:58:38 - mmengine - INFO - Epoch(train) [50][150/293] lr: 5.000000e-04 eta: 6:45:59 time: 0.606761 data_time: 0.102100 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.805504 loss: 0.000688 2022/09/23 14:59:08 - mmengine - INFO - Epoch(train) [50][200/293] lr: 5.000000e-04 eta: 6:45:46 time: 0.596936 data_time: 0.104546 memory: 14267 loss_kpt: 0.000689 acc_pose: 0.813450 loss: 0.000689 2022/09/23 14:59:37 - mmengine - INFO - Epoch(train) [50][250/293] lr: 5.000000e-04 eta: 6:45:33 time: 0.598157 data_time: 0.090496 memory: 14267 loss_kpt: 0.000697 acc_pose: 0.782506 loss: 0.000697 2022/09/23 15:00:03 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:00:03 - mmengine - INFO - Saving checkpoint at 50 epochs 2022/09/23 15:00:28 - mmengine - INFO - Epoch(val) [50][50/407] eta: 0:02:12 time: 0.371728 data_time: 0.055396 memory: 14267 2022/09/23 15:00:44 - mmengine - INFO - Epoch(val) [50][100/407] eta: 0:01:38 time: 0.320536 data_time: 0.023358 memory: 1464 2022/09/23 15:01:01 - mmengine - INFO - Epoch(val) [50][150/407] eta: 0:01:30 time: 0.350253 data_time: 0.037988 memory: 1464 2022/09/23 15:01:18 - mmengine - INFO - Epoch(val) [50][200/407] eta: 0:01:09 time: 0.334273 data_time: 0.024209 memory: 1464 2022/09/23 15:01:36 - mmengine - INFO - Epoch(val) [50][250/407] eta: 0:00:55 time: 0.351118 data_time: 0.058968 memory: 1464 2022/09/23 15:01:53 - mmengine - INFO - Epoch(val) [50][300/407] eta: 0:00:37 time: 0.347566 data_time: 0.054203 memory: 1464 2022/09/23 15:02:10 - mmengine - INFO - Epoch(val) [50][350/407] eta: 0:00:19 time: 0.341579 data_time: 0.039956 memory: 1464 2022/09/23 15:02:24 - mmengine - INFO - Epoch(val) [50][400/407] eta: 0:00:01 time: 0.283318 data_time: 0.036957 memory: 1464 2022/09/23 15:02:58 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 15:03:11 - mmengine - INFO - Epoch(val) [50][407/407] coco/AP: 0.695842 coco/AP .5: 0.884413 coco/AP .75: 0.759203 coco/AP (M): 0.651493 coco/AP (L): 0.768717 coco/AR: 0.749355 coco/AR .5: 0.921127 coco/AR .75: 0.809351 coco/AR (M): 0.699891 coco/AR (L): 0.819807 2022/09/23 15:03:11 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_40.pth is removed 2022/09/23 15:03:13 - mmengine - INFO - The best checkpoint with 0.6958 coco/AP at 50 epoch is saved to best_coco/AP_epoch_50.pth. 2022/09/23 15:03:42 - mmengine - INFO - Epoch(train) [51][50/293] lr: 5.000000e-04 eta: 6:43:44 time: 0.582860 data_time: 0.101947 memory: 14267 loss_kpt: 0.000676 acc_pose: 0.816119 loss: 0.000676 2022/09/23 15:04:12 - mmengine - INFO - Epoch(train) [51][100/293] lr: 5.000000e-04 eta: 6:43:31 time: 0.598788 data_time: 0.082668 memory: 14267 loss_kpt: 0.000699 acc_pose: 0.807538 loss: 0.000699 2022/09/23 15:04:42 - mmengine - INFO - Epoch(train) [51][150/293] lr: 5.000000e-04 eta: 6:43:17 time: 0.588498 data_time: 0.084973 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.836976 loss: 0.000688 2022/09/23 15:05:12 - mmengine - INFO - Epoch(train) [51][200/293] lr: 5.000000e-04 eta: 6:43:03 time: 0.596229 data_time: 0.091664 memory: 14267 loss_kpt: 0.000675 acc_pose: 0.819851 loss: 0.000675 2022/09/23 15:05:40 - mmengine - INFO - Epoch(train) [51][250/293] lr: 5.000000e-04 eta: 6:42:46 time: 0.577411 data_time: 0.080571 memory: 14267 loss_kpt: 0.000691 acc_pose: 0.790315 loss: 0.000691 2022/09/23 15:06:07 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:06:37 - mmengine - INFO - Epoch(train) [52][50/293] lr: 5.000000e-04 eta: 6:41:01 time: 0.598186 data_time: 0.087351 memory: 14267 loss_kpt: 0.000681 acc_pose: 0.784379 loss: 0.000681 2022/09/23 15:06:40 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:07:06 - mmengine - INFO - Epoch(train) [52][100/293] lr: 5.000000e-04 eta: 6:40:47 time: 0.589909 data_time: 0.086338 memory: 14267 loss_kpt: 0.000682 acc_pose: 0.776155 loss: 0.000682 2022/09/23 15:07:36 - mmengine - INFO - Epoch(train) [52][150/293] lr: 5.000000e-04 eta: 6:40:32 time: 0.588422 data_time: 0.093365 memory: 14267 loss_kpt: 0.000679 acc_pose: 0.773164 loss: 0.000679 2022/09/23 15:08:05 - mmengine - INFO - Epoch(train) [52][200/293] lr: 5.000000e-04 eta: 6:40:18 time: 0.596062 data_time: 0.090153 memory: 14267 loss_kpt: 0.000682 acc_pose: 0.845197 loss: 0.000682 2022/09/23 15:08:34 - mmengine - INFO - Epoch(train) [52][250/293] lr: 5.000000e-04 eta: 6:40:02 time: 0.582896 data_time: 0.074009 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.799133 loss: 0.000688 2022/09/23 15:09:00 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:09:31 - mmengine - INFO - Epoch(train) [53][50/293] lr: 5.000000e-04 eta: 6:38:21 time: 0.613650 data_time: 0.096217 memory: 14267 loss_kpt: 0.000678 acc_pose: 0.795227 loss: 0.000678 2022/09/23 15:10:01 - mmengine - INFO - Epoch(train) [53][100/293] lr: 5.000000e-04 eta: 6:38:07 time: 0.592856 data_time: 0.080579 memory: 14267 loss_kpt: 0.000684 acc_pose: 0.795354 loss: 0.000684 2022/09/23 15:10:30 - mmengine - INFO - Epoch(train) [53][150/293] lr: 5.000000e-04 eta: 6:37:52 time: 0.591488 data_time: 0.085516 memory: 14267 loss_kpt: 0.000683 acc_pose: 0.798102 loss: 0.000683 2022/09/23 15:11:00 - mmengine - INFO - Epoch(train) [53][200/293] lr: 5.000000e-04 eta: 6:37:37 time: 0.593622 data_time: 0.094137 memory: 14267 loss_kpt: 0.000679 acc_pose: 0.795256 loss: 0.000679 2022/09/23 15:11:30 - mmengine - INFO - Epoch(train) [53][250/293] lr: 5.000000e-04 eta: 6:37:23 time: 0.594152 data_time: 0.079856 memory: 14267 loss_kpt: 0.000666 acc_pose: 0.838604 loss: 0.000666 2022/09/23 15:11:53 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:12:24 - mmengine - INFO - Epoch(train) [54][50/293] lr: 5.000000e-04 eta: 6:35:42 time: 0.603407 data_time: 0.094958 memory: 14267 loss_kpt: 0.000689 acc_pose: 0.790031 loss: 0.000689 2022/09/23 15:12:53 - mmengine - INFO - Epoch(train) [54][100/293] lr: 5.000000e-04 eta: 6:35:25 time: 0.579371 data_time: 0.078520 memory: 14267 loss_kpt: 0.000685 acc_pose: 0.862223 loss: 0.000685 2022/09/23 15:13:24 - mmengine - INFO - Epoch(train) [54][150/293] lr: 5.000000e-04 eta: 6:35:15 time: 0.621940 data_time: 0.100797 memory: 14267 loss_kpt: 0.000679 acc_pose: 0.838280 loss: 0.000679 2022/09/23 15:13:52 - mmengine - INFO - Epoch(train) [54][200/293] lr: 5.000000e-04 eta: 6:34:55 time: 0.559047 data_time: 0.076035 memory: 14267 loss_kpt: 0.000688 acc_pose: 0.766117 loss: 0.000688 2022/09/23 15:14:21 - mmengine - INFO - Epoch(train) [54][250/293] lr: 5.000000e-04 eta: 6:34:40 time: 0.590742 data_time: 0.096218 memory: 14267 loss_kpt: 0.000690 acc_pose: 0.789887 loss: 0.000690 2022/09/23 15:14:45 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:15:16 - mmengine - INFO - Epoch(train) [55][50/293] lr: 5.000000e-04 eta: 6:33:02 time: 0.618398 data_time: 0.107226 memory: 14267 loss_kpt: 0.000686 acc_pose: 0.738459 loss: 0.000686 2022/09/23 15:15:46 - mmengine - INFO - Epoch(train) [55][100/293] lr: 5.000000e-04 eta: 6:32:48 time: 0.601486 data_time: 0.088768 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.834670 loss: 0.000663 2022/09/23 15:16:16 - mmengine - INFO - Epoch(train) [55][150/293] lr: 5.000000e-04 eta: 6:32:34 time: 0.594260 data_time: 0.089072 memory: 14267 loss_kpt: 0.000674 acc_pose: 0.800393 loss: 0.000674 2022/09/23 15:16:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:16:46 - mmengine - INFO - Epoch(train) [55][200/293] lr: 5.000000e-04 eta: 6:32:20 time: 0.603039 data_time: 0.090891 memory: 14267 loss_kpt: 0.000676 acc_pose: 0.793115 loss: 0.000676 2022/09/23 15:17:15 - mmengine - INFO - Epoch(train) [55][250/293] lr: 5.000000e-04 eta: 6:32:03 time: 0.578311 data_time: 0.072436 memory: 14267 loss_kpt: 0.000698 acc_pose: 0.817659 loss: 0.000698 2022/09/23 15:17:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:18:11 - mmengine - INFO - Epoch(train) [56][50/293] lr: 5.000000e-04 eta: 6:30:24 time: 0.605154 data_time: 0.095420 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.818654 loss: 0.000670 2022/09/23 15:18:39 - mmengine - INFO - Epoch(train) [56][100/293] lr: 5.000000e-04 eta: 6:30:06 time: 0.566793 data_time: 0.077719 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.824185 loss: 0.000668 2022/09/23 15:19:09 - mmengine - INFO - Epoch(train) [56][150/293] lr: 5.000000e-04 eta: 6:29:49 time: 0.582379 data_time: 0.090483 memory: 14267 loss_kpt: 0.000685 acc_pose: 0.792947 loss: 0.000685 2022/09/23 15:19:38 - mmengine - INFO - Epoch(train) [56][200/293] lr: 5.000000e-04 eta: 6:29:32 time: 0.582268 data_time: 0.092170 memory: 14267 loss_kpt: 0.000664 acc_pose: 0.805894 loss: 0.000664 2022/09/23 15:20:07 - mmengine - INFO - Epoch(train) [56][250/293] lr: 5.000000e-04 eta: 6:29:16 time: 0.583508 data_time: 0.096299 memory: 14267 loss_kpt: 0.000691 acc_pose: 0.814835 loss: 0.000691 2022/09/23 15:20:31 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:21:02 - mmengine - INFO - Epoch(train) [57][50/293] lr: 5.000000e-04 eta: 6:27:40 time: 0.613538 data_time: 0.098933 memory: 14267 loss_kpt: 0.000683 acc_pose: 0.743520 loss: 0.000683 2022/09/23 15:21:32 - mmengine - INFO - Epoch(train) [57][100/293] lr: 5.000000e-04 eta: 6:27:25 time: 0.598685 data_time: 0.087218 memory: 14267 loss_kpt: 0.000682 acc_pose: 0.818792 loss: 0.000682 2022/09/23 15:22:02 - mmengine - INFO - Epoch(train) [57][150/293] lr: 5.000000e-04 eta: 6:27:12 time: 0.607319 data_time: 0.111867 memory: 14267 loss_kpt: 0.000678 acc_pose: 0.786469 loss: 0.000678 2022/09/23 15:22:31 - mmengine - INFO - Epoch(train) [57][200/293] lr: 5.000000e-04 eta: 6:26:55 time: 0.584964 data_time: 0.077784 memory: 14267 loss_kpt: 0.000667 acc_pose: 0.796639 loss: 0.000667 2022/09/23 15:23:00 - mmengine - INFO - Epoch(train) [57][250/293] lr: 5.000000e-04 eta: 6:26:37 time: 0.570469 data_time: 0.078897 memory: 14267 loss_kpt: 0.000685 acc_pose: 0.799399 loss: 0.000685 2022/09/23 15:23:25 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:23:56 - mmengine - INFO - Epoch(train) [58][50/293] lr: 5.000000e-04 eta: 6:25:01 time: 0.609211 data_time: 0.105433 memory: 14267 loss_kpt: 0.000682 acc_pose: 0.788203 loss: 0.000682 2022/09/23 15:24:24 - mmengine - INFO - Epoch(train) [58][100/293] lr: 5.000000e-04 eta: 6:24:43 time: 0.570940 data_time: 0.080307 memory: 14267 loss_kpt: 0.000673 acc_pose: 0.813719 loss: 0.000673 2022/09/23 15:24:53 - mmengine - INFO - Epoch(train) [58][150/293] lr: 5.000000e-04 eta: 6:24:24 time: 0.570429 data_time: 0.088993 memory: 14267 loss_kpt: 0.000679 acc_pose: 0.766699 loss: 0.000679 2022/09/23 15:25:22 - mmengine - INFO - Epoch(train) [58][200/293] lr: 5.000000e-04 eta: 6:24:08 time: 0.587954 data_time: 0.095455 memory: 14267 loss_kpt: 0.000671 acc_pose: 0.742343 loss: 0.000671 2022/09/23 15:25:51 - mmengine - INFO - Epoch(train) [58][250/293] lr: 5.000000e-04 eta: 6:23:51 time: 0.581181 data_time: 0.098460 memory: 14267 loss_kpt: 0.000677 acc_pose: 0.755893 loss: 0.000677 2022/09/23 15:26:16 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:26:21 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:26:47 - mmengine - INFO - Epoch(train) [59][50/293] lr: 5.000000e-04 eta: 6:22:17 time: 0.612161 data_time: 0.097139 memory: 14267 loss_kpt: 0.000674 acc_pose: 0.815090 loss: 0.000674 2022/09/23 15:27:16 - mmengine - INFO - Epoch(train) [59][100/293] lr: 5.000000e-04 eta: 6:22:00 time: 0.582432 data_time: 0.081098 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.816627 loss: 0.000670 2022/09/23 15:27:45 - mmengine - INFO - Epoch(train) [59][150/293] lr: 5.000000e-04 eta: 6:21:43 time: 0.581844 data_time: 0.084970 memory: 14267 loss_kpt: 0.000673 acc_pose: 0.782788 loss: 0.000673 2022/09/23 15:28:15 - mmengine - INFO - Epoch(train) [59][200/293] lr: 5.000000e-04 eta: 6:21:27 time: 0.597382 data_time: 0.081284 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.801743 loss: 0.000663 2022/09/23 15:28:43 - mmengine - INFO - Epoch(train) [59][250/293] lr: 5.000000e-04 eta: 6:21:08 time: 0.568426 data_time: 0.083735 memory: 14267 loss_kpt: 0.000665 acc_pose: 0.832239 loss: 0.000665 2022/09/23 15:29:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:29:38 - mmengine - INFO - Epoch(train) [60][50/293] lr: 5.000000e-04 eta: 6:19:35 time: 0.601487 data_time: 0.090770 memory: 14267 loss_kpt: 0.000662 acc_pose: 0.769733 loss: 0.000662 2022/09/23 15:30:07 - mmengine - INFO - Epoch(train) [60][100/293] lr: 5.000000e-04 eta: 6:19:17 time: 0.580160 data_time: 0.085007 memory: 14267 loss_kpt: 0.000667 acc_pose: 0.795041 loss: 0.000667 2022/09/23 15:30:36 - mmengine - INFO - Epoch(train) [60][150/293] lr: 5.000000e-04 eta: 6:18:59 time: 0.575525 data_time: 0.091689 memory: 14267 loss_kpt: 0.000672 acc_pose: 0.783138 loss: 0.000672 2022/09/23 15:31:06 - mmengine - INFO - Epoch(train) [60][200/293] lr: 5.000000e-04 eta: 6:18:43 time: 0.590852 data_time: 0.097903 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.825271 loss: 0.000670 2022/09/23 15:31:35 - mmengine - INFO - Epoch(train) [60][250/293] lr: 5.000000e-04 eta: 6:18:26 time: 0.592886 data_time: 0.090950 memory: 14267 loss_kpt: 0.000679 acc_pose: 0.781249 loss: 0.000679 2022/09/23 15:32:01 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:32:01 - mmengine - INFO - Saving checkpoint at 60 epochs 2022/09/23 15:32:26 - mmengine - INFO - Epoch(val) [60][50/407] eta: 0:02:16 time: 0.382428 data_time: 0.075156 memory: 14267 2022/09/23 15:32:43 - mmengine - INFO - Epoch(val) [60][100/407] eta: 0:01:44 time: 0.339896 data_time: 0.024299 memory: 1464 2022/09/23 15:33:01 - mmengine - INFO - Epoch(val) [60][150/407] eta: 0:01:30 time: 0.352931 data_time: 0.041374 memory: 1464 2022/09/23 15:33:17 - mmengine - INFO - Epoch(val) [60][200/407] eta: 0:01:06 time: 0.321275 data_time: 0.029071 memory: 1464 2022/09/23 15:33:35 - mmengine - INFO - Epoch(val) [60][250/407] eta: 0:00:56 time: 0.358290 data_time: 0.043865 memory: 1464 2022/09/23 15:33:52 - mmengine - INFO - Epoch(val) [60][300/407] eta: 0:00:37 time: 0.348026 data_time: 0.038177 memory: 1464 2022/09/23 15:34:10 - mmengine - INFO - Epoch(val) [60][350/407] eta: 0:00:20 time: 0.351346 data_time: 0.042666 memory: 1464 2022/09/23 15:34:23 - mmengine - INFO - Epoch(val) [60][400/407] eta: 0:00:01 time: 0.264729 data_time: 0.033692 memory: 1464 2022/09/23 15:34:57 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 15:35:10 - mmengine - INFO - Epoch(val) [60][407/407] coco/AP: 0.701451 coco/AP .5: 0.887881 coco/AP .75: 0.765524 coco/AP (M): 0.652925 coco/AP (L): 0.780560 coco/AR: 0.753652 coco/AR .5: 0.923646 coco/AR .75: 0.811713 coco/AR (M): 0.701857 coco/AR (L): 0.827165 2022/09/23 15:35:10 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_50.pth is removed 2022/09/23 15:35:13 - mmengine - INFO - The best checkpoint with 0.7015 coco/AP at 60 epoch is saved to best_coco/AP_epoch_60.pth. 2022/09/23 15:35:42 - mmengine - INFO - Epoch(train) [61][50/293] lr: 5.000000e-04 eta: 6:16:53 time: 0.597420 data_time: 0.095251 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.822822 loss: 0.000653 2022/09/23 15:36:12 - mmengine - INFO - Epoch(train) [61][100/293] lr: 5.000000e-04 eta: 6:16:37 time: 0.591412 data_time: 0.099777 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.771057 loss: 0.000670 2022/09/23 15:36:41 - mmengine - INFO - Epoch(train) [61][150/293] lr: 5.000000e-04 eta: 6:16:19 time: 0.579327 data_time: 0.089217 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.831139 loss: 0.000670 2022/09/23 15:37:10 - mmengine - INFO - Epoch(train) [61][200/293] lr: 5.000000e-04 eta: 6:16:01 time: 0.574821 data_time: 0.086180 memory: 14267 loss_kpt: 0.000672 acc_pose: 0.778712 loss: 0.000672 2022/09/23 15:37:40 - mmengine - INFO - Epoch(train) [61][250/293] lr: 5.000000e-04 eta: 6:15:45 time: 0.599291 data_time: 0.097660 memory: 14267 loss_kpt: 0.000669 acc_pose: 0.798013 loss: 0.000669 2022/09/23 15:38:04 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:38:35 - mmengine - INFO - Epoch(train) [62][50/293] lr: 5.000000e-04 eta: 6:14:14 time: 0.606827 data_time: 0.109266 memory: 14267 loss_kpt: 0.000674 acc_pose: 0.819550 loss: 0.000674 2022/09/23 15:39:06 - mmengine - INFO - Epoch(train) [62][100/293] lr: 5.000000e-04 eta: 6:14:01 time: 0.620872 data_time: 0.106048 memory: 14267 loss_kpt: 0.000674 acc_pose: 0.789401 loss: 0.000674 2022/09/23 15:39:21 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:39:34 - mmengine - INFO - Epoch(train) [62][150/293] lr: 5.000000e-04 eta: 6:13:42 time: 0.567124 data_time: 0.090031 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.845281 loss: 0.000668 2022/09/23 15:40:04 - mmengine - INFO - Epoch(train) [62][200/293] lr: 5.000000e-04 eta: 6:13:25 time: 0.593154 data_time: 0.081771 memory: 14267 loss_kpt: 0.000662 acc_pose: 0.835115 loss: 0.000662 2022/09/23 15:40:34 - mmengine - INFO - Epoch(train) [62][250/293] lr: 5.000000e-04 eta: 6:13:10 time: 0.598782 data_time: 0.085743 memory: 14267 loss_kpt: 0.000667 acc_pose: 0.774155 loss: 0.000667 2022/09/23 15:40:58 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:41:28 - mmengine - INFO - Epoch(train) [63][50/293] lr: 5.000000e-04 eta: 6:11:39 time: 0.604274 data_time: 0.104414 memory: 14267 loss_kpt: 0.000655 acc_pose: 0.805319 loss: 0.000655 2022/09/23 15:41:58 - mmengine - INFO - Epoch(train) [63][100/293] lr: 5.000000e-04 eta: 6:11:22 time: 0.589261 data_time: 0.090841 memory: 14267 loss_kpt: 0.000654 acc_pose: 0.784490 loss: 0.000654 2022/09/23 15:42:27 - mmengine - INFO - Epoch(train) [63][150/293] lr: 5.000000e-04 eta: 6:11:05 time: 0.587553 data_time: 0.078769 memory: 14267 loss_kpt: 0.000672 acc_pose: 0.774903 loss: 0.000672 2022/09/23 15:42:55 - mmengine - INFO - Epoch(train) [63][200/293] lr: 5.000000e-04 eta: 6:10:46 time: 0.567327 data_time: 0.086889 memory: 14267 loss_kpt: 0.000672 acc_pose: 0.783222 loss: 0.000672 2022/09/23 15:43:25 - mmengine - INFO - Epoch(train) [63][250/293] lr: 5.000000e-04 eta: 6:10:30 time: 0.598916 data_time: 0.107092 memory: 14267 loss_kpt: 0.000669 acc_pose: 0.792599 loss: 0.000669 2022/09/23 15:43:51 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:44:21 - mmengine - INFO - Epoch(train) [64][50/293] lr: 5.000000e-04 eta: 6:09:01 time: 0.607952 data_time: 0.112635 memory: 14267 loss_kpt: 0.000673 acc_pose: 0.786088 loss: 0.000673 2022/09/23 15:44:51 - mmengine - INFO - Epoch(train) [64][100/293] lr: 5.000000e-04 eta: 6:08:44 time: 0.593563 data_time: 0.082881 memory: 14267 loss_kpt: 0.000666 acc_pose: 0.774648 loss: 0.000666 2022/09/23 15:45:20 - mmengine - INFO - Epoch(train) [64][150/293] lr: 5.000000e-04 eta: 6:08:26 time: 0.576806 data_time: 0.081155 memory: 14267 loss_kpt: 0.000671 acc_pose: 0.787407 loss: 0.000671 2022/09/23 15:45:49 - mmengine - INFO - Epoch(train) [64][200/293] lr: 5.000000e-04 eta: 6:08:08 time: 0.582214 data_time: 0.086207 memory: 14267 loss_kpt: 0.000676 acc_pose: 0.842774 loss: 0.000676 2022/09/23 15:46:17 - mmengine - INFO - Epoch(train) [64][250/293] lr: 5.000000e-04 eta: 6:07:48 time: 0.567809 data_time: 0.083229 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.874107 loss: 0.000670 2022/09/23 15:46:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:47:12 - mmengine - INFO - Epoch(train) [65][50/293] lr: 5.000000e-04 eta: 6:06:20 time: 0.611792 data_time: 0.104464 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.768872 loss: 0.000663 2022/09/23 15:47:40 - mmengine - INFO - Epoch(train) [65][100/293] lr: 5.000000e-04 eta: 6:06:02 time: 0.576037 data_time: 0.086606 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.818651 loss: 0.000668 2022/09/23 15:48:11 - mmengine - INFO - Epoch(train) [65][150/293] lr: 5.000000e-04 eta: 6:05:47 time: 0.608974 data_time: 0.095512 memory: 14267 loss_kpt: 0.000659 acc_pose: 0.837172 loss: 0.000659 2022/09/23 15:48:41 - mmengine - INFO - Epoch(train) [65][200/293] lr: 5.000000e-04 eta: 6:05:30 time: 0.594412 data_time: 0.096231 memory: 14267 loss_kpt: 0.000665 acc_pose: 0.826228 loss: 0.000665 2022/09/23 15:49:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:49:09 - mmengine - INFO - Epoch(train) [65][250/293] lr: 5.000000e-04 eta: 6:05:10 time: 0.569826 data_time: 0.079235 memory: 14267 loss_kpt: 0.000666 acc_pose: 0.811144 loss: 0.000666 2022/09/23 15:49:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:50:04 - mmengine - INFO - Epoch(train) [66][50/293] lr: 5.000000e-04 eta: 6:03:43 time: 0.604146 data_time: 0.098401 memory: 14267 loss_kpt: 0.000677 acc_pose: 0.786234 loss: 0.000677 2022/09/23 15:50:33 - mmengine - INFO - Epoch(train) [66][100/293] lr: 5.000000e-04 eta: 6:03:26 time: 0.590273 data_time: 0.087963 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.806360 loss: 0.000670 2022/09/23 15:51:04 - mmengine - INFO - Epoch(train) [66][150/293] lr: 5.000000e-04 eta: 6:03:10 time: 0.602505 data_time: 0.081315 memory: 14267 loss_kpt: 0.000678 acc_pose: 0.762439 loss: 0.000678 2022/09/23 15:51:33 - mmengine - INFO - Epoch(train) [66][200/293] lr: 5.000000e-04 eta: 6:02:53 time: 0.596888 data_time: 0.095870 memory: 14267 loss_kpt: 0.000669 acc_pose: 0.808762 loss: 0.000669 2022/09/23 15:52:03 - mmengine - INFO - Epoch(train) [66][250/293] lr: 5.000000e-04 eta: 6:02:37 time: 0.600218 data_time: 0.100475 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.825071 loss: 0.000670 2022/09/23 15:52:28 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:52:58 - mmengine - INFO - Epoch(train) [67][50/293] lr: 5.000000e-04 eta: 6:01:09 time: 0.592534 data_time: 0.113121 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.842020 loss: 0.000668 2022/09/23 15:53:27 - mmengine - INFO - Epoch(train) [67][100/293] lr: 5.000000e-04 eta: 6:00:50 time: 0.579897 data_time: 0.097053 memory: 14267 loss_kpt: 0.000661 acc_pose: 0.851218 loss: 0.000661 2022/09/23 15:53:56 - mmengine - INFO - Epoch(train) [67][150/293] lr: 5.000000e-04 eta: 6:00:32 time: 0.584285 data_time: 0.078041 memory: 14267 loss_kpt: 0.000667 acc_pose: 0.776083 loss: 0.000667 2022/09/23 15:54:25 - mmengine - INFO - Epoch(train) [67][200/293] lr: 5.000000e-04 eta: 6:00:14 time: 0.588394 data_time: 0.088631 memory: 14267 loss_kpt: 0.000662 acc_pose: 0.789416 loss: 0.000662 2022/09/23 15:54:55 - mmengine - INFO - Epoch(train) [67][250/293] lr: 5.000000e-04 eta: 5:59:57 time: 0.597635 data_time: 0.096377 memory: 14267 loss_kpt: 0.000670 acc_pose: 0.803064 loss: 0.000670 2022/09/23 15:55:21 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:55:52 - mmengine - INFO - Epoch(train) [68][50/293] lr: 5.000000e-04 eta: 5:58:33 time: 0.610267 data_time: 0.088451 memory: 14267 loss_kpt: 0.000647 acc_pose: 0.837816 loss: 0.000647 2022/09/23 15:56:21 - mmengine - INFO - Epoch(train) [68][100/293] lr: 5.000000e-04 eta: 5:58:14 time: 0.582151 data_time: 0.094402 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.788972 loss: 0.000663 2022/09/23 15:56:50 - mmengine - INFO - Epoch(train) [68][150/293] lr: 5.000000e-04 eta: 5:57:56 time: 0.584869 data_time: 0.086462 memory: 14267 loss_kpt: 0.000661 acc_pose: 0.789224 loss: 0.000661 2022/09/23 15:57:20 - mmengine - INFO - Epoch(train) [68][200/293] lr: 5.000000e-04 eta: 5:57:39 time: 0.596534 data_time: 0.101212 memory: 14267 loss_kpt: 0.000666 acc_pose: 0.768653 loss: 0.000666 2022/09/23 15:57:48 - mmengine - INFO - Epoch(train) [68][250/293] lr: 5.000000e-04 eta: 5:57:19 time: 0.569180 data_time: 0.074896 memory: 14267 loss_kpt: 0.000666 acc_pose: 0.815401 loss: 0.000666 2022/09/23 15:58:13 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:58:43 - mmengine - INFO - Epoch(train) [69][50/293] lr: 5.000000e-04 eta: 5:55:53 time: 0.593356 data_time: 0.100478 memory: 14267 loss_kpt: 0.000664 acc_pose: 0.834489 loss: 0.000664 2022/09/23 15:58:59 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 15:59:13 - mmengine - INFO - Epoch(train) [69][100/293] lr: 5.000000e-04 eta: 5:55:37 time: 0.603245 data_time: 0.087299 memory: 14267 loss_kpt: 0.000674 acc_pose: 0.794248 loss: 0.000674 2022/09/23 15:59:42 - mmengine - INFO - Epoch(train) [69][150/293] lr: 5.000000e-04 eta: 5:55:18 time: 0.581847 data_time: 0.099028 memory: 14267 loss_kpt: 0.000659 acc_pose: 0.803176 loss: 0.000659 2022/09/23 16:00:12 - mmengine - INFO - Epoch(train) [69][200/293] lr: 5.000000e-04 eta: 5:55:00 time: 0.591764 data_time: 0.091467 memory: 14267 loss_kpt: 0.000675 acc_pose: 0.790891 loss: 0.000675 2022/09/23 16:00:42 - mmengine - INFO - Epoch(train) [69][250/293] lr: 5.000000e-04 eta: 5:54:43 time: 0.596395 data_time: 0.084084 memory: 14267 loss_kpt: 0.000680 acc_pose: 0.832046 loss: 0.000680 2022/09/23 16:01:05 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:01:36 - mmengine - INFO - Epoch(train) [70][50/293] lr: 5.000000e-04 eta: 5:53:19 time: 0.606144 data_time: 0.098158 memory: 14267 loss_kpt: 0.000665 acc_pose: 0.804503 loss: 0.000665 2022/09/23 16:02:05 - mmengine - INFO - Epoch(train) [70][100/293] lr: 5.000000e-04 eta: 5:53:02 time: 0.592472 data_time: 0.092904 memory: 14267 loss_kpt: 0.000669 acc_pose: 0.788632 loss: 0.000669 2022/09/23 16:02:35 - mmengine - INFO - Epoch(train) [70][150/293] lr: 5.000000e-04 eta: 5:52:43 time: 0.588707 data_time: 0.075741 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.823009 loss: 0.000668 2022/09/23 16:03:04 - mmengine - INFO - Epoch(train) [70][200/293] lr: 5.000000e-04 eta: 5:52:25 time: 0.590788 data_time: 0.083253 memory: 14267 loss_kpt: 0.000669 acc_pose: 0.813755 loss: 0.000669 2022/09/23 16:03:34 - mmengine - INFO - Epoch(train) [70][250/293] lr: 5.000000e-04 eta: 5:52:07 time: 0.590158 data_time: 0.076842 memory: 14267 loss_kpt: 0.000637 acc_pose: 0.752738 loss: 0.000637 2022/09/23 16:03:59 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:03:59 - mmengine - INFO - Saving checkpoint at 70 epochs 2022/09/23 16:04:24 - mmengine - INFO - Epoch(val) [70][50/407] eta: 0:02:04 time: 0.349769 data_time: 0.048581 memory: 14267 2022/09/23 16:04:42 - mmengine - INFO - Epoch(val) [70][100/407] eta: 0:01:49 time: 0.358167 data_time: 0.051595 memory: 1464 2022/09/23 16:05:00 - mmengine - INFO - Epoch(val) [70][150/407] eta: 0:01:34 time: 0.367673 data_time: 0.051130 memory: 1464 2022/09/23 16:05:17 - mmengine - INFO - Epoch(val) [70][200/407] eta: 0:01:10 time: 0.339250 data_time: 0.022915 memory: 1464 2022/09/23 16:05:35 - mmengine - INFO - Epoch(val) [70][250/407] eta: 0:00:54 time: 0.348730 data_time: 0.025799 memory: 1464 2022/09/23 16:05:51 - mmengine - INFO - Epoch(val) [70][300/407] eta: 0:00:35 time: 0.328189 data_time: 0.027407 memory: 1464 2022/09/23 16:06:09 - mmengine - INFO - Epoch(val) [70][350/407] eta: 0:00:20 time: 0.353626 data_time: 0.046323 memory: 1464 2022/09/23 16:06:22 - mmengine - INFO - Epoch(val) [70][400/407] eta: 0:00:01 time: 0.257167 data_time: 0.018005 memory: 1464 2022/09/23 16:06:55 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 16:07:08 - mmengine - INFO - Epoch(val) [70][407/407] coco/AP: 0.703101 coco/AP .5: 0.886975 coco/AP .75: 0.767051 coco/AP (M): 0.656497 coco/AP (L): 0.780024 coco/AR: 0.756895 coco/AR .5: 0.926008 coco/AR .75: 0.816278 coco/AR (M): 0.705845 coco/AR (L): 0.829803 2022/09/23 16:07:08 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_60.pth is removed 2022/09/23 16:07:11 - mmengine - INFO - The best checkpoint with 0.7031 coco/AP at 70 epoch is saved to best_coco/AP_epoch_70.pth. 2022/09/23 16:07:40 - mmengine - INFO - Epoch(train) [71][50/293] lr: 5.000000e-04 eta: 5:50:43 time: 0.588979 data_time: 0.099676 memory: 14267 loss_kpt: 0.000655 acc_pose: 0.794459 loss: 0.000655 2022/09/23 16:08:08 - mmengine - INFO - Epoch(train) [71][100/293] lr: 5.000000e-04 eta: 5:50:22 time: 0.562840 data_time: 0.073363 memory: 14267 loss_kpt: 0.000657 acc_pose: 0.839937 loss: 0.000657 2022/09/23 16:08:38 - mmengine - INFO - Epoch(train) [71][150/293] lr: 5.000000e-04 eta: 5:50:04 time: 0.591018 data_time: 0.082306 memory: 14267 loss_kpt: 0.000673 acc_pose: 0.790003 loss: 0.000673 2022/09/23 16:09:07 - mmengine - INFO - Epoch(train) [71][200/293] lr: 5.000000e-04 eta: 5:49:46 time: 0.592754 data_time: 0.096687 memory: 14267 loss_kpt: 0.000664 acc_pose: 0.781621 loss: 0.000664 2022/09/23 16:09:37 - mmengine - INFO - Epoch(train) [71][250/293] lr: 5.000000e-04 eta: 5:49:28 time: 0.593645 data_time: 0.089577 memory: 14267 loss_kpt: 0.000665 acc_pose: 0.793636 loss: 0.000665 2022/09/23 16:10:02 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:10:34 - mmengine - INFO - Epoch(train) [72][50/293] lr: 5.000000e-04 eta: 5:48:09 time: 0.638191 data_time: 0.116374 memory: 14267 loss_kpt: 0.000652 acc_pose: 0.745931 loss: 0.000652 2022/09/23 16:11:03 - mmengine - INFO - Epoch(train) [72][100/293] lr: 5.000000e-04 eta: 5:47:50 time: 0.577794 data_time: 0.101764 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.833699 loss: 0.000668 2022/09/23 16:11:33 - mmengine - INFO - Epoch(train) [72][150/293] lr: 5.000000e-04 eta: 5:47:32 time: 0.600172 data_time: 0.084202 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.865741 loss: 0.000663 2022/09/23 16:12:01 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:12:02 - mmengine - INFO - Epoch(train) [72][200/293] lr: 5.000000e-04 eta: 5:47:13 time: 0.580509 data_time: 0.079642 memory: 14267 loss_kpt: 0.000664 acc_pose: 0.773338 loss: 0.000664 2022/09/23 16:12:31 - mmengine - INFO - Epoch(train) [72][250/293] lr: 5.000000e-04 eta: 5:46:53 time: 0.567109 data_time: 0.088209 memory: 14267 loss_kpt: 0.000673 acc_pose: 0.762852 loss: 0.000673 2022/09/23 16:12:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:13:26 - mmengine - INFO - Epoch(train) [73][50/293] lr: 5.000000e-04 eta: 5:45:31 time: 0.608266 data_time: 0.102326 memory: 14267 loss_kpt: 0.000660 acc_pose: 0.825605 loss: 0.000660 2022/09/23 16:13:56 - mmengine - INFO - Epoch(train) [73][100/293] lr: 5.000000e-04 eta: 5:45:13 time: 0.592811 data_time: 0.089425 memory: 14267 loss_kpt: 0.000648 acc_pose: 0.806133 loss: 0.000648 2022/09/23 16:14:25 - mmengine - INFO - Epoch(train) [73][150/293] lr: 5.000000e-04 eta: 5:44:55 time: 0.588228 data_time: 0.089890 memory: 14267 loss_kpt: 0.000657 acc_pose: 0.787976 loss: 0.000657 2022/09/23 16:14:54 - mmengine - INFO - Epoch(train) [73][200/293] lr: 5.000000e-04 eta: 5:44:36 time: 0.588674 data_time: 0.085563 memory: 14267 loss_kpt: 0.000665 acc_pose: 0.828185 loss: 0.000665 2022/09/23 16:15:24 - mmengine - INFO - Epoch(train) [73][250/293] lr: 5.000000e-04 eta: 5:44:19 time: 0.600554 data_time: 0.108376 memory: 14267 loss_kpt: 0.000659 acc_pose: 0.834513 loss: 0.000659 2022/09/23 16:15:49 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:16:21 - mmengine - INFO - Epoch(train) [74][50/293] lr: 5.000000e-04 eta: 5:43:01 time: 0.637002 data_time: 0.127647 memory: 14267 loss_kpt: 0.000648 acc_pose: 0.800065 loss: 0.000648 2022/09/23 16:16:51 - mmengine - INFO - Epoch(train) [74][100/293] lr: 5.000000e-04 eta: 5:42:43 time: 0.597127 data_time: 0.102929 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.844161 loss: 0.000653 2022/09/23 16:17:20 - mmengine - INFO - Epoch(train) [74][150/293] lr: 5.000000e-04 eta: 5:42:23 time: 0.575990 data_time: 0.081126 memory: 14267 loss_kpt: 0.000673 acc_pose: 0.827387 loss: 0.000673 2022/09/23 16:17:49 - mmengine - INFO - Epoch(train) [74][200/293] lr: 5.000000e-04 eta: 5:42:03 time: 0.574187 data_time: 0.087618 memory: 14267 loss_kpt: 0.000671 acc_pose: 0.799290 loss: 0.000671 2022/09/23 16:18:18 - mmengine - INFO - Epoch(train) [74][250/293] lr: 5.000000e-04 eta: 5:41:44 time: 0.582374 data_time: 0.089084 memory: 14267 loss_kpt: 0.000677 acc_pose: 0.786289 loss: 0.000677 2022/09/23 16:18:42 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:19:13 - mmengine - INFO - Epoch(train) [75][50/293] lr: 5.000000e-04 eta: 5:40:26 time: 0.627947 data_time: 0.118996 memory: 14267 loss_kpt: 0.000658 acc_pose: 0.789280 loss: 0.000658 2022/09/23 16:19:43 - mmengine - INFO - Epoch(train) [75][100/293] lr: 5.000000e-04 eta: 5:40:08 time: 0.593706 data_time: 0.093010 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.765708 loss: 0.000663 2022/09/23 16:20:13 - mmengine - INFO - Epoch(train) [75][150/293] lr: 5.000000e-04 eta: 5:39:49 time: 0.590900 data_time: 0.087171 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.783044 loss: 0.000638 2022/09/23 16:20:42 - mmengine - INFO - Epoch(train) [75][200/293] lr: 5.000000e-04 eta: 5:39:30 time: 0.586216 data_time: 0.094342 memory: 14267 loss_kpt: 0.000647 acc_pose: 0.818635 loss: 0.000647 2022/09/23 16:21:13 - mmengine - INFO - Epoch(train) [75][250/293] lr: 5.000000e-04 eta: 5:39:13 time: 0.611797 data_time: 0.104356 memory: 14267 loss_kpt: 0.000655 acc_pose: 0.788812 loss: 0.000655 2022/09/23 16:21:36 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:21:53 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:22:07 - mmengine - INFO - Epoch(train) [76][50/293] lr: 5.000000e-04 eta: 5:37:55 time: 0.620834 data_time: 0.109808 memory: 14267 loss_kpt: 0.000654 acc_pose: 0.827234 loss: 0.000654 2022/09/23 16:22:37 - mmengine - INFO - Epoch(train) [76][100/293] lr: 5.000000e-04 eta: 5:37:37 time: 0.600020 data_time: 0.100066 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.782764 loss: 0.000653 2022/09/23 16:23:07 - mmengine - INFO - Epoch(train) [76][150/293] lr: 5.000000e-04 eta: 5:37:19 time: 0.591419 data_time: 0.097831 memory: 14267 loss_kpt: 0.000656 acc_pose: 0.788986 loss: 0.000656 2022/09/23 16:23:37 - mmengine - INFO - Epoch(train) [76][200/293] lr: 5.000000e-04 eta: 5:37:00 time: 0.588943 data_time: 0.090184 memory: 14267 loss_kpt: 0.000657 acc_pose: 0.794278 loss: 0.000657 2022/09/23 16:24:06 - mmengine - INFO - Epoch(train) [76][250/293] lr: 5.000000e-04 eta: 5:36:41 time: 0.595652 data_time: 0.096302 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.810106 loss: 0.000663 2022/09/23 16:24:32 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:25:02 - mmengine - INFO - Epoch(train) [77][50/293] lr: 5.000000e-04 eta: 5:35:23 time: 0.610847 data_time: 0.123326 memory: 14267 loss_kpt: 0.000649 acc_pose: 0.810348 loss: 0.000649 2022/09/23 16:25:32 - mmengine - INFO - Epoch(train) [77][100/293] lr: 5.000000e-04 eta: 5:35:04 time: 0.587919 data_time: 0.096271 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.812238 loss: 0.000653 2022/09/23 16:26:00 - mmengine - INFO - Epoch(train) [77][150/293] lr: 5.000000e-04 eta: 5:34:42 time: 0.559111 data_time: 0.083830 memory: 14267 loss_kpt: 0.000649 acc_pose: 0.790269 loss: 0.000649 2022/09/23 16:26:29 - mmengine - INFO - Epoch(train) [77][200/293] lr: 5.000000e-04 eta: 5:34:22 time: 0.577135 data_time: 0.087980 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.783140 loss: 0.000663 2022/09/23 16:26:57 - mmengine - INFO - Epoch(train) [77][250/293] lr: 5.000000e-04 eta: 5:34:01 time: 0.572178 data_time: 0.086918 memory: 14267 loss_kpt: 0.000641 acc_pose: 0.793173 loss: 0.000641 2022/09/23 16:27:23 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:27:54 - mmengine - INFO - Epoch(train) [78][50/293] lr: 5.000000e-04 eta: 5:32:45 time: 0.627131 data_time: 0.117604 memory: 14267 loss_kpt: 0.000663 acc_pose: 0.851397 loss: 0.000663 2022/09/23 16:28:24 - mmengine - INFO - Epoch(train) [78][100/293] lr: 5.000000e-04 eta: 5:32:27 time: 0.593162 data_time: 0.109408 memory: 14267 loss_kpt: 0.000645 acc_pose: 0.783646 loss: 0.000645 2022/09/23 16:28:53 - mmengine - INFO - Epoch(train) [78][150/293] lr: 5.000000e-04 eta: 5:32:08 time: 0.590824 data_time: 0.093420 memory: 14267 loss_kpt: 0.000668 acc_pose: 0.839495 loss: 0.000668 2022/09/23 16:29:23 - mmengine - INFO - Epoch(train) [78][200/293] lr: 5.000000e-04 eta: 5:31:48 time: 0.583097 data_time: 0.102063 memory: 14267 loss_kpt: 0.000652 acc_pose: 0.796024 loss: 0.000652 2022/09/23 16:29:53 - mmengine - INFO - Epoch(train) [78][250/293] lr: 5.000000e-04 eta: 5:31:30 time: 0.601908 data_time: 0.086272 memory: 14267 loss_kpt: 0.000665 acc_pose: 0.845108 loss: 0.000665 2022/09/23 16:30:18 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:30:50 - mmengine - INFO - Epoch(train) [79][50/293] lr: 5.000000e-04 eta: 5:30:14 time: 0.628404 data_time: 0.097658 memory: 14267 loss_kpt: 0.000650 acc_pose: 0.842518 loss: 0.000650 2022/09/23 16:31:19 - mmengine - INFO - Epoch(train) [79][100/293] lr: 5.000000e-04 eta: 5:29:55 time: 0.588789 data_time: 0.103111 memory: 14267 loss_kpt: 0.000642 acc_pose: 0.781417 loss: 0.000642 2022/09/23 16:31:46 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:31:49 - mmengine - INFO - Epoch(train) [79][150/293] lr: 5.000000e-04 eta: 5:29:36 time: 0.591331 data_time: 0.107526 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.807873 loss: 0.000653 2022/09/23 16:32:18 - mmengine - INFO - Epoch(train) [79][200/293] lr: 5.000000e-04 eta: 5:29:16 time: 0.587803 data_time: 0.081142 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.809113 loss: 0.000638 2022/09/23 16:32:48 - mmengine - INFO - Epoch(train) [79][250/293] lr: 5.000000e-04 eta: 5:28:57 time: 0.590192 data_time: 0.090406 memory: 14267 loss_kpt: 0.000642 acc_pose: 0.806980 loss: 0.000642 2022/09/23 16:33:12 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:33:44 - mmengine - INFO - Epoch(train) [80][50/293] lr: 5.000000e-04 eta: 5:27:43 time: 0.631805 data_time: 0.131786 memory: 14267 loss_kpt: 0.000639 acc_pose: 0.831875 loss: 0.000639 2022/09/23 16:34:12 - mmengine - INFO - Epoch(train) [80][100/293] lr: 5.000000e-04 eta: 5:27:20 time: 0.554582 data_time: 0.085044 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.785657 loss: 0.000653 2022/09/23 16:34:41 - mmengine - INFO - Epoch(train) [80][150/293] lr: 5.000000e-04 eta: 5:27:00 time: 0.577224 data_time: 0.086249 memory: 14267 loss_kpt: 0.000669 acc_pose: 0.835085 loss: 0.000669 2022/09/23 16:35:09 - mmengine - INFO - Epoch(train) [80][200/293] lr: 5.000000e-04 eta: 5:26:39 time: 0.571366 data_time: 0.091412 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.784712 loss: 0.000653 2022/09/23 16:35:38 - mmengine - INFO - Epoch(train) [80][250/293] lr: 5.000000e-04 eta: 5:26:19 time: 0.575607 data_time: 0.081708 memory: 14267 loss_kpt: 0.000653 acc_pose: 0.778344 loss: 0.000653 2022/09/23 16:36:03 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:36:03 - mmengine - INFO - Saving checkpoint at 80 epochs 2022/09/23 16:36:29 - mmengine - INFO - Epoch(val) [80][50/407] eta: 0:02:14 time: 0.377220 data_time: 0.058279 memory: 14267 2022/09/23 16:36:47 - mmengine - INFO - Epoch(val) [80][100/407] eta: 0:01:49 time: 0.356346 data_time: 0.047092 memory: 1464 2022/09/23 16:37:05 - mmengine - INFO - Epoch(val) [80][150/407] eta: 0:01:29 time: 0.348001 data_time: 0.041411 memory: 1464 2022/09/23 16:37:22 - mmengine - INFO - Epoch(val) [80][200/407] eta: 0:01:12 time: 0.351433 data_time: 0.024668 memory: 1464 2022/09/23 16:37:40 - mmengine - INFO - Epoch(val) [80][250/407] eta: 0:00:54 time: 0.345866 data_time: 0.036456 memory: 1464 2022/09/23 16:37:57 - mmengine - INFO - Epoch(val) [80][300/407] eta: 0:00:36 time: 0.344869 data_time: 0.045038 memory: 1464 2022/09/23 16:38:14 - mmengine - INFO - Epoch(val) [80][350/407] eta: 0:00:19 time: 0.337394 data_time: 0.024290 memory: 1464 2022/09/23 16:38:26 - mmengine - INFO - Epoch(val) [80][400/407] eta: 0:00:01 time: 0.242734 data_time: 0.029672 memory: 1464 2022/09/23 16:38:59 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 16:39:12 - mmengine - INFO - Epoch(val) [80][407/407] coco/AP: 0.705445 coco/AP .5: 0.890492 coco/AP .75: 0.770076 coco/AP (M): 0.659064 coco/AP (L): 0.782334 coco/AR: 0.758580 coco/AR .5: 0.927424 coco/AR .75: 0.818955 coco/AR (M): 0.708249 coco/AR (L): 0.830769 2022/09/23 16:39:12 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_70.pth is removed 2022/09/23 16:39:15 - mmengine - INFO - The best checkpoint with 0.7054 coco/AP at 80 epoch is saved to best_coco/AP_epoch_80.pth. 2022/09/23 16:39:44 - mmengine - INFO - Epoch(train) [81][50/293] lr: 5.000000e-04 eta: 5:25:01 time: 0.591348 data_time: 0.097810 memory: 14267 loss_kpt: 0.000655 acc_pose: 0.835069 loss: 0.000655 2022/09/23 16:40:15 - mmengine - INFO - Epoch(train) [81][100/293] lr: 5.000000e-04 eta: 5:24:44 time: 0.612987 data_time: 0.112372 memory: 14267 loss_kpt: 0.000643 acc_pose: 0.847013 loss: 0.000643 2022/09/23 16:40:45 - mmengine - INFO - Epoch(train) [81][150/293] lr: 5.000000e-04 eta: 5:24:25 time: 0.600399 data_time: 0.095759 memory: 14267 loss_kpt: 0.000644 acc_pose: 0.734523 loss: 0.000644 2022/09/23 16:41:14 - mmengine - INFO - Epoch(train) [81][200/293] lr: 5.000000e-04 eta: 5:24:06 time: 0.588565 data_time: 0.103748 memory: 14267 loss_kpt: 0.000658 acc_pose: 0.733362 loss: 0.000658 2022/09/23 16:41:44 - mmengine - INFO - Epoch(train) [81][250/293] lr: 5.000000e-04 eta: 5:23:46 time: 0.582533 data_time: 0.094431 memory: 14267 loss_kpt: 0.000643 acc_pose: 0.821451 loss: 0.000643 2022/09/23 16:42:09 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:42:40 - mmengine - INFO - Epoch(train) [82][50/293] lr: 5.000000e-04 eta: 5:22:32 time: 0.624539 data_time: 0.102350 memory: 14267 loss_kpt: 0.000632 acc_pose: 0.795026 loss: 0.000632 2022/09/23 16:43:09 - mmengine - INFO - Epoch(train) [82][100/293] lr: 5.000000e-04 eta: 5:22:10 time: 0.561518 data_time: 0.103560 memory: 14267 loss_kpt: 0.000637 acc_pose: 0.802844 loss: 0.000637 2022/09/23 16:43:38 - mmengine - INFO - Epoch(train) [82][150/293] lr: 5.000000e-04 eta: 5:21:50 time: 0.589542 data_time: 0.089329 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.802388 loss: 0.000638 2022/09/23 16:44:08 - mmengine - INFO - Epoch(train) [82][200/293] lr: 5.000000e-04 eta: 5:21:31 time: 0.592536 data_time: 0.086299 memory: 14267 loss_kpt: 0.000654 acc_pose: 0.846229 loss: 0.000654 2022/09/23 16:44:37 - mmengine - INFO - Epoch(train) [82][250/293] lr: 5.000000e-04 eta: 5:21:10 time: 0.580028 data_time: 0.087239 memory: 14267 loss_kpt: 0.000642 acc_pose: 0.814491 loss: 0.000642 2022/09/23 16:44:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:45:02 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:45:33 - mmengine - INFO - Epoch(train) [83][50/293] lr: 5.000000e-04 eta: 5:19:56 time: 0.616275 data_time: 0.103626 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.840139 loss: 0.000629 2022/09/23 16:46:03 - mmengine - INFO - Epoch(train) [83][100/293] lr: 5.000000e-04 eta: 5:19:37 time: 0.599217 data_time: 0.091331 memory: 14267 loss_kpt: 0.000661 acc_pose: 0.808720 loss: 0.000661 2022/09/23 16:46:33 - mmengine - INFO - Epoch(train) [83][150/293] lr: 5.000000e-04 eta: 5:19:19 time: 0.602005 data_time: 0.093516 memory: 14267 loss_kpt: 0.000640 acc_pose: 0.794960 loss: 0.000640 2022/09/23 16:47:02 - mmengine - INFO - Epoch(train) [83][200/293] lr: 5.000000e-04 eta: 5:18:59 time: 0.585435 data_time: 0.089788 memory: 14267 loss_kpt: 0.000643 acc_pose: 0.793632 loss: 0.000643 2022/09/23 16:47:31 - mmengine - INFO - Epoch(train) [83][250/293] lr: 5.000000e-04 eta: 5:18:38 time: 0.579928 data_time: 0.088504 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.824542 loss: 0.000630 2022/09/23 16:47:57 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:48:28 - mmengine - INFO - Epoch(train) [84][50/293] lr: 5.000000e-04 eta: 5:17:25 time: 0.624791 data_time: 0.110262 memory: 14267 loss_kpt: 0.000640 acc_pose: 0.800300 loss: 0.000640 2022/09/23 16:48:57 - mmengine - INFO - Epoch(train) [84][100/293] lr: 5.000000e-04 eta: 5:17:06 time: 0.591040 data_time: 0.081324 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.795449 loss: 0.000636 2022/09/23 16:49:27 - mmengine - INFO - Epoch(train) [84][150/293] lr: 5.000000e-04 eta: 5:16:46 time: 0.590425 data_time: 0.093832 memory: 14267 loss_kpt: 0.000644 acc_pose: 0.857061 loss: 0.000644 2022/09/23 16:49:56 - mmengine - INFO - Epoch(train) [84][200/293] lr: 5.000000e-04 eta: 5:16:25 time: 0.578368 data_time: 0.075954 memory: 14267 loss_kpt: 0.000634 acc_pose: 0.805564 loss: 0.000634 2022/09/23 16:50:26 - mmengine - INFO - Epoch(train) [84][250/293] lr: 5.000000e-04 eta: 5:16:06 time: 0.598052 data_time: 0.091658 memory: 14267 loss_kpt: 0.000647 acc_pose: 0.839019 loss: 0.000647 2022/09/23 16:50:50 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:51:22 - mmengine - INFO - Epoch(train) [85][50/293] lr: 5.000000e-04 eta: 5:14:54 time: 0.627133 data_time: 0.114657 memory: 14267 loss_kpt: 0.000634 acc_pose: 0.830473 loss: 0.000634 2022/09/23 16:51:51 - mmengine - INFO - Epoch(train) [85][100/293] lr: 5.000000e-04 eta: 5:14:33 time: 0.575349 data_time: 0.089308 memory: 14267 loss_kpt: 0.000640 acc_pose: 0.827021 loss: 0.000640 2022/09/23 16:52:21 - mmengine - INFO - Epoch(train) [85][150/293] lr: 5.000000e-04 eta: 5:14:15 time: 0.614097 data_time: 0.102136 memory: 14267 loss_kpt: 0.000647 acc_pose: 0.691410 loss: 0.000647 2022/09/23 16:52:50 - mmengine - INFO - Epoch(train) [85][200/293] lr: 5.000000e-04 eta: 5:13:54 time: 0.583554 data_time: 0.087724 memory: 14267 loss_kpt: 0.000652 acc_pose: 0.843555 loss: 0.000652 2022/09/23 16:53:20 - mmengine - INFO - Epoch(train) [85][250/293] lr: 5.000000e-04 eta: 5:13:34 time: 0.581626 data_time: 0.091268 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.825442 loss: 0.000638 2022/09/23 16:53:45 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:54:16 - mmengine - INFO - Epoch(train) [86][50/293] lr: 5.000000e-04 eta: 5:12:21 time: 0.613847 data_time: 0.108113 memory: 14267 loss_kpt: 0.000633 acc_pose: 0.821238 loss: 0.000633 2022/09/23 16:54:42 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:54:45 - mmengine - INFO - Epoch(train) [86][100/293] lr: 5.000000e-04 eta: 5:12:00 time: 0.582490 data_time: 0.090907 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.818714 loss: 0.000631 2022/09/23 16:55:14 - mmengine - INFO - Epoch(train) [86][150/293] lr: 5.000000e-04 eta: 5:11:40 time: 0.582481 data_time: 0.096634 memory: 14267 loss_kpt: 0.000651 acc_pose: 0.867272 loss: 0.000651 2022/09/23 16:55:42 - mmengine - INFO - Epoch(train) [86][200/293] lr: 5.000000e-04 eta: 5:11:18 time: 0.567996 data_time: 0.077072 memory: 14267 loss_kpt: 0.000645 acc_pose: 0.786184 loss: 0.000645 2022/09/23 16:56:12 - mmengine - INFO - Epoch(train) [86][250/293] lr: 5.000000e-04 eta: 5:10:58 time: 0.588022 data_time: 0.082442 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.780734 loss: 0.000630 2022/09/23 16:56:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 16:57:08 - mmengine - INFO - Epoch(train) [87][50/293] lr: 5.000000e-04 eta: 5:09:47 time: 0.622741 data_time: 0.106296 memory: 14267 loss_kpt: 0.000647 acc_pose: 0.801001 loss: 0.000647 2022/09/23 16:57:37 - mmengine - INFO - Epoch(train) [87][100/293] lr: 5.000000e-04 eta: 5:09:25 time: 0.572310 data_time: 0.070845 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.785452 loss: 0.000626 2022/09/23 16:58:06 - mmengine - INFO - Epoch(train) [87][150/293] lr: 5.000000e-04 eta: 5:09:04 time: 0.577556 data_time: 0.094844 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.786351 loss: 0.000636 2022/09/23 16:58:35 - mmengine - INFO - Epoch(train) [87][200/293] lr: 5.000000e-04 eta: 5:08:44 time: 0.582578 data_time: 0.082492 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.858666 loss: 0.000630 2022/09/23 16:59:03 - mmengine - INFO - Epoch(train) [87][250/293] lr: 5.000000e-04 eta: 5:08:21 time: 0.556944 data_time: 0.075694 memory: 14267 loss_kpt: 0.000655 acc_pose: 0.815847 loss: 0.000655 2022/09/23 16:59:28 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:00:00 - mmengine - INFO - Epoch(train) [88][50/293] lr: 5.000000e-04 eta: 5:07:12 time: 0.641745 data_time: 0.099568 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.812931 loss: 0.000638 2022/09/23 17:00:29 - mmengine - INFO - Epoch(train) [88][100/293] lr: 5.000000e-04 eta: 5:06:50 time: 0.571060 data_time: 0.098175 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.803490 loss: 0.000622 2022/09/23 17:00:58 - mmengine - INFO - Epoch(train) [88][150/293] lr: 5.000000e-04 eta: 5:06:30 time: 0.591152 data_time: 0.090979 memory: 14267 loss_kpt: 0.000641 acc_pose: 0.732816 loss: 0.000641 2022/09/23 17:01:28 - mmengine - INFO - Epoch(train) [88][200/293] lr: 5.000000e-04 eta: 5:06:10 time: 0.597699 data_time: 0.079989 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.860644 loss: 0.000630 2022/09/23 17:01:58 - mmengine - INFO - Epoch(train) [88][250/293] lr: 5.000000e-04 eta: 5:05:50 time: 0.584382 data_time: 0.082901 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.824017 loss: 0.000636 2022/09/23 17:02:22 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:02:53 - mmengine - INFO - Epoch(train) [89][50/293] lr: 5.000000e-04 eta: 5:04:39 time: 0.615568 data_time: 0.119909 memory: 14267 loss_kpt: 0.000641 acc_pose: 0.793042 loss: 0.000641 2022/09/23 17:03:23 - mmengine - INFO - Epoch(train) [89][100/293] lr: 5.000000e-04 eta: 5:04:19 time: 0.600859 data_time: 0.098219 memory: 14267 loss_kpt: 0.000646 acc_pose: 0.817695 loss: 0.000646 2022/09/23 17:03:52 - mmengine - INFO - Epoch(train) [89][150/293] lr: 5.000000e-04 eta: 5:03:59 time: 0.588641 data_time: 0.091631 memory: 14267 loss_kpt: 0.000648 acc_pose: 0.779866 loss: 0.000648 2022/09/23 17:04:23 - mmengine - INFO - Epoch(train) [89][200/293] lr: 5.000000e-04 eta: 5:03:40 time: 0.611254 data_time: 0.097505 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.823477 loss: 0.000636 2022/09/23 17:04:32 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:04:53 - mmengine - INFO - Epoch(train) [89][250/293] lr: 5.000000e-04 eta: 5:03:20 time: 0.595286 data_time: 0.100452 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.786587 loss: 0.000635 2022/09/23 17:05:17 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:05:49 - mmengine - INFO - Epoch(train) [90][50/293] lr: 5.000000e-04 eta: 5:02:10 time: 0.619031 data_time: 0.118690 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.832080 loss: 0.000629 2022/09/23 17:06:18 - mmengine - INFO - Epoch(train) [90][100/293] lr: 5.000000e-04 eta: 5:01:49 time: 0.583904 data_time: 0.102905 memory: 14267 loss_kpt: 0.000642 acc_pose: 0.851700 loss: 0.000642 2022/09/23 17:06:47 - mmengine - INFO - Epoch(train) [90][150/293] lr: 5.000000e-04 eta: 5:01:28 time: 0.577333 data_time: 0.093256 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.825500 loss: 0.000628 2022/09/23 17:07:17 - mmengine - INFO - Epoch(train) [90][200/293] lr: 5.000000e-04 eta: 5:01:08 time: 0.606115 data_time: 0.110425 memory: 14267 loss_kpt: 0.000645 acc_pose: 0.827279 loss: 0.000645 2022/09/23 17:07:48 - mmengine - INFO - Epoch(train) [90][250/293] lr: 5.000000e-04 eta: 5:00:49 time: 0.613874 data_time: 0.091229 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.813941 loss: 0.000636 2022/09/23 17:08:13 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:08:13 - mmengine - INFO - Saving checkpoint at 90 epochs 2022/09/23 17:08:38 - mmengine - INFO - Epoch(val) [90][50/407] eta: 0:02:12 time: 0.372278 data_time: 0.046649 memory: 14267 2022/09/23 17:08:55 - mmengine - INFO - Epoch(val) [90][100/407] eta: 0:01:46 time: 0.347680 data_time: 0.044019 memory: 1464 2022/09/23 17:09:13 - mmengine - INFO - Epoch(val) [90][150/407] eta: 0:01:30 time: 0.350775 data_time: 0.029357 memory: 1464 2022/09/23 17:09:30 - mmengine - INFO - Epoch(val) [90][200/407] eta: 0:01:09 time: 0.335623 data_time: 0.037467 memory: 1464 2022/09/23 17:09:47 - mmengine - INFO - Epoch(val) [90][250/407] eta: 0:00:55 time: 0.354234 data_time: 0.048578 memory: 1464 2022/09/23 17:10:05 - mmengine - INFO - Epoch(val) [90][300/407] eta: 0:00:36 time: 0.343583 data_time: 0.041308 memory: 1464 2022/09/23 17:10:22 - mmengine - INFO - Epoch(val) [90][350/407] eta: 0:00:19 time: 0.343730 data_time: 0.044618 memory: 1464 2022/09/23 17:10:34 - mmengine - INFO - Epoch(val) [90][400/407] eta: 0:00:01 time: 0.252089 data_time: 0.023017 memory: 1464 2022/09/23 17:11:09 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 17:11:22 - mmengine - INFO - Epoch(val) [90][407/407] coco/AP: 0.713930 coco/AP .5: 0.894663 coco/AP .75: 0.779625 coco/AP (M): 0.668869 coco/AP (L): 0.788651 coco/AR: 0.766357 coco/AR .5: 0.930888 coco/AR .75: 0.826511 coco/AR (M): 0.717045 coco/AR (L): 0.836789 2022/09/23 17:11:22 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_80.pth is removed 2022/09/23 17:11:24 - mmengine - INFO - The best checkpoint with 0.7139 coco/AP at 90 epoch is saved to best_coco/AP_epoch_90.pth. 2022/09/23 17:11:54 - mmengine - INFO - Epoch(train) [91][50/293] lr: 5.000000e-04 eta: 4:59:38 time: 0.595675 data_time: 0.094951 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.820200 loss: 0.000638 2022/09/23 17:12:23 - mmengine - INFO - Epoch(train) [91][100/293] lr: 5.000000e-04 eta: 4:59:17 time: 0.578306 data_time: 0.084964 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.780474 loss: 0.000628 2022/09/23 17:12:51 - mmengine - INFO - Epoch(train) [91][150/293] lr: 5.000000e-04 eta: 4:58:54 time: 0.550753 data_time: 0.076129 memory: 14267 loss_kpt: 0.000637 acc_pose: 0.862120 loss: 0.000637 2022/09/23 17:13:20 - mmengine - INFO - Epoch(train) [91][200/293] lr: 5.000000e-04 eta: 4:58:33 time: 0.593294 data_time: 0.094315 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.793830 loss: 0.000635 2022/09/23 17:13:49 - mmengine - INFO - Epoch(train) [91][250/293] lr: 5.000000e-04 eta: 4:58:12 time: 0.578396 data_time: 0.089197 memory: 14267 loss_kpt: 0.000655 acc_pose: 0.746489 loss: 0.000655 2022/09/23 17:14:14 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:14:45 - mmengine - INFO - Epoch(train) [92][50/293] lr: 5.000000e-04 eta: 4:57:02 time: 0.614838 data_time: 0.105774 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.828378 loss: 0.000629 2022/09/23 17:15:15 - mmengine - INFO - Epoch(train) [92][100/293] lr: 5.000000e-04 eta: 4:56:43 time: 0.608194 data_time: 0.094835 memory: 14267 loss_kpt: 0.000640 acc_pose: 0.831137 loss: 0.000640 2022/09/23 17:15:44 - mmengine - INFO - Epoch(train) [92][150/293] lr: 5.000000e-04 eta: 4:56:22 time: 0.584532 data_time: 0.105378 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.825579 loss: 0.000635 2022/09/23 17:16:14 - mmengine - INFO - Epoch(train) [92][200/293] lr: 5.000000e-04 eta: 4:56:02 time: 0.600945 data_time: 0.098299 memory: 14267 loss_kpt: 0.000637 acc_pose: 0.807070 loss: 0.000637 2022/09/23 17:16:43 - mmengine - INFO - Epoch(train) [92][250/293] lr: 5.000000e-04 eta: 4:55:40 time: 0.564004 data_time: 0.090155 memory: 14267 loss_kpt: 0.000648 acc_pose: 0.794956 loss: 0.000648 2022/09/23 17:17:07 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:17:36 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:17:40 - mmengine - INFO - Epoch(train) [93][50/293] lr: 5.000000e-04 eta: 4:54:33 time: 0.652617 data_time: 0.145994 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.812918 loss: 0.000636 2022/09/23 17:18:10 - mmengine - INFO - Epoch(train) [93][100/293] lr: 5.000000e-04 eta: 4:54:13 time: 0.604183 data_time: 0.099991 memory: 14267 loss_kpt: 0.000645 acc_pose: 0.817966 loss: 0.000645 2022/09/23 17:18:40 - mmengine - INFO - Epoch(train) [93][150/293] lr: 5.000000e-04 eta: 4:53:53 time: 0.596462 data_time: 0.094926 memory: 14267 loss_kpt: 0.000640 acc_pose: 0.823556 loss: 0.000640 2022/09/23 17:19:09 - mmengine - INFO - Epoch(train) [93][200/293] lr: 5.000000e-04 eta: 4:53:32 time: 0.588273 data_time: 0.098518 memory: 14267 loss_kpt: 0.000632 acc_pose: 0.836079 loss: 0.000632 2022/09/23 17:19:38 - mmengine - INFO - Epoch(train) [93][250/293] lr: 5.000000e-04 eta: 4:53:11 time: 0.578001 data_time: 0.088627 memory: 14267 loss_kpt: 0.000634 acc_pose: 0.799378 loss: 0.000634 2022/09/23 17:20:04 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:20:36 - mmengine - INFO - Epoch(train) [94][50/293] lr: 5.000000e-04 eta: 4:52:03 time: 0.638478 data_time: 0.104833 memory: 14267 loss_kpt: 0.000640 acc_pose: 0.764684 loss: 0.000640 2022/09/23 17:21:05 - mmengine - INFO - Epoch(train) [94][100/293] lr: 5.000000e-04 eta: 4:51:42 time: 0.577977 data_time: 0.086488 memory: 14267 loss_kpt: 0.000625 acc_pose: 0.804644 loss: 0.000625 2022/09/23 17:21:34 - mmengine - INFO - Epoch(train) [94][150/293] lr: 5.000000e-04 eta: 4:51:21 time: 0.580087 data_time: 0.089843 memory: 14267 loss_kpt: 0.000641 acc_pose: 0.798031 loss: 0.000641 2022/09/23 17:22:03 - mmengine - INFO - Epoch(train) [94][200/293] lr: 5.000000e-04 eta: 4:51:00 time: 0.587316 data_time: 0.085701 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.807145 loss: 0.000622 2022/09/23 17:22:33 - mmengine - INFO - Epoch(train) [94][250/293] lr: 5.000000e-04 eta: 4:50:39 time: 0.600945 data_time: 0.106739 memory: 14267 loss_kpt: 0.000639 acc_pose: 0.809673 loss: 0.000639 2022/09/23 17:22:59 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:23:30 - mmengine - INFO - Epoch(train) [95][50/293] lr: 5.000000e-04 eta: 4:49:30 time: 0.605909 data_time: 0.090817 memory: 14267 loss_kpt: 0.000633 acc_pose: 0.780167 loss: 0.000633 2022/09/23 17:24:00 - mmengine - INFO - Epoch(train) [95][100/293] lr: 5.000000e-04 eta: 4:49:11 time: 0.615171 data_time: 0.092506 memory: 14267 loss_kpt: 0.000641 acc_pose: 0.824629 loss: 0.000641 2022/09/23 17:24:29 - mmengine - INFO - Epoch(train) [95][150/293] lr: 5.000000e-04 eta: 4:48:50 time: 0.579806 data_time: 0.091569 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.848895 loss: 0.000630 2022/09/23 17:24:59 - mmengine - INFO - Epoch(train) [95][200/293] lr: 5.000000e-04 eta: 4:48:30 time: 0.602711 data_time: 0.096633 memory: 14267 loss_kpt: 0.000637 acc_pose: 0.802091 loss: 0.000637 2022/09/23 17:25:29 - mmengine - INFO - Epoch(train) [95][250/293] lr: 5.000000e-04 eta: 4:48:08 time: 0.586269 data_time: 0.095050 memory: 14267 loss_kpt: 0.000624 acc_pose: 0.863664 loss: 0.000624 2022/09/23 17:25:54 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:26:24 - mmengine - INFO - Epoch(train) [96][50/293] lr: 5.000000e-04 eta: 4:47:00 time: 0.614061 data_time: 0.098883 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.854265 loss: 0.000635 2022/09/23 17:26:53 - mmengine - INFO - Epoch(train) [96][100/293] lr: 5.000000e-04 eta: 4:46:38 time: 0.566023 data_time: 0.076423 memory: 14267 loss_kpt: 0.000627 acc_pose: 0.814454 loss: 0.000627 2022/09/23 17:27:22 - mmengine - INFO - Epoch(train) [96][150/293] lr: 5.000000e-04 eta: 4:46:17 time: 0.593953 data_time: 0.094830 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.814794 loss: 0.000638 2022/09/23 17:27:31 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:27:53 - mmengine - INFO - Epoch(train) [96][200/293] lr: 5.000000e-04 eta: 4:45:57 time: 0.605248 data_time: 0.097712 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.822525 loss: 0.000635 2022/09/23 17:28:22 - mmengine - INFO - Epoch(train) [96][250/293] lr: 5.000000e-04 eta: 4:45:36 time: 0.591612 data_time: 0.069314 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.808559 loss: 0.000626 2022/09/23 17:28:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:29:18 - mmengine - INFO - Epoch(train) [97][50/293] lr: 5.000000e-04 eta: 4:44:29 time: 0.615550 data_time: 0.100568 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.837095 loss: 0.000628 2022/09/23 17:29:47 - mmengine - INFO - Epoch(train) [97][100/293] lr: 5.000000e-04 eta: 4:44:07 time: 0.578751 data_time: 0.080324 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.861139 loss: 0.000631 2022/09/23 17:30:17 - mmengine - INFO - Epoch(train) [97][150/293] lr: 5.000000e-04 eta: 4:43:47 time: 0.608986 data_time: 0.117418 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.794935 loss: 0.000628 2022/09/23 17:30:46 - mmengine - INFO - Epoch(train) [97][200/293] lr: 5.000000e-04 eta: 4:43:26 time: 0.579355 data_time: 0.075474 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.796226 loss: 0.000630 2022/09/23 17:31:16 - mmengine - INFO - Epoch(train) [97][250/293] lr: 5.000000e-04 eta: 4:43:05 time: 0.603798 data_time: 0.096231 memory: 14267 loss_kpt: 0.000632 acc_pose: 0.795160 loss: 0.000632 2022/09/23 17:31:42 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:32:14 - mmengine - INFO - Epoch(train) [98][50/293] lr: 5.000000e-04 eta: 4:41:59 time: 0.637063 data_time: 0.112699 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.813111 loss: 0.000631 2022/09/23 17:32:43 - mmengine - INFO - Epoch(train) [98][100/293] lr: 5.000000e-04 eta: 4:41:38 time: 0.585430 data_time: 0.089226 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.810160 loss: 0.000629 2022/09/23 17:33:12 - mmengine - INFO - Epoch(train) [98][150/293] lr: 5.000000e-04 eta: 4:41:17 time: 0.583208 data_time: 0.081260 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.805730 loss: 0.000635 2022/09/23 17:33:41 - mmengine - INFO - Epoch(train) [98][200/293] lr: 5.000000e-04 eta: 4:40:54 time: 0.573876 data_time: 0.095632 memory: 14267 loss_kpt: 0.000634 acc_pose: 0.805674 loss: 0.000634 2022/09/23 17:34:10 - mmengine - INFO - Epoch(train) [98][250/293] lr: 5.000000e-04 eta: 4:40:33 time: 0.586913 data_time: 0.082128 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.842032 loss: 0.000628 2022/09/23 17:34:35 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:35:04 - mmengine - INFO - Epoch(train) [99][50/293] lr: 5.000000e-04 eta: 4:39:25 time: 0.595153 data_time: 0.108941 memory: 14267 loss_kpt: 0.000635 acc_pose: 0.820089 loss: 0.000635 2022/09/23 17:35:34 - mmengine - INFO - Epoch(train) [99][100/293] lr: 5.000000e-04 eta: 4:39:04 time: 0.596238 data_time: 0.097388 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.824049 loss: 0.000616 2022/09/23 17:36:04 - mmengine - INFO - Epoch(train) [99][150/293] lr: 5.000000e-04 eta: 4:38:43 time: 0.593257 data_time: 0.084485 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.866231 loss: 0.000638 2022/09/23 17:36:33 - mmengine - INFO - Epoch(train) [99][200/293] lr: 5.000000e-04 eta: 4:38:22 time: 0.588802 data_time: 0.108054 memory: 14267 loss_kpt: 0.000632 acc_pose: 0.797723 loss: 0.000632 2022/09/23 17:37:03 - mmengine - INFO - Epoch(train) [99][250/293] lr: 5.000000e-04 eta: 4:38:01 time: 0.584083 data_time: 0.083452 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.811059 loss: 0.000620 2022/09/23 17:37:24 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:37:28 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:38:00 - mmengine - INFO - Epoch(train) [100][50/293] lr: 5.000000e-04 eta: 4:36:55 time: 0.629346 data_time: 0.115707 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.799627 loss: 0.000631 2022/09/23 17:38:29 - mmengine - INFO - Epoch(train) [100][100/293] lr: 5.000000e-04 eta: 4:36:34 time: 0.591597 data_time: 0.096464 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.853406 loss: 0.000626 2022/09/23 17:38:58 - mmengine - INFO - Epoch(train) [100][150/293] lr: 5.000000e-04 eta: 4:36:12 time: 0.584257 data_time: 0.092952 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.787164 loss: 0.000636 2022/09/23 17:39:28 - mmengine - INFO - Epoch(train) [100][200/293] lr: 5.000000e-04 eta: 4:35:51 time: 0.586745 data_time: 0.083434 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.836741 loss: 0.000607 2022/09/23 17:39:57 - mmengine - INFO - Epoch(train) [100][250/293] lr: 5.000000e-04 eta: 4:35:30 time: 0.592568 data_time: 0.101374 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.812584 loss: 0.000629 2022/09/23 17:40:22 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:40:22 - mmengine - INFO - Saving checkpoint at 100 epochs 2022/09/23 17:40:47 - mmengine - INFO - Epoch(val) [100][50/407] eta: 0:02:08 time: 0.359768 data_time: 0.048260 memory: 14267 2022/09/23 17:41:04 - mmengine - INFO - Epoch(val) [100][100/407] eta: 0:01:44 time: 0.340813 data_time: 0.027029 memory: 1464 2022/09/23 17:41:21 - mmengine - INFO - Epoch(val) [100][150/407] eta: 0:01:28 time: 0.345760 data_time: 0.025528 memory: 1464 2022/09/23 17:41:38 - mmengine - INFO - Epoch(val) [100][200/407] eta: 0:01:09 time: 0.334014 data_time: 0.029493 memory: 1464 2022/09/23 17:41:56 - mmengine - INFO - Epoch(val) [100][250/407] eta: 0:00:55 time: 0.355629 data_time: 0.053854 memory: 1464 2022/09/23 17:42:13 - mmengine - INFO - Epoch(val) [100][300/407] eta: 0:00:37 time: 0.346283 data_time: 0.035566 memory: 1464 2022/09/23 17:42:30 - mmengine - INFO - Epoch(val) [100][350/407] eta: 0:00:19 time: 0.345210 data_time: 0.036418 memory: 1464 2022/09/23 17:42:45 - mmengine - INFO - Epoch(val) [100][400/407] eta: 0:00:02 time: 0.288531 data_time: 0.024294 memory: 1464 2022/09/23 17:43:19 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 17:43:32 - mmengine - INFO - Epoch(val) [100][407/407] coco/AP: 0.716660 coco/AP .5: 0.895180 coco/AP .75: 0.782171 coco/AP (M): 0.671920 coco/AP (L): 0.791739 coco/AR: 0.768671 coco/AR .5: 0.930730 coco/AR .75: 0.828873 coco/AR (M): 0.719448 coco/AR (L): 0.838685 2022/09/23 17:43:32 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_90.pth is removed 2022/09/23 17:43:35 - mmengine - INFO - The best checkpoint with 0.7167 coco/AP at 100 epoch is saved to best_coco/AP_epoch_100.pth. 2022/09/23 17:44:05 - mmengine - INFO - Epoch(train) [101][50/293] lr: 5.000000e-04 eta: 4:34:23 time: 0.612322 data_time: 0.099857 memory: 14267 loss_kpt: 0.000632 acc_pose: 0.823814 loss: 0.000632 2022/09/23 17:44:34 - mmengine - INFO - Epoch(train) [101][100/293] lr: 5.000000e-04 eta: 4:34:01 time: 0.579650 data_time: 0.099302 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.824444 loss: 0.000630 2022/09/23 17:45:04 - mmengine - INFO - Epoch(train) [101][150/293] lr: 5.000000e-04 eta: 4:33:40 time: 0.585782 data_time: 0.088899 memory: 14267 loss_kpt: 0.000633 acc_pose: 0.817351 loss: 0.000633 2022/09/23 17:45:34 - mmengine - INFO - Epoch(train) [101][200/293] lr: 5.000000e-04 eta: 4:33:19 time: 0.603657 data_time: 0.094799 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.802931 loss: 0.000631 2022/09/23 17:46:04 - mmengine - INFO - Epoch(train) [101][250/293] lr: 5.000000e-04 eta: 4:32:58 time: 0.592698 data_time: 0.095429 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.815183 loss: 0.000636 2022/09/23 17:46:27 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:46:59 - mmengine - INFO - Epoch(train) [102][50/293] lr: 5.000000e-04 eta: 4:31:53 time: 0.620849 data_time: 0.100615 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.810229 loss: 0.000631 2022/09/23 17:47:27 - mmengine - INFO - Epoch(train) [102][100/293] lr: 5.000000e-04 eta: 4:31:30 time: 0.566395 data_time: 0.087540 memory: 14267 loss_kpt: 0.000630 acc_pose: 0.755058 loss: 0.000630 2022/09/23 17:47:56 - mmengine - INFO - Epoch(train) [102][150/293] lr: 5.000000e-04 eta: 4:31:08 time: 0.589019 data_time: 0.083846 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.833084 loss: 0.000621 2022/09/23 17:48:26 - mmengine - INFO - Epoch(train) [102][200/293] lr: 5.000000e-04 eta: 4:30:48 time: 0.600680 data_time: 0.101914 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.840054 loss: 0.000628 2022/09/23 17:48:55 - mmengine - INFO - Epoch(train) [102][250/293] lr: 5.000000e-04 eta: 4:30:25 time: 0.567764 data_time: 0.086555 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.811999 loss: 0.000631 2022/09/23 17:49:19 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:49:50 - mmengine - INFO - Epoch(train) [103][50/293] lr: 5.000000e-04 eta: 4:29:19 time: 0.612970 data_time: 0.112330 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.817499 loss: 0.000621 2022/09/23 17:50:21 - mmengine - INFO - Epoch(train) [103][100/293] lr: 5.000000e-04 eta: 4:28:59 time: 0.612836 data_time: 0.082148 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.859967 loss: 0.000626 2022/09/23 17:50:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:50:49 - mmengine - INFO - Epoch(train) [103][150/293] lr: 5.000000e-04 eta: 4:28:36 time: 0.561176 data_time: 0.085867 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.885703 loss: 0.000620 2022/09/23 17:51:19 - mmengine - INFO - Epoch(train) [103][200/293] lr: 5.000000e-04 eta: 4:28:15 time: 0.598527 data_time: 0.099296 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.840742 loss: 0.000621 2022/09/23 17:51:48 - mmengine - INFO - Epoch(train) [103][250/293] lr: 5.000000e-04 eta: 4:27:53 time: 0.588428 data_time: 0.104767 memory: 14267 loss_kpt: 0.000636 acc_pose: 0.851491 loss: 0.000636 2022/09/23 17:52:13 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:52:44 - mmengine - INFO - Epoch(train) [104][50/293] lr: 5.000000e-04 eta: 4:26:49 time: 0.616693 data_time: 0.086164 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.749716 loss: 0.000626 2022/09/23 17:53:14 - mmengine - INFO - Epoch(train) [104][100/293] lr: 5.000000e-04 eta: 4:26:27 time: 0.597354 data_time: 0.081927 memory: 14267 loss_kpt: 0.000634 acc_pose: 0.814733 loss: 0.000634 2022/09/23 17:53:44 - mmengine - INFO - Epoch(train) [104][150/293] lr: 5.000000e-04 eta: 4:26:06 time: 0.593621 data_time: 0.093983 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.803373 loss: 0.000616 2022/09/23 17:54:14 - mmengine - INFO - Epoch(train) [104][200/293] lr: 5.000000e-04 eta: 4:25:45 time: 0.598254 data_time: 0.092487 memory: 14267 loss_kpt: 0.000625 acc_pose: 0.834395 loss: 0.000625 2022/09/23 17:54:44 - mmengine - INFO - Epoch(train) [104][250/293] lr: 5.000000e-04 eta: 4:25:24 time: 0.606621 data_time: 0.084596 memory: 14267 loss_kpt: 0.000625 acc_pose: 0.817905 loss: 0.000625 2022/09/23 17:55:09 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:55:40 - mmengine - INFO - Epoch(train) [105][50/293] lr: 5.000000e-04 eta: 4:24:19 time: 0.615497 data_time: 0.101355 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.813330 loss: 0.000620 2022/09/23 17:56:10 - mmengine - INFO - Epoch(train) [105][100/293] lr: 5.000000e-04 eta: 4:23:58 time: 0.597854 data_time: 0.096731 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.759036 loss: 0.000629 2022/09/23 17:56:39 - mmengine - INFO - Epoch(train) [105][150/293] lr: 5.000000e-04 eta: 4:23:36 time: 0.588847 data_time: 0.089920 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.851739 loss: 0.000628 2022/09/23 17:57:09 - mmengine - INFO - Epoch(train) [105][200/293] lr: 5.000000e-04 eta: 4:23:15 time: 0.601510 data_time: 0.099095 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.864812 loss: 0.000622 2022/09/23 17:57:39 - mmengine - INFO - Epoch(train) [105][250/293] lr: 5.000000e-04 eta: 4:22:54 time: 0.590733 data_time: 0.084786 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.830128 loss: 0.000631 2022/09/23 17:58:04 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 17:58:35 - mmengine - INFO - Epoch(train) [106][50/293] lr: 5.000000e-04 eta: 4:21:50 time: 0.628091 data_time: 0.113931 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.787005 loss: 0.000622 2022/09/23 17:59:05 - mmengine - INFO - Epoch(train) [106][100/293] lr: 5.000000e-04 eta: 4:21:29 time: 0.593925 data_time: 0.102688 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.857025 loss: 0.000631 2022/09/23 17:59:34 - mmengine - INFO - Epoch(train) [106][150/293] lr: 5.000000e-04 eta: 4:21:06 time: 0.579910 data_time: 0.086612 memory: 14267 loss_kpt: 0.000619 acc_pose: 0.838750 loss: 0.000619 2022/09/23 18:00:06 - mmengine - INFO - Epoch(train) [106][200/293] lr: 5.000000e-04 eta: 4:20:47 time: 0.644416 data_time: 0.088458 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.853352 loss: 0.000626 2022/09/23 18:00:27 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:00:35 - mmengine - INFO - Epoch(train) [106][250/293] lr: 5.000000e-04 eta: 4:20:25 time: 0.579147 data_time: 0.086747 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.831488 loss: 0.000613 2022/09/23 18:01:00 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:01:31 - mmengine - INFO - Epoch(train) [107][50/293] lr: 5.000000e-04 eta: 4:19:21 time: 0.621355 data_time: 0.113028 memory: 14267 loss_kpt: 0.000629 acc_pose: 0.813712 loss: 0.000629 2022/09/23 18:02:01 - mmengine - INFO - Epoch(train) [107][100/293] lr: 5.000000e-04 eta: 4:19:00 time: 0.597534 data_time: 0.087922 memory: 14267 loss_kpt: 0.000638 acc_pose: 0.781268 loss: 0.000638 2022/09/23 18:02:30 - mmengine - INFO - Epoch(train) [107][150/293] lr: 5.000000e-04 eta: 4:18:37 time: 0.577891 data_time: 0.078912 memory: 14267 loss_kpt: 0.000627 acc_pose: 0.859234 loss: 0.000627 2022/09/23 18:03:00 - mmengine - INFO - Epoch(train) [107][200/293] lr: 5.000000e-04 eta: 4:18:16 time: 0.604540 data_time: 0.084123 memory: 14267 loss_kpt: 0.000632 acc_pose: 0.828952 loss: 0.000632 2022/09/23 18:03:30 - mmengine - INFO - Epoch(train) [107][250/293] lr: 5.000000e-04 eta: 4:17:55 time: 0.594368 data_time: 0.085785 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.794372 loss: 0.000628 2022/09/23 18:03:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:04:27 - mmengine - INFO - Epoch(train) [108][50/293] lr: 5.000000e-04 eta: 4:16:52 time: 0.630206 data_time: 0.111074 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.842535 loss: 0.000604 2022/09/23 18:04:55 - mmengine - INFO - Epoch(train) [108][100/293] lr: 5.000000e-04 eta: 4:16:29 time: 0.568392 data_time: 0.078749 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.804100 loss: 0.000622 2022/09/23 18:05:26 - mmengine - INFO - Epoch(train) [108][150/293] lr: 5.000000e-04 eta: 4:16:08 time: 0.607691 data_time: 0.112235 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.859858 loss: 0.000626 2022/09/23 18:05:56 - mmengine - INFO - Epoch(train) [108][200/293] lr: 5.000000e-04 eta: 4:15:47 time: 0.616604 data_time: 0.094419 memory: 14267 loss_kpt: 0.000619 acc_pose: 0.864970 loss: 0.000619 2022/09/23 18:06:27 - mmengine - INFO - Epoch(train) [108][250/293] lr: 5.000000e-04 eta: 4:15:26 time: 0.615996 data_time: 0.092325 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.825826 loss: 0.000621 2022/09/23 18:06:52 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:07:24 - mmengine - INFO - Epoch(train) [109][50/293] lr: 5.000000e-04 eta: 4:14:24 time: 0.633979 data_time: 0.106750 memory: 14267 loss_kpt: 0.000610 acc_pose: 0.809219 loss: 0.000610 2022/09/23 18:07:53 - mmengine - INFO - Epoch(train) [109][100/293] lr: 5.000000e-04 eta: 4:14:02 time: 0.580975 data_time: 0.093396 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.857450 loss: 0.000620 2022/09/23 18:08:23 - mmengine - INFO - Epoch(train) [109][150/293] lr: 5.000000e-04 eta: 4:13:40 time: 0.594165 data_time: 0.076356 memory: 14267 loss_kpt: 0.000618 acc_pose: 0.801126 loss: 0.000618 2022/09/23 18:08:52 - mmengine - INFO - Epoch(train) [109][200/293] lr: 5.000000e-04 eta: 4:13:18 time: 0.593946 data_time: 0.096909 memory: 14267 loss_kpt: 0.000615 acc_pose: 0.796444 loss: 0.000615 2022/09/23 18:09:22 - mmengine - INFO - Epoch(train) [109][250/293] lr: 5.000000e-04 eta: 4:12:56 time: 0.587685 data_time: 0.095183 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.789539 loss: 0.000620 2022/09/23 18:09:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:10:19 - mmengine - INFO - Epoch(train) [110][50/293] lr: 5.000000e-04 eta: 4:11:53 time: 0.628567 data_time: 0.107352 memory: 14267 loss_kpt: 0.000615 acc_pose: 0.817565 loss: 0.000615 2022/09/23 18:10:26 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:10:48 - mmengine - INFO - Epoch(train) [110][100/293] lr: 5.000000e-04 eta: 4:11:32 time: 0.598472 data_time: 0.096466 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.810265 loss: 0.000613 2022/09/23 18:11:18 - mmengine - INFO - Epoch(train) [110][150/293] lr: 5.000000e-04 eta: 4:11:10 time: 0.598969 data_time: 0.097628 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.871443 loss: 0.000613 2022/09/23 18:11:48 - mmengine - INFO - Epoch(train) [110][200/293] lr: 5.000000e-04 eta: 4:10:49 time: 0.600592 data_time: 0.079160 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.799375 loss: 0.000613 2022/09/23 18:12:19 - mmengine - INFO - Epoch(train) [110][250/293] lr: 5.000000e-04 eta: 4:10:27 time: 0.602270 data_time: 0.099465 memory: 14267 loss_kpt: 0.000623 acc_pose: 0.801285 loss: 0.000623 2022/09/23 18:12:44 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:12:44 - mmengine - INFO - Saving checkpoint at 110 epochs 2022/09/23 18:13:08 - mmengine - INFO - Epoch(val) [110][50/407] eta: 0:02:11 time: 0.367361 data_time: 0.057615 memory: 14267 2022/09/23 18:13:25 - mmengine - INFO - Epoch(val) [110][100/407] eta: 0:01:44 time: 0.339561 data_time: 0.024340 memory: 1464 2022/09/23 18:13:43 - mmengine - INFO - Epoch(val) [110][150/407] eta: 0:01:32 time: 0.360775 data_time: 0.062869 memory: 1464 2022/09/23 18:14:00 - mmengine - INFO - Epoch(val) [110][200/407] eta: 0:01:10 time: 0.339068 data_time: 0.032653 memory: 1464 2022/09/23 18:14:18 - mmengine - INFO - Epoch(val) [110][250/407] eta: 0:00:55 time: 0.350478 data_time: 0.039390 memory: 1464 2022/09/23 18:14:35 - mmengine - INFO - Epoch(val) [110][300/407] eta: 0:00:37 time: 0.348911 data_time: 0.028823 memory: 1464 2022/09/23 18:14:52 - mmengine - INFO - Epoch(val) [110][350/407] eta: 0:00:19 time: 0.342034 data_time: 0.041585 memory: 1464 2022/09/23 18:15:06 - mmengine - INFO - Epoch(val) [110][400/407] eta: 0:00:01 time: 0.278028 data_time: 0.025859 memory: 1464 2022/09/23 18:15:40 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 18:15:53 - mmengine - INFO - Epoch(val) [110][407/407] coco/AP: 0.711991 coco/AP .5: 0.889159 coco/AP .75: 0.776276 coco/AP (M): 0.665483 coco/AP (L): 0.790045 coco/AR: 0.762988 coco/AR .5: 0.926322 coco/AR .75: 0.821159 coco/AR (M): 0.712428 coco/AR (L): 0.835340 2022/09/23 18:16:25 - mmengine - INFO - Epoch(train) [111][50/293] lr: 5.000000e-04 eta: 4:09:26 time: 0.642793 data_time: 0.117823 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.847081 loss: 0.000614 2022/09/23 18:16:55 - mmengine - INFO - Epoch(train) [111][100/293] lr: 5.000000e-04 eta: 4:09:03 time: 0.588386 data_time: 0.078580 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.857651 loss: 0.000612 2022/09/23 18:17:23 - mmengine - INFO - Epoch(train) [111][150/293] lr: 5.000000e-04 eta: 4:08:41 time: 0.573452 data_time: 0.084306 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.817135 loss: 0.000614 2022/09/23 18:17:53 - mmengine - INFO - Epoch(train) [111][200/293] lr: 5.000000e-04 eta: 4:08:18 time: 0.587006 data_time: 0.090556 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.833868 loss: 0.000620 2022/09/23 18:18:22 - mmengine - INFO - Epoch(train) [111][250/293] lr: 5.000000e-04 eta: 4:07:56 time: 0.578688 data_time: 0.085691 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.816738 loss: 0.000616 2022/09/23 18:18:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:19:18 - mmengine - INFO - Epoch(train) [112][50/293] lr: 5.000000e-04 eta: 4:06:54 time: 0.628799 data_time: 0.121768 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.839658 loss: 0.000621 2022/09/23 18:19:48 - mmengine - INFO - Epoch(train) [112][100/293] lr: 5.000000e-04 eta: 4:06:32 time: 0.600230 data_time: 0.091986 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.816424 loss: 0.000604 2022/09/23 18:20:17 - mmengine - INFO - Epoch(train) [112][150/293] lr: 5.000000e-04 eta: 4:06:09 time: 0.571185 data_time: 0.093788 memory: 14267 loss_kpt: 0.000624 acc_pose: 0.821751 loss: 0.000624 2022/09/23 18:20:46 - mmengine - INFO - Epoch(train) [112][200/293] lr: 5.000000e-04 eta: 4:05:47 time: 0.586739 data_time: 0.083662 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.843861 loss: 0.000608 2022/09/23 18:21:16 - mmengine - INFO - Epoch(train) [112][250/293] lr: 5.000000e-04 eta: 4:05:25 time: 0.600930 data_time: 0.092178 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.840874 loss: 0.000613 2022/09/23 18:21:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:22:12 - mmengine - INFO - Epoch(train) [113][50/293] lr: 5.000000e-04 eta: 4:04:23 time: 0.620963 data_time: 0.110120 memory: 14267 loss_kpt: 0.000624 acc_pose: 0.781227 loss: 0.000624 2022/09/23 18:22:43 - mmengine - INFO - Epoch(train) [113][100/293] lr: 5.000000e-04 eta: 4:04:02 time: 0.609239 data_time: 0.108590 memory: 14267 loss_kpt: 0.000618 acc_pose: 0.790011 loss: 0.000618 2022/09/23 18:23:14 - mmengine - INFO - Epoch(train) [113][150/293] lr: 5.000000e-04 eta: 4:03:40 time: 0.610955 data_time: 0.110515 memory: 14267 loss_kpt: 0.000619 acc_pose: 0.817779 loss: 0.000619 2022/09/23 18:23:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:23:44 - mmengine - INFO - Epoch(train) [113][200/293] lr: 5.000000e-04 eta: 4:03:19 time: 0.604096 data_time: 0.097658 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.841112 loss: 0.000616 2022/09/23 18:24:13 - mmengine - INFO - Epoch(train) [113][250/293] lr: 5.000000e-04 eta: 4:02:57 time: 0.587922 data_time: 0.096942 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.756528 loss: 0.000616 2022/09/23 18:24:38 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:25:10 - mmengine - INFO - Epoch(train) [114][50/293] lr: 5.000000e-04 eta: 4:01:55 time: 0.630578 data_time: 0.117115 memory: 14267 loss_kpt: 0.000606 acc_pose: 0.830258 loss: 0.000606 2022/09/23 18:25:40 - mmengine - INFO - Epoch(train) [114][100/293] lr: 5.000000e-04 eta: 4:01:33 time: 0.599620 data_time: 0.094918 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.809436 loss: 0.000616 2022/09/23 18:26:10 - mmengine - INFO - Epoch(train) [114][150/293] lr: 5.000000e-04 eta: 4:01:12 time: 0.601525 data_time: 0.098362 memory: 14267 loss_kpt: 0.000624 acc_pose: 0.795341 loss: 0.000624 2022/09/23 18:26:40 - mmengine - INFO - Epoch(train) [114][200/293] lr: 5.000000e-04 eta: 4:00:50 time: 0.600942 data_time: 0.087588 memory: 14267 loss_kpt: 0.000615 acc_pose: 0.836688 loss: 0.000615 2022/09/23 18:27:09 - mmengine - INFO - Epoch(train) [114][250/293] lr: 5.000000e-04 eta: 4:00:27 time: 0.579755 data_time: 0.084370 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.819200 loss: 0.000631 2022/09/23 18:27:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:28:06 - mmengine - INFO - Epoch(train) [115][50/293] lr: 5.000000e-04 eta: 3:59:26 time: 0.632709 data_time: 0.127322 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.792451 loss: 0.000622 2022/09/23 18:28:36 - mmengine - INFO - Epoch(train) [115][100/293] lr: 5.000000e-04 eta: 3:59:04 time: 0.606214 data_time: 0.091943 memory: 14267 loss_kpt: 0.000623 acc_pose: 0.822729 loss: 0.000623 2022/09/23 18:29:07 - mmengine - INFO - Epoch(train) [115][150/293] lr: 5.000000e-04 eta: 3:58:43 time: 0.615424 data_time: 0.095307 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.836152 loss: 0.000622 2022/09/23 18:29:37 - mmengine - INFO - Epoch(train) [115][200/293] lr: 5.000000e-04 eta: 3:58:21 time: 0.605261 data_time: 0.098291 memory: 14267 loss_kpt: 0.000609 acc_pose: 0.840645 loss: 0.000609 2022/09/23 18:30:07 - mmengine - INFO - Epoch(train) [115][250/293] lr: 5.000000e-04 eta: 3:57:59 time: 0.589789 data_time: 0.091344 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.835165 loss: 0.000621 2022/09/23 18:30:32 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:31:03 - mmengine - INFO - Epoch(train) [116][50/293] lr: 5.000000e-04 eta: 3:56:57 time: 0.618568 data_time: 0.122196 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.845365 loss: 0.000612 2022/09/23 18:31:32 - mmengine - INFO - Epoch(train) [116][100/293] lr: 5.000000e-04 eta: 3:56:35 time: 0.577245 data_time: 0.082317 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.821493 loss: 0.000603 2022/09/23 18:32:02 - mmengine - INFO - Epoch(train) [116][150/293] lr: 5.000000e-04 eta: 3:56:13 time: 0.599309 data_time: 0.106730 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.813074 loss: 0.000611 2022/09/23 18:32:31 - mmengine - INFO - Epoch(train) [116][200/293] lr: 5.000000e-04 eta: 3:55:50 time: 0.589545 data_time: 0.076143 memory: 14267 loss_kpt: 0.000601 acc_pose: 0.808160 loss: 0.000601 2022/09/23 18:33:02 - mmengine - INFO - Epoch(train) [116][250/293] lr: 5.000000e-04 eta: 3:55:29 time: 0.621177 data_time: 0.115554 memory: 14267 loss_kpt: 0.000631 acc_pose: 0.814192 loss: 0.000631 2022/09/23 18:33:27 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:33:35 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:33:58 - mmengine - INFO - Epoch(train) [117][50/293] lr: 5.000000e-04 eta: 3:54:28 time: 0.625097 data_time: 0.111560 memory: 14267 loss_kpt: 0.000609 acc_pose: 0.864175 loss: 0.000609 2022/09/23 18:34:28 - mmengine - INFO - Epoch(train) [117][100/293] lr: 5.000000e-04 eta: 3:54:06 time: 0.605251 data_time: 0.080171 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.816087 loss: 0.000612 2022/09/23 18:34:58 - mmengine - INFO - Epoch(train) [117][150/293] lr: 5.000000e-04 eta: 3:53:44 time: 0.591793 data_time: 0.093762 memory: 14267 loss_kpt: 0.000620 acc_pose: 0.819121 loss: 0.000620 2022/09/23 18:35:27 - mmengine - INFO - Epoch(train) [117][200/293] lr: 5.000000e-04 eta: 3:53:21 time: 0.581704 data_time: 0.091255 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.807816 loss: 0.000622 2022/09/23 18:35:56 - mmengine - INFO - Epoch(train) [117][250/293] lr: 5.000000e-04 eta: 3:52:58 time: 0.586539 data_time: 0.094199 memory: 14267 loss_kpt: 0.000619 acc_pose: 0.806976 loss: 0.000619 2022/09/23 18:36:22 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:36:53 - mmengine - INFO - Epoch(train) [118][50/293] lr: 5.000000e-04 eta: 3:51:57 time: 0.613621 data_time: 0.109891 memory: 14267 loss_kpt: 0.000599 acc_pose: 0.835945 loss: 0.000599 2022/09/23 18:37:22 - mmengine - INFO - Epoch(train) [118][100/293] lr: 5.000000e-04 eta: 3:51:35 time: 0.599003 data_time: 0.118706 memory: 14267 loss_kpt: 0.000615 acc_pose: 0.782083 loss: 0.000615 2022/09/23 18:37:52 - mmengine - INFO - Epoch(train) [118][150/293] lr: 5.000000e-04 eta: 3:51:13 time: 0.589406 data_time: 0.074328 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.850299 loss: 0.000616 2022/09/23 18:38:21 - mmengine - INFO - Epoch(train) [118][200/293] lr: 5.000000e-04 eta: 3:50:50 time: 0.588362 data_time: 0.096041 memory: 14267 loss_kpt: 0.000601 acc_pose: 0.838646 loss: 0.000601 2022/09/23 18:38:51 - mmengine - INFO - Epoch(train) [118][250/293] lr: 5.000000e-04 eta: 3:50:28 time: 0.593262 data_time: 0.093004 memory: 14267 loss_kpt: 0.000619 acc_pose: 0.871634 loss: 0.000619 2022/09/23 18:39:17 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:39:48 - mmengine - INFO - Epoch(train) [119][50/293] lr: 5.000000e-04 eta: 3:49:27 time: 0.622405 data_time: 0.105460 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.783877 loss: 0.000612 2022/09/23 18:40:18 - mmengine - INFO - Epoch(train) [119][100/293] lr: 5.000000e-04 eta: 3:49:05 time: 0.604191 data_time: 0.098643 memory: 14267 loss_kpt: 0.000617 acc_pose: 0.828995 loss: 0.000617 2022/09/23 18:40:48 - mmengine - INFO - Epoch(train) [119][150/293] lr: 5.000000e-04 eta: 3:48:43 time: 0.586423 data_time: 0.088548 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.825482 loss: 0.000595 2022/09/23 18:41:17 - mmengine - INFO - Epoch(train) [119][200/293] lr: 5.000000e-04 eta: 3:48:20 time: 0.587880 data_time: 0.091995 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.792360 loss: 0.000622 2022/09/23 18:41:47 - mmengine - INFO - Epoch(train) [119][250/293] lr: 5.000000e-04 eta: 3:47:57 time: 0.596604 data_time: 0.089386 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.759373 loss: 0.000621 2022/09/23 18:42:12 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:42:42 - mmengine - INFO - Epoch(train) [120][50/293] lr: 5.000000e-04 eta: 3:46:57 time: 0.615246 data_time: 0.104385 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.803388 loss: 0.000613 2022/09/23 18:43:11 - mmengine - INFO - Epoch(train) [120][100/293] lr: 5.000000e-04 eta: 3:46:34 time: 0.570661 data_time: 0.081639 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.830103 loss: 0.000592 2022/09/23 18:43:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:43:40 - mmengine - INFO - Epoch(train) [120][150/293] lr: 5.000000e-04 eta: 3:46:10 time: 0.571253 data_time: 0.080326 memory: 14267 loss_kpt: 0.000623 acc_pose: 0.807063 loss: 0.000623 2022/09/23 18:44:09 - mmengine - INFO - Epoch(train) [120][200/293] lr: 5.000000e-04 eta: 3:45:48 time: 0.597125 data_time: 0.097603 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.807103 loss: 0.000621 2022/09/23 18:44:40 - mmengine - INFO - Epoch(train) [120][250/293] lr: 5.000000e-04 eta: 3:45:26 time: 0.602376 data_time: 0.085118 memory: 14267 loss_kpt: 0.000615 acc_pose: 0.809016 loss: 0.000615 2022/09/23 18:45:05 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:45:05 - mmengine - INFO - Saving checkpoint at 120 epochs 2022/09/23 18:45:31 - mmengine - INFO - Epoch(val) [120][50/407] eta: 0:02:13 time: 0.372760 data_time: 0.086961 memory: 14267 2022/09/23 18:45:47 - mmengine - INFO - Epoch(val) [120][100/407] eta: 0:01:42 time: 0.334821 data_time: 0.039347 memory: 1464 2022/09/23 18:46:04 - mmengine - INFO - Epoch(val) [120][150/407] eta: 0:01:26 time: 0.336649 data_time: 0.050813 memory: 1464 2022/09/23 18:46:21 - mmengine - INFO - Epoch(val) [120][200/407] eta: 0:01:10 time: 0.338355 data_time: 0.048862 memory: 1464 2022/09/23 18:46:39 - mmengine - INFO - Epoch(val) [120][250/407] eta: 0:00:54 time: 0.348041 data_time: 0.061062 memory: 1464 2022/09/23 18:46:55 - mmengine - INFO - Epoch(val) [120][300/407] eta: 0:00:35 time: 0.331776 data_time: 0.046866 memory: 1464 2022/09/23 18:47:12 - mmengine - INFO - Epoch(val) [120][350/407] eta: 0:00:19 time: 0.339240 data_time: 0.057286 memory: 1464 2022/09/23 18:47:26 - mmengine - INFO - Epoch(val) [120][400/407] eta: 0:00:01 time: 0.280593 data_time: 0.038608 memory: 1464 2022/09/23 18:48:00 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 18:48:14 - mmengine - INFO - Epoch(val) [120][407/407] coco/AP: 0.716695 coco/AP .5: 0.891861 coco/AP .75: 0.781810 coco/AP (M): 0.669042 coco/AP (L): 0.792930 coco/AR: 0.767821 coco/AR .5: 0.929471 coco/AR .75: 0.826354 coco/AR (M): 0.717236 coco/AR (L): 0.840171 2022/09/23 18:48:14 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_100.pth is removed 2022/09/23 18:48:16 - mmengine - INFO - The best checkpoint with 0.7167 coco/AP at 120 epoch is saved to best_coco/AP_epoch_120.pth. 2022/09/23 18:48:47 - mmengine - INFO - Epoch(train) [121][50/293] lr: 5.000000e-04 eta: 3:44:26 time: 0.616340 data_time: 0.110727 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.806731 loss: 0.000607 2022/09/23 18:49:16 - mmengine - INFO - Epoch(train) [121][100/293] lr: 5.000000e-04 eta: 3:44:03 time: 0.589261 data_time: 0.085226 memory: 14267 loss_kpt: 0.000624 acc_pose: 0.863688 loss: 0.000624 2022/09/23 18:49:45 - mmengine - INFO - Epoch(train) [121][150/293] lr: 5.000000e-04 eta: 3:43:40 time: 0.581148 data_time: 0.079789 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.856120 loss: 0.000605 2022/09/23 18:50:14 - mmengine - INFO - Epoch(train) [121][200/293] lr: 5.000000e-04 eta: 3:43:17 time: 0.584185 data_time: 0.086082 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.794675 loss: 0.000595 2022/09/23 18:50:44 - mmengine - INFO - Epoch(train) [121][250/293] lr: 5.000000e-04 eta: 3:42:55 time: 0.600474 data_time: 0.085529 memory: 14267 loss_kpt: 0.000606 acc_pose: 0.827625 loss: 0.000606 2022/09/23 18:51:09 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:51:40 - mmengine - INFO - Epoch(train) [122][50/293] lr: 5.000000e-04 eta: 3:41:55 time: 0.622896 data_time: 0.098570 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.835856 loss: 0.000611 2022/09/23 18:52:10 - mmengine - INFO - Epoch(train) [122][100/293] lr: 5.000000e-04 eta: 3:41:32 time: 0.592942 data_time: 0.087360 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.777499 loss: 0.000613 2022/09/23 18:52:39 - mmengine - INFO - Epoch(train) [122][150/293] lr: 5.000000e-04 eta: 3:41:09 time: 0.581404 data_time: 0.073554 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.826166 loss: 0.000595 2022/09/23 18:53:08 - mmengine - INFO - Epoch(train) [122][200/293] lr: 5.000000e-04 eta: 3:40:46 time: 0.588376 data_time: 0.087273 memory: 14267 loss_kpt: 0.000617 acc_pose: 0.842601 loss: 0.000617 2022/09/23 18:53:37 - mmengine - INFO - Epoch(train) [122][250/293] lr: 5.000000e-04 eta: 3:40:23 time: 0.576150 data_time: 0.073588 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.820727 loss: 0.000611 2022/09/23 18:54:04 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:54:35 - mmengine - INFO - Epoch(train) [123][50/293] lr: 5.000000e-04 eta: 3:39:24 time: 0.624894 data_time: 0.120330 memory: 14267 loss_kpt: 0.000610 acc_pose: 0.831724 loss: 0.000610 2022/09/23 18:55:03 - mmengine - INFO - Epoch(train) [123][100/293] lr: 5.000000e-04 eta: 3:39:00 time: 0.572216 data_time: 0.078012 memory: 14267 loss_kpt: 0.000617 acc_pose: 0.814209 loss: 0.000617 2022/09/23 18:55:32 - mmengine - INFO - Epoch(train) [123][150/293] lr: 5.000000e-04 eta: 3:38:37 time: 0.570183 data_time: 0.083899 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.834792 loss: 0.000607 2022/09/23 18:56:02 - mmengine - INFO - Epoch(train) [123][200/293] lr: 5.000000e-04 eta: 3:38:14 time: 0.594263 data_time: 0.081264 memory: 14267 loss_kpt: 0.000626 acc_pose: 0.795487 loss: 0.000626 2022/09/23 18:56:31 - mmengine - INFO - Epoch(train) [123][250/293] lr: 5.000000e-04 eta: 3:37:52 time: 0.595396 data_time: 0.079983 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.817885 loss: 0.000607 2022/09/23 18:56:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:56:56 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 18:57:28 - mmengine - INFO - Epoch(train) [124][50/293] lr: 5.000000e-04 eta: 3:36:53 time: 0.634310 data_time: 0.106542 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.858126 loss: 0.000600 2022/09/23 18:57:58 - mmengine - INFO - Epoch(train) [124][100/293] lr: 5.000000e-04 eta: 3:36:30 time: 0.595077 data_time: 0.095999 memory: 14267 loss_kpt: 0.000618 acc_pose: 0.831864 loss: 0.000618 2022/09/23 18:58:27 - mmengine - INFO - Epoch(train) [124][150/293] lr: 5.000000e-04 eta: 3:36:07 time: 0.589373 data_time: 0.085954 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.789154 loss: 0.000614 2022/09/23 18:58:57 - mmengine - INFO - Epoch(train) [124][200/293] lr: 5.000000e-04 eta: 3:35:45 time: 0.596572 data_time: 0.087183 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.833824 loss: 0.000611 2022/09/23 18:59:28 - mmengine - INFO - Epoch(train) [124][250/293] lr: 5.000000e-04 eta: 3:35:22 time: 0.606019 data_time: 0.105072 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.810062 loss: 0.000614 2022/09/23 18:59:52 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:00:24 - mmengine - INFO - Epoch(train) [125][50/293] lr: 5.000000e-04 eta: 3:34:24 time: 0.629293 data_time: 0.115224 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.799754 loss: 0.000605 2022/09/23 19:00:53 - mmengine - INFO - Epoch(train) [125][100/293] lr: 5.000000e-04 eta: 3:34:00 time: 0.577094 data_time: 0.069638 memory: 14267 loss_kpt: 0.000599 acc_pose: 0.831025 loss: 0.000599 2022/09/23 19:01:23 - mmengine - INFO - Epoch(train) [125][150/293] lr: 5.000000e-04 eta: 3:33:38 time: 0.605758 data_time: 0.092691 memory: 14267 loss_kpt: 0.000618 acc_pose: 0.805820 loss: 0.000618 2022/09/23 19:01:53 - mmengine - INFO - Epoch(train) [125][200/293] lr: 5.000000e-04 eta: 3:33:15 time: 0.592995 data_time: 0.089154 memory: 14267 loss_kpt: 0.000610 acc_pose: 0.836997 loss: 0.000610 2022/09/23 19:02:22 - mmengine - INFO - Epoch(train) [125][250/293] lr: 5.000000e-04 eta: 3:32:52 time: 0.583834 data_time: 0.088486 memory: 14267 loss_kpt: 0.000621 acc_pose: 0.850612 loss: 0.000621 2022/09/23 19:02:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:03:19 - mmengine - INFO - Epoch(train) [126][50/293] lr: 5.000000e-04 eta: 3:31:54 time: 0.634684 data_time: 0.102050 memory: 14267 loss_kpt: 0.000618 acc_pose: 0.810724 loss: 0.000618 2022/09/23 19:03:48 - mmengine - INFO - Epoch(train) [126][100/293] lr: 5.000000e-04 eta: 3:31:31 time: 0.596866 data_time: 0.105849 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.862149 loss: 0.000593 2022/09/23 19:04:19 - mmengine - INFO - Epoch(train) [126][150/293] lr: 5.000000e-04 eta: 3:31:09 time: 0.609541 data_time: 0.101238 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.817449 loss: 0.000600 2022/09/23 19:04:49 - mmengine - INFO - Epoch(train) [126][200/293] lr: 5.000000e-04 eta: 3:30:46 time: 0.595978 data_time: 0.096083 memory: 14267 loss_kpt: 0.000616 acc_pose: 0.860779 loss: 0.000616 2022/09/23 19:05:19 - mmengine - INFO - Epoch(train) [126][250/293] lr: 5.000000e-04 eta: 3:30:23 time: 0.598999 data_time: 0.099527 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.818768 loss: 0.000613 2022/09/23 19:05:43 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:06:15 - mmengine - INFO - Epoch(train) [127][50/293] lr: 5.000000e-04 eta: 3:29:25 time: 0.646961 data_time: 0.130938 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.873144 loss: 0.000608 2022/09/23 19:06:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:06:45 - mmengine - INFO - Epoch(train) [127][100/293] lr: 5.000000e-04 eta: 3:29:02 time: 0.583289 data_time: 0.087466 memory: 14267 loss_kpt: 0.000615 acc_pose: 0.798077 loss: 0.000615 2022/09/23 19:07:13 - mmengine - INFO - Epoch(train) [127][150/293] lr: 5.000000e-04 eta: 3:28:38 time: 0.561422 data_time: 0.071383 memory: 14267 loss_kpt: 0.000619 acc_pose: 0.845087 loss: 0.000619 2022/09/23 19:07:43 - mmengine - INFO - Epoch(train) [127][200/293] lr: 5.000000e-04 eta: 3:28:16 time: 0.601028 data_time: 0.086365 memory: 14267 loss_kpt: 0.000610 acc_pose: 0.873686 loss: 0.000610 2022/09/23 19:08:12 - mmengine - INFO - Epoch(train) [127][250/293] lr: 5.000000e-04 eta: 3:27:52 time: 0.577600 data_time: 0.077962 memory: 14267 loss_kpt: 0.000628 acc_pose: 0.831255 loss: 0.000628 2022/09/23 19:08:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:09:08 - mmengine - INFO - Epoch(train) [128][50/293] lr: 5.000000e-04 eta: 3:26:54 time: 0.627488 data_time: 0.099431 memory: 14267 loss_kpt: 0.000606 acc_pose: 0.850454 loss: 0.000606 2022/09/23 19:09:37 - mmengine - INFO - Epoch(train) [128][100/293] lr: 5.000000e-04 eta: 3:26:31 time: 0.579227 data_time: 0.088421 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.883398 loss: 0.000594 2022/09/23 19:10:07 - mmengine - INFO - Epoch(train) [128][150/293] lr: 5.000000e-04 eta: 3:26:08 time: 0.594039 data_time: 0.094570 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.860862 loss: 0.000600 2022/09/23 19:10:36 - mmengine - INFO - Epoch(train) [128][200/293] lr: 5.000000e-04 eta: 3:25:44 time: 0.582716 data_time: 0.096738 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.765316 loss: 0.000608 2022/09/23 19:11:05 - mmengine - INFO - Epoch(train) [128][250/293] lr: 5.000000e-04 eta: 3:25:21 time: 0.585943 data_time: 0.079592 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.827856 loss: 0.000600 2022/09/23 19:11:31 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:12:02 - mmengine - INFO - Epoch(train) [129][50/293] lr: 5.000000e-04 eta: 3:24:23 time: 0.623636 data_time: 0.116789 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.797730 loss: 0.000614 2022/09/23 19:12:32 - mmengine - INFO - Epoch(train) [129][100/293] lr: 5.000000e-04 eta: 3:24:00 time: 0.600551 data_time: 0.100952 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.824820 loss: 0.000604 2022/09/23 19:13:01 - mmengine - INFO - Epoch(train) [129][150/293] lr: 5.000000e-04 eta: 3:23:37 time: 0.578118 data_time: 0.114589 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.830888 loss: 0.000613 2022/09/23 19:13:29 - mmengine - INFO - Epoch(train) [129][200/293] lr: 5.000000e-04 eta: 3:23:13 time: 0.570875 data_time: 0.085862 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.866057 loss: 0.000597 2022/09/23 19:13:59 - mmengine - INFO - Epoch(train) [129][250/293] lr: 5.000000e-04 eta: 3:22:50 time: 0.594790 data_time: 0.084463 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.815112 loss: 0.000611 2022/09/23 19:14:24 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:14:56 - mmengine - INFO - Epoch(train) [130][50/293] lr: 5.000000e-04 eta: 3:21:52 time: 0.628327 data_time: 0.105294 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.793376 loss: 0.000598 2022/09/23 19:15:25 - mmengine - INFO - Epoch(train) [130][100/293] lr: 5.000000e-04 eta: 3:21:29 time: 0.593559 data_time: 0.096322 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.864013 loss: 0.000592 2022/09/23 19:15:55 - mmengine - INFO - Epoch(train) [130][150/293] lr: 5.000000e-04 eta: 3:21:06 time: 0.592570 data_time: 0.098080 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.851581 loss: 0.000603 2022/09/23 19:16:25 - mmengine - INFO - Epoch(train) [130][200/293] lr: 5.000000e-04 eta: 3:20:44 time: 0.606406 data_time: 0.095195 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.824737 loss: 0.000607 2022/09/23 19:16:27 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:16:55 - mmengine - INFO - Epoch(train) [130][250/293] lr: 5.000000e-04 eta: 3:20:21 time: 0.604509 data_time: 0.090856 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.826926 loss: 0.000612 2022/09/23 19:17:21 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:17:21 - mmengine - INFO - Saving checkpoint at 130 epochs 2022/09/23 19:17:43 - mmengine - INFO - Epoch(val) [130][50/407] eta: 0:02:07 time: 0.357506 data_time: 0.058534 memory: 14267 2022/09/23 19:18:00 - mmengine - INFO - Epoch(val) [130][100/407] eta: 0:01:43 time: 0.338378 data_time: 0.047601 memory: 1464 2022/09/23 19:18:17 - mmengine - INFO - Epoch(val) [130][150/407] eta: 0:01:27 time: 0.340177 data_time: 0.061174 memory: 1464 2022/09/23 19:18:33 - mmengine - INFO - Epoch(val) [130][200/407] eta: 0:01:06 time: 0.319833 data_time: 0.040948 memory: 1464 2022/09/23 19:18:50 - mmengine - INFO - Epoch(val) [130][250/407] eta: 0:00:53 time: 0.340674 data_time: 0.051294 memory: 1464 2022/09/23 19:19:07 - mmengine - INFO - Epoch(val) [130][300/407] eta: 0:00:36 time: 0.337970 data_time: 0.051065 memory: 1464 2022/09/23 19:19:25 - mmengine - INFO - Epoch(val) [130][350/407] eta: 0:00:20 time: 0.353909 data_time: 0.066733 memory: 1464 2022/09/23 19:19:41 - mmengine - INFO - Epoch(val) [130][400/407] eta: 0:00:02 time: 0.318844 data_time: 0.046242 memory: 1464 2022/09/23 19:20:14 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 19:20:27 - mmengine - INFO - Epoch(val) [130][407/407] coco/AP: 0.714285 coco/AP .5: 0.889162 coco/AP .75: 0.779777 coco/AP (M): 0.668938 coco/AP (L): 0.791915 coco/AR: 0.767758 coco/AR .5: 0.928999 coco/AR .75: 0.827613 coco/AR (M): 0.718957 coco/AR (L): 0.837718 2022/09/23 19:20:57 - mmengine - INFO - Epoch(train) [131][50/293] lr: 5.000000e-04 eta: 3:19:23 time: 0.610290 data_time: 0.113053 memory: 14267 loss_kpt: 0.000590 acc_pose: 0.798939 loss: 0.000590 2022/09/23 19:21:26 - mmengine - INFO - Epoch(train) [131][100/293] lr: 5.000000e-04 eta: 3:18:59 time: 0.571079 data_time: 0.078559 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.790830 loss: 0.000608 2022/09/23 19:21:56 - mmengine - INFO - Epoch(train) [131][150/293] lr: 5.000000e-04 eta: 3:18:36 time: 0.600751 data_time: 0.082736 memory: 14267 loss_kpt: 0.000601 acc_pose: 0.840159 loss: 0.000601 2022/09/23 19:22:26 - mmengine - INFO - Epoch(train) [131][200/293] lr: 5.000000e-04 eta: 3:18:13 time: 0.601400 data_time: 0.088551 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.827852 loss: 0.000614 2022/09/23 19:22:55 - mmengine - INFO - Epoch(train) [131][250/293] lr: 5.000000e-04 eta: 3:17:50 time: 0.576782 data_time: 0.080194 memory: 14267 loss_kpt: 0.000622 acc_pose: 0.814468 loss: 0.000622 2022/09/23 19:23:20 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:23:52 - mmengine - INFO - Epoch(train) [132][50/293] lr: 5.000000e-04 eta: 3:16:53 time: 0.634133 data_time: 0.113801 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.804413 loss: 0.000611 2022/09/23 19:24:22 - mmengine - INFO - Epoch(train) [132][100/293] lr: 5.000000e-04 eta: 3:16:30 time: 0.602993 data_time: 0.085706 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.766004 loss: 0.000608 2022/09/23 19:24:51 - mmengine - INFO - Epoch(train) [132][150/293] lr: 5.000000e-04 eta: 3:16:06 time: 0.584798 data_time: 0.097721 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.840221 loss: 0.000603 2022/09/23 19:25:20 - mmengine - INFO - Epoch(train) [132][200/293] lr: 5.000000e-04 eta: 3:15:43 time: 0.584628 data_time: 0.079014 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.825115 loss: 0.000607 2022/09/23 19:25:51 - mmengine - INFO - Epoch(train) [132][250/293] lr: 5.000000e-04 eta: 3:15:20 time: 0.606373 data_time: 0.081413 memory: 14267 loss_kpt: 0.000610 acc_pose: 0.808344 loss: 0.000610 2022/09/23 19:26:16 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:26:48 - mmengine - INFO - Epoch(train) [133][50/293] lr: 5.000000e-04 eta: 3:14:23 time: 0.623785 data_time: 0.094576 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.867600 loss: 0.000600 2022/09/23 19:27:18 - mmengine - INFO - Epoch(train) [133][100/293] lr: 5.000000e-04 eta: 3:14:00 time: 0.593476 data_time: 0.091931 memory: 14267 loss_kpt: 0.000609 acc_pose: 0.845933 loss: 0.000609 2022/09/23 19:27:48 - mmengine - INFO - Epoch(train) [133][150/293] lr: 5.000000e-04 eta: 3:13:37 time: 0.607269 data_time: 0.087643 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.823261 loss: 0.000611 2022/09/23 19:28:17 - mmengine - INFO - Epoch(train) [133][200/293] lr: 5.000000e-04 eta: 3:13:14 time: 0.585769 data_time: 0.080136 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.867515 loss: 0.000600 2022/09/23 19:28:47 - mmengine - INFO - Epoch(train) [133][250/293] lr: 5.000000e-04 eta: 3:12:50 time: 0.593699 data_time: 0.094421 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.843060 loss: 0.000605 2022/09/23 19:29:12 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:29:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:29:44 - mmengine - INFO - Epoch(train) [134][50/293] lr: 5.000000e-04 eta: 3:11:54 time: 0.636827 data_time: 0.106157 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.812350 loss: 0.000604 2022/09/23 19:30:13 - mmengine - INFO - Epoch(train) [134][100/293] lr: 5.000000e-04 eta: 3:11:30 time: 0.578758 data_time: 0.072277 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.798569 loss: 0.000604 2022/09/23 19:30:42 - mmengine - INFO - Epoch(train) [134][150/293] lr: 5.000000e-04 eta: 3:11:07 time: 0.585656 data_time: 0.076644 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.809097 loss: 0.000611 2022/09/23 19:31:12 - mmengine - INFO - Epoch(train) [134][200/293] lr: 5.000000e-04 eta: 3:10:43 time: 0.596256 data_time: 0.081949 memory: 14267 loss_kpt: 0.000613 acc_pose: 0.845260 loss: 0.000613 2022/09/23 19:31:41 - mmengine - INFO - Epoch(train) [134][250/293] lr: 5.000000e-04 eta: 3:10:20 time: 0.580325 data_time: 0.083072 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.844127 loss: 0.000611 2022/09/23 19:32:06 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:32:37 - mmengine - INFO - Epoch(train) [135][50/293] lr: 5.000000e-04 eta: 3:09:23 time: 0.612713 data_time: 0.127609 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.780528 loss: 0.000612 2022/09/23 19:33:06 - mmengine - INFO - Epoch(train) [135][100/293] lr: 5.000000e-04 eta: 3:08:59 time: 0.582608 data_time: 0.082260 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.855081 loss: 0.000603 2022/09/23 19:33:36 - mmengine - INFO - Epoch(train) [135][150/293] lr: 5.000000e-04 eta: 3:08:36 time: 0.609060 data_time: 0.084009 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.814078 loss: 0.000605 2022/09/23 19:34:06 - mmengine - INFO - Epoch(train) [135][200/293] lr: 5.000000e-04 eta: 3:08:13 time: 0.592901 data_time: 0.087567 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.864606 loss: 0.000612 2022/09/23 19:34:36 - mmengine - INFO - Epoch(train) [135][250/293] lr: 5.000000e-04 eta: 3:07:50 time: 0.599942 data_time: 0.097882 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.829701 loss: 0.000595 2022/09/23 19:35:01 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:35:33 - mmengine - INFO - Epoch(train) [136][50/293] lr: 5.000000e-04 eta: 3:06:53 time: 0.638186 data_time: 0.104075 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.788224 loss: 0.000611 2022/09/23 19:36:03 - mmengine - INFO - Epoch(train) [136][100/293] lr: 5.000000e-04 eta: 3:06:30 time: 0.601616 data_time: 0.087047 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.854597 loss: 0.000595 2022/09/23 19:36:33 - mmengine - INFO - Epoch(train) [136][150/293] lr: 5.000000e-04 eta: 3:06:07 time: 0.599109 data_time: 0.082052 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.864479 loss: 0.000604 2022/09/23 19:37:02 - mmengine - INFO - Epoch(train) [136][200/293] lr: 5.000000e-04 eta: 3:05:44 time: 0.583986 data_time: 0.084449 memory: 14267 loss_kpt: 0.000611 acc_pose: 0.805284 loss: 0.000611 2022/09/23 19:37:32 - mmengine - INFO - Epoch(train) [136][250/293] lr: 5.000000e-04 eta: 3:05:20 time: 0.602099 data_time: 0.077828 memory: 14267 loss_kpt: 0.000612 acc_pose: 0.765437 loss: 0.000612 2022/09/23 19:37:58 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:38:29 - mmengine - INFO - Epoch(train) [137][50/293] lr: 5.000000e-04 eta: 3:04:24 time: 0.622968 data_time: 0.097787 memory: 14267 loss_kpt: 0.000609 acc_pose: 0.833435 loss: 0.000609 2022/09/23 19:38:58 - mmengine - INFO - Epoch(train) [137][100/293] lr: 5.000000e-04 eta: 3:04:00 time: 0.578962 data_time: 0.074550 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.792590 loss: 0.000600 2022/09/23 19:39:28 - mmengine - INFO - Epoch(train) [137][150/293] lr: 5.000000e-04 eta: 3:03:37 time: 0.595602 data_time: 0.075603 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.820578 loss: 0.000598 2022/09/23 19:39:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:39:57 - mmengine - INFO - Epoch(train) [137][200/293] lr: 5.000000e-04 eta: 3:03:13 time: 0.586920 data_time: 0.100592 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.782195 loss: 0.000595 2022/09/23 19:40:27 - mmengine - INFO - Epoch(train) [137][250/293] lr: 5.000000e-04 eta: 3:02:50 time: 0.602304 data_time: 0.096154 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.837973 loss: 0.000603 2022/09/23 19:40:53 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:41:25 - mmengine - INFO - Epoch(train) [138][50/293] lr: 5.000000e-04 eta: 3:01:54 time: 0.631240 data_time: 0.092220 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.826610 loss: 0.000598 2022/09/23 19:41:55 - mmengine - INFO - Epoch(train) [138][100/293] lr: 5.000000e-04 eta: 3:01:31 time: 0.608839 data_time: 0.085552 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.820612 loss: 0.000597 2022/09/23 19:42:24 - mmengine - INFO - Epoch(train) [138][150/293] lr: 5.000000e-04 eta: 3:01:07 time: 0.569418 data_time: 0.088580 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.860275 loss: 0.000598 2022/09/23 19:42:53 - mmengine - INFO - Epoch(train) [138][200/293] lr: 5.000000e-04 eta: 3:00:43 time: 0.589748 data_time: 0.077520 memory: 14267 loss_kpt: 0.000601 acc_pose: 0.837282 loss: 0.000601 2022/09/23 19:43:24 - mmengine - INFO - Epoch(train) [138][250/293] lr: 5.000000e-04 eta: 3:00:20 time: 0.604085 data_time: 0.086474 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.831565 loss: 0.000597 2022/09/23 19:43:49 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:44:20 - mmengine - INFO - Epoch(train) [139][50/293] lr: 5.000000e-04 eta: 2:59:24 time: 0.624881 data_time: 0.113233 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.829274 loss: 0.000600 2022/09/23 19:44:50 - mmengine - INFO - Epoch(train) [139][100/293] lr: 5.000000e-04 eta: 2:59:01 time: 0.609729 data_time: 0.105563 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.825357 loss: 0.000607 2022/09/23 19:45:20 - mmengine - INFO - Epoch(train) [139][150/293] lr: 5.000000e-04 eta: 2:58:38 time: 0.590890 data_time: 0.065793 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.803649 loss: 0.000604 2022/09/23 19:45:51 - mmengine - INFO - Epoch(train) [139][200/293] lr: 5.000000e-04 eta: 2:58:15 time: 0.612723 data_time: 0.083232 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.838219 loss: 0.000600 2022/09/23 19:46:20 - mmengine - INFO - Epoch(train) [139][250/293] lr: 5.000000e-04 eta: 2:57:51 time: 0.592133 data_time: 0.078407 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.848846 loss: 0.000607 2022/09/23 19:46:44 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:47:16 - mmengine - INFO - Epoch(train) [140][50/293] lr: 5.000000e-04 eta: 2:56:55 time: 0.635726 data_time: 0.125282 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.774773 loss: 0.000597 2022/09/23 19:47:45 - mmengine - INFO - Epoch(train) [140][100/293] lr: 5.000000e-04 eta: 2:56:32 time: 0.583453 data_time: 0.082949 memory: 14267 loss_kpt: 0.000586 acc_pose: 0.885277 loss: 0.000586 2022/09/23 19:48:16 - mmengine - INFO - Epoch(train) [140][150/293] lr: 5.000000e-04 eta: 2:56:08 time: 0.606384 data_time: 0.094629 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.837205 loss: 0.000596 2022/09/23 19:48:45 - mmengine - INFO - Epoch(train) [140][200/293] lr: 5.000000e-04 eta: 2:55:45 time: 0.594955 data_time: 0.089321 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.828912 loss: 0.000604 2022/09/23 19:49:16 - mmengine - INFO - Epoch(train) [140][250/293] lr: 5.000000e-04 eta: 2:55:22 time: 0.606843 data_time: 0.087986 memory: 14267 loss_kpt: 0.000614 acc_pose: 0.825318 loss: 0.000614 2022/09/23 19:49:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:49:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:49:41 - mmengine - INFO - Saving checkpoint at 140 epochs 2022/09/23 19:50:05 - mmengine - INFO - Epoch(val) [140][50/407] eta: 0:02:09 time: 0.364096 data_time: 0.077038 memory: 14267 2022/09/23 19:50:23 - mmengine - INFO - Epoch(val) [140][100/407] eta: 0:01:51 time: 0.363186 data_time: 0.062246 memory: 1464 2022/09/23 19:50:41 - mmengine - INFO - Epoch(val) [140][150/407] eta: 0:01:29 time: 0.346917 data_time: 0.063163 memory: 1464 2022/09/23 19:50:58 - mmengine - INFO - Epoch(val) [140][200/407] eta: 0:01:11 time: 0.345024 data_time: 0.065722 memory: 1464 2022/09/23 19:51:15 - mmengine - INFO - Epoch(val) [140][250/407] eta: 0:00:54 time: 0.344831 data_time: 0.064322 memory: 1464 2022/09/23 19:51:32 - mmengine - INFO - Epoch(val) [140][300/407] eta: 0:00:36 time: 0.343217 data_time: 0.063516 memory: 1464 2022/09/23 19:51:49 - mmengine - INFO - Epoch(val) [140][350/407] eta: 0:00:19 time: 0.334391 data_time: 0.050097 memory: 1464 2022/09/23 19:52:04 - mmengine - INFO - Epoch(val) [140][400/407] eta: 0:00:02 time: 0.290364 data_time: 0.046600 memory: 1464 2022/09/23 19:52:37 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 19:52:50 - mmengine - INFO - Epoch(val) [140][407/407] coco/AP: 0.714733 coco/AP .5: 0.892550 coco/AP .75: 0.780556 coco/AP (M): 0.669110 coco/AP (L): 0.790700 coco/AR: 0.767632 coco/AR .5: 0.930888 coco/AR .75: 0.827298 coco/AR (M): 0.717482 coco/AR (L): 0.839019 2022/09/23 19:53:21 - mmengine - INFO - Epoch(train) [141][50/293] lr: 5.000000e-04 eta: 2:54:26 time: 0.623399 data_time: 0.091810 memory: 14267 loss_kpt: 0.000599 acc_pose: 0.699891 loss: 0.000599 2022/09/23 19:53:51 - mmengine - INFO - Epoch(train) [141][100/293] lr: 5.000000e-04 eta: 2:54:02 time: 0.595170 data_time: 0.084953 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.777513 loss: 0.000604 2022/09/23 19:54:20 - mmengine - INFO - Epoch(train) [141][150/293] lr: 5.000000e-04 eta: 2:53:38 time: 0.570910 data_time: 0.089381 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.827045 loss: 0.000600 2022/09/23 19:54:49 - mmengine - INFO - Epoch(train) [141][200/293] lr: 5.000000e-04 eta: 2:53:15 time: 0.586044 data_time: 0.072982 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.841844 loss: 0.000597 2022/09/23 19:55:19 - mmengine - INFO - Epoch(train) [141][250/293] lr: 5.000000e-04 eta: 2:52:51 time: 0.599603 data_time: 0.083489 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.793339 loss: 0.000607 2022/09/23 19:55:44 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:56:16 - mmengine - INFO - Epoch(train) [142][50/293] lr: 5.000000e-04 eta: 2:51:56 time: 0.633663 data_time: 0.117361 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.862692 loss: 0.000594 2022/09/23 19:56:47 - mmengine - INFO - Epoch(train) [142][100/293] lr: 5.000000e-04 eta: 2:51:32 time: 0.606059 data_time: 0.102020 memory: 14267 loss_kpt: 0.000606 acc_pose: 0.800516 loss: 0.000606 2022/09/23 19:57:16 - mmengine - INFO - Epoch(train) [142][150/293] lr: 5.000000e-04 eta: 2:51:09 time: 0.587893 data_time: 0.095617 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.811870 loss: 0.000602 2022/09/23 19:57:46 - mmengine - INFO - Epoch(train) [142][200/293] lr: 5.000000e-04 eta: 2:50:45 time: 0.595898 data_time: 0.085417 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.811059 loss: 0.000602 2022/09/23 19:58:15 - mmengine - INFO - Epoch(train) [142][250/293] lr: 5.000000e-04 eta: 2:50:21 time: 0.588564 data_time: 0.093972 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.856050 loss: 0.000602 2022/09/23 19:58:41 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 19:59:11 - mmengine - INFO - Epoch(train) [143][50/293] lr: 5.000000e-04 eta: 2:49:25 time: 0.594255 data_time: 0.098701 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.760696 loss: 0.000605 2022/09/23 19:59:41 - mmengine - INFO - Epoch(train) [143][100/293] lr: 5.000000e-04 eta: 2:49:02 time: 0.603013 data_time: 0.070860 memory: 14267 loss_kpt: 0.000591 acc_pose: 0.819924 loss: 0.000591 2022/09/23 20:00:11 - mmengine - INFO - Epoch(train) [143][150/293] lr: 5.000000e-04 eta: 2:48:38 time: 0.599491 data_time: 0.079119 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.835376 loss: 0.000608 2022/09/23 20:00:41 - mmengine - INFO - Epoch(train) [143][200/293] lr: 5.000000e-04 eta: 2:48:15 time: 0.604646 data_time: 0.084034 memory: 14267 loss_kpt: 0.000606 acc_pose: 0.809360 loss: 0.000606 2022/09/23 20:01:11 - mmengine - INFO - Epoch(train) [143][250/293] lr: 5.000000e-04 eta: 2:47:51 time: 0.589420 data_time: 0.085438 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.857930 loss: 0.000593 2022/09/23 20:01:37 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:02:09 - mmengine - INFO - Epoch(train) [144][50/293] lr: 5.000000e-04 eta: 2:46:56 time: 0.637710 data_time: 0.111486 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.843739 loss: 0.000605 2022/09/23 20:02:39 - mmengine - INFO - Epoch(train) [144][100/293] lr: 5.000000e-04 eta: 2:46:33 time: 0.600602 data_time: 0.096055 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.844593 loss: 0.000592 2022/09/23 20:02:39 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:03:09 - mmengine - INFO - Epoch(train) [144][150/293] lr: 5.000000e-04 eta: 2:46:09 time: 0.593374 data_time: 0.099765 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.835519 loss: 0.000593 2022/09/23 20:03:39 - mmengine - INFO - Epoch(train) [144][200/293] lr: 5.000000e-04 eta: 2:45:46 time: 0.606841 data_time: 0.094816 memory: 14267 loss_kpt: 0.000609 acc_pose: 0.768629 loss: 0.000609 2022/09/23 20:04:09 - mmengine - INFO - Epoch(train) [144][250/293] lr: 5.000000e-04 eta: 2:45:22 time: 0.608778 data_time: 0.092416 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.788403 loss: 0.000592 2022/09/23 20:04:35 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:05:06 - mmengine - INFO - Epoch(train) [145][50/293] lr: 5.000000e-04 eta: 2:44:27 time: 0.619169 data_time: 0.123550 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.830678 loss: 0.000593 2022/09/23 20:05:36 - mmengine - INFO - Epoch(train) [145][100/293] lr: 5.000000e-04 eta: 2:44:03 time: 0.595263 data_time: 0.095860 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.828895 loss: 0.000605 2022/09/23 20:06:06 - mmengine - INFO - Epoch(train) [145][150/293] lr: 5.000000e-04 eta: 2:43:40 time: 0.605025 data_time: 0.073592 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.827982 loss: 0.000602 2022/09/23 20:06:36 - mmengine - INFO - Epoch(train) [145][200/293] lr: 5.000000e-04 eta: 2:43:16 time: 0.600207 data_time: 0.089709 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.840948 loss: 0.000592 2022/09/23 20:07:07 - mmengine - INFO - Epoch(train) [145][250/293] lr: 5.000000e-04 eta: 2:42:53 time: 0.617493 data_time: 0.090413 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.826946 loss: 0.000596 2022/09/23 20:07:34 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:08:06 - mmengine - INFO - Epoch(train) [146][50/293] lr: 5.000000e-04 eta: 2:41:58 time: 0.638697 data_time: 0.106700 memory: 14267 loss_kpt: 0.000589 acc_pose: 0.852695 loss: 0.000589 2022/09/23 20:08:36 - mmengine - INFO - Epoch(train) [146][100/293] lr: 5.000000e-04 eta: 2:41:35 time: 0.600605 data_time: 0.087988 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.825660 loss: 0.000605 2022/09/23 20:09:07 - mmengine - INFO - Epoch(train) [146][150/293] lr: 5.000000e-04 eta: 2:41:12 time: 0.616780 data_time: 0.102494 memory: 14267 loss_kpt: 0.000591 acc_pose: 0.764030 loss: 0.000591 2022/09/23 20:09:36 - mmengine - INFO - Epoch(train) [146][200/293] lr: 5.000000e-04 eta: 2:40:48 time: 0.591403 data_time: 0.083917 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.829261 loss: 0.000598 2022/09/23 20:10:06 - mmengine - INFO - Epoch(train) [146][250/293] lr: 5.000000e-04 eta: 2:40:24 time: 0.600296 data_time: 0.092002 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.809550 loss: 0.000593 2022/09/23 20:10:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:11:01 - mmengine - INFO - Epoch(train) [147][50/293] lr: 5.000000e-04 eta: 2:39:29 time: 0.622606 data_time: 0.105088 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.820808 loss: 0.000593 2022/09/23 20:11:32 - mmengine - INFO - Epoch(train) [147][100/293] lr: 5.000000e-04 eta: 2:39:06 time: 0.606965 data_time: 0.089439 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.850765 loss: 0.000585 2022/09/23 20:12:02 - mmengine - INFO - Epoch(train) [147][150/293] lr: 5.000000e-04 eta: 2:38:42 time: 0.602514 data_time: 0.088837 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.821145 loss: 0.000592 2022/09/23 20:12:32 - mmengine - INFO - Epoch(train) [147][200/293] lr: 5.000000e-04 eta: 2:38:19 time: 0.609043 data_time: 0.104864 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.828016 loss: 0.000608 2022/09/23 20:12:45 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:13:02 - mmengine - INFO - Epoch(train) [147][250/293] lr: 5.000000e-04 eta: 2:37:55 time: 0.595406 data_time: 0.083766 memory: 14267 loss_kpt: 0.000589 acc_pose: 0.854002 loss: 0.000589 2022/09/23 20:13:28 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:14:01 - mmengine - INFO - Epoch(train) [148][50/293] lr: 5.000000e-04 eta: 2:37:01 time: 0.648966 data_time: 0.125640 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.810576 loss: 0.000594 2022/09/23 20:14:30 - mmengine - INFO - Epoch(train) [148][100/293] lr: 5.000000e-04 eta: 2:36:37 time: 0.583160 data_time: 0.079596 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.838390 loss: 0.000585 2022/09/23 20:15:00 - mmengine - INFO - Epoch(train) [148][150/293] lr: 5.000000e-04 eta: 2:36:13 time: 0.591494 data_time: 0.081567 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.768369 loss: 0.000603 2022/09/23 20:15:29 - mmengine - INFO - Epoch(train) [148][200/293] lr: 5.000000e-04 eta: 2:35:49 time: 0.588682 data_time: 0.097343 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.793715 loss: 0.000598 2022/09/23 20:15:59 - mmengine - INFO - Epoch(train) [148][250/293] lr: 5.000000e-04 eta: 2:35:25 time: 0.592292 data_time: 0.095690 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.834277 loss: 0.000595 2022/09/23 20:16:24 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:16:56 - mmengine - INFO - Epoch(train) [149][50/293] lr: 5.000000e-04 eta: 2:34:30 time: 0.621742 data_time: 0.092549 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.848744 loss: 0.000595 2022/09/23 20:17:25 - mmengine - INFO - Epoch(train) [149][100/293] lr: 5.000000e-04 eta: 2:34:06 time: 0.584152 data_time: 0.100327 memory: 14267 loss_kpt: 0.000591 acc_pose: 0.845903 loss: 0.000591 2022/09/23 20:17:57 - mmengine - INFO - Epoch(train) [149][150/293] lr: 5.000000e-04 eta: 2:33:43 time: 0.635814 data_time: 0.098196 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.866540 loss: 0.000585 2022/09/23 20:18:26 - mmengine - INFO - Epoch(train) [149][200/293] lr: 5.000000e-04 eta: 2:33:19 time: 0.586620 data_time: 0.091214 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.831996 loss: 0.000598 2022/09/23 20:18:58 - mmengine - INFO - Epoch(train) [149][250/293] lr: 5.000000e-04 eta: 2:32:56 time: 0.632927 data_time: 0.099533 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.842505 loss: 0.000605 2022/09/23 20:19:23 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:19:55 - mmengine - INFO - Epoch(train) [150][50/293] lr: 5.000000e-04 eta: 2:32:02 time: 0.633644 data_time: 0.108625 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.808794 loss: 0.000600 2022/09/23 20:20:25 - mmengine - INFO - Epoch(train) [150][100/293] lr: 5.000000e-04 eta: 2:31:38 time: 0.600062 data_time: 0.089584 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.780077 loss: 0.000592 2022/09/23 20:20:55 - mmengine - INFO - Epoch(train) [150][150/293] lr: 5.000000e-04 eta: 2:31:14 time: 0.592516 data_time: 0.074651 memory: 14267 loss_kpt: 0.000604 acc_pose: 0.811419 loss: 0.000604 2022/09/23 20:21:26 - mmengine - INFO - Epoch(train) [150][200/293] lr: 5.000000e-04 eta: 2:30:51 time: 0.615718 data_time: 0.103567 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.840102 loss: 0.000597 2022/09/23 20:21:55 - mmengine - INFO - Epoch(train) [150][250/293] lr: 5.000000e-04 eta: 2:30:27 time: 0.593309 data_time: 0.092390 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.821237 loss: 0.000602 2022/09/23 20:22:21 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:22:21 - mmengine - INFO - Saving checkpoint at 150 epochs 2022/09/23 20:22:45 - mmengine - INFO - Epoch(val) [150][50/407] eta: 0:02:10 time: 0.365100 data_time: 0.086573 memory: 14267 2022/09/23 20:23:02 - mmengine - INFO - Epoch(val) [150][100/407] eta: 0:01:44 time: 0.340857 data_time: 0.073123 memory: 1464 2022/09/23 20:23:19 - mmengine - INFO - Epoch(val) [150][150/407] eta: 0:01:29 time: 0.346603 data_time: 0.050517 memory: 1464 2022/09/23 20:23:36 - mmengine - INFO - Epoch(val) [150][200/407] eta: 0:01:08 time: 0.330710 data_time: 0.052652 memory: 1464 2022/09/23 20:23:53 - mmengine - INFO - Epoch(val) [150][250/407] eta: 0:00:54 time: 0.348965 data_time: 0.076027 memory: 1464 2022/09/23 20:24:09 - mmengine - INFO - Epoch(val) [150][300/407] eta: 0:00:34 time: 0.326833 data_time: 0.053696 memory: 1464 2022/09/23 20:24:27 - mmengine - INFO - Epoch(val) [150][350/407] eta: 0:00:19 time: 0.347996 data_time: 0.077741 memory: 1464 2022/09/23 20:24:42 - mmengine - INFO - Epoch(val) [150][400/407] eta: 0:00:02 time: 0.307953 data_time: 0.046953 memory: 1464 2022/09/23 20:25:16 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 20:25:29 - mmengine - INFO - Epoch(val) [150][407/407] coco/AP: 0.718589 coco/AP .5: 0.893797 coco/AP .75: 0.781320 coco/AP (M): 0.674164 coco/AP (L): 0.794273 coco/AR: 0.771474 coco/AR .5: 0.931518 coco/AR .75: 0.829975 coco/AR (M): 0.723627 coco/AR (L): 0.840320 2022/09/23 20:25:30 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_120.pth is removed 2022/09/23 20:25:32 - mmengine - INFO - The best checkpoint with 0.7186 coco/AP at 150 epoch is saved to best_coco/AP_epoch_150.pth. 2022/09/23 20:26:02 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:26:02 - mmengine - INFO - Epoch(train) [151][50/293] lr: 5.000000e-04 eta: 2:29:32 time: 0.602549 data_time: 0.101640 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.840672 loss: 0.000596 2022/09/23 20:26:33 - mmengine - INFO - Epoch(train) [151][100/293] lr: 5.000000e-04 eta: 2:29:08 time: 0.615024 data_time: 0.111705 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.789692 loss: 0.000598 2022/09/23 20:27:02 - mmengine - INFO - Epoch(train) [151][150/293] lr: 5.000000e-04 eta: 2:28:44 time: 0.594325 data_time: 0.083566 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.796051 loss: 0.000602 2022/09/23 20:27:33 - mmengine - INFO - Epoch(train) [151][200/293] lr: 5.000000e-04 eta: 2:28:21 time: 0.616217 data_time: 0.088906 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.830046 loss: 0.000593 2022/09/23 20:28:03 - mmengine - INFO - Epoch(train) [151][250/293] lr: 5.000000e-04 eta: 2:27:57 time: 0.603589 data_time: 0.097485 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.757215 loss: 0.000598 2022/09/23 20:28:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:29:02 - mmengine - INFO - Epoch(train) [152][50/293] lr: 5.000000e-04 eta: 2:27:03 time: 0.643756 data_time: 0.123107 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.814516 loss: 0.000597 2022/09/23 20:29:32 - mmengine - INFO - Epoch(train) [152][100/293] lr: 5.000000e-04 eta: 2:26:40 time: 0.614455 data_time: 0.100325 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.831434 loss: 0.000592 2022/09/23 20:30:02 - mmengine - INFO - Epoch(train) [152][150/293] lr: 5.000000e-04 eta: 2:26:16 time: 0.600599 data_time: 0.086243 memory: 14267 loss_kpt: 0.000591 acc_pose: 0.863903 loss: 0.000591 2022/09/23 20:30:33 - mmengine - INFO - Epoch(train) [152][200/293] lr: 5.000000e-04 eta: 2:25:52 time: 0.611808 data_time: 0.092858 memory: 14267 loss_kpt: 0.000603 acc_pose: 0.884306 loss: 0.000603 2022/09/23 20:31:03 - mmengine - INFO - Epoch(train) [152][250/293] lr: 5.000000e-04 eta: 2:25:29 time: 0.608502 data_time: 0.078285 memory: 14267 loss_kpt: 0.000606 acc_pose: 0.768457 loss: 0.000606 2022/09/23 20:31:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:32:01 - mmengine - INFO - Epoch(train) [153][50/293] lr: 5.000000e-04 eta: 2:24:35 time: 0.625737 data_time: 0.099373 memory: 14267 loss_kpt: 0.000581 acc_pose: 0.832830 loss: 0.000581 2022/09/23 20:32:30 - mmengine - INFO - Epoch(train) [153][100/293] lr: 5.000000e-04 eta: 2:24:11 time: 0.593392 data_time: 0.086384 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.833696 loss: 0.000605 2022/09/23 20:33:01 - mmengine - INFO - Epoch(train) [153][150/293] lr: 5.000000e-04 eta: 2:23:47 time: 0.605588 data_time: 0.089434 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.827233 loss: 0.000598 2022/09/23 20:33:31 - mmengine - INFO - Epoch(train) [153][200/293] lr: 5.000000e-04 eta: 2:23:23 time: 0.599124 data_time: 0.099000 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.844741 loss: 0.000602 2022/09/23 20:34:00 - mmengine - INFO - Epoch(train) [153][250/293] lr: 5.000000e-04 eta: 2:22:59 time: 0.592546 data_time: 0.089486 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.796302 loss: 0.000605 2022/09/23 20:34:26 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:34:57 - mmengine - INFO - Epoch(train) [154][50/293] lr: 5.000000e-04 eta: 2:22:05 time: 0.620159 data_time: 0.110201 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.858189 loss: 0.000593 2022/09/23 20:35:27 - mmengine - INFO - Epoch(train) [154][100/293] lr: 5.000000e-04 eta: 2:21:41 time: 0.590218 data_time: 0.085433 memory: 14267 loss_kpt: 0.000581 acc_pose: 0.829250 loss: 0.000581 2022/09/23 20:35:56 - mmengine - INFO - Epoch(train) [154][150/293] lr: 5.000000e-04 eta: 2:21:16 time: 0.588299 data_time: 0.082528 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.850564 loss: 0.000588 2022/09/23 20:36:09 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:36:27 - mmengine - INFO - Epoch(train) [154][200/293] lr: 5.000000e-04 eta: 2:20:53 time: 0.619237 data_time: 0.096534 memory: 14267 loss_kpt: 0.000598 acc_pose: 0.828570 loss: 0.000598 2022/09/23 20:36:57 - mmengine - INFO - Epoch(train) [154][250/293] lr: 5.000000e-04 eta: 2:20:29 time: 0.588530 data_time: 0.085652 memory: 14267 loss_kpt: 0.000587 acc_pose: 0.806825 loss: 0.000587 2022/09/23 20:37:24 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:37:55 - mmengine - INFO - Epoch(train) [155][50/293] lr: 5.000000e-04 eta: 2:19:35 time: 0.623665 data_time: 0.101823 memory: 14267 loss_kpt: 0.000607 acc_pose: 0.847527 loss: 0.000607 2022/09/23 20:38:24 - mmengine - INFO - Epoch(train) [155][100/293] lr: 5.000000e-04 eta: 2:19:10 time: 0.570923 data_time: 0.079665 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.843396 loss: 0.000608 2022/09/23 20:38:53 - mmengine - INFO - Epoch(train) [155][150/293] lr: 5.000000e-04 eta: 2:18:46 time: 0.588493 data_time: 0.064659 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.864594 loss: 0.000585 2022/09/23 20:39:23 - mmengine - INFO - Epoch(train) [155][200/293] lr: 5.000000e-04 eta: 2:18:22 time: 0.591413 data_time: 0.089628 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.853144 loss: 0.000582 2022/09/23 20:39:52 - mmengine - INFO - Epoch(train) [155][250/293] lr: 5.000000e-04 eta: 2:17:58 time: 0.579327 data_time: 0.079611 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.831081 loss: 0.000593 2022/09/23 20:40:17 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:40:48 - mmengine - INFO - Epoch(train) [156][50/293] lr: 5.000000e-04 eta: 2:17:04 time: 0.619416 data_time: 0.109713 memory: 14267 loss_kpt: 0.000591 acc_pose: 0.834740 loss: 0.000591 2022/09/23 20:41:18 - mmengine - INFO - Epoch(train) [156][100/293] lr: 5.000000e-04 eta: 2:16:40 time: 0.602950 data_time: 0.084227 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.835071 loss: 0.000588 2022/09/23 20:41:48 - mmengine - INFO - Epoch(train) [156][150/293] lr: 5.000000e-04 eta: 2:16:16 time: 0.592694 data_time: 0.091667 memory: 14267 loss_kpt: 0.000601 acc_pose: 0.830078 loss: 0.000601 2022/09/23 20:42:17 - mmengine - INFO - Epoch(train) [156][200/293] lr: 5.000000e-04 eta: 2:15:51 time: 0.575702 data_time: 0.087147 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.846608 loss: 0.000597 2022/09/23 20:42:46 - mmengine - INFO - Epoch(train) [156][250/293] lr: 5.000000e-04 eta: 2:15:27 time: 0.584608 data_time: 0.082438 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.861977 loss: 0.000588 2022/09/23 20:43:11 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:43:42 - mmengine - INFO - Epoch(train) [157][50/293] lr: 5.000000e-04 eta: 2:14:34 time: 0.636001 data_time: 0.115955 memory: 14267 loss_kpt: 0.000583 acc_pose: 0.852045 loss: 0.000583 2022/09/23 20:44:12 - mmengine - INFO - Epoch(train) [157][100/293] lr: 5.000000e-04 eta: 2:14:10 time: 0.588762 data_time: 0.093369 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.746041 loss: 0.000585 2022/09/23 20:44:41 - mmengine - INFO - Epoch(train) [157][150/293] lr: 5.000000e-04 eta: 2:13:45 time: 0.588133 data_time: 0.088283 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.828974 loss: 0.000605 2022/09/23 20:45:13 - mmengine - INFO - Epoch(train) [157][200/293] lr: 5.000000e-04 eta: 2:13:22 time: 0.630303 data_time: 0.100469 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.846144 loss: 0.000596 2022/09/23 20:45:42 - mmengine - INFO - Epoch(train) [157][250/293] lr: 5.000000e-04 eta: 2:12:57 time: 0.589819 data_time: 0.080897 memory: 14267 loss_kpt: 0.000590 acc_pose: 0.822909 loss: 0.000590 2022/09/23 20:46:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:46:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:46:39 - mmengine - INFO - Epoch(train) [158][50/293] lr: 5.000000e-04 eta: 2:12:04 time: 0.623229 data_time: 0.117417 memory: 14267 loss_kpt: 0.000584 acc_pose: 0.839213 loss: 0.000584 2022/09/23 20:47:10 - mmengine - INFO - Epoch(train) [158][100/293] lr: 5.000000e-04 eta: 2:11:40 time: 0.615803 data_time: 0.087809 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.844088 loss: 0.000595 2022/09/23 20:47:39 - mmengine - INFO - Epoch(train) [158][150/293] lr: 5.000000e-04 eta: 2:11:16 time: 0.574463 data_time: 0.083106 memory: 14267 loss_kpt: 0.000583 acc_pose: 0.865400 loss: 0.000583 2022/09/23 20:48:10 - mmengine - INFO - Epoch(train) [158][200/293] lr: 5.000000e-04 eta: 2:10:52 time: 0.610918 data_time: 0.119927 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.794157 loss: 0.000594 2022/09/23 20:48:38 - mmengine - INFO - Epoch(train) [158][250/293] lr: 5.000000e-04 eta: 2:10:27 time: 0.577284 data_time: 0.084159 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.787470 loss: 0.000594 2022/09/23 20:49:04 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:49:35 - mmengine - INFO - Epoch(train) [159][50/293] lr: 5.000000e-04 eta: 2:09:34 time: 0.628600 data_time: 0.120047 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.846869 loss: 0.000600 2022/09/23 20:50:05 - mmengine - INFO - Epoch(train) [159][100/293] lr: 5.000000e-04 eta: 2:09:10 time: 0.582559 data_time: 0.092452 memory: 14267 loss_kpt: 0.000574 acc_pose: 0.812994 loss: 0.000574 2022/09/23 20:50:34 - mmengine - INFO - Epoch(train) [159][150/293] lr: 5.000000e-04 eta: 2:08:46 time: 0.591560 data_time: 0.080759 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.787480 loss: 0.000596 2022/09/23 20:51:04 - mmengine - INFO - Epoch(train) [159][200/293] lr: 5.000000e-04 eta: 2:08:21 time: 0.591030 data_time: 0.091276 memory: 14267 loss_kpt: 0.000600 acc_pose: 0.825018 loss: 0.000600 2022/09/23 20:51:33 - mmengine - INFO - Epoch(train) [159][250/293] lr: 5.000000e-04 eta: 2:07:57 time: 0.592919 data_time: 0.093043 memory: 14267 loss_kpt: 0.000610 acc_pose: 0.836361 loss: 0.000610 2022/09/23 20:51:58 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:52:30 - mmengine - INFO - Epoch(train) [160][50/293] lr: 5.000000e-04 eta: 2:07:04 time: 0.632108 data_time: 0.098271 memory: 14267 loss_kpt: 0.000570 acc_pose: 0.849593 loss: 0.000570 2022/09/23 20:53:00 - mmengine - INFO - Epoch(train) [160][100/293] lr: 5.000000e-04 eta: 2:06:40 time: 0.594744 data_time: 0.073226 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.810970 loss: 0.000585 2022/09/23 20:53:29 - mmengine - INFO - Epoch(train) [160][150/293] lr: 5.000000e-04 eta: 2:06:16 time: 0.592947 data_time: 0.094143 memory: 14267 loss_kpt: 0.000583 acc_pose: 0.834544 loss: 0.000583 2022/09/23 20:53:59 - mmengine - INFO - Epoch(train) [160][200/293] lr: 5.000000e-04 eta: 2:05:51 time: 0.598417 data_time: 0.094228 memory: 14267 loss_kpt: 0.000569 acc_pose: 0.852440 loss: 0.000569 2022/09/23 20:54:29 - mmengine - INFO - Epoch(train) [160][250/293] lr: 5.000000e-04 eta: 2:05:27 time: 0.607048 data_time: 0.090419 memory: 14267 loss_kpt: 0.000602 acc_pose: 0.824776 loss: 0.000602 2022/09/23 20:54:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:54:55 - mmengine - INFO - Saving checkpoint at 160 epochs 2022/09/23 20:55:19 - mmengine - INFO - Epoch(val) [160][50/407] eta: 0:02:17 time: 0.385982 data_time: 0.096702 memory: 14267 2022/09/23 20:55:36 - mmengine - INFO - Epoch(val) [160][100/407] eta: 0:01:41 time: 0.331117 data_time: 0.046254 memory: 1464 2022/09/23 20:55:53 - mmengine - INFO - Epoch(val) [160][150/407] eta: 0:01:27 time: 0.339441 data_time: 0.045761 memory: 1464 2022/09/23 20:56:09 - mmengine - INFO - Epoch(val) [160][200/407] eta: 0:01:06 time: 0.320514 data_time: 0.035193 memory: 1464 2022/09/23 20:56:26 - mmengine - INFO - Epoch(val) [160][250/407] eta: 0:00:55 time: 0.350372 data_time: 0.044189 memory: 1464 2022/09/23 20:56:43 - mmengine - INFO - Epoch(val) [160][300/407] eta: 0:00:36 time: 0.341260 data_time: 0.051312 memory: 1464 2022/09/23 20:57:00 - mmengine - INFO - Epoch(val) [160][350/407] eta: 0:00:19 time: 0.343700 data_time: 0.055325 memory: 1464 2022/09/23 20:57:15 - mmengine - INFO - Epoch(val) [160][400/407] eta: 0:00:01 time: 0.283826 data_time: 0.056269 memory: 1464 2022/09/23 20:57:48 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 20:58:01 - mmengine - INFO - Epoch(val) [160][407/407] coco/AP: 0.722748 coco/AP .5: 0.897681 coco/AP .75: 0.787369 coco/AP (M): 0.677886 coco/AP (L): 0.798550 coco/AR: 0.775110 coco/AR .5: 0.935139 coco/AR .75: 0.834540 coco/AR (M): 0.725922 coco/AR (L): 0.844816 2022/09/23 20:58:01 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_150.pth is removed 2022/09/23 20:58:03 - mmengine - INFO - The best checkpoint with 0.7227 coco/AP at 160 epoch is saved to best_coco/AP_epoch_160.pth. 2022/09/23 20:58:33 - mmengine - INFO - Epoch(train) [161][50/293] lr: 5.000000e-04 eta: 2:04:34 time: 0.590294 data_time: 0.078941 memory: 14267 loss_kpt: 0.000587 acc_pose: 0.828435 loss: 0.000587 2022/09/23 20:59:03 - mmengine - INFO - Epoch(train) [161][100/293] lr: 5.000000e-04 eta: 2:04:10 time: 0.602500 data_time: 0.083921 memory: 14267 loss_kpt: 0.000595 acc_pose: 0.873408 loss: 0.000595 2022/09/23 20:59:14 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 20:59:32 - mmengine - INFO - Epoch(train) [161][150/293] lr: 5.000000e-04 eta: 2:03:45 time: 0.590632 data_time: 0.085897 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.841488 loss: 0.000594 2022/09/23 21:00:02 - mmengine - INFO - Epoch(train) [161][200/293] lr: 5.000000e-04 eta: 2:03:21 time: 0.583410 data_time: 0.091067 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.825236 loss: 0.000596 2022/09/23 21:00:32 - mmengine - INFO - Epoch(train) [161][250/293] lr: 5.000000e-04 eta: 2:02:57 time: 0.611543 data_time: 0.108034 memory: 14267 loss_kpt: 0.000589 acc_pose: 0.804421 loss: 0.000589 2022/09/23 21:00:57 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:01:28 - mmengine - INFO - Epoch(train) [162][50/293] lr: 5.000000e-04 eta: 2:02:04 time: 0.619558 data_time: 0.124491 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.837606 loss: 0.000593 2022/09/23 21:01:58 - mmengine - INFO - Epoch(train) [162][100/293] lr: 5.000000e-04 eta: 2:01:40 time: 0.611750 data_time: 0.101055 memory: 14267 loss_kpt: 0.000596 acc_pose: 0.808529 loss: 0.000596 2022/09/23 21:02:28 - mmengine - INFO - Epoch(train) [162][150/293] lr: 5.000000e-04 eta: 2:01:16 time: 0.597620 data_time: 0.091576 memory: 14267 loss_kpt: 0.000594 acc_pose: 0.852561 loss: 0.000594 2022/09/23 21:02:59 - mmengine - INFO - Epoch(train) [162][200/293] lr: 5.000000e-04 eta: 2:00:51 time: 0.610259 data_time: 0.096413 memory: 14267 loss_kpt: 0.000605 acc_pose: 0.833032 loss: 0.000605 2022/09/23 21:03:28 - mmengine - INFO - Epoch(train) [162][250/293] lr: 5.000000e-04 eta: 2:00:27 time: 0.584330 data_time: 0.079621 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.860734 loss: 0.000588 2022/09/23 21:03:53 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:04:25 - mmengine - INFO - Epoch(train) [163][50/293] lr: 5.000000e-04 eta: 1:59:34 time: 0.625522 data_time: 0.103120 memory: 14267 loss_kpt: 0.000583 acc_pose: 0.802444 loss: 0.000583 2022/09/23 21:04:55 - mmengine - INFO - Epoch(train) [163][100/293] lr: 5.000000e-04 eta: 1:59:10 time: 0.601277 data_time: 0.101309 memory: 14267 loss_kpt: 0.000576 acc_pose: 0.806111 loss: 0.000576 2022/09/23 21:05:25 - mmengine - INFO - Epoch(train) [163][150/293] lr: 5.000000e-04 eta: 1:58:46 time: 0.604564 data_time: 0.097531 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.779972 loss: 0.000588 2022/09/23 21:05:54 - mmengine - INFO - Epoch(train) [163][200/293] lr: 5.000000e-04 eta: 1:58:21 time: 0.575384 data_time: 0.087064 memory: 14267 loss_kpt: 0.000584 acc_pose: 0.836418 loss: 0.000584 2022/09/23 21:06:23 - mmengine - INFO - Epoch(train) [163][250/293] lr: 5.000000e-04 eta: 1:57:57 time: 0.583719 data_time: 0.084769 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.807774 loss: 0.000597 2022/09/23 21:06:48 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:07:20 - mmengine - INFO - Epoch(train) [164][50/293] lr: 5.000000e-04 eta: 1:57:04 time: 0.625991 data_time: 0.117227 memory: 14267 loss_kpt: 0.000571 acc_pose: 0.831044 loss: 0.000571 2022/09/23 21:07:49 - mmengine - INFO - Epoch(train) [164][100/293] lr: 5.000000e-04 eta: 1:56:40 time: 0.590619 data_time: 0.089016 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.820903 loss: 0.000582 2022/09/23 21:08:18 - mmengine - INFO - Epoch(train) [164][150/293] lr: 5.000000e-04 eta: 1:56:15 time: 0.576809 data_time: 0.075870 memory: 14267 loss_kpt: 0.000587 acc_pose: 0.832177 loss: 0.000587 2022/09/23 21:08:48 - mmengine - INFO - Epoch(train) [164][200/293] lr: 5.000000e-04 eta: 1:55:51 time: 0.589731 data_time: 0.097811 memory: 14267 loss_kpt: 0.000579 acc_pose: 0.846077 loss: 0.000579 2022/09/23 21:09:11 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:09:17 - mmengine - INFO - Epoch(train) [164][250/293] lr: 5.000000e-04 eta: 1:55:26 time: 0.590384 data_time: 0.085118 memory: 14267 loss_kpt: 0.000581 acc_pose: 0.834641 loss: 0.000581 2022/09/23 21:09:43 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:10:15 - mmengine - INFO - Epoch(train) [165][50/293] lr: 5.000000e-04 eta: 1:54:34 time: 0.636417 data_time: 0.104293 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.856590 loss: 0.000588 2022/09/23 21:10:45 - mmengine - INFO - Epoch(train) [165][100/293] lr: 5.000000e-04 eta: 1:54:10 time: 0.599837 data_time: 0.107010 memory: 14267 loss_kpt: 0.000586 acc_pose: 0.815025 loss: 0.000586 2022/09/23 21:11:16 - mmengine - INFO - Epoch(train) [165][150/293] lr: 5.000000e-04 eta: 1:53:46 time: 0.615170 data_time: 0.097528 memory: 14267 loss_kpt: 0.000586 acc_pose: 0.822705 loss: 0.000586 2022/09/23 21:11:46 - mmengine - INFO - Epoch(train) [165][200/293] lr: 5.000000e-04 eta: 1:53:21 time: 0.603953 data_time: 0.096167 memory: 14267 loss_kpt: 0.000590 acc_pose: 0.823428 loss: 0.000590 2022/09/23 21:12:15 - mmengine - INFO - Epoch(train) [165][250/293] lr: 5.000000e-04 eta: 1:52:57 time: 0.586296 data_time: 0.083087 memory: 14267 loss_kpt: 0.000588 acc_pose: 0.804656 loss: 0.000588 2022/09/23 21:12:40 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:13:11 - mmengine - INFO - Epoch(train) [166][50/293] lr: 5.000000e-04 eta: 1:52:04 time: 0.624320 data_time: 0.111619 memory: 14267 loss_kpt: 0.000584 acc_pose: 0.852985 loss: 0.000584 2022/09/23 21:13:41 - mmengine - INFO - Epoch(train) [166][100/293] lr: 5.000000e-04 eta: 1:51:40 time: 0.587997 data_time: 0.093045 memory: 14267 loss_kpt: 0.000591 acc_pose: 0.864237 loss: 0.000591 2022/09/23 21:14:12 - mmengine - INFO - Epoch(train) [166][150/293] lr: 5.000000e-04 eta: 1:51:16 time: 0.626886 data_time: 0.094822 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.807338 loss: 0.000582 2022/09/23 21:14:42 - mmengine - INFO - Epoch(train) [166][200/293] lr: 5.000000e-04 eta: 1:50:51 time: 0.587854 data_time: 0.086618 memory: 14267 loss_kpt: 0.000580 acc_pose: 0.870250 loss: 0.000580 2022/09/23 21:15:11 - mmengine - INFO - Epoch(train) [166][250/293] lr: 5.000000e-04 eta: 1:50:27 time: 0.579220 data_time: 0.092991 memory: 14267 loss_kpt: 0.000573 acc_pose: 0.833613 loss: 0.000573 2022/09/23 21:15:36 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:16:08 - mmengine - INFO - Epoch(train) [167][50/293] lr: 5.000000e-04 eta: 1:49:35 time: 0.635474 data_time: 0.114363 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.807349 loss: 0.000582 2022/09/23 21:16:37 - mmengine - INFO - Epoch(train) [167][100/293] lr: 5.000000e-04 eta: 1:49:10 time: 0.595387 data_time: 0.096609 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.862061 loss: 0.000592 2022/09/23 21:17:06 - mmengine - INFO - Epoch(train) [167][150/293] lr: 5.000000e-04 eta: 1:48:45 time: 0.578505 data_time: 0.069514 memory: 14267 loss_kpt: 0.000578 acc_pose: 0.844894 loss: 0.000578 2022/09/23 21:17:36 - mmengine - INFO - Epoch(train) [167][200/293] lr: 5.000000e-04 eta: 1:48:21 time: 0.600594 data_time: 0.090226 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.854991 loss: 0.000582 2022/09/23 21:18:06 - mmengine - INFO - Epoch(train) [167][250/293] lr: 5.000000e-04 eta: 1:47:56 time: 0.595529 data_time: 0.093846 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.881773 loss: 0.000582 2022/09/23 21:18:31 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:19:03 - mmengine - INFO - Epoch(train) [168][50/293] lr: 5.000000e-04 eta: 1:47:05 time: 0.631255 data_time: 0.107903 memory: 14267 loss_kpt: 0.000576 acc_pose: 0.852579 loss: 0.000576 2022/09/23 21:19:15 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:19:33 - mmengine - INFO - Epoch(train) [168][100/293] lr: 5.000000e-04 eta: 1:46:40 time: 0.596597 data_time: 0.102595 memory: 14267 loss_kpt: 0.000597 acc_pose: 0.815677 loss: 0.000597 2022/09/23 21:20:03 - mmengine - INFO - Epoch(train) [168][150/293] lr: 5.000000e-04 eta: 1:46:16 time: 0.598990 data_time: 0.083563 memory: 14267 loss_kpt: 0.000581 acc_pose: 0.863518 loss: 0.000581 2022/09/23 21:20:33 - mmengine - INFO - Epoch(train) [168][200/293] lr: 5.000000e-04 eta: 1:45:51 time: 0.604354 data_time: 0.094352 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.863322 loss: 0.000585 2022/09/23 21:21:03 - mmengine - INFO - Epoch(train) [168][250/293] lr: 5.000000e-04 eta: 1:45:27 time: 0.604417 data_time: 0.098604 memory: 14267 loss_kpt: 0.000583 acc_pose: 0.809251 loss: 0.000583 2022/09/23 21:21:28 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:21:59 - mmengine - INFO - Epoch(train) [169][50/293] lr: 5.000000e-04 eta: 1:44:35 time: 0.621574 data_time: 0.109374 memory: 14267 loss_kpt: 0.000578 acc_pose: 0.846170 loss: 0.000578 2022/09/23 21:22:29 - mmengine - INFO - Epoch(train) [169][100/293] lr: 5.000000e-04 eta: 1:44:10 time: 0.587321 data_time: 0.084668 memory: 14267 loss_kpt: 0.000593 acc_pose: 0.855567 loss: 0.000593 2022/09/23 21:22:58 - mmengine - INFO - Epoch(train) [169][150/293] lr: 5.000000e-04 eta: 1:43:46 time: 0.595456 data_time: 0.090114 memory: 14267 loss_kpt: 0.000608 acc_pose: 0.843993 loss: 0.000608 2022/09/23 21:23:29 - mmengine - INFO - Epoch(train) [169][200/293] lr: 5.000000e-04 eta: 1:43:21 time: 0.603545 data_time: 0.091056 memory: 14267 loss_kpt: 0.000570 acc_pose: 0.858777 loss: 0.000570 2022/09/23 21:23:58 - mmengine - INFO - Epoch(train) [169][250/293] lr: 5.000000e-04 eta: 1:42:57 time: 0.586301 data_time: 0.088629 memory: 14267 loss_kpt: 0.000587 acc_pose: 0.855202 loss: 0.000587 2022/09/23 21:24:23 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:24:54 - mmengine - INFO - Epoch(train) [170][50/293] lr: 5.000000e-04 eta: 1:42:05 time: 0.623009 data_time: 0.130577 memory: 14267 loss_kpt: 0.000583 acc_pose: 0.835272 loss: 0.000583 2022/09/23 21:25:23 - mmengine - INFO - Epoch(train) [170][100/293] lr: 5.000000e-04 eta: 1:41:40 time: 0.587055 data_time: 0.104551 memory: 14267 loss_kpt: 0.000589 acc_pose: 0.816715 loss: 0.000589 2022/09/23 21:25:52 - mmengine - INFO - Epoch(train) [170][150/293] lr: 5.000000e-04 eta: 1:41:15 time: 0.579381 data_time: 0.078694 memory: 14267 loss_kpt: 0.000592 acc_pose: 0.839265 loss: 0.000592 2022/09/23 21:26:22 - mmengine - INFO - Epoch(train) [170][200/293] lr: 5.000000e-04 eta: 1:40:51 time: 0.596913 data_time: 0.090844 memory: 14267 loss_kpt: 0.000585 acc_pose: 0.870608 loss: 0.000585 2022/09/23 21:26:51 - mmengine - INFO - Epoch(train) [170][250/293] lr: 5.000000e-04 eta: 1:40:26 time: 0.587043 data_time: 0.089210 memory: 14267 loss_kpt: 0.000582 acc_pose: 0.796795 loss: 0.000582 2022/09/23 21:27:17 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:27:17 - mmengine - INFO - Saving checkpoint at 170 epochs 2022/09/23 21:27:40 - mmengine - INFO - Epoch(val) [170][50/407] eta: 0:02:11 time: 0.369592 data_time: 0.078310 memory: 14267 2022/09/23 21:27:58 - mmengine - INFO - Epoch(val) [170][100/407] eta: 0:01:46 time: 0.347999 data_time: 0.065381 memory: 1464 2022/09/23 21:28:15 - mmengine - INFO - Epoch(val) [170][150/407] eta: 0:01:29 time: 0.348994 data_time: 0.060498 memory: 1464 2022/09/23 21:28:31 - mmengine - INFO - Epoch(val) [170][200/407] eta: 0:01:07 time: 0.324801 data_time: 0.035359 memory: 1464 2022/09/23 21:28:48 - mmengine - INFO - Epoch(val) [170][250/407] eta: 0:00:53 time: 0.338867 data_time: 0.048563 memory: 1464 2022/09/23 21:29:05 - mmengine - INFO - Epoch(val) [170][300/407] eta: 0:00:36 time: 0.339475 data_time: 0.033070 memory: 1464 2022/09/23 21:29:23 - mmengine - INFO - Epoch(val) [170][350/407] eta: 0:00:19 time: 0.345804 data_time: 0.051936 memory: 1464 2022/09/23 21:29:37 - mmengine - INFO - Epoch(val) [170][400/407] eta: 0:00:01 time: 0.283839 data_time: 0.043694 memory: 1464 2022/09/23 21:30:10 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 21:30:23 - mmengine - INFO - Epoch(val) [170][407/407] coco/AP: 0.722733 coco/AP .5: 0.897220 coco/AP .75: 0.787338 coco/AP (M): 0.678581 coco/AP (L): 0.797421 coco/AR: 0.775472 coco/AR .5: 0.935139 coco/AR .75: 0.833123 coco/AR (M): 0.726414 coco/AR (L): 0.845857 2022/09/23 21:30:54 - mmengine - INFO - Epoch(train) [171][50/293] lr: 5.000000e-05 eta: 1:39:35 time: 0.622494 data_time: 0.106106 memory: 14267 loss_kpt: 0.000587 acc_pose: 0.874229 loss: 0.000587 2022/09/23 21:31:23 - mmengine - INFO - Epoch(train) [171][100/293] lr: 5.000000e-05 eta: 1:39:10 time: 0.582977 data_time: 0.084255 memory: 14267 loss_kpt: 0.000578 acc_pose: 0.835405 loss: 0.000578 2022/09/23 21:31:52 - mmengine - INFO - Epoch(train) [171][150/293] lr: 5.000000e-05 eta: 1:38:45 time: 0.581982 data_time: 0.094191 memory: 14267 loss_kpt: 0.000574 acc_pose: 0.831194 loss: 0.000574 2022/09/23 21:32:17 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:32:23 - mmengine - INFO - Epoch(train) [171][200/293] lr: 5.000000e-05 eta: 1:38:21 time: 0.610226 data_time: 0.087634 memory: 14267 loss_kpt: 0.000567 acc_pose: 0.835496 loss: 0.000567 2022/09/23 21:32:53 - mmengine - INFO - Epoch(train) [171][250/293] lr: 5.000000e-05 eta: 1:37:56 time: 0.602080 data_time: 0.074231 memory: 14267 loss_kpt: 0.000572 acc_pose: 0.854439 loss: 0.000572 2022/09/23 21:33:18 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:33:48 - mmengine - INFO - Epoch(train) [172][50/293] lr: 5.000000e-05 eta: 1:37:04 time: 0.609358 data_time: 0.104765 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.803982 loss: 0.000550 2022/09/23 21:34:18 - mmengine - INFO - Epoch(train) [172][100/293] lr: 5.000000e-05 eta: 1:36:40 time: 0.602863 data_time: 0.091314 memory: 14267 loss_kpt: 0.000572 acc_pose: 0.799941 loss: 0.000572 2022/09/23 21:34:49 - mmengine - INFO - Epoch(train) [172][150/293] lr: 5.000000e-05 eta: 1:36:15 time: 0.600912 data_time: 0.091658 memory: 14267 loss_kpt: 0.000558 acc_pose: 0.831818 loss: 0.000558 2022/09/23 21:35:17 - mmengine - INFO - Epoch(train) [172][200/293] lr: 5.000000e-05 eta: 1:35:50 time: 0.573311 data_time: 0.084178 memory: 14267 loss_kpt: 0.000565 acc_pose: 0.863010 loss: 0.000565 2022/09/23 21:35:47 - mmengine - INFO - Epoch(train) [172][250/293] lr: 5.000000e-05 eta: 1:35:26 time: 0.598725 data_time: 0.109460 memory: 14267 loss_kpt: 0.000565 acc_pose: 0.859180 loss: 0.000565 2022/09/23 21:36:12 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:36:43 - mmengine - INFO - Epoch(train) [173][50/293] lr: 5.000000e-05 eta: 1:34:34 time: 0.625118 data_time: 0.118837 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.845662 loss: 0.000548 2022/09/23 21:37:14 - mmengine - INFO - Epoch(train) [173][100/293] lr: 5.000000e-05 eta: 1:34:10 time: 0.601537 data_time: 0.091102 memory: 14267 loss_kpt: 0.000565 acc_pose: 0.844339 loss: 0.000565 2022/09/23 21:37:44 - mmengine - INFO - Epoch(train) [173][150/293] lr: 5.000000e-05 eta: 1:33:45 time: 0.609074 data_time: 0.088358 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.874062 loss: 0.000549 2022/09/23 21:38:14 - mmengine - INFO - Epoch(train) [173][200/293] lr: 5.000000e-05 eta: 1:33:20 time: 0.593701 data_time: 0.095670 memory: 14267 loss_kpt: 0.000568 acc_pose: 0.825414 loss: 0.000568 2022/09/23 21:38:43 - mmengine - INFO - Epoch(train) [173][250/293] lr: 5.000000e-05 eta: 1:32:56 time: 0.594275 data_time: 0.096465 memory: 14267 loss_kpt: 0.000562 acc_pose: 0.869009 loss: 0.000562 2022/09/23 21:39:08 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:39:39 - mmengine - INFO - Epoch(train) [174][50/293] lr: 5.000000e-05 eta: 1:32:05 time: 0.616060 data_time: 0.098266 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.815716 loss: 0.000545 2022/09/23 21:40:09 - mmengine - INFO - Epoch(train) [174][100/293] lr: 5.000000e-05 eta: 1:31:40 time: 0.614579 data_time: 0.095078 memory: 14267 loss_kpt: 0.000575 acc_pose: 0.868779 loss: 0.000575 2022/09/23 21:40:39 - mmengine - INFO - Epoch(train) [174][150/293] lr: 5.000000e-05 eta: 1:31:15 time: 0.596742 data_time: 0.097783 memory: 14267 loss_kpt: 0.000556 acc_pose: 0.839369 loss: 0.000556 2022/09/23 21:41:10 - mmengine - INFO - Epoch(train) [174][200/293] lr: 5.000000e-05 eta: 1:30:51 time: 0.610202 data_time: 0.087233 memory: 14267 loss_kpt: 0.000558 acc_pose: 0.828945 loss: 0.000558 2022/09/23 21:41:40 - mmengine - INFO - Epoch(train) [174][250/293] lr: 5.000000e-05 eta: 1:30:26 time: 0.600813 data_time: 0.086844 memory: 14267 loss_kpt: 0.000556 acc_pose: 0.856295 loss: 0.000556 2022/09/23 21:42:06 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:42:18 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:42:38 - mmengine - INFO - Epoch(train) [175][50/293] lr: 5.000000e-05 eta: 1:29:35 time: 0.642000 data_time: 0.118718 memory: 14267 loss_kpt: 0.000558 acc_pose: 0.837388 loss: 0.000558 2022/09/23 21:43:07 - mmengine - INFO - Epoch(train) [175][100/293] lr: 5.000000e-05 eta: 1:29:10 time: 0.581807 data_time: 0.091126 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.865152 loss: 0.000548 2022/09/23 21:43:37 - mmengine - INFO - Epoch(train) [175][150/293] lr: 5.000000e-05 eta: 1:28:46 time: 0.602806 data_time: 0.100910 memory: 14267 loss_kpt: 0.000566 acc_pose: 0.863014 loss: 0.000566 2022/09/23 21:44:08 - mmengine - INFO - Epoch(train) [175][200/293] lr: 5.000000e-05 eta: 1:28:21 time: 0.612294 data_time: 0.113391 memory: 14267 loss_kpt: 0.000555 acc_pose: 0.788225 loss: 0.000555 2022/09/23 21:44:38 - mmengine - INFO - Epoch(train) [175][250/293] lr: 5.000000e-05 eta: 1:27:56 time: 0.606149 data_time: 0.095462 memory: 14267 loss_kpt: 0.000563 acc_pose: 0.803192 loss: 0.000563 2022/09/23 21:45:03 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:45:34 - mmengine - INFO - Epoch(train) [176][50/293] lr: 5.000000e-05 eta: 1:27:05 time: 0.628835 data_time: 0.106994 memory: 14267 loss_kpt: 0.000560 acc_pose: 0.854848 loss: 0.000560 2022/09/23 21:46:05 - mmengine - INFO - Epoch(train) [176][100/293] lr: 5.000000e-05 eta: 1:26:41 time: 0.606712 data_time: 0.101187 memory: 14267 loss_kpt: 0.000559 acc_pose: 0.876235 loss: 0.000559 2022/09/23 21:46:33 - mmengine - INFO - Epoch(train) [176][150/293] lr: 5.000000e-05 eta: 1:26:16 time: 0.577116 data_time: 0.073547 memory: 14267 loss_kpt: 0.000551 acc_pose: 0.827293 loss: 0.000551 2022/09/23 21:47:05 - mmengine - INFO - Epoch(train) [176][200/293] lr: 5.000000e-05 eta: 1:25:51 time: 0.624195 data_time: 0.090012 memory: 14267 loss_kpt: 0.000538 acc_pose: 0.904824 loss: 0.000538 2022/09/23 21:47:35 - mmengine - INFO - Epoch(train) [176][250/293] lr: 5.000000e-05 eta: 1:25:27 time: 0.610516 data_time: 0.112864 memory: 14267 loss_kpt: 0.000542 acc_pose: 0.860248 loss: 0.000542 2022/09/23 21:48:01 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:48:33 - mmengine - INFO - Epoch(train) [177][50/293] lr: 5.000000e-05 eta: 1:24:36 time: 0.626783 data_time: 0.101598 memory: 14267 loss_kpt: 0.000558 acc_pose: 0.845305 loss: 0.000558 2022/09/23 21:49:03 - mmengine - INFO - Epoch(train) [177][100/293] lr: 5.000000e-05 eta: 1:24:11 time: 0.607361 data_time: 0.093684 memory: 14267 loss_kpt: 0.000570 acc_pose: 0.849732 loss: 0.000570 2022/09/23 21:49:33 - mmengine - INFO - Epoch(train) [177][150/293] lr: 5.000000e-05 eta: 1:23:46 time: 0.595071 data_time: 0.077579 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.850695 loss: 0.000549 2022/09/23 21:50:03 - mmengine - INFO - Epoch(train) [177][200/293] lr: 5.000000e-05 eta: 1:23:22 time: 0.612477 data_time: 0.105204 memory: 14267 loss_kpt: 0.000562 acc_pose: 0.832839 loss: 0.000562 2022/09/23 21:50:33 - mmengine - INFO - Epoch(train) [177][250/293] lr: 5.000000e-05 eta: 1:22:57 time: 0.597950 data_time: 0.094973 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.823416 loss: 0.000550 2022/09/23 21:50:59 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:51:31 - mmengine - INFO - Epoch(train) [178][50/293] lr: 5.000000e-05 eta: 1:22:06 time: 0.626656 data_time: 0.108212 memory: 14267 loss_kpt: 0.000556 acc_pose: 0.893361 loss: 0.000556 2022/09/23 21:52:01 - mmengine - INFO - Epoch(train) [178][100/293] lr: 5.000000e-05 eta: 1:21:41 time: 0.603817 data_time: 0.086286 memory: 14267 loss_kpt: 0.000542 acc_pose: 0.828976 loss: 0.000542 2022/09/23 21:52:25 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:52:30 - mmengine - INFO - Epoch(train) [178][150/293] lr: 5.000000e-05 eta: 1:21:16 time: 0.591326 data_time: 0.089376 memory: 14267 loss_kpt: 0.000551 acc_pose: 0.882326 loss: 0.000551 2022/09/23 21:53:00 - mmengine - INFO - Epoch(train) [178][200/293] lr: 5.000000e-05 eta: 1:20:52 time: 0.599323 data_time: 0.083944 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.837940 loss: 0.000541 2022/09/23 21:53:30 - mmengine - INFO - Epoch(train) [178][250/293] lr: 5.000000e-05 eta: 1:20:27 time: 0.590505 data_time: 0.087703 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.887199 loss: 0.000543 2022/09/23 21:53:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:54:26 - mmengine - INFO - Epoch(train) [179][50/293] lr: 5.000000e-05 eta: 1:19:36 time: 0.627759 data_time: 0.102089 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.881009 loss: 0.000537 2022/09/23 21:54:55 - mmengine - INFO - Epoch(train) [179][100/293] lr: 5.000000e-05 eta: 1:19:11 time: 0.577968 data_time: 0.075046 memory: 14267 loss_kpt: 0.000559 acc_pose: 0.824230 loss: 0.000559 2022/09/23 21:55:25 - mmengine - INFO - Epoch(train) [179][150/293] lr: 5.000000e-05 eta: 1:18:46 time: 0.604828 data_time: 0.090086 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.902676 loss: 0.000541 2022/09/23 21:55:55 - mmengine - INFO - Epoch(train) [179][200/293] lr: 5.000000e-05 eta: 1:18:21 time: 0.583351 data_time: 0.086337 memory: 14267 loss_kpt: 0.000561 acc_pose: 0.839456 loss: 0.000561 2022/09/23 21:56:24 - mmengine - INFO - Epoch(train) [179][250/293] lr: 5.000000e-05 eta: 1:17:56 time: 0.596692 data_time: 0.072685 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.849820 loss: 0.000544 2022/09/23 21:56:49 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:57:21 - mmengine - INFO - Epoch(train) [180][50/293] lr: 5.000000e-05 eta: 1:17:06 time: 0.630128 data_time: 0.122327 memory: 14267 loss_kpt: 0.000551 acc_pose: 0.854893 loss: 0.000551 2022/09/23 21:57:51 - mmengine - INFO - Epoch(train) [180][100/293] lr: 5.000000e-05 eta: 1:16:41 time: 0.606708 data_time: 0.076816 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.841960 loss: 0.000546 2022/09/23 21:58:22 - mmengine - INFO - Epoch(train) [180][150/293] lr: 5.000000e-05 eta: 1:16:16 time: 0.616881 data_time: 0.101342 memory: 14267 loss_kpt: 0.000556 acc_pose: 0.830272 loss: 0.000556 2022/09/23 21:58:52 - mmengine - INFO - Epoch(train) [180][200/293] lr: 5.000000e-05 eta: 1:15:52 time: 0.595826 data_time: 0.115332 memory: 14267 loss_kpt: 0.000551 acc_pose: 0.812666 loss: 0.000551 2022/09/23 21:59:22 - mmengine - INFO - Epoch(train) [180][250/293] lr: 5.000000e-05 eta: 1:15:27 time: 0.600833 data_time: 0.089993 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.793934 loss: 0.000547 2022/09/23 21:59:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 21:59:47 - mmengine - INFO - Saving checkpoint at 180 epochs 2022/09/23 22:00:11 - mmengine - INFO - Epoch(val) [180][50/407] eta: 0:02:11 time: 0.367572 data_time: 0.083242 memory: 14267 2022/09/23 22:00:28 - mmengine - INFO - Epoch(val) [180][100/407] eta: 0:01:46 time: 0.345449 data_time: 0.063366 memory: 1464 2022/09/23 22:00:45 - mmengine - INFO - Epoch(val) [180][150/407] eta: 0:01:25 time: 0.333832 data_time: 0.068776 memory: 1464 2022/09/23 22:01:02 - mmengine - INFO - Epoch(val) [180][200/407] eta: 0:01:12 time: 0.349085 data_time: 0.049313 memory: 1464 2022/09/23 22:01:20 - mmengine - INFO - Epoch(val) [180][250/407] eta: 0:00:54 time: 0.350251 data_time: 0.050603 memory: 1464 2022/09/23 22:01:37 - mmengine - INFO - Epoch(val) [180][300/407] eta: 0:00:36 time: 0.345680 data_time: 0.068107 memory: 1464 2022/09/23 22:01:54 - mmengine - INFO - Epoch(val) [180][350/407] eta: 0:00:19 time: 0.343921 data_time: 0.039257 memory: 1464 2022/09/23 22:02:08 - mmengine - INFO - Epoch(val) [180][400/407] eta: 0:00:01 time: 0.270759 data_time: 0.030157 memory: 1464 2022/09/23 22:02:41 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 22:02:54 - mmengine - INFO - Epoch(val) [180][407/407] coco/AP: 0.731941 coco/AP .5: 0.899930 coco/AP .75: 0.799154 coco/AP (M): 0.687287 coco/AP (L): 0.806148 coco/AR: 0.782950 coco/AR .5: 0.937028 coco/AR .75: 0.843356 coco/AR (M): 0.734745 coco/AR (L): 0.852211 2022/09/23 22:02:54 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_160.pth is removed 2022/09/23 22:02:56 - mmengine - INFO - The best checkpoint with 0.7319 coco/AP at 180 epoch is saved to best_coco/AP_epoch_180.pth. 2022/09/23 22:03:27 - mmengine - INFO - Epoch(train) [181][50/293] lr: 5.000000e-05 eta: 1:14:36 time: 0.602659 data_time: 0.098249 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.878686 loss: 0.000543 2022/09/23 22:03:56 - mmengine - INFO - Epoch(train) [181][100/293] lr: 5.000000e-05 eta: 1:14:11 time: 0.586523 data_time: 0.086435 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.849637 loss: 0.000544 2022/09/23 22:04:25 - mmengine - INFO - Epoch(train) [181][150/293] lr: 5.000000e-05 eta: 1:13:46 time: 0.585740 data_time: 0.083043 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.841767 loss: 0.000546 2022/09/23 22:04:55 - mmengine - INFO - Epoch(train) [181][200/293] lr: 5.000000e-05 eta: 1:13:21 time: 0.587407 data_time: 0.094470 memory: 14267 loss_kpt: 0.000554 acc_pose: 0.847585 loss: 0.000554 2022/09/23 22:05:25 - mmengine - INFO - Epoch(train) [181][250/293] lr: 5.000000e-05 eta: 1:12:56 time: 0.604697 data_time: 0.077885 memory: 14267 loss_kpt: 0.000552 acc_pose: 0.857363 loss: 0.000552 2022/09/23 22:05:31 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:05:50 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:06:20 - mmengine - INFO - Epoch(train) [182][50/293] lr: 5.000000e-05 eta: 1:12:06 time: 0.602665 data_time: 0.088433 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.915895 loss: 0.000549 2022/09/23 22:06:50 - mmengine - INFO - Epoch(train) [182][100/293] lr: 5.000000e-05 eta: 1:11:41 time: 0.609784 data_time: 0.096265 memory: 14267 loss_kpt: 0.000555 acc_pose: 0.847039 loss: 0.000555 2022/09/23 22:07:21 - mmengine - INFO - Epoch(train) [182][150/293] lr: 5.000000e-05 eta: 1:11:16 time: 0.613842 data_time: 0.086898 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.890965 loss: 0.000540 2022/09/23 22:07:50 - mmengine - INFO - Epoch(train) [182][200/293] lr: 5.000000e-05 eta: 1:10:51 time: 0.576878 data_time: 0.085845 memory: 14267 loss_kpt: 0.000564 acc_pose: 0.871480 loss: 0.000564 2022/09/23 22:08:20 - mmengine - INFO - Epoch(train) [182][250/293] lr: 5.000000e-05 eta: 1:10:26 time: 0.592545 data_time: 0.084825 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.845391 loss: 0.000547 2022/09/23 22:08:44 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:09:16 - mmengine - INFO - Epoch(train) [183][50/293] lr: 5.000000e-05 eta: 1:09:36 time: 0.637114 data_time: 0.113035 memory: 14267 loss_kpt: 0.000552 acc_pose: 0.852543 loss: 0.000552 2022/09/23 22:09:47 - mmengine - INFO - Epoch(train) [183][100/293] lr: 5.000000e-05 eta: 1:09:11 time: 0.610633 data_time: 0.106418 memory: 14267 loss_kpt: 0.000565 acc_pose: 0.814623 loss: 0.000565 2022/09/23 22:10:17 - mmengine - INFO - Epoch(train) [183][150/293] lr: 5.000000e-05 eta: 1:08:46 time: 0.596994 data_time: 0.088779 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.871143 loss: 0.000544 2022/09/23 22:10:46 - mmengine - INFO - Epoch(train) [183][200/293] lr: 5.000000e-05 eta: 1:08:21 time: 0.584180 data_time: 0.095827 memory: 14267 loss_kpt: 0.000559 acc_pose: 0.855918 loss: 0.000559 2022/09/23 22:11:15 - mmengine - INFO - Epoch(train) [183][250/293] lr: 5.000000e-05 eta: 1:07:56 time: 0.583906 data_time: 0.083223 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.876282 loss: 0.000541 2022/09/23 22:11:40 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:12:11 - mmengine - INFO - Epoch(train) [184][50/293] lr: 5.000000e-05 eta: 1:07:06 time: 0.624476 data_time: 0.087672 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.823945 loss: 0.000548 2022/09/23 22:12:42 - mmengine - INFO - Epoch(train) [184][100/293] lr: 5.000000e-05 eta: 1:06:41 time: 0.600782 data_time: 0.096255 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.861808 loss: 0.000547 2022/09/23 22:13:10 - mmengine - INFO - Epoch(train) [184][150/293] lr: 5.000000e-05 eta: 1:06:16 time: 0.579227 data_time: 0.080945 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.878414 loss: 0.000543 2022/09/23 22:13:40 - mmengine - INFO - Epoch(train) [184][200/293] lr: 5.000000e-05 eta: 1:05:51 time: 0.596809 data_time: 0.099706 memory: 14267 loss_kpt: 0.000552 acc_pose: 0.816627 loss: 0.000552 2022/09/23 22:14:10 - mmengine - INFO - Epoch(train) [184][250/293] lr: 5.000000e-05 eta: 1:05:26 time: 0.584451 data_time: 0.101513 memory: 14267 loss_kpt: 0.000561 acc_pose: 0.835465 loss: 0.000561 2022/09/23 22:14:35 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:15:06 - mmengine - INFO - Epoch(train) [185][50/293] lr: 5.000000e-05 eta: 1:04:36 time: 0.615005 data_time: 0.097951 memory: 14267 loss_kpt: 0.000551 acc_pose: 0.845228 loss: 0.000551 2022/09/23 22:15:29 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:15:37 - mmengine - INFO - Epoch(train) [185][100/293] lr: 5.000000e-05 eta: 1:04:11 time: 0.618046 data_time: 0.084031 memory: 14267 loss_kpt: 0.000555 acc_pose: 0.792703 loss: 0.000555 2022/09/23 22:16:06 - mmengine - INFO - Epoch(train) [185][150/293] lr: 5.000000e-05 eta: 1:03:46 time: 0.586196 data_time: 0.079747 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.872099 loss: 0.000547 2022/09/23 22:16:36 - mmengine - INFO - Epoch(train) [185][200/293] lr: 5.000000e-05 eta: 1:03:21 time: 0.597340 data_time: 0.095385 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.850913 loss: 0.000546 2022/09/23 22:17:07 - mmengine - INFO - Epoch(train) [185][250/293] lr: 5.000000e-05 eta: 1:02:56 time: 0.604765 data_time: 0.100594 memory: 14267 loss_kpt: 0.000553 acc_pose: 0.867094 loss: 0.000553 2022/09/23 22:17:32 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:18:05 - mmengine - INFO - Epoch(train) [186][50/293] lr: 5.000000e-05 eta: 1:02:06 time: 0.651185 data_time: 0.111909 memory: 14267 loss_kpt: 0.000553 acc_pose: 0.836916 loss: 0.000553 2022/09/23 22:18:34 - mmengine - INFO - Epoch(train) [186][100/293] lr: 5.000000e-05 eta: 1:01:41 time: 0.583711 data_time: 0.073020 memory: 14267 loss_kpt: 0.000554 acc_pose: 0.883625 loss: 0.000554 2022/09/23 22:19:04 - mmengine - INFO - Epoch(train) [186][150/293] lr: 5.000000e-05 eta: 1:01:16 time: 0.592077 data_time: 0.076399 memory: 14267 loss_kpt: 0.000558 acc_pose: 0.804033 loss: 0.000558 2022/09/23 22:19:34 - mmengine - INFO - Epoch(train) [186][200/293] lr: 5.000000e-05 eta: 1:00:51 time: 0.611731 data_time: 0.086712 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.852805 loss: 0.000548 2022/09/23 22:20:04 - mmengine - INFO - Epoch(train) [186][250/293] lr: 5.000000e-05 eta: 1:00:26 time: 0.587358 data_time: 0.092580 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.885235 loss: 0.000550 2022/09/23 22:20:30 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:21:01 - mmengine - INFO - Epoch(train) [187][50/293] lr: 5.000000e-05 eta: 0:59:36 time: 0.632269 data_time: 0.117143 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.856083 loss: 0.000545 2022/09/23 22:21:30 - mmengine - INFO - Epoch(train) [187][100/293] lr: 5.000000e-05 eta: 0:59:11 time: 0.574517 data_time: 0.077581 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.886010 loss: 0.000550 2022/09/23 22:22:02 - mmengine - INFO - Epoch(train) [187][150/293] lr: 5.000000e-05 eta: 0:58:46 time: 0.629833 data_time: 0.093398 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.885427 loss: 0.000550 2022/09/23 22:22:32 - mmengine - INFO - Epoch(train) [187][200/293] lr: 5.000000e-05 eta: 0:58:21 time: 0.599622 data_time: 0.094610 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.898497 loss: 0.000543 2022/09/23 22:23:01 - mmengine - INFO - Epoch(train) [187][250/293] lr: 5.000000e-05 eta: 0:57:56 time: 0.588859 data_time: 0.087348 memory: 14267 loss_kpt: 0.000552 acc_pose: 0.827306 loss: 0.000552 2022/09/23 22:23:26 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:23:58 - mmengine - INFO - Epoch(train) [188][50/293] lr: 5.000000e-05 eta: 0:57:06 time: 0.638422 data_time: 0.107148 memory: 14267 loss_kpt: 0.000538 acc_pose: 0.819588 loss: 0.000538 2022/09/23 22:24:27 - mmengine - INFO - Epoch(train) [188][100/293] lr: 5.000000e-05 eta: 0:56:41 time: 0.584976 data_time: 0.085709 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.843793 loss: 0.000548 2022/09/23 22:24:57 - mmengine - INFO - Epoch(train) [188][150/293] lr: 5.000000e-05 eta: 0:56:16 time: 0.602633 data_time: 0.080842 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.864377 loss: 0.000544 2022/09/23 22:25:28 - mmengine - INFO - Epoch(train) [188][200/293] lr: 5.000000e-05 eta: 0:55:51 time: 0.600602 data_time: 0.085314 memory: 14267 loss_kpt: 0.000553 acc_pose: 0.876775 loss: 0.000553 2022/09/23 22:25:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:25:57 - mmengine - INFO - Epoch(train) [188][250/293] lr: 5.000000e-05 eta: 0:55:25 time: 0.593139 data_time: 0.092197 memory: 14267 loss_kpt: 0.000556 acc_pose: 0.788581 loss: 0.000556 2022/09/23 22:26:23 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:26:55 - mmengine - INFO - Epoch(train) [189][50/293] lr: 5.000000e-05 eta: 0:54:36 time: 0.643989 data_time: 0.101883 memory: 14267 loss_kpt: 0.000555 acc_pose: 0.871623 loss: 0.000555 2022/09/23 22:27:25 - mmengine - INFO - Epoch(train) [189][100/293] lr: 5.000000e-05 eta: 0:54:11 time: 0.588356 data_time: 0.083809 memory: 14267 loss_kpt: 0.000533 acc_pose: 0.844781 loss: 0.000533 2022/09/23 22:27:54 - mmengine - INFO - Epoch(train) [189][150/293] lr: 5.000000e-05 eta: 0:53:46 time: 0.591365 data_time: 0.089580 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.901312 loss: 0.000543 2022/09/23 22:28:23 - mmengine - INFO - Epoch(train) [189][200/293] lr: 5.000000e-05 eta: 0:53:20 time: 0.572796 data_time: 0.082555 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.823143 loss: 0.000550 2022/09/23 22:28:54 - mmengine - INFO - Epoch(train) [189][250/293] lr: 5.000000e-05 eta: 0:52:55 time: 0.612865 data_time: 0.089854 memory: 14267 loss_kpt: 0.000553 acc_pose: 0.891948 loss: 0.000553 2022/09/23 22:29:20 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:29:52 - mmengine - INFO - Epoch(train) [190][50/293] lr: 5.000000e-05 eta: 0:52:06 time: 0.629346 data_time: 0.117947 memory: 14267 loss_kpt: 0.000561 acc_pose: 0.863318 loss: 0.000561 2022/09/23 22:30:21 - mmengine - INFO - Epoch(train) [190][100/293] lr: 5.000000e-05 eta: 0:51:41 time: 0.592675 data_time: 0.092583 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.841849 loss: 0.000544 2022/09/23 22:30:51 - mmengine - INFO - Epoch(train) [190][150/293] lr: 5.000000e-05 eta: 0:51:16 time: 0.597862 data_time: 0.076999 memory: 14267 loss_kpt: 0.000553 acc_pose: 0.839539 loss: 0.000553 2022/09/23 22:31:22 - mmengine - INFO - Epoch(train) [190][200/293] lr: 5.000000e-05 eta: 0:50:50 time: 0.604408 data_time: 0.091040 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.848311 loss: 0.000537 2022/09/23 22:31:51 - mmengine - INFO - Epoch(train) [190][250/293] lr: 5.000000e-05 eta: 0:50:25 time: 0.586365 data_time: 0.084205 memory: 14267 loss_kpt: 0.000551 acc_pose: 0.853329 loss: 0.000551 2022/09/23 22:32:15 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:32:15 - mmengine - INFO - Saving checkpoint at 190 epochs 2022/09/23 22:32:39 - mmengine - INFO - Epoch(val) [190][50/407] eta: 0:02:09 time: 0.362326 data_time: 0.080189 memory: 14267 2022/09/23 22:32:56 - mmengine - INFO - Epoch(val) [190][100/407] eta: 0:01:48 time: 0.352526 data_time: 0.081596 memory: 1464 2022/09/23 22:33:14 - mmengine - INFO - Epoch(val) [190][150/407] eta: 0:01:29 time: 0.349654 data_time: 0.062516 memory: 1464 2022/09/23 22:33:30 - mmengine - INFO - Epoch(val) [190][200/407] eta: 0:01:08 time: 0.330600 data_time: 0.056523 memory: 1464 2022/09/23 22:33:48 - mmengine - INFO - Epoch(val) [190][250/407] eta: 0:00:54 time: 0.344307 data_time: 0.062584 memory: 1464 2022/09/23 22:34:04 - mmengine - INFO - Epoch(val) [190][300/407] eta: 0:00:35 time: 0.333243 data_time: 0.052028 memory: 1464 2022/09/23 22:34:21 - mmengine - INFO - Epoch(val) [190][350/407] eta: 0:00:19 time: 0.343923 data_time: 0.046816 memory: 1464 2022/09/23 22:34:35 - mmengine - INFO - Epoch(val) [190][400/407] eta: 0:00:01 time: 0.273155 data_time: 0.034774 memory: 1464 2022/09/23 22:35:09 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 22:35:22 - mmengine - INFO - Epoch(val) [190][407/407] coco/AP: 0.734511 coco/AP .5: 0.900631 coco/AP .75: 0.800069 coco/AP (M): 0.690170 coco/AP (L): 0.808426 coco/AR: 0.785548 coco/AR .5: 0.937500 coco/AR .75: 0.843356 coco/AR (M): 0.737449 coco/AR (L): 0.854701 2022/09/23 22:35:22 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_180.pth is removed 2022/09/23 22:35:24 - mmengine - INFO - The best checkpoint with 0.7345 coco/AP at 190 epoch is saved to best_coco/AP_epoch_190.pth. 2022/09/23 22:35:55 - mmengine - INFO - Epoch(train) [191][50/293] lr: 5.000000e-05 eta: 0:49:36 time: 0.610557 data_time: 0.099435 memory: 14267 loss_kpt: 0.000533 acc_pose: 0.856115 loss: 0.000533 2022/09/23 22:36:25 - mmengine - INFO - Epoch(train) [191][100/293] lr: 5.000000e-05 eta: 0:49:11 time: 0.598789 data_time: 0.090855 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.894625 loss: 0.000543 2022/09/23 22:36:56 - mmengine - INFO - Epoch(train) [191][150/293] lr: 5.000000e-05 eta: 0:48:46 time: 0.623975 data_time: 0.085986 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.865083 loss: 0.000550 2022/09/23 22:37:26 - mmengine - INFO - Epoch(train) [191][200/293] lr: 5.000000e-05 eta: 0:48:20 time: 0.607788 data_time: 0.094188 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.862183 loss: 0.000537 2022/09/23 22:37:56 - mmengine - INFO - Epoch(train) [191][250/293] lr: 5.000000e-05 eta: 0:47:55 time: 0.589044 data_time: 0.092084 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.888957 loss: 0.000548 2022/09/23 22:38:21 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:38:44 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:38:51 - mmengine - INFO - Epoch(train) [192][50/293] lr: 5.000000e-05 eta: 0:47:06 time: 0.615540 data_time: 0.105960 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.891675 loss: 0.000547 2022/09/23 22:39:22 - mmengine - INFO - Epoch(train) [192][100/293] lr: 5.000000e-05 eta: 0:46:41 time: 0.613735 data_time: 0.083438 memory: 14267 loss_kpt: 0.000558 acc_pose: 0.835796 loss: 0.000558 2022/09/23 22:39:52 - mmengine - INFO - Epoch(train) [192][150/293] lr: 5.000000e-05 eta: 0:46:15 time: 0.592693 data_time: 0.079317 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.873010 loss: 0.000540 2022/09/23 22:40:22 - mmengine - INFO - Epoch(train) [192][200/293] lr: 5.000000e-05 eta: 0:45:50 time: 0.608959 data_time: 0.098767 memory: 14267 loss_kpt: 0.000538 acc_pose: 0.833673 loss: 0.000538 2022/09/23 22:40:52 - mmengine - INFO - Epoch(train) [192][250/293] lr: 5.000000e-05 eta: 0:45:25 time: 0.587784 data_time: 0.091715 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.801554 loss: 0.000547 2022/09/23 22:41:18 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:41:49 - mmengine - INFO - Epoch(train) [193][50/293] lr: 5.000000e-05 eta: 0:44:36 time: 0.626931 data_time: 0.111427 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.826755 loss: 0.000541 2022/09/23 22:42:19 - mmengine - INFO - Epoch(train) [193][100/293] lr: 5.000000e-05 eta: 0:44:11 time: 0.597594 data_time: 0.085541 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.841193 loss: 0.000543 2022/09/23 22:42:48 - mmengine - INFO - Epoch(train) [193][150/293] lr: 5.000000e-05 eta: 0:43:45 time: 0.581757 data_time: 0.074745 memory: 14267 loss_kpt: 0.000532 acc_pose: 0.876993 loss: 0.000532 2022/09/23 22:43:18 - mmengine - INFO - Epoch(train) [193][200/293] lr: 5.000000e-05 eta: 0:43:20 time: 0.591223 data_time: 0.084339 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.893579 loss: 0.000549 2022/09/23 22:43:48 - mmengine - INFO - Epoch(train) [193][250/293] lr: 5.000000e-05 eta: 0:42:55 time: 0.603060 data_time: 0.077193 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.834064 loss: 0.000545 2022/09/23 22:44:13 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:44:44 - mmengine - INFO - Epoch(train) [194][50/293] lr: 5.000000e-05 eta: 0:42:06 time: 0.627897 data_time: 0.101046 memory: 14267 loss_kpt: 0.000539 acc_pose: 0.826014 loss: 0.000539 2022/09/23 22:45:15 - mmengine - INFO - Epoch(train) [194][100/293] lr: 5.000000e-05 eta: 0:41:41 time: 0.609553 data_time: 0.085161 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.830960 loss: 0.000545 2022/09/23 22:45:45 - mmengine - INFO - Epoch(train) [194][150/293] lr: 5.000000e-05 eta: 0:41:15 time: 0.607798 data_time: 0.093616 memory: 14267 loss_kpt: 0.000530 acc_pose: 0.873456 loss: 0.000530 2022/09/23 22:46:15 - mmengine - INFO - Epoch(train) [194][200/293] lr: 5.000000e-05 eta: 0:40:50 time: 0.602804 data_time: 0.085784 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.824468 loss: 0.000550 2022/09/23 22:46:46 - mmengine - INFO - Epoch(train) [194][250/293] lr: 5.000000e-05 eta: 0:40:25 time: 0.604012 data_time: 0.090221 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.854578 loss: 0.000550 2022/09/23 22:47:11 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:47:42 - mmengine - INFO - Epoch(train) [195][50/293] lr: 5.000000e-05 eta: 0:39:36 time: 0.621447 data_time: 0.116762 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.872583 loss: 0.000549 2022/09/23 22:48:12 - mmengine - INFO - Epoch(train) [195][100/293] lr: 5.000000e-05 eta: 0:39:11 time: 0.597218 data_time: 0.089796 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.863825 loss: 0.000537 2022/09/23 22:48:41 - mmengine - INFO - Epoch(train) [195][150/293] lr: 5.000000e-05 eta: 0:38:45 time: 0.569738 data_time: 0.064750 memory: 14267 loss_kpt: 0.000536 acc_pose: 0.853819 loss: 0.000536 2022/09/23 22:48:45 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:49:11 - mmengine - INFO - Epoch(train) [195][200/293] lr: 5.000000e-05 eta: 0:38:20 time: 0.599809 data_time: 0.083405 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.836025 loss: 0.000550 2022/09/23 22:49:41 - mmengine - INFO - Epoch(train) [195][250/293] lr: 5.000000e-05 eta: 0:37:55 time: 0.599460 data_time: 0.101740 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.876664 loss: 0.000544 2022/09/23 22:50:07 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:50:37 - mmengine - INFO - Epoch(train) [196][50/293] lr: 5.000000e-05 eta: 0:37:06 time: 0.615009 data_time: 0.120970 memory: 14267 loss_kpt: 0.000538 acc_pose: 0.915130 loss: 0.000538 2022/09/23 22:51:08 - mmengine - INFO - Epoch(train) [196][100/293] lr: 5.000000e-05 eta: 0:36:40 time: 0.613140 data_time: 0.077229 memory: 14267 loss_kpt: 0.000550 acc_pose: 0.858624 loss: 0.000550 2022/09/23 22:51:37 - mmengine - INFO - Epoch(train) [196][150/293] lr: 5.000000e-05 eta: 0:36:15 time: 0.584643 data_time: 0.080119 memory: 14267 loss_kpt: 0.000533 acc_pose: 0.837204 loss: 0.000533 2022/09/23 22:52:07 - mmengine - INFO - Epoch(train) [196][200/293] lr: 5.000000e-05 eta: 0:35:50 time: 0.596421 data_time: 0.090676 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.804769 loss: 0.000543 2022/09/23 22:52:37 - mmengine - INFO - Epoch(train) [196][250/293] lr: 5.000000e-05 eta: 0:35:24 time: 0.604791 data_time: 0.091316 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.857410 loss: 0.000546 2022/09/23 22:53:03 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:53:35 - mmengine - INFO - Epoch(train) [197][50/293] lr: 5.000000e-05 eta: 0:34:36 time: 0.625654 data_time: 0.104454 memory: 14267 loss_kpt: 0.000533 acc_pose: 0.828585 loss: 0.000533 2022/09/23 22:54:04 - mmengine - INFO - Epoch(train) [197][100/293] lr: 5.000000e-05 eta: 0:34:10 time: 0.581208 data_time: 0.079635 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.879984 loss: 0.000549 2022/09/23 22:54:34 - mmengine - INFO - Epoch(train) [197][150/293] lr: 5.000000e-05 eta: 0:33:45 time: 0.597875 data_time: 0.079356 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.858363 loss: 0.000537 2022/09/23 22:55:03 - mmengine - INFO - Epoch(train) [197][200/293] lr: 5.000000e-05 eta: 0:33:19 time: 0.590069 data_time: 0.092559 memory: 14267 loss_kpt: 0.000553 acc_pose: 0.862162 loss: 0.000553 2022/09/23 22:55:34 - mmengine - INFO - Epoch(train) [197][250/293] lr: 5.000000e-05 eta: 0:32:54 time: 0.607136 data_time: 0.098626 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.839272 loss: 0.000546 2022/09/23 22:55:59 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:56:31 - mmengine - INFO - Epoch(train) [198][50/293] lr: 5.000000e-05 eta: 0:32:05 time: 0.641953 data_time: 0.102715 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.835162 loss: 0.000544 2022/09/23 22:57:00 - mmengine - INFO - Epoch(train) [198][100/293] lr: 5.000000e-05 eta: 0:31:40 time: 0.581925 data_time: 0.086016 memory: 14267 loss_kpt: 0.000561 acc_pose: 0.865411 loss: 0.000561 2022/09/23 22:57:29 - mmengine - INFO - Epoch(train) [198][150/293] lr: 5.000000e-05 eta: 0:31:15 time: 0.579290 data_time: 0.096366 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.852514 loss: 0.000540 2022/09/23 22:58:00 - mmengine - INFO - Epoch(train) [198][200/293] lr: 5.000000e-05 eta: 0:30:49 time: 0.611431 data_time: 0.088260 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.862765 loss: 0.000537 2022/09/23 22:58:29 - mmengine - INFO - Epoch(train) [198][250/293] lr: 5.000000e-05 eta: 0:30:24 time: 0.583498 data_time: 0.091537 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.840090 loss: 0.000546 2022/09/23 22:58:47 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:58:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 22:59:27 - mmengine - INFO - Epoch(train) [199][50/293] lr: 5.000000e-05 eta: 0:29:35 time: 0.642209 data_time: 0.106041 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.878448 loss: 0.000534 2022/09/23 22:59:58 - mmengine - INFO - Epoch(train) [199][100/293] lr: 5.000000e-05 eta: 0:29:10 time: 0.609389 data_time: 0.081208 memory: 14267 loss_kpt: 0.000539 acc_pose: 0.845575 loss: 0.000539 2022/09/23 23:00:28 - mmengine - INFO - Epoch(train) [199][150/293] lr: 5.000000e-05 eta: 0:28:45 time: 0.603874 data_time: 0.082554 memory: 14267 loss_kpt: 0.000536 acc_pose: 0.828831 loss: 0.000536 2022/09/23 23:00:58 - mmengine - INFO - Epoch(train) [199][200/293] lr: 5.000000e-05 eta: 0:28:19 time: 0.600414 data_time: 0.096546 memory: 14267 loss_kpt: 0.000535 acc_pose: 0.837516 loss: 0.000535 2022/09/23 23:01:28 - mmengine - INFO - Epoch(train) [199][250/293] lr: 5.000000e-05 eta: 0:27:54 time: 0.603821 data_time: 0.095126 memory: 14267 loss_kpt: 0.000549 acc_pose: 0.847666 loss: 0.000549 2022/09/23 23:01:53 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:02:25 - mmengine - INFO - Epoch(train) [200][50/293] lr: 5.000000e-05 eta: 0:27:05 time: 0.628265 data_time: 0.105883 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.873520 loss: 0.000534 2022/09/23 23:02:54 - mmengine - INFO - Epoch(train) [200][100/293] lr: 5.000000e-05 eta: 0:26:40 time: 0.586511 data_time: 0.083143 memory: 14267 loss_kpt: 0.000529 acc_pose: 0.846164 loss: 0.000529 2022/09/23 23:03:25 - mmengine - INFO - Epoch(train) [200][150/293] lr: 5.000000e-05 eta: 0:26:15 time: 0.613763 data_time: 0.095587 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.816391 loss: 0.000537 2022/09/23 23:03:54 - mmengine - INFO - Epoch(train) [200][200/293] lr: 5.000000e-05 eta: 0:25:49 time: 0.587147 data_time: 0.090713 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.837960 loss: 0.000541 2022/09/23 23:04:23 - mmengine - INFO - Epoch(train) [200][250/293] lr: 5.000000e-05 eta: 0:25:24 time: 0.585576 data_time: 0.085893 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.832668 loss: 0.000545 2022/09/23 23:04:49 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:04:49 - mmengine - INFO - Saving checkpoint at 200 epochs 2022/09/23 23:05:13 - mmengine - INFO - Epoch(val) [200][50/407] eta: 0:02:07 time: 0.358445 data_time: 0.070120 memory: 14267 2022/09/23 23:05:30 - mmengine - INFO - Epoch(val) [200][100/407] eta: 0:01:46 time: 0.345492 data_time: 0.059445 memory: 1464 2022/09/23 23:05:47 - mmengine - INFO - Epoch(val) [200][150/407] eta: 0:01:24 time: 0.329501 data_time: 0.043056 memory: 1464 2022/09/23 23:06:04 - mmengine - INFO - Epoch(val) [200][200/407] eta: 0:01:08 time: 0.330848 data_time: 0.037534 memory: 1464 2022/09/23 23:06:20 - mmengine - INFO - Epoch(val) [200][250/407] eta: 0:00:51 time: 0.329058 data_time: 0.040657 memory: 1464 2022/09/23 23:06:38 - mmengine - INFO - Epoch(val) [200][300/407] eta: 0:00:38 time: 0.356491 data_time: 0.066915 memory: 1464 2022/09/23 23:06:54 - mmengine - INFO - Epoch(val) [200][350/407] eta: 0:00:18 time: 0.333317 data_time: 0.052057 memory: 1464 2022/09/23 23:07:09 - mmengine - INFO - Epoch(val) [200][400/407] eta: 0:00:02 time: 0.289500 data_time: 0.057405 memory: 1464 2022/09/23 23:07:42 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 23:07:56 - mmengine - INFO - Epoch(val) [200][407/407] coco/AP: 0.735123 coco/AP .5: 0.901683 coco/AP .75: 0.801081 coco/AP (M): 0.689967 coco/AP (L): 0.808741 coco/AR: 0.785973 coco/AR .5: 0.937972 coco/AR .75: 0.844616 coco/AR (M): 0.738405 coco/AR (L): 0.853995 2022/09/23 23:07:56 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220923/res50_dark_384/best_coco/AP_epoch_190.pth is removed 2022/09/23 23:07:58 - mmengine - INFO - The best checkpoint with 0.7351 coco/AP at 200 epoch is saved to best_coco/AP_epoch_200.pth. 2022/09/23 23:08:29 - mmengine - INFO - Epoch(train) [201][50/293] lr: 5.000000e-06 eta: 0:24:35 time: 0.617842 data_time: 0.099147 memory: 14267 loss_kpt: 0.000533 acc_pose: 0.866211 loss: 0.000533 2022/09/23 23:09:00 - mmengine - INFO - Epoch(train) [201][100/293] lr: 5.000000e-06 eta: 0:24:10 time: 0.615023 data_time: 0.088823 memory: 14267 loss_kpt: 0.000542 acc_pose: 0.842715 loss: 0.000542 2022/09/23 23:09:29 - mmengine - INFO - Epoch(train) [201][150/293] lr: 5.000000e-06 eta: 0:23:44 time: 0.594523 data_time: 0.080498 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.866038 loss: 0.000541 2022/09/23 23:10:00 - mmengine - INFO - Epoch(train) [201][200/293] lr: 5.000000e-06 eta: 0:23:19 time: 0.605803 data_time: 0.091534 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.838274 loss: 0.000543 2022/09/23 23:10:28 - mmengine - INFO - Epoch(train) [201][250/293] lr: 5.000000e-06 eta: 0:22:53 time: 0.573934 data_time: 0.081117 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.839787 loss: 0.000544 2022/09/23 23:10:54 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:11:25 - mmengine - INFO - Epoch(train) [202][50/293] lr: 5.000000e-06 eta: 0:22:05 time: 0.620636 data_time: 0.125118 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.890272 loss: 0.000548 2022/09/23 23:11:56 - mmengine - INFO - Epoch(train) [202][100/293] lr: 5.000000e-06 eta: 0:21:40 time: 0.606099 data_time: 0.090695 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.838176 loss: 0.000543 2022/09/23 23:12:00 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:12:26 - mmengine - INFO - Epoch(train) [202][150/293] lr: 5.000000e-06 eta: 0:21:14 time: 0.602280 data_time: 0.082559 memory: 14267 loss_kpt: 0.000548 acc_pose: 0.817146 loss: 0.000548 2022/09/23 23:12:55 - mmengine - INFO - Epoch(train) [202][200/293] lr: 5.000000e-06 eta: 0:20:49 time: 0.595459 data_time: 0.100177 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.819377 loss: 0.000545 2022/09/23 23:13:25 - mmengine - INFO - Epoch(train) [202][250/293] lr: 5.000000e-06 eta: 0:20:23 time: 0.593009 data_time: 0.085521 memory: 14267 loss_kpt: 0.000542 acc_pose: 0.823719 loss: 0.000542 2022/09/23 23:13:51 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:14:21 - mmengine - INFO - Epoch(train) [203][50/293] lr: 5.000000e-06 eta: 0:19:35 time: 0.603534 data_time: 0.110831 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.874136 loss: 0.000540 2022/09/23 23:14:52 - mmengine - INFO - Epoch(train) [203][100/293] lr: 5.000000e-06 eta: 0:19:10 time: 0.614503 data_time: 0.097313 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.867037 loss: 0.000537 2022/09/23 23:15:21 - mmengine - INFO - Epoch(train) [203][150/293] lr: 5.000000e-06 eta: 0:18:44 time: 0.590461 data_time: 0.097363 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.830726 loss: 0.000540 2022/09/23 23:15:51 - mmengine - INFO - Epoch(train) [203][200/293] lr: 5.000000e-06 eta: 0:18:19 time: 0.582456 data_time: 0.080259 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.852726 loss: 0.000546 2022/09/23 23:16:20 - mmengine - INFO - Epoch(train) [203][250/293] lr: 5.000000e-06 eta: 0:17:53 time: 0.591781 data_time: 0.090955 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.825262 loss: 0.000544 2022/09/23 23:16:46 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:17:17 - mmengine - INFO - Epoch(train) [204][50/293] lr: 5.000000e-06 eta: 0:17:05 time: 0.633081 data_time: 0.102758 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.881732 loss: 0.000543 2022/09/23 23:17:47 - mmengine - INFO - Epoch(train) [204][100/293] lr: 5.000000e-06 eta: 0:16:39 time: 0.595743 data_time: 0.122040 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.817384 loss: 0.000537 2022/09/23 23:18:17 - mmengine - INFO - Epoch(train) [204][150/293] lr: 5.000000e-06 eta: 0:16:14 time: 0.600074 data_time: 0.085103 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.853781 loss: 0.000534 2022/09/23 23:18:47 - mmengine - INFO - Epoch(train) [204][200/293] lr: 5.000000e-06 eta: 0:15:48 time: 0.589006 data_time: 0.090763 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.839458 loss: 0.000547 2022/09/23 23:19:16 - mmengine - INFO - Epoch(train) [204][250/293] lr: 5.000000e-06 eta: 0:15:23 time: 0.595105 data_time: 0.084070 memory: 14267 loss_kpt: 0.000552 acc_pose: 0.885903 loss: 0.000552 2022/09/23 23:19:42 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:20:14 - mmengine - INFO - Epoch(train) [205][50/293] lr: 5.000000e-06 eta: 0:14:35 time: 0.622530 data_time: 0.104413 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.859811 loss: 0.000543 2022/09/23 23:20:43 - mmengine - INFO - Epoch(train) [205][100/293] lr: 5.000000e-06 eta: 0:14:09 time: 0.598333 data_time: 0.091830 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.857430 loss: 0.000540 2022/09/23 23:21:13 - mmengine - INFO - Epoch(train) [205][150/293] lr: 5.000000e-06 eta: 0:13:44 time: 0.594601 data_time: 0.076554 memory: 14267 loss_kpt: 0.000540 acc_pose: 0.858557 loss: 0.000540 2022/09/23 23:21:44 - mmengine - INFO - Epoch(train) [205][200/293] lr: 5.000000e-06 eta: 0:13:18 time: 0.614633 data_time: 0.103131 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.832731 loss: 0.000534 2022/09/23 23:22:00 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:22:13 - mmengine - INFO - Epoch(train) [205][250/293] lr: 5.000000e-06 eta: 0:12:53 time: 0.584570 data_time: 0.079545 memory: 14267 loss_kpt: 0.000547 acc_pose: 0.829231 loss: 0.000547 2022/09/23 23:22:38 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:23:08 - mmengine - INFO - Epoch(train) [206][50/293] lr: 5.000000e-06 eta: 0:12:05 time: 0.609519 data_time: 0.100733 memory: 14267 loss_kpt: 0.000528 acc_pose: 0.846337 loss: 0.000528 2022/09/23 23:23:39 - mmengine - INFO - Epoch(train) [206][100/293] lr: 5.000000e-06 eta: 0:11:39 time: 0.603504 data_time: 0.077146 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.842780 loss: 0.000544 2022/09/23 23:24:08 - mmengine - INFO - Epoch(train) [206][150/293] lr: 5.000000e-06 eta: 0:11:13 time: 0.597078 data_time: 0.092049 memory: 14267 loss_kpt: 0.000544 acc_pose: 0.884678 loss: 0.000544 2022/09/23 23:24:38 - mmengine - INFO - Epoch(train) [206][200/293] lr: 5.000000e-06 eta: 0:10:48 time: 0.590893 data_time: 0.075210 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.875760 loss: 0.000537 2022/09/23 23:25:08 - mmengine - INFO - Epoch(train) [206][250/293] lr: 5.000000e-06 eta: 0:10:22 time: 0.594430 data_time: 0.082681 memory: 14267 loss_kpt: 0.000555 acc_pose: 0.801419 loss: 0.000555 2022/09/23 23:25:33 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:26:05 - mmengine - INFO - Epoch(train) [207][50/293] lr: 5.000000e-06 eta: 0:09:34 time: 0.640513 data_time: 0.117791 memory: 14267 loss_kpt: 0.000531 acc_pose: 0.812719 loss: 0.000531 2022/09/23 23:26:34 - mmengine - INFO - Epoch(train) [207][100/293] lr: 5.000000e-06 eta: 0:09:09 time: 0.591070 data_time: 0.100209 memory: 14267 loss_kpt: 0.000532 acc_pose: 0.855919 loss: 0.000532 2022/09/23 23:27:03 - mmengine - INFO - Epoch(train) [207][150/293] lr: 5.000000e-06 eta: 0:08:43 time: 0.573851 data_time: 0.085671 memory: 14267 loss_kpt: 0.000536 acc_pose: 0.841556 loss: 0.000536 2022/09/23 23:27:32 - mmengine - INFO - Epoch(train) [207][200/293] lr: 5.000000e-06 eta: 0:08:18 time: 0.569236 data_time: 0.070512 memory: 14267 loss_kpt: 0.000546 acc_pose: 0.858024 loss: 0.000546 2022/09/23 23:28:02 - mmengine - INFO - Epoch(train) [207][250/293] lr: 5.000000e-06 eta: 0:07:52 time: 0.610397 data_time: 0.099324 memory: 14267 loss_kpt: 0.000532 acc_pose: 0.899091 loss: 0.000532 2022/09/23 23:28:27 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:28:56 - mmengine - INFO - Epoch(train) [208][50/293] lr: 5.000000e-06 eta: 0:07:04 time: 0.591016 data_time: 0.086383 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.837860 loss: 0.000541 2022/09/23 23:29:26 - mmengine - INFO - Epoch(train) [208][100/293] lr: 5.000000e-06 eta: 0:06:39 time: 0.594179 data_time: 0.094304 memory: 14267 loss_kpt: 0.000531 acc_pose: 0.830508 loss: 0.000531 2022/09/23 23:29:56 - mmengine - INFO - Epoch(train) [208][150/293] lr: 5.000000e-06 eta: 0:06:13 time: 0.605632 data_time: 0.097809 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.850858 loss: 0.000541 2022/09/23 23:30:26 - mmengine - INFO - Epoch(train) [208][200/293] lr: 5.000000e-06 eta: 0:05:48 time: 0.601722 data_time: 0.096248 memory: 14267 loss_kpt: 0.000545 acc_pose: 0.824611 loss: 0.000545 2022/09/23 23:30:55 - mmengine - INFO - Epoch(train) [208][250/293] lr: 5.000000e-06 eta: 0:05:22 time: 0.581891 data_time: 0.088459 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.874629 loss: 0.000534 2022/09/23 23:31:20 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:31:52 - mmengine - INFO - Epoch(train) [209][50/293] lr: 5.000000e-06 eta: 0:04:34 time: 0.626231 data_time: 0.088239 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.859206 loss: 0.000534 2022/09/23 23:31:55 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:32:22 - mmengine - INFO - Epoch(train) [209][100/293] lr: 5.000000e-06 eta: 0:04:09 time: 0.610554 data_time: 0.080947 memory: 14267 loss_kpt: 0.000552 acc_pose: 0.874784 loss: 0.000552 2022/09/23 23:32:53 - mmengine - INFO - Epoch(train) [209][150/293] lr: 5.000000e-06 eta: 0:03:43 time: 0.610352 data_time: 0.092786 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.877531 loss: 0.000534 2022/09/23 23:33:24 - mmengine - INFO - Epoch(train) [209][200/293] lr: 5.000000e-06 eta: 0:03:17 time: 0.629522 data_time: 0.095922 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.853454 loss: 0.000534 2022/09/23 23:33:54 - mmengine - INFO - Epoch(train) [209][250/293] lr: 5.000000e-06 eta: 0:02:52 time: 0.598052 data_time: 0.083194 memory: 14267 loss_kpt: 0.000543 acc_pose: 0.840684 loss: 0.000543 2022/09/23 23:34:20 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:34:52 - mmengine - INFO - Epoch(train) [210][50/293] lr: 5.000000e-06 eta: 0:02:04 time: 0.656096 data_time: 0.124874 memory: 14267 loss_kpt: 0.000534 acc_pose: 0.804289 loss: 0.000534 2022/09/23 23:35:22 - mmengine - INFO - Epoch(train) [210][100/293] lr: 5.000000e-06 eta: 0:01:38 time: 0.581759 data_time: 0.075480 memory: 14267 loss_kpt: 0.000537 acc_pose: 0.846252 loss: 0.000537 2022/09/23 23:35:50 - mmengine - INFO - Epoch(train) [210][150/293] lr: 5.000000e-06 eta: 0:01:13 time: 0.570539 data_time: 0.086887 memory: 14267 loss_kpt: 0.000533 acc_pose: 0.863887 loss: 0.000533 2022/09/23 23:36:19 - mmengine - INFO - Epoch(train) [210][200/293] lr: 5.000000e-06 eta: 0:00:47 time: 0.585734 data_time: 0.093883 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.867006 loss: 0.000541 2022/09/23 23:36:49 - mmengine - INFO - Epoch(train) [210][250/293] lr: 5.000000e-06 eta: 0:00:22 time: 0.592006 data_time: 0.085980 memory: 14267 loss_kpt: 0.000541 acc_pose: 0.873392 loss: 0.000541 2022/09/23 23:37:14 - mmengine - INFO - Exp name: td-hm_res50_dark-8xb64-210e_coco-384x288_20220923_121844 2022/09/23 23:37:14 - mmengine - INFO - Saving checkpoint at 210 epochs 2022/09/23 23:37:38 - mmengine - INFO - Epoch(val) [210][50/407] eta: 0:02:10 time: 0.364804 data_time: 0.082972 memory: 14267 2022/09/23 23:37:55 - mmengine - INFO - Epoch(val) [210][100/407] eta: 0:01:45 time: 0.343751 data_time: 0.047991 memory: 1464 2022/09/23 23:38:12 - mmengine - INFO - Epoch(val) [210][150/407] eta: 0:01:26 time: 0.335431 data_time: 0.060224 memory: 1464 2022/09/23 23:38:28 - mmengine - INFO - Epoch(val) [210][200/407] eta: 0:01:09 time: 0.337444 data_time: 0.053360 memory: 1464 2022/09/23 23:38:46 - mmengine - INFO - Epoch(val) [210][250/407] eta: 0:00:54 time: 0.344547 data_time: 0.053241 memory: 1464 2022/09/23 23:39:03 - mmengine - INFO - Epoch(val) [210][300/407] eta: 0:00:36 time: 0.337539 data_time: 0.037385 memory: 1464 2022/09/23 23:39:19 - mmengine - INFO - Epoch(val) [210][350/407] eta: 0:00:18 time: 0.333139 data_time: 0.049458 memory: 1464 2022/09/23 23:39:33 - mmengine - INFO - Epoch(val) [210][400/407] eta: 0:00:01 time: 0.278393 data_time: 0.027899 memory: 1464 2022/09/23 23:40:07 - mmengine - INFO - Evaluating CocoMetric... 2022/09/23 23:40:20 - mmengine - INFO - Epoch(val) [210][407/407] coco/AP: 0.735023 coco/AP .5: 0.901337 coco/AP .75: 0.801081 coco/AP (M): 0.692199 coco/AP (L): 0.807296 coco/AR: 0.786414 coco/AR .5: 0.938130 coco/AR .75: 0.844773 coco/AR (M): 0.739716 coco/AR (L): 0.853586