2022/09/26 22:17:27 - mmengine - INFO - ------------------------------------------------------------ System environment: sys.platform: linux Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0] CUDA available: True numpy_random_seed: 1430764832 GPU 0,1,2,3,4,5,6,7: NVIDIA A100-SXM4-80GB CUDA_HOME: /mnt/lustre/share/cuda-11.3 NVCC: Cuda compilation tools, release 11.3, V11.3.109 GCC: gcc (GCC) 5.4.0 PyTorch: 1.12.1 PyTorch compiling details: PyTorch built with: - GCC 9.3 - C++ Version: 201402 - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815) - OpenMP 201511 (a.k.a. OpenMP 4.5) - LAPACK is enabled (usually provided by MKL) - NNPACK is enabled - CPU capability usage: AVX2 - CUDA Runtime 11.3 - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37 - CuDNN 8.3.2 (built against CUDA 11.5) - Magma 2.5.2 - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, TorchVision: 0.13.1 OpenCV: 4.6.0 MMEngine: 0.1.0 Runtime environment: cudnn_benchmark: False mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: None Distributed launcher: slurm Distributed training: True GPU number: 8 ------------------------------------------------------------ 2022/09/26 22:17:28 - mmengine - INFO - Config: default_scope = 'mmpose' default_hooks = dict( timer=dict(type='IterTimerHook'), logger=dict(type='LoggerHook', interval=50), param_scheduler=dict(type='ParamSchedulerHook'), checkpoint=dict( type='CheckpointHook', interval=10, max_keep_ckpts=1, save_best='coco/AP', rule='greater'), sampler_seed=dict(type='DistSamplerSeedHook'), visualization=dict(type='PoseVisualizationHook', enable=False)) custom_hooks = [dict(type='SyncBuffersHook')] env_cfg = dict( cudnn_benchmark=False, mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0), dist_cfg=dict(backend='nccl')) vis_backends = [dict(type='LocalVisBackend')] visualizer = dict( type='PoseLocalVisualizer', vis_backends=[dict(type='LocalVisBackend')], name='visualizer') log_processor = dict( type='LogProcessor', window_size=50, by_epoch=True, num_digits=6) log_level = 'INFO' load_from = None resume = False file_client_args = dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' })) train_cfg = dict(by_epoch=True, max_epochs=210, val_interval=10) val_cfg = dict() test_cfg = dict() optim_wrapper = dict( optimizer=dict(type='Adam', lr=0.0005), type='AmpOptimWrapper', loss_scale='dynamic') param_scheduler = [ dict( type='LinearLR', begin=0, end=500, start_factor=0.001, by_epoch=False), dict( type='MultiStepLR', begin=0, end=210, milestones=[170, 200], gamma=0.1, by_epoch=True) ] auto_scale_lr = dict(base_batch_size=512) codec = dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2) model = dict( type='TopdownPoseEstimator', data_preprocessor=dict( type='PoseDataPreprocessor', mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], bgr_to_rgb=True), backbone=dict( type='ResNet', depth=50, init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')), head=dict( type='HeatmapHead', in_channels=2048, out_channels=17, loss=dict(type='KeypointMSELoss', use_target_weight=True), decoder=dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2)), test_cfg=dict(flip_test=True, flip_mode='heatmap', shift_heatmap=True)) dataset_type = 'CocoDataset' data_mode = 'topdown' data_root = 'data/coco/' train_pipeline = [ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='RandomFlip', direction='horizontal'), dict(type='RandomHalfBody'), dict(type='RandomBBoxTransform'), dict(type='TopdownAffine', input_size=(192, 256)), dict( type='GenerateTarget', target_type='heatmap', encoder=dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2)), dict(type='PackPoseInputs') ] val_pipeline = [ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(192, 256)), dict(type='PackPoseInputs') ] train_dataloader = dict( batch_size=64, num_workers=2, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=True), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_train2017.json', data_prefix=dict(img='train2017/'), pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='RandomFlip', direction='horizontal'), dict(type='RandomHalfBody'), dict(type='RandomBBoxTransform'), dict(type='TopdownAffine', input_size=(192, 256)), dict( type='GenerateTarget', target_type='heatmap', encoder=dict( type='MSRAHeatmap', input_size=(192, 256), heatmap_size=(48, 64), sigma=2)), dict(type='PackPoseInputs') ])) val_dataloader = dict( batch_size=32, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_val2017.json', bbox_file= 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json', data_prefix=dict(img='val2017/'), test_mode=True, pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(192, 256)), dict(type='PackPoseInputs') ])) test_dataloader = dict( batch_size=32, num_workers=2, persistent_workers=True, drop_last=False, sampler=dict(type='DefaultSampler', shuffle=False, round_up=False), dataset=dict( type='CocoDataset', data_root='data/coco/', data_mode='topdown', ann_file='annotations/person_keypoints_val2017.json', bbox_file= 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json', data_prefix=dict(img='val2017/'), test_mode=True, pipeline=[ dict( type='LoadImage', file_client_args=dict( backend='petrel', path_mapping=dict({ './data/': 's3://openmmlab/datasets/detection/', 'data/': 's3://openmmlab/datasets/detection/' }))), dict(type='GetBBoxCenterScale'), dict(type='TopdownAffine', input_size=(192, 256)), dict(type='PackPoseInputs') ])) val_evaluator = dict( type='CocoMetric', ann_file='data/coco/annotations/person_keypoints_val2017.json') test_evaluator = dict( type='CocoMetric', ann_file='data/coco/annotations/person_keypoints_val2017.json') launcher = 'slurm' work_dir = '/mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/' 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "data sampler" registry tree. As a workaround, the current "data sampler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optimizer wrapper constructor" registry tree. As a workaround, the current "optimizer wrapper constructor" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optimizer" registry tree. As a workaround, the current "optimizer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "optim_wrapper" registry tree. As a workaround, the current "optim_wrapper" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:12 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "parameter scheduler" registry tree. As a workaround, the current "parameter scheduler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:15 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "data sampler" registry tree. As a workaround, the current "data sampler" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:17 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:20 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. 2022/09/26 22:18:20 - mmengine - WARNING - Failed to search registry with scope "mmpose" in the "weight initializer" registry tree. As a workaround, the current "weight initializer" registry in "mmengine" is used to build instance. This may cause unexpected failure when running the built modules. Please check whether "mmpose" is a correct scope, or whether the registry is initialized. Name of parameter - Initialization information backbone.conv1.weight - torch.Size([64, 3, 7, 7]): PretrainedInit: load from torchvision://resnet50 backbone.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.conv1.weight - torch.Size([64, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.downsample.0.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.downsample.1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.0.downsample.1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.1.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.conv1.weight - torch.Size([64, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn1.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn1.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.conv2.weight - torch.Size([64, 64, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn2.weight - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn2.bias - torch.Size([64]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.conv3.weight - torch.Size([256, 64, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn3.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer1.2.bn3.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.conv1.weight - torch.Size([128, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.downsample.0.weight - torch.Size([512, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.downsample.1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.0.downsample.1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.1.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.2.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.conv1.weight - torch.Size([128, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn1.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn1.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.conv2.weight - torch.Size([128, 128, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn2.weight - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn2.bias - torch.Size([128]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.conv3.weight - torch.Size([512, 128, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn3.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer2.3.bn3.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.conv1.weight - torch.Size([256, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.downsample.0.weight - torch.Size([1024, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.downsample.1.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.0.downsample.1.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.1.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.2.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.3.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.4.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.conv1.weight - torch.Size([256, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn1.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn1.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.conv2.weight - torch.Size([256, 256, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn2.weight - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn2.bias - torch.Size([256]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.conv3.weight - torch.Size([1024, 256, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn3.weight - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer3.5.bn3.bias - torch.Size([1024]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.conv1.weight - torch.Size([512, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.downsample.0.weight - torch.Size([2048, 1024, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.downsample.1.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.0.downsample.1.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.1.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.conv1.weight - torch.Size([512, 2048, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn1.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn1.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.conv2.weight - torch.Size([512, 512, 3, 3]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn2.weight - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn2.bias - torch.Size([512]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.conv3.weight - torch.Size([2048, 512, 1, 1]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn3.weight - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 backbone.layer4.2.bn3.bias - torch.Size([2048]): PretrainedInit: load from torchvision://resnet50 head.deconv_layers.0.weight - torch.Size([2048, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.1.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.1.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.3.weight - torch.Size([256, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.4.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.4.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.6.weight - torch.Size([256, 256, 4, 4]): NormalInit: mean=0, std=0.001, bias=0 head.deconv_layers.7.weight - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.deconv_layers.7.bias - torch.Size([256]): The value is the same before and after calling `init_weights` of TopdownPoseEstimator head.final_layer.weight - torch.Size([17, 256, 1, 1]): NormalInit: mean=0, std=0.001, bias=0 head.final_layer.bias - torch.Size([17]): NormalInit: mean=0, std=0.001, bias=0 2022/09/26 22:18:20 - mmengine - INFO - Checkpoints will be saved to /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16 by HardDiskBackend. 2022/09/26 22:19:32 - mmengine - INFO - Epoch(train) [1][50/293] lr: 4.954910e-05 eta: 1 day, 0:48:03 time: 1.452233 data_time: 0.340492 memory: 3840 loss_kpt: 0.002157 acc_pose: 0.122956 loss: 0.002157 2022/09/26 22:20:11 - mmengine - INFO - Epoch(train) [1][100/293] lr: 9.959920e-05 eta: 19:03:24 time: 0.781344 data_time: 0.104576 memory: 3840 loss_kpt: 0.001870 acc_pose: 0.310418 loss: 0.001870 2022/09/26 22:20:47 - mmengine - INFO - Epoch(train) [1][150/293] lr: 1.496493e-04 eta: 16:47:14 time: 0.720205 data_time: 0.087930 memory: 3840 loss_kpt: 0.001591 acc_pose: 0.413150 loss: 0.001591 2022/09/26 22:21:21 - mmengine - INFO - Epoch(train) [1][200/293] lr: 1.996994e-04 eta: 15:27:29 time: 0.675715 data_time: 0.097495 memory: 3840 loss_kpt: 0.001432 acc_pose: 0.524372 loss: 0.001432 2022/09/26 22:21:55 - mmengine - INFO - Epoch(train) [1][250/293] lr: 2.497495e-04 eta: 14:38:07 time: 0.669433 data_time: 0.112479 memory: 3840 loss_kpt: 0.001341 acc_pose: 0.554222 loss: 0.001341 2022/09/26 22:22:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:22:53 - mmengine - INFO - Epoch(train) [2][50/293] lr: 3.428427e-04 eta: 12:10:51 time: 0.617549 data_time: 0.126059 memory: 3840 loss_kpt: 0.001262 acc_pose: 0.570928 loss: 0.001262 2022/09/26 22:23:21 - mmengine - INFO - Epoch(train) [2][100/293] lr: 3.928928e-04 eta: 11:50:51 time: 0.566923 data_time: 0.081224 memory: 3840 loss_kpt: 0.001234 acc_pose: 0.604993 loss: 0.001234 2022/09/26 22:23:50 - mmengine - INFO - Epoch(train) [2][150/293] lr: 4.429429e-04 eta: 11:37:18 time: 0.584741 data_time: 0.083014 memory: 3840 loss_kpt: 0.001212 acc_pose: 0.585488 loss: 0.001212 2022/09/26 22:24:19 - mmengine - INFO - Epoch(train) [2][200/293] lr: 4.929930e-04 eta: 11:24:56 time: 0.570719 data_time: 0.086202 memory: 3840 loss_kpt: 0.001194 acc_pose: 0.567544 loss: 0.001194 2022/09/26 22:24:49 - mmengine - INFO - Epoch(train) [2][250/293] lr: 5.000000e-04 eta: 11:17:39 time: 0.601404 data_time: 0.105014 memory: 3840 loss_kpt: 0.001191 acc_pose: 0.625220 loss: 0.001191 2022/09/26 22:25:13 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:25:43 - mmengine - INFO - Epoch(train) [3][50/293] lr: 5.000000e-04 eta: 10:25:26 time: 0.598535 data_time: 0.111291 memory: 3840 loss_kpt: 0.001157 acc_pose: 0.635466 loss: 0.001157 2022/09/26 22:26:12 - mmengine - INFO - Epoch(train) [3][100/293] lr: 5.000000e-04 eta: 10:22:12 time: 0.579423 data_time: 0.091099 memory: 3840 loss_kpt: 0.001137 acc_pose: 0.638121 loss: 0.001137 2022/09/26 22:26:41 - mmengine - INFO - Epoch(train) [3][150/293] lr: 5.000000e-04 eta: 10:18:37 time: 0.568885 data_time: 0.077998 memory: 3840 loss_kpt: 0.001124 acc_pose: 0.624368 loss: 0.001124 2022/09/26 22:27:10 - mmengine - INFO - Epoch(train) [3][200/293] lr: 5.000000e-04 eta: 10:16:05 time: 0.579136 data_time: 0.082655 memory: 3840 loss_kpt: 0.001102 acc_pose: 0.627094 loss: 0.001102 2022/09/26 22:27:39 - mmengine - INFO - Epoch(train) [3][250/293] lr: 5.000000e-04 eta: 10:14:14 time: 0.586577 data_time: 0.098171 memory: 3840 loss_kpt: 0.001105 acc_pose: 0.600158 loss: 0.001105 2022/09/26 22:28:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:28:34 - mmengine - INFO - Epoch(train) [4][50/293] lr: 5.000000e-04 eta: 9:44:03 time: 0.591344 data_time: 0.122509 memory: 3840 loss_kpt: 0.001072 acc_pose: 0.673740 loss: 0.001072 2022/09/26 22:29:03 - mmengine - INFO - Epoch(train) [4][100/293] lr: 5.000000e-04 eta: 9:43:53 time: 0.584558 data_time: 0.149540 memory: 3840 loss_kpt: 0.001060 acc_pose: 0.642271 loss: 0.001060 2022/09/26 22:29:15 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:29:31 - mmengine - INFO - Epoch(train) [4][150/293] lr: 5.000000e-04 eta: 9:42:53 time: 0.567914 data_time: 0.159486 memory: 3840 loss_kpt: 0.001076 acc_pose: 0.629977 loss: 0.001076 2022/09/26 22:30:01 - mmengine - INFO - Epoch(train) [4][200/293] lr: 5.000000e-04 eta: 9:42:46 time: 0.585984 data_time: 0.086350 memory: 3840 loss_kpt: 0.001060 acc_pose: 0.641930 loss: 0.001060 2022/09/26 22:30:29 - mmengine - INFO - Epoch(train) [4][250/293] lr: 5.000000e-04 eta: 9:42:17 time: 0.578417 data_time: 0.079070 memory: 3840 loss_kpt: 0.001047 acc_pose: 0.653587 loss: 0.001047 2022/09/26 22:30:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:31:24 - mmengine - INFO - Epoch(train) [5][50/293] lr: 5.000000e-04 eta: 9:22:42 time: 0.621465 data_time: 0.192462 memory: 3840 loss_kpt: 0.001025 acc_pose: 0.629680 loss: 0.001025 2022/09/26 22:31:53 - mmengine - INFO - Epoch(train) [5][100/293] lr: 5.000000e-04 eta: 9:23:13 time: 0.584697 data_time: 0.133437 memory: 3840 loss_kpt: 0.001003 acc_pose: 0.647833 loss: 0.001003 2022/09/26 22:32:21 - mmengine - INFO - Epoch(train) [5][150/293] lr: 5.000000e-04 eta: 9:22:59 time: 0.566913 data_time: 0.087020 memory: 3840 loss_kpt: 0.001021 acc_pose: 0.700394 loss: 0.001021 2022/09/26 22:32:51 - mmengine - INFO - Epoch(train) [5][200/293] lr: 5.000000e-04 eta: 9:23:54 time: 0.599051 data_time: 0.116677 memory: 3840 loss_kpt: 0.001006 acc_pose: 0.637652 loss: 0.001006 2022/09/26 22:33:20 - mmengine - INFO - Epoch(train) [5][250/293] lr: 5.000000e-04 eta: 9:24:07 time: 0.581834 data_time: 0.121210 memory: 3840 loss_kpt: 0.001011 acc_pose: 0.635938 loss: 0.001011 2022/09/26 22:33:44 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:34:14 - mmengine - INFO - Epoch(train) [6][50/293] lr: 5.000000e-04 eta: 9:08:19 time: 0.595176 data_time: 0.168041 memory: 3840 loss_kpt: 0.001006 acc_pose: 0.681672 loss: 0.001006 2022/09/26 22:34:42 - mmengine - INFO - Epoch(train) [6][100/293] lr: 5.000000e-04 eta: 9:08:15 time: 0.560432 data_time: 0.169697 memory: 3840 loss_kpt: 0.001001 acc_pose: 0.711888 loss: 0.001001 2022/09/26 22:35:11 - mmengine - INFO - Epoch(train) [6][150/293] lr: 5.000000e-04 eta: 9:08:45 time: 0.579186 data_time: 0.090579 memory: 3840 loss_kpt: 0.000986 acc_pose: 0.691760 loss: 0.000986 2022/09/26 22:35:40 - mmengine - INFO - Epoch(train) [6][200/293] lr: 5.000000e-04 eta: 9:09:01 time: 0.573975 data_time: 0.091370 memory: 3840 loss_kpt: 0.000972 acc_pose: 0.586794 loss: 0.000972 2022/09/26 22:36:10 - mmengine - INFO - Epoch(train) [6][250/293] lr: 5.000000e-04 eta: 9:10:04 time: 0.601980 data_time: 0.078877 memory: 3840 loss_kpt: 0.000997 acc_pose: 0.660731 loss: 0.000997 2022/09/26 22:36:35 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:37:05 - mmengine - INFO - Epoch(train) [7][50/293] lr: 5.000000e-04 eta: 8:57:14 time: 0.591276 data_time: 0.092889 memory: 3840 loss_kpt: 0.000979 acc_pose: 0.645079 loss: 0.000979 2022/09/26 22:37:33 - mmengine - INFO - Epoch(train) [7][100/293] lr: 5.000000e-04 eta: 8:57:26 time: 0.564310 data_time: 0.075074 memory: 3840 loss_kpt: 0.000968 acc_pose: 0.657573 loss: 0.000968 2022/09/26 22:38:03 - mmengine - INFO - Epoch(train) [7][150/293] lr: 5.000000e-04 eta: 8:58:31 time: 0.598883 data_time: 0.202125 memory: 3840 loss_kpt: 0.000953 acc_pose: 0.659303 loss: 0.000953 2022/09/26 22:38:32 - mmengine - INFO - Epoch(train) [7][200/293] lr: 5.000000e-04 eta: 8:59:08 time: 0.584564 data_time: 0.126394 memory: 3840 loss_kpt: 0.000954 acc_pose: 0.701180 loss: 0.000954 2022/09/26 22:38:56 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:39:01 - mmengine - INFO - Epoch(train) [7][250/293] lr: 5.000000e-04 eta: 8:59:44 time: 0.585116 data_time: 0.081107 memory: 3840 loss_kpt: 0.000947 acc_pose: 0.673846 loss: 0.000947 2022/09/26 22:39:26 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:39:55 - mmengine - INFO - Epoch(train) [8][50/293] lr: 5.000000e-04 eta: 8:48:23 time: 0.566281 data_time: 0.161445 memory: 3840 loss_kpt: 0.000962 acc_pose: 0.711251 loss: 0.000962 2022/09/26 22:40:24 - mmengine - INFO - Epoch(train) [8][100/293] lr: 5.000000e-04 eta: 8:49:04 time: 0.582739 data_time: 0.104722 memory: 3840 loss_kpt: 0.000948 acc_pose: 0.677159 loss: 0.000948 2022/09/26 22:40:51 - mmengine - INFO - Epoch(train) [8][150/293] lr: 5.000000e-04 eta: 8:48:55 time: 0.547796 data_time: 0.073425 memory: 3840 loss_kpt: 0.000949 acc_pose: 0.722278 loss: 0.000949 2022/09/26 22:41:21 - mmengine - INFO - Epoch(train) [8][200/293] lr: 5.000000e-04 eta: 8:49:49 time: 0.596284 data_time: 0.118220 memory: 3840 loss_kpt: 0.000949 acc_pose: 0.741239 loss: 0.000949 2022/09/26 22:41:50 - mmengine - INFO - Epoch(train) [8][250/293] lr: 5.000000e-04 eta: 8:50:19 time: 0.580557 data_time: 0.089575 memory: 3840 loss_kpt: 0.000943 acc_pose: 0.688683 loss: 0.000943 2022/09/26 22:42:14 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:42:44 - mmengine - INFO - Epoch(train) [9][50/293] lr: 5.000000e-04 eta: 8:41:21 time: 0.603533 data_time: 0.122554 memory: 3840 loss_kpt: 0.000933 acc_pose: 0.723331 loss: 0.000933 2022/09/26 22:43:16 - mmengine - INFO - Epoch(train) [9][100/293] lr: 5.000000e-04 eta: 8:42:53 time: 0.627025 data_time: 0.106294 memory: 3840 loss_kpt: 0.000937 acc_pose: 0.685983 loss: 0.000937 2022/09/26 22:43:44 - mmengine - INFO - Epoch(train) [9][150/293] lr: 5.000000e-04 eta: 8:43:08 time: 0.566373 data_time: 0.073138 memory: 3840 loss_kpt: 0.000929 acc_pose: 0.695293 loss: 0.000929 2022/09/26 22:44:11 - mmengine - INFO - Epoch(train) [9][200/293] lr: 5.000000e-04 eta: 8:42:59 time: 0.546403 data_time: 0.090892 memory: 3840 loss_kpt: 0.000936 acc_pose: 0.717699 loss: 0.000936 2022/09/26 22:44:40 - mmengine - INFO - Epoch(train) [9][250/293] lr: 5.000000e-04 eta: 8:43:27 time: 0.580267 data_time: 0.076081 memory: 3840 loss_kpt: 0.000945 acc_pose: 0.713643 loss: 0.000945 2022/09/26 22:45:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:45:34 - mmengine - INFO - Epoch(train) [10][50/293] lr: 5.000000e-04 eta: 8:35:19 time: 0.590592 data_time: 0.096677 memory: 3840 loss_kpt: 0.000919 acc_pose: 0.701593 loss: 0.000919 2022/09/26 22:46:04 - mmengine - INFO - Epoch(train) [10][100/293] lr: 5.000000e-04 eta: 8:36:02 time: 0.590177 data_time: 0.102644 memory: 3840 loss_kpt: 0.000938 acc_pose: 0.688547 loss: 0.000938 2022/09/26 22:46:33 - mmengine - INFO - Epoch(train) [10][150/293] lr: 5.000000e-04 eta: 8:36:34 time: 0.581719 data_time: 0.091473 memory: 3840 loss_kpt: 0.000920 acc_pose: 0.723395 loss: 0.000920 2022/09/26 22:47:01 - mmengine - INFO - Epoch(train) [10][200/293] lr: 5.000000e-04 eta: 8:36:38 time: 0.557718 data_time: 0.082705 memory: 3840 loss_kpt: 0.000913 acc_pose: 0.719968 loss: 0.000913 2022/09/26 22:47:31 - mmengine - INFO - Epoch(train) [10][250/293] lr: 5.000000e-04 eta: 8:37:25 time: 0.600113 data_time: 0.182873 memory: 3840 loss_kpt: 0.000911 acc_pose: 0.728988 loss: 0.000911 2022/09/26 22:47:54 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:47:54 - mmengine - INFO - Saving checkpoint at 10 epochs 2022/09/26 22:48:20 - mmengine - INFO - Epoch(val) [10][50/407] eta: 0:02:41 time: 0.451619 data_time: 0.311027 memory: 3840 2022/09/26 22:48:35 - mmengine - INFO - Epoch(val) [10][100/407] eta: 0:01:34 time: 0.306406 data_time: 0.157338 memory: 1017 2022/09/26 22:48:50 - mmengine - INFO - Epoch(val) [10][150/407] eta: 0:01:16 time: 0.296272 data_time: 0.150348 memory: 1017 2022/09/26 22:49:05 - mmengine - INFO - Epoch(val) [10][200/407] eta: 0:01:04 time: 0.309989 data_time: 0.182766 memory: 1017 2022/09/26 22:49:20 - mmengine - INFO - Epoch(val) [10][250/407] eta: 0:00:45 time: 0.292576 data_time: 0.181168 memory: 1017 2022/09/26 22:49:35 - mmengine - INFO - Epoch(val) [10][300/407] eta: 0:00:32 time: 0.303228 data_time: 0.171984 memory: 1017 2022/09/26 22:49:51 - mmengine - INFO - Epoch(val) [10][350/407] eta: 0:00:17 time: 0.311368 data_time: 0.173726 memory: 1017 2022/09/26 22:50:05 - mmengine - INFO - Epoch(val) [10][400/407] eta: 0:00:01 time: 0.282760 data_time: 0.137608 memory: 1017 2022/09/26 22:50:43 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 22:50:57 - mmengine - INFO - Epoch(val) [10][407/407] coco/AP: 0.595220 coco/AP .5: 0.850600 coco/AP .75: 0.662966 coco/AP (M): 0.558146 coco/AP (L): 0.660337 coco/AR: 0.660390 coco/AR .5: 0.894679 coco/AR .75: 0.726385 coco/AR (M): 0.614122 coco/AR (L): 0.725678 2022/09/26 22:50:59 - mmengine - INFO - The best checkpoint with 0.5952 coco/AP at 10 epoch is saved to best_coco/AP_epoch_10.pth. 2022/09/26 22:51:29 - mmengine - INFO - Epoch(train) [11][50/293] lr: 5.000000e-04 eta: 8:30:11 time: 0.592607 data_time: 0.137623 memory: 3840 loss_kpt: 0.000924 acc_pose: 0.713683 loss: 0.000924 2022/09/26 22:51:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:51:57 - mmengine - INFO - Epoch(train) [11][100/293] lr: 5.000000e-04 eta: 8:30:28 time: 0.568399 data_time: 0.089859 memory: 3840 loss_kpt: 0.000904 acc_pose: 0.731475 loss: 0.000904 2022/09/26 22:52:26 - mmengine - INFO - Epoch(train) [11][150/293] lr: 5.000000e-04 eta: 8:30:49 time: 0.572589 data_time: 0.111921 memory: 3840 loss_kpt: 0.000900 acc_pose: 0.683936 loss: 0.000900 2022/09/26 22:52:55 - mmengine - INFO - Epoch(train) [11][200/293] lr: 5.000000e-04 eta: 8:31:18 time: 0.583731 data_time: 0.158575 memory: 3840 loss_kpt: 0.000932 acc_pose: 0.744068 loss: 0.000932 2022/09/26 22:53:24 - mmengine - INFO - Epoch(train) [11][250/293] lr: 5.000000e-04 eta: 8:31:44 time: 0.582040 data_time: 0.131932 memory: 3840 loss_kpt: 0.000907 acc_pose: 0.725846 loss: 0.000907 2022/09/26 22:53:48 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:54:18 - mmengine - INFO - Epoch(train) [12][50/293] lr: 5.000000e-04 eta: 8:25:11 time: 0.592090 data_time: 0.089811 memory: 3840 loss_kpt: 0.000909 acc_pose: 0.734941 loss: 0.000909 2022/09/26 22:54:47 - mmengine - INFO - Epoch(train) [12][100/293] lr: 5.000000e-04 eta: 8:25:41 time: 0.584281 data_time: 0.089051 memory: 3840 loss_kpt: 0.000889 acc_pose: 0.755206 loss: 0.000889 2022/09/26 22:55:17 - mmengine - INFO - Epoch(train) [12][150/293] lr: 5.000000e-04 eta: 8:26:10 time: 0.585946 data_time: 0.166967 memory: 3840 loss_kpt: 0.000887 acc_pose: 0.760309 loss: 0.000887 2022/09/26 22:55:46 - mmengine - INFO - Epoch(train) [12][200/293] lr: 5.000000e-04 eta: 8:26:38 time: 0.585586 data_time: 0.191384 memory: 3840 loss_kpt: 0.000916 acc_pose: 0.742648 loss: 0.000916 2022/09/26 22:56:15 - mmengine - INFO - Epoch(train) [12][250/293] lr: 5.000000e-04 eta: 8:27:07 time: 0.589306 data_time: 0.163667 memory: 3840 loss_kpt: 0.000887 acc_pose: 0.751456 loss: 0.000887 2022/09/26 22:56:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 22:57:11 - mmengine - INFO - Epoch(train) [13][50/293] lr: 5.000000e-04 eta: 8:21:28 time: 0.617340 data_time: 0.176411 memory: 3840 loss_kpt: 0.000901 acc_pose: 0.735922 loss: 0.000901 2022/09/26 22:57:40 - mmengine - INFO - Epoch(train) [13][100/293] lr: 5.000000e-04 eta: 8:21:56 time: 0.585988 data_time: 0.150975 memory: 3840 loss_kpt: 0.000875 acc_pose: 0.742815 loss: 0.000875 2022/09/26 22:58:08 - mmengine - INFO - Epoch(train) [13][150/293] lr: 5.000000e-04 eta: 8:22:03 time: 0.562249 data_time: 0.096617 memory: 3840 loss_kpt: 0.000893 acc_pose: 0.738761 loss: 0.000893 2022/09/26 22:58:39 - mmengine - INFO - Epoch(train) [13][200/293] lr: 5.000000e-04 eta: 8:22:46 time: 0.609566 data_time: 0.115995 memory: 3840 loss_kpt: 0.000898 acc_pose: 0.743743 loss: 0.000898 2022/09/26 22:59:23 - mmengine - INFO - Epoch(train) [13][250/293] lr: 5.000000e-04 eta: 8:27:04 time: 0.891640 data_time: 0.115679 memory: 3840 loss_kpt: 0.000898 acc_pose: 0.745491 loss: 0.000898 2022/09/26 22:59:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:00:24 - mmengine - INFO - Epoch(train) [14][50/293] lr: 5.000000e-04 eta: 8:21:45 time: 0.618975 data_time: 0.103742 memory: 3840 loss_kpt: 0.000869 acc_pose: 0.712667 loss: 0.000869 2022/09/26 23:00:54 - mmengine - INFO - Epoch(train) [14][100/293] lr: 5.000000e-04 eta: 8:22:16 time: 0.599317 data_time: 0.101378 memory: 3840 loss_kpt: 0.000874 acc_pose: 0.736973 loss: 0.000874 2022/09/26 23:01:23 - mmengine - INFO - Epoch(train) [14][150/293] lr: 5.000000e-04 eta: 8:22:26 time: 0.572837 data_time: 0.108883 memory: 3840 loss_kpt: 0.000881 acc_pose: 0.688631 loss: 0.000881 2022/09/26 23:01:47 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:01:52 - mmengine - INFO - Epoch(train) [14][200/293] lr: 5.000000e-04 eta: 8:22:43 time: 0.583364 data_time: 0.096127 memory: 3840 loss_kpt: 0.000876 acc_pose: 0.700618 loss: 0.000876 2022/09/26 23:02:21 - mmengine - INFO - Epoch(train) [14][250/293] lr: 5.000000e-04 eta: 8:23:00 time: 0.586274 data_time: 0.097568 memory: 3840 loss_kpt: 0.000877 acc_pose: 0.719785 loss: 0.000877 2022/09/26 23:02:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:03:17 - mmengine - INFO - Epoch(train) [15][50/293] lr: 5.000000e-04 eta: 8:17:59 time: 0.611874 data_time: 0.130765 memory: 3840 loss_kpt: 0.000876 acc_pose: 0.755933 loss: 0.000876 2022/09/26 23:03:46 - mmengine - INFO - Epoch(train) [15][100/293] lr: 5.000000e-04 eta: 8:18:20 time: 0.588918 data_time: 0.084660 memory: 3840 loss_kpt: 0.000867 acc_pose: 0.740740 loss: 0.000867 2022/09/26 23:04:15 - mmengine - INFO - Epoch(train) [15][150/293] lr: 5.000000e-04 eta: 8:18:35 time: 0.583224 data_time: 0.094171 memory: 3840 loss_kpt: 0.000869 acc_pose: 0.758447 loss: 0.000869 2022/09/26 23:04:45 - mmengine - INFO - Epoch(train) [15][200/293] lr: 5.000000e-04 eta: 8:18:53 time: 0.588020 data_time: 0.091932 memory: 3840 loss_kpt: 0.000877 acc_pose: 0.687275 loss: 0.000877 2022/09/26 23:05:13 - mmengine - INFO - Epoch(train) [15][250/293] lr: 5.000000e-04 eta: 8:18:55 time: 0.565973 data_time: 0.119311 memory: 3840 loss_kpt: 0.000872 acc_pose: 0.746547 loss: 0.000872 2022/09/26 23:05:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:06:08 - mmengine - INFO - Epoch(train) [16][50/293] lr: 5.000000e-04 eta: 8:14:15 time: 0.613057 data_time: 0.109588 memory: 3840 loss_kpt: 0.000879 acc_pose: 0.706133 loss: 0.000879 2022/09/26 23:06:38 - mmengine - INFO - Epoch(train) [16][100/293] lr: 5.000000e-04 eta: 8:14:35 time: 0.593177 data_time: 0.161401 memory: 3840 loss_kpt: 0.000872 acc_pose: 0.826315 loss: 0.000872 2022/09/26 23:07:07 - mmengine - INFO - Epoch(train) [16][150/293] lr: 5.000000e-04 eta: 8:14:48 time: 0.581513 data_time: 0.091324 memory: 3840 loss_kpt: 0.000867 acc_pose: 0.744650 loss: 0.000867 2022/09/26 23:07:35 - mmengine - INFO - Epoch(train) [16][200/293] lr: 5.000000e-04 eta: 8:14:42 time: 0.554178 data_time: 0.111729 memory: 3840 loss_kpt: 0.000863 acc_pose: 0.788705 loss: 0.000863 2022/09/26 23:08:03 - mmengine - INFO - Epoch(train) [16][250/293] lr: 5.000000e-04 eta: 8:14:49 time: 0.574760 data_time: 0.167184 memory: 3840 loss_kpt: 0.000869 acc_pose: 0.744285 loss: 0.000869 2022/09/26 23:08:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:08:59 - mmengine - INFO - Epoch(train) [17][50/293] lr: 5.000000e-04 eta: 8:10:20 time: 0.603451 data_time: 0.110942 memory: 3840 loss_kpt: 0.000856 acc_pose: 0.776982 loss: 0.000856 2022/09/26 23:09:29 - mmengine - INFO - Epoch(train) [17][100/293] lr: 5.000000e-04 eta: 8:10:40 time: 0.595242 data_time: 0.094283 memory: 3840 loss_kpt: 0.000866 acc_pose: 0.744962 loss: 0.000866 2022/09/26 23:09:58 - mmengine - INFO - Epoch(train) [17][150/293] lr: 5.000000e-04 eta: 8:10:51 time: 0.582642 data_time: 0.096921 memory: 3840 loss_kpt: 0.000850 acc_pose: 0.773897 loss: 0.000850 2022/09/26 23:10:27 - mmengine - INFO - Epoch(train) [17][200/293] lr: 5.000000e-04 eta: 8:11:06 time: 0.589302 data_time: 0.112759 memory: 3840 loss_kpt: 0.000864 acc_pose: 0.702753 loss: 0.000864 2022/09/26 23:10:57 - mmengine - INFO - Epoch(train) [17][250/293] lr: 5.000000e-04 eta: 8:11:21 time: 0.592246 data_time: 0.083637 memory: 3840 loss_kpt: 0.000863 acc_pose: 0.756682 loss: 0.000863 2022/09/26 23:11:21 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:11:32 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:11:51 - mmengine - INFO - Epoch(train) [18][50/293] lr: 5.000000e-04 eta: 8:07:06 time: 0.600937 data_time: 0.104373 memory: 3840 loss_kpt: 0.000852 acc_pose: 0.718635 loss: 0.000852 2022/09/26 23:12:19 - mmengine - INFO - Epoch(train) [18][100/293] lr: 5.000000e-04 eta: 8:07:07 time: 0.565233 data_time: 0.079183 memory: 3840 loss_kpt: 0.000859 acc_pose: 0.660398 loss: 0.000859 2022/09/26 23:12:48 - mmengine - INFO - Epoch(train) [18][150/293] lr: 5.000000e-04 eta: 8:07:16 time: 0.582378 data_time: 0.120796 memory: 3840 loss_kpt: 0.000849 acc_pose: 0.716120 loss: 0.000849 2022/09/26 23:13:17 - mmengine - INFO - Epoch(train) [18][200/293] lr: 5.000000e-04 eta: 8:07:21 time: 0.573911 data_time: 0.086050 memory: 3840 loss_kpt: 0.000845 acc_pose: 0.786903 loss: 0.000845 2022/09/26 23:13:46 - mmengine - INFO - Epoch(train) [18][250/293] lr: 5.000000e-04 eta: 8:07:30 time: 0.583493 data_time: 0.173450 memory: 3840 loss_kpt: 0.000857 acc_pose: 0.753448 loss: 0.000857 2022/09/26 23:14:11 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:14:42 - mmengine - INFO - Epoch(train) [19][50/293] lr: 5.000000e-04 eta: 8:03:38 time: 0.618557 data_time: 0.107613 memory: 3840 loss_kpt: 0.000841 acc_pose: 0.741689 loss: 0.000841 2022/09/26 23:15:11 - mmengine - INFO - Epoch(train) [19][100/293] lr: 5.000000e-04 eta: 8:03:45 time: 0.579328 data_time: 0.079893 memory: 3840 loss_kpt: 0.000835 acc_pose: 0.774226 loss: 0.000835 2022/09/26 23:15:40 - mmengine - INFO - Epoch(train) [19][150/293] lr: 5.000000e-04 eta: 8:03:50 time: 0.576441 data_time: 0.100489 memory: 3840 loss_kpt: 0.000848 acc_pose: 0.744057 loss: 0.000848 2022/09/26 23:16:10 - mmengine - INFO - Epoch(train) [19][200/293] lr: 5.000000e-04 eta: 8:04:01 time: 0.589210 data_time: 0.115818 memory: 3840 loss_kpt: 0.000836 acc_pose: 0.754357 loss: 0.000836 2022/09/26 23:16:38 - mmengine - INFO - Epoch(train) [19][250/293] lr: 5.000000e-04 eta: 8:03:59 time: 0.566673 data_time: 0.175970 memory: 3840 loss_kpt: 0.000834 acc_pose: 0.756545 loss: 0.000834 2022/09/26 23:17:02 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:17:32 - mmengine - INFO - Epoch(train) [20][50/293] lr: 5.000000e-04 eta: 8:00:11 time: 0.603429 data_time: 0.128289 memory: 3840 loss_kpt: 0.000837 acc_pose: 0.745177 loss: 0.000837 2022/09/26 23:18:00 - mmengine - INFO - Epoch(train) [20][100/293] lr: 5.000000e-04 eta: 8:00:11 time: 0.566808 data_time: 0.073968 memory: 3840 loss_kpt: 0.000839 acc_pose: 0.745427 loss: 0.000839 2022/09/26 23:18:28 - mmengine - INFO - Epoch(train) [20][150/293] lr: 5.000000e-04 eta: 8:00:07 time: 0.559454 data_time: 0.086306 memory: 3840 loss_kpt: 0.000844 acc_pose: 0.745678 loss: 0.000844 2022/09/26 23:18:57 - mmengine - INFO - Epoch(train) [20][200/293] lr: 5.000000e-04 eta: 8:00:14 time: 0.585783 data_time: 0.095260 memory: 3840 loss_kpt: 0.000842 acc_pose: 0.771600 loss: 0.000842 2022/09/26 23:19:27 - mmengine - INFO - Epoch(train) [20][250/293] lr: 5.000000e-04 eta: 8:00:24 time: 0.591109 data_time: 0.076720 memory: 3840 loss_kpt: 0.000843 acc_pose: 0.695769 loss: 0.000843 2022/09/26 23:19:52 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:19:52 - mmengine - INFO - Saving checkpoint at 20 epochs 2022/09/26 23:20:14 - mmengine - INFO - Epoch(val) [20][50/407] eta: 0:01:53 time: 0.318430 data_time: 0.175196 memory: 3840 2022/09/26 23:20:30 - mmengine - INFO - Epoch(val) [20][100/407] eta: 0:01:34 time: 0.308566 data_time: 0.173891 memory: 1017 2022/09/26 23:20:45 - mmengine - INFO - Epoch(val) [20][150/407] eta: 0:01:16 time: 0.296471 data_time: 0.162340 memory: 1017 2022/09/26 23:21:00 - mmengine - INFO - Epoch(val) [20][200/407] eta: 0:01:05 time: 0.314041 data_time: 0.182530 memory: 1017 2022/09/26 23:21:16 - mmengine - INFO - Epoch(val) [20][250/407] eta: 0:00:47 time: 0.302473 data_time: 0.176494 memory: 1017 2022/09/26 23:21:30 - mmengine - INFO - Epoch(val) [20][300/407] eta: 0:00:31 time: 0.291737 data_time: 0.158810 memory: 1017 2022/09/26 23:21:45 - mmengine - INFO - Epoch(val) [20][350/407] eta: 0:00:17 time: 0.304643 data_time: 0.143933 memory: 1017 2022/09/26 23:21:58 - mmengine - INFO - Epoch(val) [20][400/407] eta: 0:00:01 time: 0.260077 data_time: 0.146691 memory: 1017 2022/09/26 23:22:31 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 23:22:45 - mmengine - INFO - Epoch(val) [20][407/407] coco/AP: 0.621551 coco/AP .5: 0.857774 coco/AP .75: 0.695278 coco/AP (M): 0.585829 coco/AP (L): 0.687199 coco/AR: 0.683344 coco/AR .5: 0.901291 coco/AR .75: 0.753463 coco/AR (M): 0.638323 coco/AR (L): 0.747009 2022/09/26 23:22:45 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_10.pth is removed 2022/09/26 23:22:47 - mmengine - INFO - The best checkpoint with 0.6216 coco/AP at 20 epoch is saved to best_coco/AP_epoch_20.pth. 2022/09/26 23:23:17 - mmengine - INFO - Epoch(train) [21][50/293] lr: 5.000000e-04 eta: 7:56:43 time: 0.595380 data_time: 0.132306 memory: 3840 loss_kpt: 0.000836 acc_pose: 0.744494 loss: 0.000836 2022/09/26 23:23:46 - mmengine - INFO - Epoch(train) [21][100/293] lr: 5.000000e-04 eta: 7:56:55 time: 0.593920 data_time: 0.094105 memory: 3840 loss_kpt: 0.000828 acc_pose: 0.758720 loss: 0.000828 2022/09/26 23:24:09 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:24:15 - mmengine - INFO - Epoch(train) [21][150/293] lr: 5.000000e-04 eta: 7:56:53 time: 0.567345 data_time: 0.087650 memory: 3840 loss_kpt: 0.000827 acc_pose: 0.757298 loss: 0.000827 2022/09/26 23:24:43 - mmengine - INFO - Epoch(train) [21][200/293] lr: 5.000000e-04 eta: 7:56:53 time: 0.570869 data_time: 0.102128 memory: 3840 loss_kpt: 0.000841 acc_pose: 0.751539 loss: 0.000841 2022/09/26 23:25:14 - mmengine - INFO - Epoch(train) [21][250/293] lr: 5.000000e-04 eta: 7:57:09 time: 0.607107 data_time: 0.078104 memory: 3840 loss_kpt: 0.000829 acc_pose: 0.845912 loss: 0.000829 2022/09/26 23:25:39 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:26:09 - mmengine - INFO - Epoch(train) [22][50/293] lr: 5.000000e-04 eta: 7:53:43 time: 0.607229 data_time: 0.215312 memory: 3840 loss_kpt: 0.000821 acc_pose: 0.727053 loss: 0.000821 2022/09/26 23:26:39 - mmengine - INFO - Epoch(train) [22][100/293] lr: 5.000000e-04 eta: 7:53:48 time: 0.584174 data_time: 0.191876 memory: 3840 loss_kpt: 0.000843 acc_pose: 0.724371 loss: 0.000843 2022/09/26 23:27:07 - mmengine - INFO - Epoch(train) [22][150/293] lr: 5.000000e-04 eta: 7:53:43 time: 0.560445 data_time: 0.102403 memory: 3840 loss_kpt: 0.000814 acc_pose: 0.737892 loss: 0.000814 2022/09/26 23:27:36 - mmengine - INFO - Epoch(train) [22][200/293] lr: 5.000000e-04 eta: 7:53:48 time: 0.585531 data_time: 0.154338 memory: 3840 loss_kpt: 0.000839 acc_pose: 0.737418 loss: 0.000839 2022/09/26 23:28:05 - mmengine - INFO - Epoch(train) [22][250/293] lr: 5.000000e-04 eta: 7:53:46 time: 0.571491 data_time: 0.145586 memory: 3840 loss_kpt: 0.000828 acc_pose: 0.729952 loss: 0.000828 2022/09/26 23:28:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:28:59 - mmengine - INFO - Epoch(train) [23][50/293] lr: 5.000000e-04 eta: 7:50:27 time: 0.601080 data_time: 0.110940 memory: 3840 loss_kpt: 0.000834 acc_pose: 0.748803 loss: 0.000834 2022/09/26 23:29:27 - mmengine - INFO - Epoch(train) [23][100/293] lr: 5.000000e-04 eta: 7:50:19 time: 0.556105 data_time: 0.075184 memory: 3840 loss_kpt: 0.000822 acc_pose: 0.754322 loss: 0.000822 2022/09/26 23:29:56 - mmengine - INFO - Epoch(train) [23][150/293] lr: 5.000000e-04 eta: 7:50:23 time: 0.584919 data_time: 0.076852 memory: 3840 loss_kpt: 0.000808 acc_pose: 0.739491 loss: 0.000808 2022/09/26 23:30:26 - mmengine - INFO - Epoch(train) [23][200/293] lr: 5.000000e-04 eta: 7:50:29 time: 0.589195 data_time: 0.131647 memory: 3840 loss_kpt: 0.000826 acc_pose: 0.746562 loss: 0.000826 2022/09/26 23:30:55 - mmengine - INFO - Epoch(train) [23][250/293] lr: 5.000000e-04 eta: 7:50:33 time: 0.588308 data_time: 0.118517 memory: 3840 loss_kpt: 0.000819 acc_pose: 0.772521 loss: 0.000819 2022/09/26 23:31:21 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:31:50 - mmengine - INFO - Epoch(train) [24][50/293] lr: 5.000000e-04 eta: 7:47:18 time: 0.592595 data_time: 0.102137 memory: 3840 loss_kpt: 0.000816 acc_pose: 0.782242 loss: 0.000816 2022/09/26 23:32:20 - mmengine - INFO - Epoch(train) [24][100/293] lr: 5.000000e-04 eta: 7:47:21 time: 0.583960 data_time: 0.075941 memory: 3840 loss_kpt: 0.000822 acc_pose: 0.726105 loss: 0.000822 2022/09/26 23:32:48 - mmengine - INFO - Epoch(train) [24][150/293] lr: 5.000000e-04 eta: 7:47:20 time: 0.574239 data_time: 0.103457 memory: 3840 loss_kpt: 0.000810 acc_pose: 0.730548 loss: 0.000810 2022/09/26 23:33:17 - mmengine - INFO - Epoch(train) [24][200/293] lr: 5.000000e-04 eta: 7:47:20 time: 0.578380 data_time: 0.092384 memory: 3840 loss_kpt: 0.000819 acc_pose: 0.760801 loss: 0.000819 2022/09/26 23:33:45 - mmengine - INFO - Epoch(train) [24][250/293] lr: 5.000000e-04 eta: 7:47:11 time: 0.556082 data_time: 0.077089 memory: 3840 loss_kpt: 0.000817 acc_pose: 0.764304 loss: 0.000817 2022/09/26 23:33:52 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:34:09 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:34:39 - mmengine - INFO - Epoch(train) [25][50/293] lr: 5.000000e-04 eta: 7:44:01 time: 0.586553 data_time: 0.093074 memory: 3840 loss_kpt: 0.000813 acc_pose: 0.708596 loss: 0.000813 2022/09/26 23:35:07 - mmengine - INFO - Epoch(train) [25][100/293] lr: 5.000000e-04 eta: 7:44:01 time: 0.578975 data_time: 0.092339 memory: 3840 loss_kpt: 0.000823 acc_pose: 0.755414 loss: 0.000823 2022/09/26 23:35:36 - mmengine - INFO - Epoch(train) [25][150/293] lr: 5.000000e-04 eta: 7:44:00 time: 0.575418 data_time: 0.135217 memory: 3840 loss_kpt: 0.000802 acc_pose: 0.769027 loss: 0.000802 2022/09/26 23:36:04 - mmengine - INFO - Epoch(train) [25][200/293] lr: 5.000000e-04 eta: 7:43:48 time: 0.548307 data_time: 0.091878 memory: 3840 loss_kpt: 0.000808 acc_pose: 0.724137 loss: 0.000808 2022/09/26 23:36:32 - mmengine - INFO - Epoch(train) [25][250/293] lr: 5.000000e-04 eta: 7:43:44 time: 0.569851 data_time: 0.117129 memory: 3840 loss_kpt: 0.000821 acc_pose: 0.749725 loss: 0.000821 2022/09/26 23:36:56 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:37:27 - mmengine - INFO - Epoch(train) [26][50/293] lr: 5.000000e-04 eta: 7:40:49 time: 0.609651 data_time: 0.100102 memory: 3840 loss_kpt: 0.000843 acc_pose: 0.775667 loss: 0.000843 2022/09/26 23:37:54 - mmengine - INFO - Epoch(train) [26][100/293] lr: 5.000000e-04 eta: 7:40:40 time: 0.553721 data_time: 0.078904 memory: 3840 loss_kpt: 0.000811 acc_pose: 0.762211 loss: 0.000811 2022/09/26 23:38:23 - mmengine - INFO - Epoch(train) [26][150/293] lr: 5.000000e-04 eta: 7:40:34 time: 0.564717 data_time: 0.084210 memory: 3840 loss_kpt: 0.000825 acc_pose: 0.644889 loss: 0.000825 2022/09/26 23:38:52 - mmengine - INFO - Epoch(train) [26][200/293] lr: 5.000000e-04 eta: 7:40:32 time: 0.576912 data_time: 0.157062 memory: 3840 loss_kpt: 0.000836 acc_pose: 0.748860 loss: 0.000836 2022/09/26 23:39:20 - mmengine - INFO - Epoch(train) [26][250/293] lr: 5.000000e-04 eta: 7:40:25 time: 0.564648 data_time: 0.079830 memory: 3840 loss_kpt: 0.000820 acc_pose: 0.787526 loss: 0.000820 2022/09/26 23:39:44 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:40:14 - mmengine - INFO - Epoch(train) [27][50/293] lr: 5.000000e-04 eta: 7:37:32 time: 0.596810 data_time: 0.197178 memory: 3840 loss_kpt: 0.000803 acc_pose: 0.771965 loss: 0.000803 2022/09/26 23:40:42 - mmengine - INFO - Epoch(train) [27][100/293] lr: 5.000000e-04 eta: 7:37:24 time: 0.558343 data_time: 0.085727 memory: 3840 loss_kpt: 0.000812 acc_pose: 0.715685 loss: 0.000812 2022/09/26 23:41:10 - mmengine - INFO - Epoch(train) [27][150/293] lr: 5.000000e-04 eta: 7:37:19 time: 0.568563 data_time: 0.102890 memory: 3840 loss_kpt: 0.000811 acc_pose: 0.752861 loss: 0.000811 2022/09/26 23:41:40 - mmengine - INFO - Epoch(train) [27][200/293] lr: 5.000000e-04 eta: 7:37:26 time: 0.606344 data_time: 0.111529 memory: 3840 loss_kpt: 0.000808 acc_pose: 0.761383 loss: 0.000808 2022/09/26 23:42:08 - mmengine - INFO - Epoch(train) [27][250/293] lr: 5.000000e-04 eta: 7:37:15 time: 0.553872 data_time: 0.083450 memory: 3840 loss_kpt: 0.000812 acc_pose: 0.769661 loss: 0.000812 2022/09/26 23:42:34 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:43:04 - mmengine - INFO - Epoch(train) [28][50/293] lr: 5.000000e-04 eta: 7:34:31 time: 0.604697 data_time: 0.098588 memory: 3840 loss_kpt: 0.000814 acc_pose: 0.691951 loss: 0.000814 2022/09/26 23:43:26 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:43:32 - mmengine - INFO - Epoch(train) [28][100/293] lr: 5.000000e-04 eta: 7:34:23 time: 0.560104 data_time: 0.089088 memory: 3840 loss_kpt: 0.000796 acc_pose: 0.755751 loss: 0.000796 2022/09/26 23:44:01 - mmengine - INFO - Epoch(train) [28][150/293] lr: 5.000000e-04 eta: 7:34:22 time: 0.583897 data_time: 0.214673 memory: 3840 loss_kpt: 0.000823 acc_pose: 0.663399 loss: 0.000823 2022/09/26 23:44:30 - mmengine - INFO - Epoch(train) [28][200/293] lr: 5.000000e-04 eta: 7:34:16 time: 0.568026 data_time: 0.176777 memory: 3840 loss_kpt: 0.000801 acc_pose: 0.828192 loss: 0.000801 2022/09/26 23:44:58 - mmengine - INFO - Epoch(train) [28][250/293] lr: 5.000000e-04 eta: 7:34:10 time: 0.569894 data_time: 0.098414 memory: 3840 loss_kpt: 0.000819 acc_pose: 0.783906 loss: 0.000819 2022/09/26 23:45:23 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:45:53 - mmengine - INFO - Epoch(train) [29][50/293] lr: 5.000000e-04 eta: 7:31:32 time: 0.608253 data_time: 0.109101 memory: 3840 loss_kpt: 0.000798 acc_pose: 0.729407 loss: 0.000798 2022/09/26 23:46:22 - mmengine - INFO - Epoch(train) [29][100/293] lr: 5.000000e-04 eta: 7:31:30 time: 0.583619 data_time: 0.086614 memory: 3840 loss_kpt: 0.000801 acc_pose: 0.784001 loss: 0.000801 2022/09/26 23:46:51 - mmengine - INFO - Epoch(train) [29][150/293] lr: 5.000000e-04 eta: 7:31:24 time: 0.567265 data_time: 0.078571 memory: 3840 loss_kpt: 0.000797 acc_pose: 0.739720 loss: 0.000797 2022/09/26 23:47:21 - mmengine - INFO - Epoch(train) [29][200/293] lr: 5.000000e-04 eta: 7:31:26 time: 0.597117 data_time: 0.111801 memory: 3840 loss_kpt: 0.000816 acc_pose: 0.723108 loss: 0.000816 2022/09/26 23:47:49 - mmengine - INFO - Epoch(train) [29][250/293] lr: 5.000000e-04 eta: 7:31:20 time: 0.570816 data_time: 0.080259 memory: 3840 loss_kpt: 0.000817 acc_pose: 0.738249 loss: 0.000817 2022/09/26 23:48:14 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:48:44 - mmengine - INFO - Epoch(train) [30][50/293] lr: 5.000000e-04 eta: 7:28:46 time: 0.605865 data_time: 0.180157 memory: 3840 loss_kpt: 0.000815 acc_pose: 0.757374 loss: 0.000815 2022/09/26 23:49:12 - mmengine - INFO - Epoch(train) [30][100/293] lr: 5.000000e-04 eta: 7:28:36 time: 0.559721 data_time: 0.078632 memory: 3840 loss_kpt: 0.000804 acc_pose: 0.759667 loss: 0.000804 2022/09/26 23:49:42 - mmengine - INFO - Epoch(train) [30][150/293] lr: 5.000000e-04 eta: 7:28:34 time: 0.585062 data_time: 0.106373 memory: 3840 loss_kpt: 0.000805 acc_pose: 0.781801 loss: 0.000805 2022/09/26 23:50:09 - mmengine - INFO - Epoch(train) [30][200/293] lr: 5.000000e-04 eta: 7:28:20 time: 0.546035 data_time: 0.080867 memory: 3840 loss_kpt: 0.000799 acc_pose: 0.692706 loss: 0.000799 2022/09/26 23:50:36 - mmengine - INFO - Epoch(train) [30][250/293] lr: 5.000000e-04 eta: 7:28:08 time: 0.552370 data_time: 0.070234 memory: 3840 loss_kpt: 0.000800 acc_pose: 0.778377 loss: 0.000800 2022/09/26 23:51:01 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:51:01 - mmengine - INFO - Saving checkpoint at 30 epochs 2022/09/26 23:51:23 - mmengine - INFO - Epoch(val) [30][50/407] eta: 0:01:58 time: 0.331594 data_time: 0.192641 memory: 3840 2022/09/26 23:51:40 - mmengine - INFO - Epoch(val) [30][100/407] eta: 0:01:41 time: 0.331976 data_time: 0.207045 memory: 1017 2022/09/26 23:51:57 - mmengine - INFO - Epoch(val) [30][150/407] eta: 0:01:28 time: 0.343288 data_time: 0.215657 memory: 1017 2022/09/26 23:52:12 - mmengine - INFO - Epoch(val) [30][200/407] eta: 0:01:01 time: 0.296096 data_time: 0.165984 memory: 1017 2022/09/26 23:52:28 - mmengine - INFO - Epoch(val) [30][250/407] eta: 0:00:50 time: 0.323614 data_time: 0.175032 memory: 1017 2022/09/26 23:52:43 - mmengine - INFO - Epoch(val) [30][300/407] eta: 0:00:33 time: 0.310316 data_time: 0.166453 memory: 1017 2022/09/26 23:52:58 - mmengine - INFO - Epoch(val) [30][350/407] eta: 0:00:16 time: 0.297753 data_time: 0.151557 memory: 1017 2022/09/26 23:53:09 - mmengine - INFO - Epoch(val) [30][400/407] eta: 0:00:01 time: 0.225128 data_time: 0.138274 memory: 1017 2022/09/26 23:53:42 - mmengine - INFO - Evaluating CocoMetric... 2022/09/26 23:53:55 - mmengine - INFO - Epoch(val) [30][407/407] coco/AP: 0.643586 coco/AP .5: 0.867513 coco/AP .75: 0.715848 coco/AP (M): 0.608900 coco/AP (L): 0.708966 coco/AR: 0.706502 coco/AR .5: 0.911524 coco/AR .75: 0.773615 coco/AR (M): 0.661513 coco/AR (L): 0.769602 2022/09/26 23:53:55 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_20.pth is removed 2022/09/26 23:53:58 - mmengine - INFO - The best checkpoint with 0.6436 coco/AP at 30 epoch is saved to best_coco/AP_epoch_30.pth. 2022/09/26 23:54:27 - mmengine - INFO - Epoch(train) [31][50/293] lr: 5.000000e-04 eta: 7:25:33 time: 0.588271 data_time: 0.143245 memory: 3840 loss_kpt: 0.000789 acc_pose: 0.781960 loss: 0.000789 2022/09/26 23:54:56 - mmengine - INFO - Epoch(train) [31][100/293] lr: 5.000000e-04 eta: 7:25:29 time: 0.578791 data_time: 0.163293 memory: 3840 loss_kpt: 0.000799 acc_pose: 0.796157 loss: 0.000799 2022/09/26 23:55:25 - mmengine - INFO - Epoch(train) [31][150/293] lr: 5.000000e-04 eta: 7:25:24 time: 0.578839 data_time: 0.077696 memory: 3840 loss_kpt: 0.000815 acc_pose: 0.732233 loss: 0.000815 2022/09/26 23:55:53 - mmengine - INFO - Epoch(train) [31][200/293] lr: 5.000000e-04 eta: 7:25:16 time: 0.565794 data_time: 0.072251 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.715181 loss: 0.000791 2022/09/26 23:55:59 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:56:22 - mmengine - INFO - Epoch(train) [31][250/293] lr: 5.000000e-04 eta: 7:25:13 time: 0.585358 data_time: 0.108174 memory: 3840 loss_kpt: 0.000805 acc_pose: 0.773000 loss: 0.000805 2022/09/26 23:56:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/26 23:57:16 - mmengine - INFO - Epoch(train) [32][50/293] lr: 5.000000e-04 eta: 7:22:43 time: 0.590614 data_time: 0.170125 memory: 3840 loss_kpt: 0.000799 acc_pose: 0.799078 loss: 0.000799 2022/09/26 23:57:45 - mmengine - INFO - Epoch(train) [32][100/293] lr: 5.000000e-04 eta: 7:22:39 time: 0.579544 data_time: 0.083388 memory: 3840 loss_kpt: 0.000802 acc_pose: 0.684285 loss: 0.000802 2022/09/26 23:58:14 - mmengine - INFO - Epoch(train) [32][150/293] lr: 5.000000e-04 eta: 7:22:33 time: 0.575978 data_time: 0.097680 memory: 3840 loss_kpt: 0.000781 acc_pose: 0.785618 loss: 0.000781 2022/09/26 23:58:41 - mmengine - INFO - Epoch(train) [32][200/293] lr: 5.000000e-04 eta: 7:22:20 time: 0.551963 data_time: 0.157257 memory: 3840 loss_kpt: 0.000780 acc_pose: 0.789702 loss: 0.000780 2022/09/26 23:59:10 - mmengine - INFO - Epoch(train) [32][250/293] lr: 5.000000e-04 eta: 7:22:12 time: 0.571090 data_time: 0.104095 memory: 3840 loss_kpt: 0.000792 acc_pose: 0.787163 loss: 0.000792 2022/09/26 23:59:34 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:00:06 - mmengine - INFO - Epoch(train) [33][50/293] lr: 5.000000e-04 eta: 7:19:56 time: 0.625444 data_time: 0.178700 memory: 3840 loss_kpt: 0.000785 acc_pose: 0.698842 loss: 0.000785 2022/09/27 00:00:34 - mmengine - INFO - Epoch(train) [33][100/293] lr: 5.000000e-04 eta: 7:19:47 time: 0.566096 data_time: 0.081937 memory: 3840 loss_kpt: 0.000794 acc_pose: 0.803471 loss: 0.000794 2022/09/27 00:01:03 - mmengine - INFO - Epoch(train) [33][150/293] lr: 5.000000e-04 eta: 7:19:42 time: 0.580771 data_time: 0.087930 memory: 3840 loss_kpt: 0.000784 acc_pose: 0.768342 loss: 0.000784 2022/09/27 00:01:33 - mmengine - INFO - Epoch(train) [33][200/293] lr: 5.000000e-04 eta: 7:19:42 time: 0.602129 data_time: 0.122082 memory: 3840 loss_kpt: 0.000785 acc_pose: 0.748203 loss: 0.000785 2022/09/27 00:02:02 - mmengine - INFO - Epoch(train) [33][250/293] lr: 5.000000e-04 eta: 7:19:37 time: 0.582033 data_time: 0.091526 memory: 3840 loss_kpt: 0.000798 acc_pose: 0.723626 loss: 0.000798 2022/09/27 00:02:26 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:02:56 - mmengine - INFO - Epoch(train) [34][50/293] lr: 5.000000e-04 eta: 7:17:16 time: 0.592091 data_time: 0.099536 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.819860 loss: 0.000791 2022/09/27 00:03:26 - mmengine - INFO - Epoch(train) [34][100/293] lr: 5.000000e-04 eta: 7:17:14 time: 0.594823 data_time: 0.083262 memory: 3840 loss_kpt: 0.000778 acc_pose: 0.738425 loss: 0.000778 2022/09/27 00:03:56 - mmengine - INFO - Epoch(train) [34][150/293] lr: 5.000000e-04 eta: 7:17:12 time: 0.595747 data_time: 0.080135 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.801872 loss: 0.000791 2022/09/27 00:04:24 - mmengine - INFO - Epoch(train) [34][200/293] lr: 5.000000e-04 eta: 7:17:00 time: 0.559364 data_time: 0.084632 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.816705 loss: 0.000791 2022/09/27 00:04:52 - mmengine - INFO - Epoch(train) [34][250/293] lr: 5.000000e-04 eta: 7:16:52 time: 0.573260 data_time: 0.087502 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.753842 loss: 0.000791 2022/09/27 00:05:17 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:05:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:05:48 - mmengine - INFO - Epoch(train) [35][50/293] lr: 5.000000e-04 eta: 7:14:39 time: 0.612890 data_time: 0.117788 memory: 3840 loss_kpt: 0.000802 acc_pose: 0.732419 loss: 0.000802 2022/09/27 00:06:17 - mmengine - INFO - Epoch(train) [35][100/293] lr: 5.000000e-04 eta: 7:14:33 time: 0.582074 data_time: 0.139095 memory: 3840 loss_kpt: 0.000792 acc_pose: 0.775328 loss: 0.000792 2022/09/27 00:06:47 - mmengine - INFO - Epoch(train) [35][150/293] lr: 5.000000e-04 eta: 7:14:31 time: 0.597583 data_time: 0.235367 memory: 3840 loss_kpt: 0.000774 acc_pose: 0.726523 loss: 0.000774 2022/09/27 00:07:15 - mmengine - INFO - Epoch(train) [35][200/293] lr: 5.000000e-04 eta: 7:14:21 time: 0.569258 data_time: 0.176150 memory: 3840 loss_kpt: 0.000776 acc_pose: 0.785440 loss: 0.000776 2022/09/27 00:07:44 - mmengine - INFO - Epoch(train) [35][250/293] lr: 5.000000e-04 eta: 7:14:14 time: 0.579828 data_time: 0.213043 memory: 3840 loss_kpt: 0.000780 acc_pose: 0.743738 loss: 0.000780 2022/09/27 00:08:08 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:08:38 - mmengine - INFO - Epoch(train) [36][50/293] lr: 5.000000e-04 eta: 7:12:01 time: 0.598922 data_time: 0.156804 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.775693 loss: 0.000791 2022/09/27 00:09:07 - mmengine - INFO - Epoch(train) [36][100/293] lr: 5.000000e-04 eta: 7:11:53 time: 0.576185 data_time: 0.107853 memory: 3840 loss_kpt: 0.000776 acc_pose: 0.808764 loss: 0.000776 2022/09/27 00:09:37 - mmengine - INFO - Epoch(train) [36][150/293] lr: 5.000000e-04 eta: 7:11:52 time: 0.603482 data_time: 0.160118 memory: 3840 loss_kpt: 0.000790 acc_pose: 0.754703 loss: 0.000790 2022/09/27 00:10:08 - mmengine - INFO - Epoch(train) [36][200/293] lr: 5.000000e-04 eta: 7:11:52 time: 0.612308 data_time: 0.099325 memory: 3840 loss_kpt: 0.000805 acc_pose: 0.790665 loss: 0.000805 2022/09/27 00:10:38 - mmengine - INFO - Epoch(train) [36][250/293] lr: 5.000000e-04 eta: 7:11:49 time: 0.599273 data_time: 0.152343 memory: 3840 loss_kpt: 0.000790 acc_pose: 0.771746 loss: 0.000790 2022/09/27 00:11:02 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:11:32 - mmengine - INFO - Epoch(train) [37][50/293] lr: 5.000000e-04 eta: 7:09:39 time: 0.598775 data_time: 0.150624 memory: 3840 loss_kpt: 0.000759 acc_pose: 0.809292 loss: 0.000759 2022/09/27 00:12:01 - mmengine - INFO - Epoch(train) [37][100/293] lr: 5.000000e-04 eta: 7:09:29 time: 0.571968 data_time: 0.086098 memory: 3840 loss_kpt: 0.000779 acc_pose: 0.768023 loss: 0.000779 2022/09/27 00:12:30 - mmengine - INFO - Epoch(train) [37][150/293] lr: 5.000000e-04 eta: 7:09:22 time: 0.581821 data_time: 0.097451 memory: 3840 loss_kpt: 0.000787 acc_pose: 0.763299 loss: 0.000787 2022/09/27 00:12:58 - mmengine - INFO - Epoch(train) [37][200/293] lr: 5.000000e-04 eta: 7:09:10 time: 0.565733 data_time: 0.122343 memory: 3840 loss_kpt: 0.000760 acc_pose: 0.799105 loss: 0.000760 2022/09/27 00:13:27 - mmengine - INFO - Epoch(train) [37][250/293] lr: 5.000000e-04 eta: 7:09:02 time: 0.580119 data_time: 0.082675 memory: 3840 loss_kpt: 0.000788 acc_pose: 0.726601 loss: 0.000788 2022/09/27 00:13:52 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:14:22 - mmengine - INFO - Epoch(train) [38][50/293] lr: 5.000000e-04 eta: 7:06:55 time: 0.598592 data_time: 0.148023 memory: 3840 loss_kpt: 0.000787 acc_pose: 0.780340 loss: 0.000787 2022/09/27 00:14:51 - mmengine - INFO - Epoch(train) [38][100/293] lr: 5.000000e-04 eta: 7:06:46 time: 0.578886 data_time: 0.238924 memory: 3840 loss_kpt: 0.000763 acc_pose: 0.782994 loss: 0.000763 2022/09/27 00:15:22 - mmengine - INFO - Epoch(train) [38][150/293] lr: 5.000000e-04 eta: 7:06:45 time: 0.611865 data_time: 0.147505 memory: 3840 loss_kpt: 0.000779 acc_pose: 0.736488 loss: 0.000779 2022/09/27 00:15:26 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:15:50 - mmengine - INFO - Epoch(train) [38][200/293] lr: 5.000000e-04 eta: 7:06:32 time: 0.559712 data_time: 0.086109 memory: 3840 loss_kpt: 0.000772 acc_pose: 0.781141 loss: 0.000772 2022/09/27 00:16:19 - mmengine - INFO - Epoch(train) [38][250/293] lr: 5.000000e-04 eta: 7:06:25 time: 0.587915 data_time: 0.106890 memory: 3840 loss_kpt: 0.000763 acc_pose: 0.792461 loss: 0.000763 2022/09/27 00:16:43 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:17:13 - mmengine - INFO - Epoch(train) [39][50/293] lr: 5.000000e-04 eta: 7:04:22 time: 0.607286 data_time: 0.133942 memory: 3840 loss_kpt: 0.000769 acc_pose: 0.753998 loss: 0.000769 2022/09/27 00:17:43 - mmengine - INFO - Epoch(train) [39][100/293] lr: 5.000000e-04 eta: 7:04:14 time: 0.582246 data_time: 0.088967 memory: 3840 loss_kpt: 0.000784 acc_pose: 0.779828 loss: 0.000784 2022/09/27 00:18:11 - mmengine - INFO - Epoch(train) [39][150/293] lr: 5.000000e-04 eta: 7:04:03 time: 0.570072 data_time: 0.097670 memory: 3840 loss_kpt: 0.000780 acc_pose: 0.719094 loss: 0.000780 2022/09/27 00:18:39 - mmengine - INFO - Epoch(train) [39][200/293] lr: 5.000000e-04 eta: 7:03:51 time: 0.565391 data_time: 0.085332 memory: 3840 loss_kpt: 0.000787 acc_pose: 0.763385 loss: 0.000787 2022/09/27 00:19:08 - mmengine - INFO - Epoch(train) [39][250/293] lr: 5.000000e-04 eta: 7:03:42 time: 0.579404 data_time: 0.160015 memory: 3840 loss_kpt: 0.000766 acc_pose: 0.800495 loss: 0.000766 2022/09/27 00:19:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:20:04 - mmengine - INFO - Epoch(train) [40][50/293] lr: 5.000000e-04 eta: 7:01:44 time: 0.620024 data_time: 0.108348 memory: 3840 loss_kpt: 0.000782 acc_pose: 0.799901 loss: 0.000782 2022/09/27 00:20:33 - mmengine - INFO - Epoch(train) [40][100/293] lr: 5.000000e-04 eta: 7:01:35 time: 0.581237 data_time: 0.097635 memory: 3840 loss_kpt: 0.000784 acc_pose: 0.743671 loss: 0.000784 2022/09/27 00:21:01 - mmengine - INFO - Epoch(train) [40][150/293] lr: 5.000000e-04 eta: 7:01:22 time: 0.560023 data_time: 0.079551 memory: 3840 loss_kpt: 0.000791 acc_pose: 0.760900 loss: 0.000791 2022/09/27 00:21:30 - mmengine - INFO - Epoch(train) [40][200/293] lr: 5.000000e-04 eta: 7:01:14 time: 0.590348 data_time: 0.120220 memory: 3840 loss_kpt: 0.000768 acc_pose: 0.741501 loss: 0.000768 2022/09/27 00:21:59 - mmengine - INFO - Epoch(train) [40][250/293] lr: 5.000000e-04 eta: 7:01:06 time: 0.585390 data_time: 0.153045 memory: 3840 loss_kpt: 0.000770 acc_pose: 0.787806 loss: 0.000770 2022/09/27 00:22:26 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:22:26 - mmengine - INFO - Saving checkpoint at 40 epochs 2022/09/27 00:22:47 - mmengine - INFO - Epoch(val) [40][50/407] eta: 0:01:53 time: 0.317261 data_time: 0.174929 memory: 3840 2022/09/27 00:23:03 - mmengine - INFO - Epoch(val) [40][100/407] eta: 0:01:33 time: 0.304984 data_time: 0.165050 memory: 1017 2022/09/27 00:23:18 - mmengine - INFO - Epoch(val) [40][150/407] eta: 0:01:21 time: 0.315198 data_time: 0.175622 memory: 1017 2022/09/27 00:23:34 - mmengine - INFO - Epoch(val) [40][200/407] eta: 0:01:05 time: 0.318806 data_time: 0.175745 memory: 1017 2022/09/27 00:23:49 - mmengine - INFO - Epoch(val) [40][250/407] eta: 0:00:45 time: 0.289777 data_time: 0.157452 memory: 1017 2022/09/27 00:24:05 - mmengine - INFO - Epoch(val) [40][300/407] eta: 0:00:35 time: 0.330370 data_time: 0.196526 memory: 1017 2022/09/27 00:24:21 - mmengine - INFO - Epoch(val) [40][350/407] eta: 0:00:18 time: 0.319669 data_time: 0.196952 memory: 1017 2022/09/27 00:24:34 - mmengine - INFO - Epoch(val) [40][400/407] eta: 0:00:01 time: 0.245151 data_time: 0.152551 memory: 1017 2022/09/27 00:25:07 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 00:25:20 - mmengine - INFO - Epoch(val) [40][407/407] coco/AP: 0.658549 coco/AP .5: 0.873688 coco/AP .75: 0.734402 coco/AP (M): 0.624352 coco/AP (L): 0.721831 coco/AR: 0.718278 coco/AR .5: 0.913728 coco/AR .75: 0.787626 coco/AR (M): 0.675061 coco/AR (L): 0.779970 2022/09/27 00:25:21 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_30.pth is removed 2022/09/27 00:25:23 - mmengine - INFO - The best checkpoint with 0.6585 coco/AP at 40 epoch is saved to best_coco/AP_epoch_40.pth. 2022/09/27 00:25:52 - mmengine - INFO - Epoch(train) [41][50/293] lr: 5.000000e-04 eta: 6:59:03 time: 0.584419 data_time: 0.156348 memory: 3840 loss_kpt: 0.000759 acc_pose: 0.751374 loss: 0.000759 2022/09/27 00:26:21 - mmengine - INFO - Epoch(train) [41][100/293] lr: 5.000000e-04 eta: 6:58:53 time: 0.577458 data_time: 0.080340 memory: 3840 loss_kpt: 0.000772 acc_pose: 0.818084 loss: 0.000772 2022/09/27 00:26:50 - mmengine - INFO - Epoch(train) [41][150/293] lr: 5.000000e-04 eta: 6:58:44 time: 0.583373 data_time: 0.151138 memory: 3840 loss_kpt: 0.000767 acc_pose: 0.756256 loss: 0.000767 2022/09/27 00:27:19 - mmengine - INFO - Epoch(train) [41][200/293] lr: 5.000000e-04 eta: 6:58:36 time: 0.588107 data_time: 0.113543 memory: 3840 loss_kpt: 0.000778 acc_pose: 0.812193 loss: 0.000778 2022/09/27 00:27:48 - mmengine - INFO - Epoch(train) [41][250/293] lr: 5.000000e-04 eta: 6:58:26 time: 0.581538 data_time: 0.085694 memory: 3840 loss_kpt: 0.000769 acc_pose: 0.779867 loss: 0.000769 2022/09/27 00:28:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:28:13 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:28:45 - mmengine - INFO - Epoch(train) [42][50/293] lr: 5.000000e-04 eta: 6:56:36 time: 0.636078 data_time: 0.109988 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.765806 loss: 0.000761 2022/09/27 00:29:13 - mmengine - INFO - Epoch(train) [42][100/293] lr: 5.000000e-04 eta: 6:56:21 time: 0.553207 data_time: 0.097857 memory: 3840 loss_kpt: 0.000778 acc_pose: 0.763567 loss: 0.000778 2022/09/27 00:29:41 - mmengine - INFO - Epoch(train) [42][150/293] lr: 5.000000e-04 eta: 6:56:09 time: 0.574403 data_time: 0.111343 memory: 3840 loss_kpt: 0.000771 acc_pose: 0.804700 loss: 0.000771 2022/09/27 00:30:10 - mmengine - INFO - Epoch(train) [42][200/293] lr: 5.000000e-04 eta: 6:55:58 time: 0.574584 data_time: 0.087014 memory: 3840 loss_kpt: 0.000773 acc_pose: 0.743214 loss: 0.000773 2022/09/27 00:30:38 - mmengine - INFO - Epoch(train) [42][250/293] lr: 5.000000e-04 eta: 6:55:44 time: 0.562815 data_time: 0.076825 memory: 3840 loss_kpt: 0.000785 acc_pose: 0.752745 loss: 0.000785 2022/09/27 00:31:03 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:31:35 - mmengine - INFO - Epoch(train) [43][50/293] lr: 5.000000e-04 eta: 6:53:54 time: 0.626207 data_time: 0.115630 memory: 3840 loss_kpt: 0.000746 acc_pose: 0.771849 loss: 0.000746 2022/09/27 00:32:03 - mmengine - INFO - Epoch(train) [43][100/293] lr: 5.000000e-04 eta: 6:53:41 time: 0.566483 data_time: 0.084120 memory: 3840 loss_kpt: 0.000765 acc_pose: 0.778402 loss: 0.000765 2022/09/27 00:32:32 - mmengine - INFO - Epoch(train) [43][150/293] lr: 5.000000e-04 eta: 6:53:30 time: 0.575577 data_time: 0.092098 memory: 3840 loss_kpt: 0.000775 acc_pose: 0.786423 loss: 0.000775 2022/09/27 00:33:01 - mmengine - INFO - Epoch(train) [43][200/293] lr: 5.000000e-04 eta: 6:53:18 time: 0.574936 data_time: 0.148579 memory: 3840 loss_kpt: 0.000782 acc_pose: 0.735639 loss: 0.000782 2022/09/27 00:33:29 - mmengine - INFO - Epoch(train) [43][250/293] lr: 5.000000e-04 eta: 6:53:05 time: 0.570364 data_time: 0.087877 memory: 3840 loss_kpt: 0.000771 acc_pose: 0.761798 loss: 0.000771 2022/09/27 00:33:54 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:34:23 - mmengine - INFO - Epoch(train) [44][50/293] lr: 5.000000e-04 eta: 6:51:11 time: 0.593644 data_time: 0.093616 memory: 3840 loss_kpt: 0.000784 acc_pose: 0.830800 loss: 0.000784 2022/09/27 00:34:52 - mmengine - INFO - Epoch(train) [44][100/293] lr: 5.000000e-04 eta: 6:50:57 time: 0.563721 data_time: 0.115867 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.762230 loss: 0.000761 2022/09/27 00:35:21 - mmengine - INFO - Epoch(train) [44][150/293] lr: 5.000000e-04 eta: 6:50:49 time: 0.596507 data_time: 0.081884 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.781289 loss: 0.000761 2022/09/27 00:35:51 - mmengine - INFO - Epoch(train) [44][200/293] lr: 5.000000e-04 eta: 6:50:41 time: 0.592899 data_time: 0.142401 memory: 3840 loss_kpt: 0.000758 acc_pose: 0.773292 loss: 0.000758 2022/09/27 00:36:20 - mmengine - INFO - Epoch(train) [44][250/293] lr: 5.000000e-04 eta: 6:50:31 time: 0.586425 data_time: 0.136099 memory: 3840 loss_kpt: 0.000781 acc_pose: 0.775545 loss: 0.000781 2022/09/27 00:36:45 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:37:15 - mmengine - INFO - Epoch(train) [45][50/293] lr: 5.000000e-04 eta: 6:48:41 time: 0.605165 data_time: 0.130036 memory: 3840 loss_kpt: 0.000767 acc_pose: 0.734964 loss: 0.000767 2022/09/27 00:37:45 - mmengine - INFO - Epoch(train) [45][100/293] lr: 5.000000e-04 eta: 6:48:32 time: 0.593699 data_time: 0.102702 memory: 3840 loss_kpt: 0.000743 acc_pose: 0.802998 loss: 0.000743 2022/09/27 00:37:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:38:13 - mmengine - INFO - Epoch(train) [45][150/293] lr: 5.000000e-04 eta: 6:48:20 time: 0.575978 data_time: 0.155079 memory: 3840 loss_kpt: 0.000777 acc_pose: 0.749052 loss: 0.000777 2022/09/27 00:38:44 - mmengine - INFO - Epoch(train) [45][200/293] lr: 5.000000e-04 eta: 6:48:13 time: 0.604622 data_time: 0.088955 memory: 3840 loss_kpt: 0.000788 acc_pose: 0.735501 loss: 0.000788 2022/09/27 00:39:12 - mmengine - INFO - Epoch(train) [45][250/293] lr: 5.000000e-04 eta: 6:48:00 time: 0.570860 data_time: 0.089030 memory: 3840 loss_kpt: 0.000766 acc_pose: 0.818427 loss: 0.000766 2022/09/27 00:39:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:40:06 - mmengine - INFO - Epoch(train) [46][50/293] lr: 5.000000e-04 eta: 6:46:06 time: 0.577189 data_time: 0.103816 memory: 3840 loss_kpt: 0.000749 acc_pose: 0.803049 loss: 0.000749 2022/09/27 00:40:33 - mmengine - INFO - Epoch(train) [46][100/293] lr: 5.000000e-04 eta: 6:45:50 time: 0.552227 data_time: 0.084817 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.777580 loss: 0.000761 2022/09/27 00:41:02 - mmengine - INFO - Epoch(train) [46][150/293] lr: 5.000000e-04 eta: 6:45:36 time: 0.568531 data_time: 0.096924 memory: 3840 loss_kpt: 0.000755 acc_pose: 0.758711 loss: 0.000755 2022/09/27 00:41:31 - mmengine - INFO - Epoch(train) [46][200/293] lr: 5.000000e-04 eta: 6:45:23 time: 0.573572 data_time: 0.081073 memory: 3840 loss_kpt: 0.000758 acc_pose: 0.813692 loss: 0.000758 2022/09/27 00:41:59 - mmengine - INFO - Epoch(train) [46][250/293] lr: 5.000000e-04 eta: 6:45:09 time: 0.566266 data_time: 0.080872 memory: 3840 loss_kpt: 0.000764 acc_pose: 0.781389 loss: 0.000764 2022/09/27 00:42:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:42:51 - mmengine - INFO - Epoch(train) [47][50/293] lr: 5.000000e-04 eta: 6:43:17 time: 0.572605 data_time: 0.093255 memory: 3840 loss_kpt: 0.000779 acc_pose: 0.749340 loss: 0.000779 2022/09/27 00:43:19 - mmengine - INFO - Epoch(train) [47][100/293] lr: 5.000000e-04 eta: 6:43:03 time: 0.572119 data_time: 0.088136 memory: 3840 loss_kpt: 0.000774 acc_pose: 0.762003 loss: 0.000774 2022/09/27 00:43:47 - mmengine - INFO - Epoch(train) [47][150/293] lr: 5.000000e-04 eta: 6:42:48 time: 0.558104 data_time: 0.087854 memory: 3840 loss_kpt: 0.000767 acc_pose: 0.743216 loss: 0.000767 2022/09/27 00:44:15 - mmengine - INFO - Epoch(train) [47][200/293] lr: 5.000000e-04 eta: 6:42:33 time: 0.561913 data_time: 0.111497 memory: 3840 loss_kpt: 0.000752 acc_pose: 0.754537 loss: 0.000752 2022/09/27 00:44:44 - mmengine - INFO - Epoch(train) [47][250/293] lr: 5.000000e-04 eta: 6:42:18 time: 0.563957 data_time: 0.105575 memory: 3840 loss_kpt: 0.000741 acc_pose: 0.764320 loss: 0.000741 2022/09/27 00:45:08 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:45:38 - mmengine - INFO - Epoch(train) [48][50/293] lr: 5.000000e-04 eta: 6:40:32 time: 0.598963 data_time: 0.207487 memory: 3840 loss_kpt: 0.000750 acc_pose: 0.780924 loss: 0.000750 2022/09/27 00:46:05 - mmengine - INFO - Epoch(train) [48][100/293] lr: 5.000000e-04 eta: 6:40:14 time: 0.546334 data_time: 0.204812 memory: 3840 loss_kpt: 0.000742 acc_pose: 0.785074 loss: 0.000742 2022/09/27 00:46:32 - mmengine - INFO - Epoch(train) [48][150/293] lr: 5.000000e-04 eta: 6:39:55 time: 0.541758 data_time: 0.161054 memory: 3840 loss_kpt: 0.000758 acc_pose: 0.782639 loss: 0.000758 2022/09/27 00:46:59 - mmengine - INFO - Epoch(train) [48][200/293] lr: 5.000000e-04 eta: 6:39:37 time: 0.545846 data_time: 0.102147 memory: 3840 loss_kpt: 0.000746 acc_pose: 0.701142 loss: 0.000746 2022/09/27 00:47:16 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:47:27 - mmengine - INFO - Epoch(train) [48][250/293] lr: 5.000000e-04 eta: 6:39:21 time: 0.554713 data_time: 0.099306 memory: 3840 loss_kpt: 0.000742 acc_pose: 0.789225 loss: 0.000742 2022/09/27 00:47:51 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:48:21 - mmengine - INFO - Epoch(train) [49][50/293] lr: 5.000000e-04 eta: 6:37:36 time: 0.597544 data_time: 0.144116 memory: 3840 loss_kpt: 0.000768 acc_pose: 0.777158 loss: 0.000768 2022/09/27 00:48:50 - mmengine - INFO - Epoch(train) [49][100/293] lr: 5.000000e-04 eta: 6:37:22 time: 0.565524 data_time: 0.100015 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.733500 loss: 0.000761 2022/09/27 00:49:18 - mmengine - INFO - Epoch(train) [49][150/293] lr: 5.000000e-04 eta: 6:37:06 time: 0.557884 data_time: 0.072020 memory: 3840 loss_kpt: 0.000748 acc_pose: 0.760696 loss: 0.000748 2022/09/27 00:49:46 - mmengine - INFO - Epoch(train) [49][200/293] lr: 5.000000e-04 eta: 6:36:51 time: 0.567104 data_time: 0.112428 memory: 3840 loss_kpt: 0.000753 acc_pose: 0.786058 loss: 0.000753 2022/09/27 00:50:14 - mmengine - INFO - Epoch(train) [49][250/293] lr: 5.000000e-04 eta: 6:36:37 time: 0.572421 data_time: 0.076728 memory: 3840 loss_kpt: 0.000765 acc_pose: 0.792096 loss: 0.000765 2022/09/27 00:50:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:51:07 - mmengine - INFO - Epoch(train) [50][50/293] lr: 5.000000e-04 eta: 6:34:54 time: 0.593752 data_time: 0.101912 memory: 3840 loss_kpt: 0.000735 acc_pose: 0.789937 loss: 0.000735 2022/09/27 00:51:34 - mmengine - INFO - Epoch(train) [50][100/293] lr: 5.000000e-04 eta: 6:34:35 time: 0.540812 data_time: 0.080804 memory: 3840 loss_kpt: 0.000763 acc_pose: 0.783750 loss: 0.000763 2022/09/27 00:52:03 - mmengine - INFO - Epoch(train) [50][150/293] lr: 5.000000e-04 eta: 6:34:22 time: 0.575534 data_time: 0.095782 memory: 3840 loss_kpt: 0.000763 acc_pose: 0.772841 loss: 0.000763 2022/09/27 00:52:31 - mmengine - INFO - Epoch(train) [50][200/293] lr: 5.000000e-04 eta: 6:34:07 time: 0.566141 data_time: 0.099860 memory: 3840 loss_kpt: 0.000757 acc_pose: 0.757184 loss: 0.000757 2022/09/27 00:53:00 - mmengine - INFO - Epoch(train) [50][250/293] lr: 5.000000e-04 eta: 6:33:52 time: 0.567802 data_time: 0.148840 memory: 3840 loss_kpt: 0.000747 acc_pose: 0.794896 loss: 0.000747 2022/09/27 00:53:23 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:53:23 - mmengine - INFO - Saving checkpoint at 50 epochs 2022/09/27 00:53:46 - mmengine - INFO - Epoch(val) [50][50/407] eta: 0:01:57 time: 0.329356 data_time: 0.179804 memory: 3840 2022/09/27 00:54:01 - mmengine - INFO - Epoch(val) [50][100/407] eta: 0:01:34 time: 0.306955 data_time: 0.171750 memory: 1017 2022/09/27 00:54:16 - mmengine - INFO - Epoch(val) [50][150/407] eta: 0:01:16 time: 0.297481 data_time: 0.150024 memory: 1017 2022/09/27 00:54:32 - mmengine - INFO - Epoch(val) [50][200/407] eta: 0:01:04 time: 0.310520 data_time: 0.169837 memory: 1017 2022/09/27 00:54:47 - mmengine - INFO - Epoch(val) [50][250/407] eta: 0:00:47 time: 0.305138 data_time: 0.164724 memory: 1017 2022/09/27 00:55:03 - mmengine - INFO - Epoch(val) [50][300/407] eta: 0:00:33 time: 0.314908 data_time: 0.180930 memory: 1017 2022/09/27 00:55:18 - mmengine - INFO - Epoch(val) [50][350/407] eta: 0:00:17 time: 0.308254 data_time: 0.156459 memory: 1017 2022/09/27 00:55:30 - mmengine - INFO - Epoch(val) [50][400/407] eta: 0:00:01 time: 0.241681 data_time: 0.129805 memory: 1017 2022/09/27 00:56:04 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 00:56:18 - mmengine - INFO - Epoch(val) [50][407/407] coco/AP: 0.669224 coco/AP .5: 0.877466 coco/AP .75: 0.742536 coco/AP (M): 0.631552 coco/AP (L): 0.735119 coco/AR: 0.727739 coco/AR .5: 0.917979 coco/AR .75: 0.795497 coco/AR (M): 0.683584 coco/AR (L): 0.790301 2022/09/27 00:56:18 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_40.pth is removed 2022/09/27 00:56:20 - mmengine - INFO - The best checkpoint with 0.6692 coco/AP at 50 epoch is saved to best_coco/AP_epoch_50.pth. 2022/09/27 00:56:48 - mmengine - INFO - Epoch(train) [51][50/293] lr: 5.000000e-04 eta: 6:32:06 time: 0.566215 data_time: 0.195666 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.771860 loss: 0.000761 2022/09/27 00:57:17 - mmengine - INFO - Epoch(train) [51][100/293] lr: 5.000000e-04 eta: 6:31:51 time: 0.566122 data_time: 0.143276 memory: 3840 loss_kpt: 0.000762 acc_pose: 0.759814 loss: 0.000762 2022/09/27 00:57:46 - mmengine - INFO - Epoch(train) [51][150/293] lr: 5.000000e-04 eta: 6:31:38 time: 0.579406 data_time: 0.069251 memory: 3840 loss_kpt: 0.000762 acc_pose: 0.787023 loss: 0.000762 2022/09/27 00:58:14 - mmengine - INFO - Epoch(train) [51][200/293] lr: 5.000000e-04 eta: 6:31:22 time: 0.561614 data_time: 0.086941 memory: 3840 loss_kpt: 0.000752 acc_pose: 0.783144 loss: 0.000752 2022/09/27 00:58:42 - mmengine - INFO - Epoch(train) [51][250/293] lr: 5.000000e-04 eta: 6:31:06 time: 0.559437 data_time: 0.081011 memory: 3840 loss_kpt: 0.000757 acc_pose: 0.807087 loss: 0.000757 2022/09/27 00:59:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 00:59:35 - mmengine - INFO - Epoch(train) [52][50/293] lr: 5.000000e-04 eta: 6:29:24 time: 0.584102 data_time: 0.101104 memory: 3840 loss_kpt: 0.000734 acc_pose: 0.824707 loss: 0.000734 2022/09/27 00:59:39 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:00:03 - mmengine - INFO - Epoch(train) [52][100/293] lr: 5.000000e-04 eta: 6:29:09 time: 0.563209 data_time: 0.075125 memory: 3840 loss_kpt: 0.000753 acc_pose: 0.743475 loss: 0.000753 2022/09/27 01:00:32 - mmengine - INFO - Epoch(train) [52][150/293] lr: 5.000000e-04 eta: 6:28:55 time: 0.574610 data_time: 0.096038 memory: 3840 loss_kpt: 0.000743 acc_pose: 0.784614 loss: 0.000743 2022/09/27 01:01:00 - mmengine - INFO - Epoch(train) [52][200/293] lr: 5.000000e-04 eta: 6:28:40 time: 0.573229 data_time: 0.079023 memory: 3840 loss_kpt: 0.000751 acc_pose: 0.814155 loss: 0.000751 2022/09/27 01:01:29 - mmengine - INFO - Epoch(train) [52][250/293] lr: 5.000000e-04 eta: 6:28:26 time: 0.574779 data_time: 0.111530 memory: 3840 loss_kpt: 0.000756 acc_pose: 0.789542 loss: 0.000756 2022/09/27 01:01:54 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:02:23 - mmengine - INFO - Epoch(train) [53][50/293] lr: 5.000000e-04 eta: 6:26:46 time: 0.580029 data_time: 0.108092 memory: 3840 loss_kpt: 0.000747 acc_pose: 0.757734 loss: 0.000747 2022/09/27 01:02:51 - mmengine - INFO - Epoch(train) [53][100/293] lr: 5.000000e-04 eta: 6:26:31 time: 0.570981 data_time: 0.127186 memory: 3840 loss_kpt: 0.000771 acc_pose: 0.783603 loss: 0.000771 2022/09/27 01:03:19 - mmengine - INFO - Epoch(train) [53][150/293] lr: 5.000000e-04 eta: 6:26:13 time: 0.547909 data_time: 0.192989 memory: 3840 loss_kpt: 0.000758 acc_pose: 0.796473 loss: 0.000758 2022/09/27 01:03:46 - mmengine - INFO - Epoch(train) [53][200/293] lr: 5.000000e-04 eta: 6:25:55 time: 0.550257 data_time: 0.114486 memory: 3840 loss_kpt: 0.000750 acc_pose: 0.789872 loss: 0.000750 2022/09/27 01:04:15 - mmengine - INFO - Epoch(train) [53][250/293] lr: 5.000000e-04 eta: 6:25:40 time: 0.573797 data_time: 0.075888 memory: 3840 loss_kpt: 0.000758 acc_pose: 0.743842 loss: 0.000758 2022/09/27 01:04:38 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:05:06 - mmengine - INFO - Epoch(train) [54][50/293] lr: 5.000000e-04 eta: 6:24:00 time: 0.569242 data_time: 0.090855 memory: 3840 loss_kpt: 0.000750 acc_pose: 0.799848 loss: 0.000750 2022/09/27 01:05:34 - mmengine - INFO - Epoch(train) [54][100/293] lr: 5.000000e-04 eta: 6:23:43 time: 0.558591 data_time: 0.081119 memory: 3840 loss_kpt: 0.000752 acc_pose: 0.735824 loss: 0.000752 2022/09/27 01:06:02 - mmengine - INFO - Epoch(train) [54][150/293] lr: 5.000000e-04 eta: 6:23:27 time: 0.561984 data_time: 0.076982 memory: 3840 loss_kpt: 0.000738 acc_pose: 0.787666 loss: 0.000738 2022/09/27 01:06:30 - mmengine - INFO - Epoch(train) [54][200/293] lr: 5.000000e-04 eta: 6:23:09 time: 0.551716 data_time: 0.090361 memory: 3840 loss_kpt: 0.000750 acc_pose: 0.768797 loss: 0.000750 2022/09/27 01:06:58 - mmengine - INFO - Epoch(train) [54][250/293] lr: 5.000000e-04 eta: 6:22:53 time: 0.563916 data_time: 0.081904 memory: 3840 loss_kpt: 0.000737 acc_pose: 0.809925 loss: 0.000737 2022/09/27 01:07:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:07:51 - mmengine - INFO - Epoch(train) [55][50/293] lr: 5.000000e-04 eta: 6:21:15 time: 0.578619 data_time: 0.095871 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.724567 loss: 0.000719 2022/09/27 01:08:19 - mmengine - INFO - Epoch(train) [55][100/293] lr: 5.000000e-04 eta: 6:20:59 time: 0.565011 data_time: 0.074683 memory: 3840 loss_kpt: 0.000738 acc_pose: 0.728727 loss: 0.000738 2022/09/27 01:08:47 - mmengine - INFO - Epoch(train) [55][150/293] lr: 5.000000e-04 eta: 6:20:44 time: 0.567830 data_time: 0.081894 memory: 3840 loss_kpt: 0.000754 acc_pose: 0.806017 loss: 0.000754 2022/09/27 01:09:03 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:09:16 - mmengine - INFO - Epoch(train) [55][200/293] lr: 5.000000e-04 eta: 6:20:28 time: 0.564917 data_time: 0.078996 memory: 3840 loss_kpt: 0.000746 acc_pose: 0.756172 loss: 0.000746 2022/09/27 01:09:46 - mmengine - INFO - Epoch(train) [55][250/293] lr: 5.000000e-04 eta: 6:20:17 time: 0.603557 data_time: 0.111343 memory: 3840 loss_kpt: 0.000737 acc_pose: 0.769085 loss: 0.000737 2022/09/27 01:10:10 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:10:39 - mmengine - INFO - Epoch(train) [56][50/293] lr: 5.000000e-04 eta: 6:18:41 time: 0.579458 data_time: 0.092856 memory: 3840 loss_kpt: 0.000736 acc_pose: 0.803586 loss: 0.000736 2022/09/27 01:11:07 - mmengine - INFO - Epoch(train) [56][100/293] lr: 5.000000e-04 eta: 6:18:26 time: 0.572917 data_time: 0.079960 memory: 3840 loss_kpt: 0.000742 acc_pose: 0.800836 loss: 0.000742 2022/09/27 01:11:36 - mmengine - INFO - Epoch(train) [56][150/293] lr: 5.000000e-04 eta: 6:18:09 time: 0.564509 data_time: 0.082528 memory: 3840 loss_kpt: 0.000736 acc_pose: 0.841510 loss: 0.000736 2022/09/27 01:12:06 - mmengine - INFO - Epoch(train) [56][200/293] lr: 5.000000e-04 eta: 6:17:59 time: 0.611030 data_time: 0.081158 memory: 3840 loss_kpt: 0.000742 acc_pose: 0.837244 loss: 0.000742 2022/09/27 01:12:34 - mmengine - INFO - Epoch(train) [56][250/293] lr: 5.000000e-04 eta: 6:17:42 time: 0.559023 data_time: 0.121789 memory: 3840 loss_kpt: 0.000747 acc_pose: 0.810503 loss: 0.000747 2022/09/27 01:12:58 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:13:29 - mmengine - INFO - Epoch(train) [57][50/293] lr: 5.000000e-04 eta: 6:16:11 time: 0.607872 data_time: 0.098514 memory: 3840 loss_kpt: 0.000743 acc_pose: 0.756327 loss: 0.000743 2022/09/27 01:13:57 - mmengine - INFO - Epoch(train) [57][100/293] lr: 5.000000e-04 eta: 6:15:53 time: 0.554074 data_time: 0.131633 memory: 3840 loss_kpt: 0.000746 acc_pose: 0.760419 loss: 0.000746 2022/09/27 01:14:26 - mmengine - INFO - Epoch(train) [57][150/293] lr: 5.000000e-04 eta: 6:15:40 time: 0.588976 data_time: 0.101676 memory: 3840 loss_kpt: 0.000744 acc_pose: 0.781896 loss: 0.000744 2022/09/27 01:14:54 - mmengine - INFO - Epoch(train) [57][200/293] lr: 5.000000e-04 eta: 6:15:22 time: 0.553314 data_time: 0.081996 memory: 3840 loss_kpt: 0.000745 acc_pose: 0.791686 loss: 0.000745 2022/09/27 01:15:22 - mmengine - INFO - Epoch(train) [57][250/293] lr: 5.000000e-04 eta: 6:15:05 time: 0.559445 data_time: 0.086338 memory: 3840 loss_kpt: 0.000737 acc_pose: 0.746324 loss: 0.000737 2022/09/27 01:15:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:16:16 - mmengine - INFO - Epoch(train) [58][50/293] lr: 5.000000e-04 eta: 6:13:33 time: 0.591535 data_time: 0.189848 memory: 3840 loss_kpt: 0.000751 acc_pose: 0.782482 loss: 0.000751 2022/09/27 01:16:45 - mmengine - INFO - Epoch(train) [58][100/293] lr: 5.000000e-04 eta: 6:13:17 time: 0.572468 data_time: 0.105613 memory: 3840 loss_kpt: 0.000761 acc_pose: 0.768397 loss: 0.000761 2022/09/27 01:17:13 - mmengine - INFO - Epoch(train) [58][150/293] lr: 5.000000e-04 eta: 6:13:01 time: 0.569471 data_time: 0.122595 memory: 3840 loss_kpt: 0.000738 acc_pose: 0.750490 loss: 0.000738 2022/09/27 01:17:41 - mmengine - INFO - Epoch(train) [58][200/293] lr: 5.000000e-04 eta: 6:12:44 time: 0.558514 data_time: 0.089499 memory: 3840 loss_kpt: 0.000733 acc_pose: 0.786306 loss: 0.000733 2022/09/27 01:18:09 - mmengine - INFO - Epoch(train) [58][250/293] lr: 5.000000e-04 eta: 6:12:27 time: 0.563085 data_time: 0.108416 memory: 3840 loss_kpt: 0.000755 acc_pose: 0.782250 loss: 0.000755 2022/09/27 01:18:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:18:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:19:04 - mmengine - INFO - Epoch(train) [59][50/293] lr: 5.000000e-04 eta: 6:10:59 time: 0.614141 data_time: 0.115039 memory: 3840 loss_kpt: 0.000731 acc_pose: 0.782228 loss: 0.000731 2022/09/27 01:19:32 - mmengine - INFO - Epoch(train) [59][100/293] lr: 5.000000e-04 eta: 6:10:43 time: 0.567636 data_time: 0.077090 memory: 3840 loss_kpt: 0.000743 acc_pose: 0.747805 loss: 0.000743 2022/09/27 01:20:02 - mmengine - INFO - Epoch(train) [59][150/293] lr: 5.000000e-04 eta: 6:10:30 time: 0.599109 data_time: 0.099194 memory: 3840 loss_kpt: 0.000739 acc_pose: 0.781062 loss: 0.000739 2022/09/27 01:20:29 - mmengine - INFO - Epoch(train) [59][200/293] lr: 5.000000e-04 eta: 6:10:11 time: 0.546288 data_time: 0.178884 memory: 3840 loss_kpt: 0.000741 acc_pose: 0.721969 loss: 0.000741 2022/09/27 01:20:58 - mmengine - INFO - Epoch(train) [59][250/293] lr: 5.000000e-04 eta: 6:09:56 time: 0.578208 data_time: 0.068678 memory: 3840 loss_kpt: 0.000742 acc_pose: 0.786540 loss: 0.000742 2022/09/27 01:21:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:21:51 - mmengine - INFO - Epoch(train) [60][50/293] lr: 5.000000e-04 eta: 6:08:25 time: 0.584461 data_time: 0.093402 memory: 3840 loss_kpt: 0.000729 acc_pose: 0.771883 loss: 0.000729 2022/09/27 01:22:19 - mmengine - INFO - Epoch(train) [60][100/293] lr: 5.000000e-04 eta: 6:08:08 time: 0.566016 data_time: 0.097984 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.736322 loss: 0.000730 2022/09/27 01:22:48 - mmengine - INFO - Epoch(train) [60][150/293] lr: 5.000000e-04 eta: 6:07:52 time: 0.565142 data_time: 0.094424 memory: 3840 loss_kpt: 0.000735 acc_pose: 0.811581 loss: 0.000735 2022/09/27 01:23:16 - mmengine - INFO - Epoch(train) [60][200/293] lr: 5.000000e-04 eta: 6:07:34 time: 0.559644 data_time: 0.082841 memory: 3840 loss_kpt: 0.000747 acc_pose: 0.764673 loss: 0.000747 2022/09/27 01:23:43 - mmengine - INFO - Epoch(train) [60][250/293] lr: 5.000000e-04 eta: 6:07:15 time: 0.550852 data_time: 0.077092 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.740743 loss: 0.000730 2022/09/27 01:24:07 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:24:07 - mmengine - INFO - Saving checkpoint at 60 epochs 2022/09/27 01:24:29 - mmengine - INFO - Epoch(val) [60][50/407] eta: 0:01:53 time: 0.317775 data_time: 0.201663 memory: 3840 2022/09/27 01:24:44 - mmengine - INFO - Epoch(val) [60][100/407] eta: 0:01:29 time: 0.292860 data_time: 0.157740 memory: 1017 2022/09/27 01:24:58 - mmengine - INFO - Epoch(val) [60][150/407] eta: 0:01:15 time: 0.292314 data_time: 0.179154 memory: 1017 2022/09/27 01:25:14 - mmengine - INFO - Epoch(val) [60][200/407] eta: 0:01:04 time: 0.311071 data_time: 0.184067 memory: 1017 2022/09/27 01:25:30 - mmengine - INFO - Epoch(val) [60][250/407] eta: 0:00:48 time: 0.312072 data_time: 0.183981 memory: 1017 2022/09/27 01:25:45 - mmengine - INFO - Epoch(val) [60][300/407] eta: 0:00:32 time: 0.301055 data_time: 0.178834 memory: 1017 2022/09/27 01:26:01 - mmengine - INFO - Epoch(val) [60][350/407] eta: 0:00:18 time: 0.325309 data_time: 0.193516 memory: 1017 2022/09/27 01:26:13 - mmengine - INFO - Epoch(val) [60][400/407] eta: 0:00:01 time: 0.244158 data_time: 0.145397 memory: 1017 2022/09/27 01:26:47 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 01:27:00 - mmengine - INFO - Epoch(val) [60][407/407] coco/AP: 0.677274 coco/AP .5: 0.883325 coco/AP .75: 0.755438 coco/AP (M): 0.644687 coco/AP (L): 0.740687 coco/AR: 0.736634 coco/AR .5: 0.925220 coco/AR .75: 0.808249 coco/AR (M): 0.694947 coco/AR (L): 0.796284 2022/09/27 01:27:01 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_50.pth is removed 2022/09/27 01:27:03 - mmengine - INFO - The best checkpoint with 0.6773 coco/AP at 60 epoch is saved to best_coco/AP_epoch_60.pth. 2022/09/27 01:27:30 - mmengine - INFO - Epoch(train) [61][50/293] lr: 5.000000e-04 eta: 6:05:41 time: 0.548449 data_time: 0.103748 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.749998 loss: 0.000730 2022/09/27 01:27:59 - mmengine - INFO - Epoch(train) [61][100/293] lr: 5.000000e-04 eta: 6:05:24 time: 0.565355 data_time: 0.082775 memory: 3840 loss_kpt: 0.000734 acc_pose: 0.771664 loss: 0.000734 2022/09/27 01:28:26 - mmengine - INFO - Epoch(train) [61][150/293] lr: 5.000000e-04 eta: 6:05:06 time: 0.554895 data_time: 0.086071 memory: 3840 loss_kpt: 0.000727 acc_pose: 0.771970 loss: 0.000727 2022/09/27 01:28:55 - mmengine - INFO - Epoch(train) [61][200/293] lr: 5.000000e-04 eta: 6:04:49 time: 0.566718 data_time: 0.075521 memory: 3840 loss_kpt: 0.000717 acc_pose: 0.824717 loss: 0.000717 2022/09/27 01:29:23 - mmengine - INFO - Epoch(train) [61][250/293] lr: 5.000000e-04 eta: 6:04:31 time: 0.558455 data_time: 0.069064 memory: 3840 loss_kpt: 0.000725 acc_pose: 0.799432 loss: 0.000725 2022/09/27 01:29:47 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:30:16 - mmengine - INFO - Epoch(train) [62][50/293] lr: 5.000000e-04 eta: 6:03:03 time: 0.586333 data_time: 0.095552 memory: 3840 loss_kpt: 0.000733 acc_pose: 0.837009 loss: 0.000733 2022/09/27 01:30:44 - mmengine - INFO - Epoch(train) [62][100/293] lr: 5.000000e-04 eta: 6:02:45 time: 0.561382 data_time: 0.177109 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.793416 loss: 0.000719 2022/09/27 01:30:59 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:31:12 - mmengine - INFO - Epoch(train) [62][150/293] lr: 5.000000e-04 eta: 6:02:26 time: 0.551175 data_time: 0.188194 memory: 3840 loss_kpt: 0.000732 acc_pose: 0.764218 loss: 0.000732 2022/09/27 01:31:41 - mmengine - INFO - Epoch(train) [62][200/293] lr: 5.000000e-04 eta: 6:02:11 time: 0.583875 data_time: 0.114629 memory: 3840 loss_kpt: 0.000731 acc_pose: 0.814814 loss: 0.000731 2022/09/27 01:32:08 - mmengine - INFO - Epoch(train) [62][250/293] lr: 5.000000e-04 eta: 6:01:52 time: 0.547603 data_time: 0.181070 memory: 3840 loss_kpt: 0.000727 acc_pose: 0.783951 loss: 0.000727 2022/09/27 01:32:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:33:03 - mmengine - INFO - Epoch(train) [63][50/293] lr: 5.000000e-04 eta: 6:00:26 time: 0.600649 data_time: 0.117493 memory: 3840 loss_kpt: 0.000721 acc_pose: 0.746849 loss: 0.000721 2022/09/27 01:33:31 - mmengine - INFO - Epoch(train) [63][100/293] lr: 5.000000e-04 eta: 6:00:09 time: 0.566007 data_time: 0.096836 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.729340 loss: 0.000730 2022/09/27 01:33:59 - mmengine - INFO - Epoch(train) [63][150/293] lr: 5.000000e-04 eta: 5:59:52 time: 0.564955 data_time: 0.098026 memory: 3840 loss_kpt: 0.000741 acc_pose: 0.827827 loss: 0.000741 2022/09/27 01:34:27 - mmengine - INFO - Epoch(train) [63][200/293] lr: 5.000000e-04 eta: 5:59:33 time: 0.545812 data_time: 0.132946 memory: 3840 loss_kpt: 0.000739 acc_pose: 0.787464 loss: 0.000739 2022/09/27 01:34:55 - mmengine - INFO - Epoch(train) [63][250/293] lr: 5.000000e-04 eta: 5:59:16 time: 0.567940 data_time: 0.167457 memory: 3840 loss_kpt: 0.000734 acc_pose: 0.803895 loss: 0.000734 2022/09/27 01:35:20 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:35:49 - mmengine - INFO - Epoch(train) [64][50/293] lr: 5.000000e-04 eta: 5:57:47 time: 0.571692 data_time: 0.226211 memory: 3840 loss_kpt: 0.000740 acc_pose: 0.761629 loss: 0.000740 2022/09/27 01:36:17 - mmengine - INFO - Epoch(train) [64][100/293] lr: 5.000000e-04 eta: 5:57:30 time: 0.561062 data_time: 0.198690 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.804789 loss: 0.000730 2022/09/27 01:36:46 - mmengine - INFO - Epoch(train) [64][150/293] lr: 5.000000e-04 eta: 5:57:14 time: 0.578708 data_time: 0.103193 memory: 3840 loss_kpt: 0.000732 acc_pose: 0.819124 loss: 0.000732 2022/09/27 01:37:15 - mmengine - INFO - Epoch(train) [64][200/293] lr: 5.000000e-04 eta: 5:56:59 time: 0.584875 data_time: 0.198309 memory: 3840 loss_kpt: 0.000733 acc_pose: 0.762682 loss: 0.000733 2022/09/27 01:37:43 - mmengine - INFO - Epoch(train) [64][250/293] lr: 5.000000e-04 eta: 5:56:41 time: 0.564304 data_time: 0.142365 memory: 3840 loss_kpt: 0.000734 acc_pose: 0.772343 loss: 0.000734 2022/09/27 01:38:07 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:38:36 - mmengine - INFO - Epoch(train) [65][50/293] lr: 5.000000e-04 eta: 5:55:16 time: 0.592225 data_time: 0.159965 memory: 3840 loss_kpt: 0.000738 acc_pose: 0.796291 loss: 0.000738 2022/09/27 01:39:06 - mmengine - INFO - Epoch(train) [65][100/293] lr: 5.000000e-04 eta: 5:55:01 time: 0.581661 data_time: 0.176625 memory: 3840 loss_kpt: 0.000720 acc_pose: 0.815341 loss: 0.000720 2022/09/27 01:39:35 - mmengine - INFO - Epoch(train) [65][150/293] lr: 5.000000e-04 eta: 5:54:46 time: 0.590609 data_time: 0.126586 memory: 3840 loss_kpt: 0.000731 acc_pose: 0.777730 loss: 0.000731 2022/09/27 01:40:03 - mmengine - INFO - Epoch(train) [65][200/293] lr: 5.000000e-04 eta: 5:54:28 time: 0.561334 data_time: 0.076245 memory: 3840 loss_kpt: 0.000708 acc_pose: 0.769905 loss: 0.000708 2022/09/27 01:40:30 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:40:31 - mmengine - INFO - Epoch(train) [65][250/293] lr: 5.000000e-04 eta: 5:54:10 time: 0.560227 data_time: 0.088857 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.814877 loss: 0.000718 2022/09/27 01:40:55 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:41:26 - mmengine - INFO - Epoch(train) [66][50/293] lr: 5.000000e-04 eta: 5:52:48 time: 0.608017 data_time: 0.222901 memory: 3840 loss_kpt: 0.000736 acc_pose: 0.782370 loss: 0.000736 2022/09/27 01:41:54 - mmengine - INFO - Epoch(train) [66][100/293] lr: 5.000000e-04 eta: 5:52:30 time: 0.561828 data_time: 0.121605 memory: 3840 loss_kpt: 0.000717 acc_pose: 0.784202 loss: 0.000717 2022/09/27 01:42:21 - mmengine - INFO - Epoch(train) [66][150/293] lr: 5.000000e-04 eta: 5:52:11 time: 0.551934 data_time: 0.068956 memory: 3840 loss_kpt: 0.000729 acc_pose: 0.744640 loss: 0.000729 2022/09/27 01:42:49 - mmengine - INFO - Epoch(train) [66][200/293] lr: 5.000000e-04 eta: 5:51:51 time: 0.545773 data_time: 0.185387 memory: 3840 loss_kpt: 0.000728 acc_pose: 0.796854 loss: 0.000728 2022/09/27 01:43:18 - mmengine - INFO - Epoch(train) [66][250/293] lr: 5.000000e-04 eta: 5:51:35 time: 0.583627 data_time: 0.076162 memory: 3840 loss_kpt: 0.000742 acc_pose: 0.798175 loss: 0.000742 2022/09/27 01:43:42 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:44:12 - mmengine - INFO - Epoch(train) [67][50/293] lr: 5.000000e-04 eta: 5:50:13 time: 0.597758 data_time: 0.098738 memory: 3840 loss_kpt: 0.000723 acc_pose: 0.763847 loss: 0.000723 2022/09/27 01:44:41 - mmengine - INFO - Epoch(train) [67][100/293] lr: 5.000000e-04 eta: 5:49:57 time: 0.584844 data_time: 0.136035 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.772282 loss: 0.000730 2022/09/27 01:45:09 - mmengine - INFO - Epoch(train) [67][150/293] lr: 5.000000e-04 eta: 5:49:38 time: 0.553413 data_time: 0.162127 memory: 3840 loss_kpt: 0.000729 acc_pose: 0.774924 loss: 0.000729 2022/09/27 01:45:37 - mmengine - INFO - Epoch(train) [67][200/293] lr: 5.000000e-04 eta: 5:49:19 time: 0.550755 data_time: 0.104929 memory: 3840 loss_kpt: 0.000728 acc_pose: 0.788068 loss: 0.000728 2022/09/27 01:46:05 - mmengine - INFO - Epoch(train) [67][250/293] lr: 5.000000e-04 eta: 5:49:01 time: 0.567015 data_time: 0.170186 memory: 3840 loss_kpt: 0.000731 acc_pose: 0.823687 loss: 0.000731 2022/09/27 01:46:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:46:59 - mmengine - INFO - Epoch(train) [68][50/293] lr: 5.000000e-04 eta: 5:47:39 time: 0.599394 data_time: 0.125908 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.747956 loss: 0.000718 2022/09/27 01:47:27 - mmengine - INFO - Epoch(train) [68][100/293] lr: 5.000000e-04 eta: 5:47:22 time: 0.569015 data_time: 0.131106 memory: 3840 loss_kpt: 0.000725 acc_pose: 0.796080 loss: 0.000725 2022/09/27 01:47:57 - mmengine - INFO - Epoch(train) [68][150/293] lr: 5.000000e-04 eta: 5:47:08 time: 0.599682 data_time: 0.093829 memory: 3840 loss_kpt: 0.000721 acc_pose: 0.781004 loss: 0.000721 2022/09/27 01:48:26 - mmengine - INFO - Epoch(train) [68][200/293] lr: 5.000000e-04 eta: 5:46:51 time: 0.572747 data_time: 0.092638 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.814240 loss: 0.000718 2022/09/27 01:48:54 - mmengine - INFO - Epoch(train) [68][250/293] lr: 5.000000e-04 eta: 5:46:33 time: 0.568317 data_time: 0.162386 memory: 3840 loss_kpt: 0.000724 acc_pose: 0.791255 loss: 0.000724 2022/09/27 01:49:18 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:49:48 - mmengine - INFO - Epoch(train) [69][50/293] lr: 5.000000e-04 eta: 5:45:12 time: 0.598888 data_time: 0.111398 memory: 3840 loss_kpt: 0.000720 acc_pose: 0.804648 loss: 0.000720 2022/09/27 01:50:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:50:18 - mmengine - INFO - Epoch(train) [69][100/293] lr: 5.000000e-04 eta: 5:44:57 time: 0.597072 data_time: 0.094572 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.758021 loss: 0.000719 2022/09/27 01:50:49 - mmengine - INFO - Epoch(train) [69][150/293] lr: 5.000000e-04 eta: 5:44:44 time: 0.608548 data_time: 0.083718 memory: 3840 loss_kpt: 0.000727 acc_pose: 0.750093 loss: 0.000727 2022/09/27 01:52:55 - mmengine - INFO - Epoch(train) [69][200/293] lr: 5.000000e-04 eta: 5:47:48 time: 2.527896 data_time: 0.607302 memory: 3840 loss_kpt: 0.000723 acc_pose: 0.774674 loss: 0.000723 2022/09/27 01:54:10 - mmengine - INFO - Epoch(train) [69][250/293] lr: 5.000000e-04 eta: 5:49:05 time: 1.508159 data_time: 0.338007 memory: 3840 loss_kpt: 0.000728 acc_pose: 0.833133 loss: 0.000728 2022/09/27 01:54:41 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:55:12 - mmengine - INFO - Epoch(train) [70][50/293] lr: 5.000000e-04 eta: 5:47:45 time: 0.611486 data_time: 0.137309 memory: 3840 loss_kpt: 0.000711 acc_pose: 0.730009 loss: 0.000711 2022/09/27 01:55:43 - mmengine - INFO - Epoch(train) [70][100/293] lr: 5.000000e-04 eta: 5:47:30 time: 0.614232 data_time: 0.101886 memory: 3840 loss_kpt: 0.000722 acc_pose: 0.765912 loss: 0.000722 2022/09/27 01:56:12 - mmengine - INFO - Epoch(train) [70][150/293] lr: 5.000000e-04 eta: 5:47:12 time: 0.577612 data_time: 0.082384 memory: 3840 loss_kpt: 0.000722 acc_pose: 0.792252 loss: 0.000722 2022/09/27 01:56:41 - mmengine - INFO - Epoch(train) [70][200/293] lr: 5.000000e-04 eta: 5:46:54 time: 0.577964 data_time: 0.078855 memory: 3840 loss_kpt: 0.000723 acc_pose: 0.783493 loss: 0.000723 2022/09/27 01:57:10 - mmengine - INFO - Epoch(train) [70][250/293] lr: 5.000000e-04 eta: 5:46:38 time: 0.597053 data_time: 0.074635 memory: 3840 loss_kpt: 0.000731 acc_pose: 0.810637 loss: 0.000731 2022/09/27 01:57:36 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 01:57:36 - mmengine - INFO - Saving checkpoint at 70 epochs 2022/09/27 01:57:58 - mmengine - INFO - Epoch(val) [70][50/407] eta: 0:01:54 time: 0.319808 data_time: 0.191646 memory: 3840 2022/09/27 01:58:12 - mmengine - INFO - Epoch(val) [70][100/407] eta: 0:01:28 time: 0.288070 data_time: 0.164774 memory: 1017 2022/09/27 01:58:28 - mmengine - INFO - Epoch(val) [70][150/407] eta: 0:01:19 time: 0.308805 data_time: 0.178472 memory: 1017 2022/09/27 01:58:43 - mmengine - INFO - Epoch(val) [70][200/407] eta: 0:01:03 time: 0.306887 data_time: 0.176358 memory: 1017 2022/09/27 01:58:59 - mmengine - INFO - Epoch(val) [70][250/407] eta: 0:00:50 time: 0.322277 data_time: 0.181816 memory: 1017 2022/09/27 01:59:15 - mmengine - INFO - Epoch(val) [70][300/407] eta: 0:00:33 time: 0.311361 data_time: 0.186497 memory: 1017 2022/09/27 01:59:31 - mmengine - INFO - Epoch(val) [70][350/407] eta: 0:00:18 time: 0.317699 data_time: 0.177064 memory: 1017 2022/09/27 01:59:43 - mmengine - INFO - Epoch(val) [70][400/407] eta: 0:00:01 time: 0.250588 data_time: 0.146825 memory: 1017 2022/09/27 02:00:18 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 02:00:31 - mmengine - INFO - Epoch(val) [70][407/407] coco/AP: 0.681014 coco/AP .5: 0.884291 coco/AP .75: 0.759779 coco/AP (M): 0.644918 coco/AP (L): 0.746227 coco/AR: 0.740822 coco/AR .5: 0.925535 coco/AR .75: 0.813287 coco/AR (M): 0.696968 coco/AR (L): 0.803567 2022/09/27 02:00:31 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_60.pth is removed 2022/09/27 02:00:34 - mmengine - INFO - The best checkpoint with 0.6810 coco/AP at 70 epoch is saved to best_coco/AP_epoch_70.pth. 2022/09/27 02:01:02 - mmengine - INFO - Epoch(train) [71][50/293] lr: 5.000000e-04 eta: 5:45:13 time: 0.557439 data_time: 0.144946 memory: 3840 loss_kpt: 0.000725 acc_pose: 0.805812 loss: 0.000725 2022/09/27 02:01:30 - mmengine - INFO - Epoch(train) [71][100/293] lr: 5.000000e-04 eta: 5:44:54 time: 0.573185 data_time: 0.112724 memory: 3840 loss_kpt: 0.000724 acc_pose: 0.786061 loss: 0.000724 2022/09/27 02:01:59 - mmengine - INFO - Epoch(train) [71][150/293] lr: 5.000000e-04 eta: 5:44:35 time: 0.572837 data_time: 0.169051 memory: 3840 loss_kpt: 0.000726 acc_pose: 0.772889 loss: 0.000726 2022/09/27 02:02:27 - mmengine - INFO - Epoch(train) [71][200/293] lr: 5.000000e-04 eta: 5:44:16 time: 0.570923 data_time: 0.093186 memory: 3840 loss_kpt: 0.000720 acc_pose: 0.788666 loss: 0.000720 2022/09/27 02:02:56 - mmengine - INFO - Epoch(train) [71][250/293] lr: 5.000000e-04 eta: 5:43:57 time: 0.563309 data_time: 0.080179 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.756652 loss: 0.000710 2022/09/27 02:03:20 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:03:49 - mmengine - INFO - Epoch(train) [72][50/293] lr: 5.000000e-04 eta: 5:42:34 time: 0.579553 data_time: 0.116534 memory: 3840 loss_kpt: 0.000734 acc_pose: 0.821421 loss: 0.000734 2022/09/27 02:04:18 - mmengine - INFO - Epoch(train) [72][100/293] lr: 5.000000e-04 eta: 5:42:17 time: 0.586206 data_time: 0.161395 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.751748 loss: 0.000713 2022/09/27 02:04:45 - mmengine - INFO - Epoch(train) [72][150/293] lr: 5.000000e-04 eta: 5:41:55 time: 0.540267 data_time: 0.087935 memory: 3840 loss_kpt: 0.000714 acc_pose: 0.829452 loss: 0.000714 2022/09/27 02:05:12 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:05:14 - mmengine - INFO - Epoch(train) [72][200/293] lr: 5.000000e-04 eta: 5:41:36 time: 0.565875 data_time: 0.114050 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.790652 loss: 0.000719 2022/09/27 02:05:42 - mmengine - INFO - Epoch(train) [72][250/293] lr: 5.000000e-04 eta: 5:41:17 time: 0.570525 data_time: 0.094853 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.808355 loss: 0.000705 2022/09/27 02:06:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:06:36 - mmengine - INFO - Epoch(train) [73][50/293] lr: 5.000000e-04 eta: 5:39:56 time: 0.592944 data_time: 0.097659 memory: 3840 loss_kpt: 0.000709 acc_pose: 0.801428 loss: 0.000709 2022/09/27 02:07:04 - mmengine - INFO - Epoch(train) [73][100/293] lr: 5.000000e-04 eta: 5:39:37 time: 0.569526 data_time: 0.081569 memory: 3840 loss_kpt: 0.000729 acc_pose: 0.786756 loss: 0.000729 2022/09/27 02:07:33 - mmengine - INFO - Epoch(train) [73][150/293] lr: 5.000000e-04 eta: 5:39:19 time: 0.578717 data_time: 0.085851 memory: 3840 loss_kpt: 0.000723 acc_pose: 0.791829 loss: 0.000723 2022/09/27 02:08:00 - mmengine - INFO - Epoch(train) [73][200/293] lr: 5.000000e-04 eta: 5:38:57 time: 0.536449 data_time: 0.072643 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.789065 loss: 0.000713 2022/09/27 02:08:29 - mmengine - INFO - Epoch(train) [73][250/293] lr: 5.000000e-04 eta: 5:38:38 time: 0.581977 data_time: 0.146240 memory: 3840 loss_kpt: 0.000712 acc_pose: 0.730821 loss: 0.000712 2022/09/27 02:08:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:09:22 - mmengine - INFO - Epoch(train) [74][50/293] lr: 5.000000e-04 eta: 5:37:19 time: 0.592025 data_time: 0.180460 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.818693 loss: 0.000718 2022/09/27 02:09:51 - mmengine - INFO - Epoch(train) [74][100/293] lr: 5.000000e-04 eta: 5:37:00 time: 0.572669 data_time: 0.083904 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.855580 loss: 0.000698 2022/09/27 02:10:19 - mmengine - INFO - Epoch(train) [74][150/293] lr: 5.000000e-04 eta: 5:36:40 time: 0.557754 data_time: 0.100666 memory: 3840 loss_kpt: 0.000706 acc_pose: 0.819916 loss: 0.000706 2022/09/27 02:10:47 - mmengine - INFO - Epoch(train) [74][200/293] lr: 5.000000e-04 eta: 5:36:20 time: 0.569556 data_time: 0.090585 memory: 3840 loss_kpt: 0.000733 acc_pose: 0.814026 loss: 0.000733 2022/09/27 02:11:15 - mmengine - INFO - Epoch(train) [74][250/293] lr: 5.000000e-04 eta: 5:36:00 time: 0.554357 data_time: 0.078750 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.828651 loss: 0.000719 2022/09/27 02:11:39 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:12:08 - mmengine - INFO - Epoch(train) [75][50/293] lr: 5.000000e-04 eta: 5:34:40 time: 0.589445 data_time: 0.150146 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.812101 loss: 0.000713 2022/09/27 02:12:37 - mmengine - INFO - Epoch(train) [75][100/293] lr: 5.000000e-04 eta: 5:34:22 time: 0.578802 data_time: 0.089297 memory: 3840 loss_kpt: 0.000715 acc_pose: 0.836494 loss: 0.000715 2022/09/27 02:13:05 - mmengine - INFO - Epoch(train) [75][150/293] lr: 5.000000e-04 eta: 5:34:01 time: 0.552274 data_time: 0.074833 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.802847 loss: 0.000710 2022/09/27 02:13:33 - mmengine - INFO - Epoch(train) [75][200/293] lr: 5.000000e-04 eta: 5:33:41 time: 0.565554 data_time: 0.102153 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.788723 loss: 0.000718 2022/09/27 02:14:01 - mmengine - INFO - Epoch(train) [75][250/293] lr: 5.000000e-04 eta: 5:33:21 time: 0.556365 data_time: 0.146897 memory: 3840 loss_kpt: 0.000735 acc_pose: 0.817517 loss: 0.000735 2022/09/27 02:14:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:14:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:14:55 - mmengine - INFO - Epoch(train) [76][50/293] lr: 5.000000e-04 eta: 5:32:04 time: 0.605631 data_time: 0.101868 memory: 3840 loss_kpt: 0.000701 acc_pose: 0.809987 loss: 0.000701 2022/09/27 02:15:23 - mmengine - INFO - Epoch(train) [76][100/293] lr: 5.000000e-04 eta: 5:31:42 time: 0.548056 data_time: 0.105347 memory: 3840 loss_kpt: 0.000715 acc_pose: 0.800509 loss: 0.000715 2022/09/27 02:15:51 - mmengine - INFO - Epoch(train) [76][150/293] lr: 5.000000e-04 eta: 5:31:23 time: 0.568552 data_time: 0.130281 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.768832 loss: 0.000713 2022/09/27 02:16:20 - mmengine - INFO - Epoch(train) [76][200/293] lr: 5.000000e-04 eta: 5:31:04 time: 0.576005 data_time: 0.157589 memory: 3840 loss_kpt: 0.000709 acc_pose: 0.849100 loss: 0.000709 2022/09/27 02:16:49 - mmengine - INFO - Epoch(train) [76][250/293] lr: 5.000000e-04 eta: 5:30:46 time: 0.582089 data_time: 0.103264 memory: 3840 loss_kpt: 0.000721 acc_pose: 0.790422 loss: 0.000721 2022/09/27 02:17:13 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:17:43 - mmengine - INFO - Epoch(train) [77][50/293] lr: 5.000000e-04 eta: 5:29:28 time: 0.587924 data_time: 0.221028 memory: 3840 loss_kpt: 0.000730 acc_pose: 0.745297 loss: 0.000730 2022/09/27 02:18:12 - mmengine - INFO - Epoch(train) [77][100/293] lr: 5.000000e-04 eta: 5:29:09 time: 0.579950 data_time: 0.164775 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.800021 loss: 0.000718 2022/09/27 02:18:40 - mmengine - INFO - Epoch(train) [77][150/293] lr: 5.000000e-04 eta: 5:28:49 time: 0.565061 data_time: 0.114380 memory: 3840 loss_kpt: 0.000711 acc_pose: 0.810705 loss: 0.000711 2022/09/27 02:19:09 - mmengine - INFO - Epoch(train) [77][200/293] lr: 5.000000e-04 eta: 5:28:31 time: 0.582884 data_time: 0.073980 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.765640 loss: 0.000718 2022/09/27 02:19:37 - mmengine - INFO - Epoch(train) [77][250/293] lr: 5.000000e-04 eta: 5:28:10 time: 0.549840 data_time: 0.082501 memory: 3840 loss_kpt: 0.000706 acc_pose: 0.838527 loss: 0.000706 2022/09/27 02:20:01 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:20:31 - mmengine - INFO - Epoch(train) [78][50/293] lr: 5.000000e-04 eta: 5:26:54 time: 0.600711 data_time: 0.104884 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.789187 loss: 0.000719 2022/09/27 02:21:00 - mmengine - INFO - Epoch(train) [78][100/293] lr: 5.000000e-04 eta: 5:26:35 time: 0.580418 data_time: 0.092087 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.747694 loss: 0.000713 2022/09/27 02:21:28 - mmengine - INFO - Epoch(train) [78][150/293] lr: 5.000000e-04 eta: 5:26:15 time: 0.567041 data_time: 0.079074 memory: 3840 loss_kpt: 0.000718 acc_pose: 0.801704 loss: 0.000718 2022/09/27 02:21:57 - mmengine - INFO - Epoch(train) [78][200/293] lr: 5.000000e-04 eta: 5:25:56 time: 0.573852 data_time: 0.073722 memory: 3840 loss_kpt: 0.000722 acc_pose: 0.833525 loss: 0.000722 2022/09/27 02:22:25 - mmengine - INFO - Epoch(train) [78][250/293] lr: 5.000000e-04 eta: 5:25:36 time: 0.563932 data_time: 0.090109 memory: 3840 loss_kpt: 0.000720 acc_pose: 0.784524 loss: 0.000720 2022/09/27 02:22:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:23:19 - mmengine - INFO - Epoch(train) [79][50/293] lr: 5.000000e-04 eta: 5:24:20 time: 0.593132 data_time: 0.104396 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.752913 loss: 0.000719 2022/09/27 02:23:47 - mmengine - INFO - Epoch(train) [79][100/293] lr: 5.000000e-04 eta: 5:24:00 time: 0.574888 data_time: 0.118406 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.826098 loss: 0.000710 2022/09/27 02:24:13 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:24:15 - mmengine - INFO - Epoch(train) [79][150/293] lr: 5.000000e-04 eta: 5:23:39 time: 0.547496 data_time: 0.195226 memory: 3840 loss_kpt: 0.000700 acc_pose: 0.785099 loss: 0.000700 2022/09/27 02:24:43 - mmengine - INFO - Epoch(train) [79][200/293] lr: 5.000000e-04 eta: 5:23:19 time: 0.567151 data_time: 0.074946 memory: 3840 loss_kpt: 0.000716 acc_pose: 0.788962 loss: 0.000716 2022/09/27 02:25:12 - mmengine - INFO - Epoch(train) [79][250/293] lr: 5.000000e-04 eta: 5:23:00 time: 0.580599 data_time: 0.174919 memory: 3840 loss_kpt: 0.000716 acc_pose: 0.818838 loss: 0.000716 2022/09/27 02:25:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:26:08 - mmengine - INFO - Epoch(train) [80][50/293] lr: 5.000000e-04 eta: 5:21:46 time: 0.609984 data_time: 0.091320 memory: 3840 loss_kpt: 0.000717 acc_pose: 0.749613 loss: 0.000717 2022/09/27 02:26:35 - mmengine - INFO - Epoch(train) [80][100/293] lr: 5.000000e-04 eta: 5:21:25 time: 0.557920 data_time: 0.085140 memory: 3840 loss_kpt: 0.000712 acc_pose: 0.747671 loss: 0.000712 2022/09/27 02:27:05 - mmengine - INFO - Epoch(train) [80][150/293] lr: 5.000000e-04 eta: 5:21:07 time: 0.581257 data_time: 0.126879 memory: 3840 loss_kpt: 0.000723 acc_pose: 0.791846 loss: 0.000723 2022/09/27 02:27:32 - mmengine - INFO - Epoch(train) [80][200/293] lr: 5.000000e-04 eta: 5:20:44 time: 0.541255 data_time: 0.074440 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.768408 loss: 0.000705 2022/09/27 02:27:59 - mmengine - INFO - Epoch(train) [80][250/293] lr: 5.000000e-04 eta: 5:20:24 time: 0.557145 data_time: 0.093870 memory: 3840 loss_kpt: 0.000709 acc_pose: 0.743884 loss: 0.000709 2022/09/27 02:28:23 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:28:23 - mmengine - INFO - Saving checkpoint at 80 epochs 2022/09/27 02:28:45 - mmengine - INFO - Epoch(val) [80][50/407] eta: 0:02:00 time: 0.336164 data_time: 0.191246 memory: 3840 2022/09/27 02:29:01 - mmengine - INFO - Epoch(val) [80][100/407] eta: 0:01:33 time: 0.303451 data_time: 0.174930 memory: 1017 2022/09/27 02:29:17 - mmengine - INFO - Epoch(val) [80][150/407] eta: 0:01:22 time: 0.321539 data_time: 0.170814 memory: 1017 2022/09/27 02:29:33 - mmengine - INFO - Epoch(val) [80][200/407] eta: 0:01:08 time: 0.331024 data_time: 0.202518 memory: 1017 2022/09/27 02:29:49 - mmengine - INFO - Epoch(val) [80][250/407] eta: 0:00:48 time: 0.306232 data_time: 0.190655 memory: 1017 2022/09/27 02:30:03 - mmengine - INFO - Epoch(val) [80][300/407] eta: 0:00:31 time: 0.291789 data_time: 0.163876 memory: 1017 2022/09/27 02:30:19 - mmengine - INFO - Epoch(val) [80][350/407] eta: 0:00:17 time: 0.312187 data_time: 0.183204 memory: 1017 2022/09/27 02:30:31 - mmengine - INFO - Epoch(val) [80][400/407] eta: 0:00:01 time: 0.239275 data_time: 0.148871 memory: 1017 2022/09/27 02:31:04 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 02:31:17 - mmengine - INFO - Epoch(val) [80][407/407] coco/AP: 0.683408 coco/AP .5: 0.882880 coco/AP .75: 0.763228 coco/AP (M): 0.648607 coco/AP (L): 0.747914 coco/AR: 0.743451 coco/AR .5: 0.923174 coco/AR .75: 0.815806 coco/AR (M): 0.700137 coco/AR (L): 0.805500 2022/09/27 02:31:18 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_70.pth is removed 2022/09/27 02:31:20 - mmengine - INFO - The best checkpoint with 0.6834 coco/AP at 80 epoch is saved to best_coco/AP_epoch_80.pth. 2022/09/27 02:31:48 - mmengine - INFO - Epoch(train) [81][50/293] lr: 5.000000e-04 eta: 5:19:06 time: 0.561271 data_time: 0.228623 memory: 3840 loss_kpt: 0.000702 acc_pose: 0.769630 loss: 0.000702 2022/09/27 02:32:16 - mmengine - INFO - Epoch(train) [81][100/293] lr: 5.000000e-04 eta: 5:18:47 time: 0.573453 data_time: 0.167673 memory: 3840 loss_kpt: 0.000706 acc_pose: 0.817224 loss: 0.000706 2022/09/27 02:32:45 - mmengine - INFO - Epoch(train) [81][150/293] lr: 5.000000e-04 eta: 5:18:27 time: 0.566751 data_time: 0.077547 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.825028 loss: 0.000697 2022/09/27 02:33:14 - mmengine - INFO - Epoch(train) [81][200/293] lr: 5.000000e-04 eta: 5:18:08 time: 0.586502 data_time: 0.081140 memory: 3840 loss_kpt: 0.000709 acc_pose: 0.817868 loss: 0.000709 2022/09/27 02:33:42 - mmengine - INFO - Epoch(train) [81][250/293] lr: 5.000000e-04 eta: 5:17:47 time: 0.560162 data_time: 0.077803 memory: 3840 loss_kpt: 0.000703 acc_pose: 0.783139 loss: 0.000703 2022/09/27 02:34:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:34:36 - mmengine - INFO - Epoch(train) [82][50/293] lr: 5.000000e-04 eta: 5:16:34 time: 0.604567 data_time: 0.124199 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.813126 loss: 0.000705 2022/09/27 02:35:05 - mmengine - INFO - Epoch(train) [82][100/293] lr: 5.000000e-04 eta: 5:16:14 time: 0.567949 data_time: 0.097083 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.827484 loss: 0.000713 2022/09/27 02:35:33 - mmengine - INFO - Epoch(train) [82][150/293] lr: 5.000000e-04 eta: 5:15:54 time: 0.565044 data_time: 0.084507 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.791715 loss: 0.000694 2022/09/27 02:36:02 - mmengine - INFO - Epoch(train) [82][200/293] lr: 5.000000e-04 eta: 5:15:34 time: 0.574712 data_time: 0.082872 memory: 3840 loss_kpt: 0.000711 acc_pose: 0.816384 loss: 0.000711 2022/09/27 02:36:30 - mmengine - INFO - Epoch(train) [82][250/293] lr: 5.000000e-04 eta: 5:15:15 time: 0.576582 data_time: 0.076164 memory: 3840 loss_kpt: 0.000700 acc_pose: 0.762031 loss: 0.000700 2022/09/27 02:36:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:36:56 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:37:26 - mmengine - INFO - Epoch(train) [83][50/293] lr: 5.000000e-04 eta: 5:14:02 time: 0.603328 data_time: 0.156409 memory: 3840 loss_kpt: 0.000703 acc_pose: 0.787137 loss: 0.000703 2022/09/27 02:37:53 - mmengine - INFO - Epoch(train) [83][100/293] lr: 5.000000e-04 eta: 5:13:39 time: 0.531324 data_time: 0.124992 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.782783 loss: 0.000713 2022/09/27 02:38:22 - mmengine - INFO - Epoch(train) [83][150/293] lr: 5.000000e-04 eta: 5:13:19 time: 0.571686 data_time: 0.111282 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.815607 loss: 0.000710 2022/09/27 02:38:51 - mmengine - INFO - Epoch(train) [83][200/293] lr: 5.000000e-04 eta: 5:13:00 time: 0.587504 data_time: 0.171426 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.780087 loss: 0.000694 2022/09/27 02:39:20 - mmengine - INFO - Epoch(train) [83][250/293] lr: 5.000000e-04 eta: 5:12:41 time: 0.581006 data_time: 0.155527 memory: 3840 loss_kpt: 0.000702 acc_pose: 0.793406 loss: 0.000702 2022/09/27 02:39:44 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:40:13 - mmengine - INFO - Epoch(train) [84][50/293] lr: 5.000000e-04 eta: 5:11:27 time: 0.584657 data_time: 0.122119 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.790304 loss: 0.000705 2022/09/27 02:40:41 - mmengine - INFO - Epoch(train) [84][100/293] lr: 5.000000e-04 eta: 5:11:06 time: 0.556779 data_time: 0.094075 memory: 3840 loss_kpt: 0.000696 acc_pose: 0.788545 loss: 0.000696 2022/09/27 02:41:09 - mmengine - INFO - Epoch(train) [84][150/293] lr: 5.000000e-04 eta: 5:10:45 time: 0.556992 data_time: 0.139979 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.763054 loss: 0.000710 2022/09/27 02:41:37 - mmengine - INFO - Epoch(train) [84][200/293] lr: 5.000000e-04 eta: 5:10:25 time: 0.570923 data_time: 0.102690 memory: 3840 loss_kpt: 0.000711 acc_pose: 0.819017 loss: 0.000711 2022/09/27 02:42:06 - mmengine - INFO - Epoch(train) [84][250/293] lr: 5.000000e-04 eta: 5:10:05 time: 0.566399 data_time: 0.082579 memory: 3840 loss_kpt: 0.000711 acc_pose: 0.750489 loss: 0.000711 2022/09/27 02:42:30 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:42:59 - mmengine - INFO - Epoch(train) [85][50/293] lr: 5.000000e-04 eta: 5:08:52 time: 0.580940 data_time: 0.092376 memory: 3840 loss_kpt: 0.000695 acc_pose: 0.799309 loss: 0.000695 2022/09/27 02:43:26 - mmengine - INFO - Epoch(train) [85][100/293] lr: 5.000000e-04 eta: 5:08:29 time: 0.540414 data_time: 0.111348 memory: 3840 loss_kpt: 0.000700 acc_pose: 0.770641 loss: 0.000700 2022/09/27 02:43:55 - mmengine - INFO - Epoch(train) [85][150/293] lr: 5.000000e-04 eta: 5:08:10 time: 0.579416 data_time: 0.165089 memory: 3840 loss_kpt: 0.000716 acc_pose: 0.851045 loss: 0.000716 2022/09/27 02:44:24 - mmengine - INFO - Epoch(train) [85][200/293] lr: 5.000000e-04 eta: 5:07:49 time: 0.566839 data_time: 0.080884 memory: 3840 loss_kpt: 0.000722 acc_pose: 0.849933 loss: 0.000722 2022/09/27 02:44:53 - mmengine - INFO - Epoch(train) [85][250/293] lr: 5.000000e-04 eta: 5:07:30 time: 0.578232 data_time: 0.094343 memory: 3840 loss_kpt: 0.000708 acc_pose: 0.816055 loss: 0.000708 2022/09/27 02:45:17 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:45:48 - mmengine - INFO - Epoch(train) [86][50/293] lr: 5.000000e-04 eta: 5:06:20 time: 0.618789 data_time: 0.157278 memory: 3840 loss_kpt: 0.000696 acc_pose: 0.794340 loss: 0.000696 2022/09/27 02:46:13 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:46:16 - mmengine - INFO - Epoch(train) [86][100/293] lr: 5.000000e-04 eta: 5:05:59 time: 0.558783 data_time: 0.080658 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.847666 loss: 0.000698 2022/09/27 02:46:44 - mmengine - INFO - Epoch(train) [86][150/293] lr: 5.000000e-04 eta: 5:05:38 time: 0.567024 data_time: 0.087121 memory: 3840 loss_kpt: 0.000696 acc_pose: 0.835385 loss: 0.000696 2022/09/27 02:47:12 - mmengine - INFO - Epoch(train) [86][200/293] lr: 5.000000e-04 eta: 5:05:16 time: 0.547544 data_time: 0.161864 memory: 3840 loss_kpt: 0.000708 acc_pose: 0.810025 loss: 0.000708 2022/09/27 02:47:40 - mmengine - INFO - Epoch(train) [86][250/293] lr: 5.000000e-04 eta: 5:04:56 time: 0.568324 data_time: 0.097258 memory: 3840 loss_kpt: 0.000719 acc_pose: 0.791580 loss: 0.000719 2022/09/27 02:48:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:48:32 - mmengine - INFO - Epoch(train) [87][50/293] lr: 5.000000e-04 eta: 5:03:43 time: 0.572473 data_time: 0.165830 memory: 3840 loss_kpt: 0.000701 acc_pose: 0.811861 loss: 0.000701 2022/09/27 02:49:01 - mmengine - INFO - Epoch(train) [87][100/293] lr: 5.000000e-04 eta: 5:03:22 time: 0.563757 data_time: 0.176940 memory: 3840 loss_kpt: 0.000709 acc_pose: 0.796748 loss: 0.000709 2022/09/27 02:49:28 - mmengine - INFO - Epoch(train) [87][150/293] lr: 5.000000e-04 eta: 5:03:01 time: 0.554088 data_time: 0.159189 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.813956 loss: 0.000697 2022/09/27 02:49:57 - mmengine - INFO - Epoch(train) [87][200/293] lr: 5.000000e-04 eta: 5:02:41 time: 0.581680 data_time: 0.084636 memory: 3840 loss_kpt: 0.000699 acc_pose: 0.785234 loss: 0.000699 2022/09/27 02:50:25 - mmengine - INFO - Epoch(train) [87][250/293] lr: 5.000000e-04 eta: 5:02:19 time: 0.545381 data_time: 0.067419 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.829184 loss: 0.000693 2022/09/27 02:50:50 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:51:19 - mmengine - INFO - Epoch(train) [88][50/293] lr: 5.000000e-04 eta: 5:01:08 time: 0.585271 data_time: 0.097125 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.797781 loss: 0.000694 2022/09/27 02:51:48 - mmengine - INFO - Epoch(train) [88][100/293] lr: 5.000000e-04 eta: 5:00:47 time: 0.562817 data_time: 0.089131 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.814431 loss: 0.000705 2022/09/27 02:52:16 - mmengine - INFO - Epoch(train) [88][150/293] lr: 5.000000e-04 eta: 5:00:26 time: 0.565430 data_time: 0.077223 memory: 3840 loss_kpt: 0.000703 acc_pose: 0.821474 loss: 0.000703 2022/09/27 02:52:43 - mmengine - INFO - Epoch(train) [88][200/293] lr: 5.000000e-04 eta: 5:00:05 time: 0.553243 data_time: 0.093077 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.834214 loss: 0.000693 2022/09/27 02:53:13 - mmengine - INFO - Epoch(train) [88][250/293] lr: 5.000000e-04 eta: 4:59:45 time: 0.584213 data_time: 0.095958 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.806932 loss: 0.000710 2022/09/27 02:53:38 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:54:07 - mmengine - INFO - Epoch(train) [89][50/293] lr: 5.000000e-04 eta: 4:58:34 time: 0.575564 data_time: 0.089226 memory: 3840 loss_kpt: 0.000695 acc_pose: 0.852935 loss: 0.000695 2022/09/27 02:54:36 - mmengine - INFO - Epoch(train) [89][100/293] lr: 5.000000e-04 eta: 4:58:14 time: 0.582585 data_time: 0.138627 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.793219 loss: 0.000697 2022/09/27 02:55:04 - mmengine - INFO - Epoch(train) [89][150/293] lr: 5.000000e-04 eta: 4:57:53 time: 0.561753 data_time: 0.158424 memory: 3840 loss_kpt: 0.000715 acc_pose: 0.776226 loss: 0.000715 2022/09/27 02:55:40 - mmengine - INFO - Epoch(train) [89][200/293] lr: 5.000000e-04 eta: 4:57:44 time: 0.730043 data_time: 0.094938 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.820253 loss: 0.000693 2022/09/27 02:55:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:56:09 - mmengine - INFO - Epoch(train) [89][250/293] lr: 5.000000e-04 eta: 4:57:24 time: 0.579859 data_time: 0.078215 memory: 3840 loss_kpt: 0.000708 acc_pose: 0.805070 loss: 0.000708 2022/09/27 02:56:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:57:05 - mmengine - INFO - Epoch(train) [90][50/293] lr: 5.000000e-04 eta: 4:56:17 time: 0.633395 data_time: 0.100077 memory: 3840 loss_kpt: 0.000709 acc_pose: 0.830362 loss: 0.000709 2022/09/27 02:57:33 - mmengine - INFO - Epoch(train) [90][100/293] lr: 5.000000e-04 eta: 4:55:55 time: 0.555552 data_time: 0.089261 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.809030 loss: 0.000697 2022/09/27 02:58:02 - mmengine - INFO - Epoch(train) [90][150/293] lr: 5.000000e-04 eta: 4:55:36 time: 0.581995 data_time: 0.120175 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.798017 loss: 0.000692 2022/09/27 02:58:30 - mmengine - INFO - Epoch(train) [90][200/293] lr: 5.000000e-04 eta: 4:55:14 time: 0.562956 data_time: 0.080914 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.811841 loss: 0.000705 2022/09/27 02:58:58 - mmengine - INFO - Epoch(train) [90][250/293] lr: 5.000000e-04 eta: 4:54:53 time: 0.557235 data_time: 0.078186 memory: 3840 loss_kpt: 0.000711 acc_pose: 0.756111 loss: 0.000711 2022/09/27 02:59:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 02:59:22 - mmengine - INFO - Saving checkpoint at 90 epochs 2022/09/27 02:59:43 - mmengine - INFO - Epoch(val) [90][50/407] eta: 0:01:57 time: 0.329689 data_time: 0.200197 memory: 3840 2022/09/27 02:59:59 - mmengine - INFO - Epoch(val) [90][100/407] eta: 0:01:33 time: 0.304395 data_time: 0.173263 memory: 1017 2022/09/27 03:00:15 - mmengine - INFO - Epoch(val) [90][150/407] eta: 0:01:24 time: 0.328543 data_time: 0.188608 memory: 1017 2022/09/27 03:00:30 - mmengine - INFO - Epoch(val) [90][200/407] eta: 0:01:02 time: 0.302676 data_time: 0.189312 memory: 1017 2022/09/27 03:00:46 - mmengine - INFO - Epoch(val) [90][250/407] eta: 0:00:48 time: 0.308646 data_time: 0.163957 memory: 1017 2022/09/27 03:01:01 - mmengine - INFO - Epoch(val) [90][300/407] eta: 0:00:33 time: 0.309911 data_time: 0.179934 memory: 1017 2022/09/27 03:01:16 - mmengine - INFO - Epoch(val) [90][350/407] eta: 0:00:17 time: 0.299023 data_time: 0.164760 memory: 1017 2022/09/27 03:01:28 - mmengine - INFO - Epoch(val) [90][400/407] eta: 0:00:01 time: 0.233584 data_time: 0.139536 memory: 1017 2022/09/27 03:02:01 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 03:02:14 - mmengine - INFO - Epoch(val) [90][407/407] coco/AP: 0.684917 coco/AP .5: 0.883931 coco/AP .75: 0.762475 coco/AP (M): 0.647105 coco/AP (L): 0.752343 coco/AR: 0.743482 coco/AR .5: 0.922544 coco/AR .75: 0.814232 coco/AR (M): 0.697460 coco/AR (L): 0.809327 2022/09/27 03:02:14 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_80.pth is removed 2022/09/27 03:02:16 - mmengine - INFO - The best checkpoint with 0.6849 coco/AP at 90 epoch is saved to best_coco/AP_epoch_90.pth. 2022/09/27 03:02:44 - mmengine - INFO - Epoch(train) [91][50/293] lr: 5.000000e-04 eta: 4:53:41 time: 0.553315 data_time: 0.175863 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.826010 loss: 0.000697 2022/09/27 03:03:13 - mmengine - INFO - Epoch(train) [91][100/293] lr: 5.000000e-04 eta: 4:53:20 time: 0.568186 data_time: 0.091999 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.725354 loss: 0.000697 2022/09/27 03:03:41 - mmengine - INFO - Epoch(train) [91][150/293] lr: 5.000000e-04 eta: 4:53:00 time: 0.569860 data_time: 0.104208 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.827746 loss: 0.000692 2022/09/27 03:04:09 - mmengine - INFO - Epoch(train) [91][200/293] lr: 5.000000e-04 eta: 4:52:39 time: 0.565477 data_time: 0.155990 memory: 3840 loss_kpt: 0.000700 acc_pose: 0.830317 loss: 0.000700 2022/09/27 03:04:37 - mmengine - INFO - Epoch(train) [91][250/293] lr: 5.000000e-04 eta: 4:52:17 time: 0.548626 data_time: 0.147928 memory: 3840 loss_kpt: 0.000721 acc_pose: 0.797163 loss: 0.000721 2022/09/27 03:05:01 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:05:31 - mmengine - INFO - Epoch(train) [92][50/293] lr: 5.000000e-04 eta: 4:51:08 time: 0.591622 data_time: 0.111235 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.808564 loss: 0.000698 2022/09/27 03:05:59 - mmengine - INFO - Epoch(train) [92][100/293] lr: 5.000000e-04 eta: 4:50:46 time: 0.561871 data_time: 0.077108 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.792962 loss: 0.000694 2022/09/27 03:06:29 - mmengine - INFO - Epoch(train) [92][150/293] lr: 5.000000e-04 eta: 4:50:28 time: 0.601717 data_time: 0.083683 memory: 3840 loss_kpt: 0.000691 acc_pose: 0.815582 loss: 0.000691 2022/09/27 03:06:56 - mmengine - INFO - Epoch(train) [92][200/293] lr: 5.000000e-04 eta: 4:50:06 time: 0.554043 data_time: 0.127172 memory: 3840 loss_kpt: 0.000713 acc_pose: 0.766699 loss: 0.000713 2022/09/27 03:07:24 - mmengine - INFO - Epoch(train) [92][250/293] lr: 5.000000e-04 eta: 4:49:44 time: 0.556905 data_time: 0.084154 memory: 3840 loss_kpt: 0.000702 acc_pose: 0.832770 loss: 0.000702 2022/09/27 03:07:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:08:14 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:08:18 - mmengine - INFO - Epoch(train) [93][50/293] lr: 5.000000e-04 eta: 4:48:35 time: 0.580480 data_time: 0.125213 memory: 3840 loss_kpt: 0.000703 acc_pose: 0.789622 loss: 0.000703 2022/09/27 03:08:46 - mmengine - INFO - Epoch(train) [93][100/293] lr: 5.000000e-04 eta: 4:48:14 time: 0.567634 data_time: 0.119473 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.800586 loss: 0.000694 2022/09/27 03:09:14 - mmengine - INFO - Epoch(train) [93][150/293] lr: 5.000000e-04 eta: 4:47:53 time: 0.561696 data_time: 0.129385 memory: 3840 loss_kpt: 0.000702 acc_pose: 0.821931 loss: 0.000702 2022/09/27 03:09:42 - mmengine - INFO - Epoch(train) [93][200/293] lr: 5.000000e-04 eta: 4:47:32 time: 0.563362 data_time: 0.129597 memory: 3840 loss_kpt: 0.000710 acc_pose: 0.809738 loss: 0.000710 2022/09/27 03:10:11 - mmengine - INFO - Epoch(train) [93][250/293] lr: 5.000000e-04 eta: 4:47:12 time: 0.579410 data_time: 0.094790 memory: 3840 loss_kpt: 0.000696 acc_pose: 0.762181 loss: 0.000696 2022/09/27 03:10:36 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:11:05 - mmengine - INFO - Epoch(train) [94][50/293] lr: 5.000000e-04 eta: 4:46:04 time: 0.594951 data_time: 0.095172 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.792108 loss: 0.000697 2022/09/27 03:11:34 - mmengine - INFO - Epoch(train) [94][100/293] lr: 5.000000e-04 eta: 4:45:43 time: 0.565687 data_time: 0.159313 memory: 3840 loss_kpt: 0.000701 acc_pose: 0.783634 loss: 0.000701 2022/09/27 03:12:02 - mmengine - INFO - Epoch(train) [94][150/293] lr: 5.000000e-04 eta: 4:45:22 time: 0.571250 data_time: 0.155745 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.823701 loss: 0.000694 2022/09/27 03:12:31 - mmengine - INFO - Epoch(train) [94][200/293] lr: 5.000000e-04 eta: 4:45:02 time: 0.581857 data_time: 0.131234 memory: 3840 loss_kpt: 0.000691 acc_pose: 0.824003 loss: 0.000691 2022/09/27 03:12:59 - mmengine - INFO - Epoch(train) [94][250/293] lr: 5.000000e-04 eta: 4:44:39 time: 0.547215 data_time: 0.166009 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.824982 loss: 0.000698 2022/09/27 03:13:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:13:51 - mmengine - INFO - Epoch(train) [95][50/293] lr: 5.000000e-04 eta: 4:43:32 time: 0.592149 data_time: 0.097323 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.796913 loss: 0.000694 2022/09/27 03:14:20 - mmengine - INFO - Epoch(train) [95][100/293] lr: 5.000000e-04 eta: 4:43:12 time: 0.581044 data_time: 0.065533 memory: 3840 loss_kpt: 0.000714 acc_pose: 0.721408 loss: 0.000714 2022/09/27 03:14:49 - mmengine - INFO - Epoch(train) [95][150/293] lr: 5.000000e-04 eta: 4:42:51 time: 0.575480 data_time: 0.078392 memory: 3840 loss_kpt: 0.000703 acc_pose: 0.740589 loss: 0.000703 2022/09/27 03:15:16 - mmengine - INFO - Epoch(train) [95][200/293] lr: 5.000000e-04 eta: 4:42:29 time: 0.544612 data_time: 0.128125 memory: 3840 loss_kpt: 0.000688 acc_pose: 0.826278 loss: 0.000688 2022/09/27 03:15:46 - mmengine - INFO - Epoch(train) [95][250/293] lr: 5.000000e-04 eta: 4:42:09 time: 0.584330 data_time: 0.159912 memory: 3840 loss_kpt: 0.000675 acc_pose: 0.828253 loss: 0.000675 2022/09/27 03:16:09 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:16:40 - mmengine - INFO - Epoch(train) [96][50/293] lr: 5.000000e-04 eta: 4:41:02 time: 0.604708 data_time: 0.091317 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.811720 loss: 0.000681 2022/09/27 03:17:09 - mmengine - INFO - Epoch(train) [96][100/293] lr: 5.000000e-04 eta: 4:40:42 time: 0.578246 data_time: 0.121790 memory: 3840 loss_kpt: 0.000685 acc_pose: 0.793267 loss: 0.000685 2022/09/27 03:17:37 - mmengine - INFO - Epoch(train) [96][150/293] lr: 5.000000e-04 eta: 4:40:21 time: 0.574483 data_time: 0.080705 memory: 3840 loss_kpt: 0.000687 acc_pose: 0.794033 loss: 0.000687 2022/09/27 03:17:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:18:06 - mmengine - INFO - Epoch(train) [96][200/293] lr: 5.000000e-04 eta: 4:40:00 time: 0.564939 data_time: 0.083998 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.779607 loss: 0.000692 2022/09/27 03:18:35 - mmengine - INFO - Epoch(train) [96][250/293] lr: 5.000000e-04 eta: 4:39:40 time: 0.585163 data_time: 0.089548 memory: 3840 loss_kpt: 0.000705 acc_pose: 0.821011 loss: 0.000705 2022/09/27 03:18:58 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:19:28 - mmengine - INFO - Epoch(train) [97][50/293] lr: 5.000000e-04 eta: 4:38:32 time: 0.584717 data_time: 0.084925 memory: 3840 loss_kpt: 0.000701 acc_pose: 0.770740 loss: 0.000701 2022/09/27 03:19:56 - mmengine - INFO - Epoch(train) [97][100/293] lr: 5.000000e-04 eta: 4:38:11 time: 0.560560 data_time: 0.095163 memory: 3840 loss_kpt: 0.000688 acc_pose: 0.800935 loss: 0.000688 2022/09/27 03:20:24 - mmengine - INFO - Epoch(train) [97][150/293] lr: 5.000000e-04 eta: 4:37:49 time: 0.557519 data_time: 0.150328 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.751255 loss: 0.000686 2022/09/27 03:20:52 - mmengine - INFO - Epoch(train) [97][200/293] lr: 5.000000e-04 eta: 4:37:28 time: 0.565596 data_time: 0.073106 memory: 3840 loss_kpt: 0.000691 acc_pose: 0.774022 loss: 0.000691 2022/09/27 03:21:21 - mmengine - INFO - Epoch(train) [97][250/293] lr: 5.000000e-04 eta: 4:37:07 time: 0.585064 data_time: 0.138116 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.788882 loss: 0.000692 2022/09/27 03:21:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:22:15 - mmengine - INFO - Epoch(train) [98][50/293] lr: 5.000000e-04 eta: 4:36:00 time: 0.579132 data_time: 0.091723 memory: 3840 loss_kpt: 0.000695 acc_pose: 0.817142 loss: 0.000695 2022/09/27 03:22:44 - mmengine - INFO - Epoch(train) [98][100/293] lr: 5.000000e-04 eta: 4:35:40 time: 0.575428 data_time: 0.097313 memory: 3840 loss_kpt: 0.000688 acc_pose: 0.831479 loss: 0.000688 2022/09/27 03:23:11 - mmengine - INFO - Epoch(train) [98][150/293] lr: 5.000000e-04 eta: 4:35:17 time: 0.552717 data_time: 0.076943 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.837603 loss: 0.000689 2022/09/27 03:23:40 - mmengine - INFO - Epoch(train) [98][200/293] lr: 5.000000e-04 eta: 4:34:57 time: 0.578058 data_time: 0.115349 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.766249 loss: 0.000698 2022/09/27 03:24:08 - mmengine - INFO - Epoch(train) [98][250/293] lr: 5.000000e-04 eta: 4:34:35 time: 0.549998 data_time: 0.100408 memory: 3840 loss_kpt: 0.000699 acc_pose: 0.820011 loss: 0.000699 2022/09/27 03:24:31 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:25:01 - mmengine - INFO - Epoch(train) [99][50/293] lr: 5.000000e-04 eta: 4:33:29 time: 0.600765 data_time: 0.181476 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.802163 loss: 0.000686 2022/09/27 03:25:28 - mmengine - INFO - Epoch(train) [99][100/293] lr: 5.000000e-04 eta: 4:33:05 time: 0.525321 data_time: 0.068574 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.826085 loss: 0.000693 2022/09/27 03:25:57 - mmengine - INFO - Epoch(train) [99][150/293] lr: 5.000000e-04 eta: 4:32:45 time: 0.585169 data_time: 0.112441 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.760829 loss: 0.000698 2022/09/27 03:26:24 - mmengine - INFO - Epoch(train) [99][200/293] lr: 5.000000e-04 eta: 4:32:23 time: 0.547024 data_time: 0.091616 memory: 3840 loss_kpt: 0.000690 acc_pose: 0.803145 loss: 0.000690 2022/09/27 03:26:52 - mmengine - INFO - Epoch(train) [99][250/293] lr: 5.000000e-04 eta: 4:32:00 time: 0.552422 data_time: 0.088535 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.776125 loss: 0.000679 2022/09/27 03:27:12 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:27:16 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:27:46 - mmengine - INFO - Epoch(train) [100][50/293] lr: 5.000000e-04 eta: 4:30:55 time: 0.603090 data_time: 0.100483 memory: 3840 loss_kpt: 0.000701 acc_pose: 0.837660 loss: 0.000701 2022/09/27 03:28:15 - mmengine - INFO - Epoch(train) [100][100/293] lr: 5.000000e-04 eta: 4:30:34 time: 0.568060 data_time: 0.174267 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.743297 loss: 0.000694 2022/09/27 03:28:43 - mmengine - INFO - Epoch(train) [100][150/293] lr: 5.000000e-04 eta: 4:30:13 time: 0.573629 data_time: 0.101981 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.841805 loss: 0.000686 2022/09/27 03:29:11 - mmengine - INFO - Epoch(train) [100][200/293] lr: 5.000000e-04 eta: 4:29:51 time: 0.555893 data_time: 0.072666 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.815310 loss: 0.000693 2022/09/27 03:29:40 - mmengine - INFO - Epoch(train) [100][250/293] lr: 5.000000e-04 eta: 4:29:31 time: 0.581803 data_time: 0.088560 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.803292 loss: 0.000692 2022/09/27 03:30:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:30:04 - mmengine - INFO - Saving checkpoint at 100 epochs 2022/09/27 03:30:26 - mmengine - INFO - Epoch(val) [100][50/407] eta: 0:01:58 time: 0.331060 data_time: 0.190132 memory: 3840 2022/09/27 03:30:41 - mmengine - INFO - Epoch(val) [100][100/407] eta: 0:01:31 time: 0.299570 data_time: 0.174823 memory: 1017 2022/09/27 03:30:58 - mmengine - INFO - Epoch(val) [100][150/407] eta: 0:01:23 time: 0.326239 data_time: 0.194340 memory: 1017 2022/09/27 03:31:13 - mmengine - INFO - Epoch(val) [100][200/407] eta: 0:01:02 time: 0.303046 data_time: 0.165645 memory: 1017 2022/09/27 03:31:29 - mmengine - INFO - Epoch(val) [100][250/407] eta: 0:00:49 time: 0.318225 data_time: 0.186099 memory: 1017 2022/09/27 03:31:45 - mmengine - INFO - Epoch(val) [100][300/407] eta: 0:00:34 time: 0.321709 data_time: 0.184605 memory: 1017 2022/09/27 03:32:02 - mmengine - INFO - Epoch(val) [100][350/407] eta: 0:00:19 time: 0.338048 data_time: 0.204871 memory: 1017 2022/09/27 03:32:12 - mmengine - INFO - Epoch(val) [100][400/407] eta: 0:00:01 time: 0.204430 data_time: 0.118966 memory: 1017 2022/09/27 03:32:45 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 03:32:58 - mmengine - INFO - Epoch(val) [100][407/407] coco/AP: 0.694608 coco/AP .5: 0.888124 coco/AP .75: 0.775993 coco/AP (M): 0.659670 coco/AP (L): 0.761066 coco/AR: 0.753668 coco/AR .5: 0.928054 coco/AR .75: 0.826669 coco/AR (M): 0.710025 coco/AR (L): 0.816091 2022/09/27 03:32:58 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_90.pth is removed 2022/09/27 03:33:00 - mmengine - INFO - The best checkpoint with 0.6946 coco/AP at 100 epoch is saved to best_coco/AP_epoch_100.pth. 2022/09/27 03:33:29 - mmengine - INFO - Epoch(train) [101][50/293] lr: 5.000000e-04 eta: 4:28:24 time: 0.572697 data_time: 0.170550 memory: 3840 loss_kpt: 0.000690 acc_pose: 0.790249 loss: 0.000690 2022/09/27 03:33:58 - mmengine - INFO - Epoch(train) [101][100/293] lr: 5.000000e-04 eta: 4:28:04 time: 0.581353 data_time: 0.106893 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.774546 loss: 0.000689 2022/09/27 03:34:26 - mmengine - INFO - Epoch(train) [101][150/293] lr: 5.000000e-04 eta: 4:27:42 time: 0.562126 data_time: 0.154170 memory: 3840 loss_kpt: 0.000702 acc_pose: 0.743096 loss: 0.000702 2022/09/27 03:34:54 - mmengine - INFO - Epoch(train) [101][200/293] lr: 5.000000e-04 eta: 4:27:20 time: 0.560700 data_time: 0.149251 memory: 3840 loss_kpt: 0.000702 acc_pose: 0.802081 loss: 0.000702 2022/09/27 03:35:24 - mmengine - INFO - Epoch(train) [101][250/293] lr: 5.000000e-04 eta: 4:27:00 time: 0.589984 data_time: 0.097902 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.824660 loss: 0.000682 2022/09/27 03:35:47 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:36:17 - mmengine - INFO - Epoch(train) [102][50/293] lr: 5.000000e-04 eta: 4:25:56 time: 0.598547 data_time: 0.138565 memory: 3840 loss_kpt: 0.000690 acc_pose: 0.814103 loss: 0.000690 2022/09/27 03:36:45 - mmengine - INFO - Epoch(train) [102][100/293] lr: 5.000000e-04 eta: 4:25:34 time: 0.560509 data_time: 0.131630 memory: 3840 loss_kpt: 0.000687 acc_pose: 0.858364 loss: 0.000687 2022/09/27 03:37:13 - mmengine - INFO - Epoch(train) [102][150/293] lr: 5.000000e-04 eta: 4:25:12 time: 0.556181 data_time: 0.199740 memory: 3840 loss_kpt: 0.000685 acc_pose: 0.721858 loss: 0.000685 2022/09/27 03:37:40 - mmengine - INFO - Epoch(train) [102][200/293] lr: 5.000000e-04 eta: 4:24:49 time: 0.543351 data_time: 0.111532 memory: 3840 loss_kpt: 0.000699 acc_pose: 0.806227 loss: 0.000699 2022/09/27 03:38:09 - mmengine - INFO - Epoch(train) [102][250/293] lr: 5.000000e-04 eta: 4:24:28 time: 0.583265 data_time: 0.129834 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.815106 loss: 0.000697 2022/09/27 03:38:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:39:03 - mmengine - INFO - Epoch(train) [103][50/293] lr: 5.000000e-04 eta: 4:23:25 time: 0.604848 data_time: 0.107503 memory: 3840 loss_kpt: 0.000699 acc_pose: 0.820787 loss: 0.000699 2022/09/27 03:39:32 - mmengine - INFO - Epoch(train) [103][100/293] lr: 5.000000e-04 eta: 4:23:03 time: 0.569199 data_time: 0.087435 memory: 3840 loss_kpt: 0.000698 acc_pose: 0.822149 loss: 0.000698 2022/09/27 03:39:39 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:40:00 - mmengine - INFO - Epoch(train) [103][150/293] lr: 5.000000e-04 eta: 4:22:42 time: 0.573165 data_time: 0.078153 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.828901 loss: 0.000693 2022/09/27 03:40:28 - mmengine - INFO - Epoch(train) [103][200/293] lr: 5.000000e-04 eta: 4:22:19 time: 0.541978 data_time: 0.083544 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.781528 loss: 0.000693 2022/09/27 03:40:55 - mmengine - INFO - Epoch(train) [103][250/293] lr: 5.000000e-04 eta: 4:21:57 time: 0.548237 data_time: 0.107814 memory: 3840 loss_kpt: 0.000676 acc_pose: 0.793927 loss: 0.000676 2022/09/27 03:41:19 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:41:49 - mmengine - INFO - Epoch(train) [104][50/293] lr: 5.000000e-04 eta: 4:20:53 time: 0.604763 data_time: 0.094549 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.821132 loss: 0.000689 2022/09/27 03:42:17 - mmengine - INFO - Epoch(train) [104][100/293] lr: 5.000000e-04 eta: 4:20:31 time: 0.562831 data_time: 0.078686 memory: 3840 loss_kpt: 0.000678 acc_pose: 0.823007 loss: 0.000678 2022/09/27 03:42:45 - mmengine - INFO - Epoch(train) [104][150/293] lr: 5.000000e-04 eta: 4:20:09 time: 0.550057 data_time: 0.086071 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.810665 loss: 0.000692 2022/09/27 03:43:13 - mmengine - INFO - Epoch(train) [104][200/293] lr: 5.000000e-04 eta: 4:19:47 time: 0.564606 data_time: 0.072792 memory: 3840 loss_kpt: 0.000712 acc_pose: 0.800725 loss: 0.000712 2022/09/27 03:43:42 - mmengine - INFO - Epoch(train) [104][250/293] lr: 5.000000e-04 eta: 4:19:26 time: 0.586081 data_time: 0.086051 memory: 3840 loss_kpt: 0.000684 acc_pose: 0.799041 loss: 0.000684 2022/09/27 03:44:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:44:37 - mmengine - INFO - Epoch(train) [105][50/293] lr: 5.000000e-04 eta: 4:18:24 time: 0.613702 data_time: 0.096152 memory: 3840 loss_kpt: 0.000690 acc_pose: 0.741665 loss: 0.000690 2022/09/27 03:45:05 - mmengine - INFO - Epoch(train) [105][100/293] lr: 5.000000e-04 eta: 4:18:02 time: 0.555620 data_time: 0.081392 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.785477 loss: 0.000689 2022/09/27 03:45:32 - mmengine - INFO - Epoch(train) [105][150/293] lr: 5.000000e-04 eta: 4:17:39 time: 0.547163 data_time: 0.105675 memory: 3840 loss_kpt: 0.000695 acc_pose: 0.814036 loss: 0.000695 2022/09/27 03:46:01 - mmengine - INFO - Epoch(train) [105][200/293] lr: 5.000000e-04 eta: 4:17:17 time: 0.567019 data_time: 0.080828 memory: 3840 loss_kpt: 0.000680 acc_pose: 0.786258 loss: 0.000680 2022/09/27 03:46:29 - mmengine - INFO - Epoch(train) [105][250/293] lr: 5.000000e-04 eta: 4:16:56 time: 0.571818 data_time: 0.123070 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.801941 loss: 0.000686 2022/09/27 03:46:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:47:22 - mmengine - INFO - Epoch(train) [106][50/293] lr: 5.000000e-04 eta: 4:15:52 time: 0.581027 data_time: 0.111693 memory: 3840 loss_kpt: 0.000684 acc_pose: 0.819169 loss: 0.000684 2022/09/27 03:47:50 - mmengine - INFO - Epoch(train) [106][100/293] lr: 5.000000e-04 eta: 4:15:30 time: 0.559132 data_time: 0.085636 memory: 3840 loss_kpt: 0.000685 acc_pose: 0.825372 loss: 0.000685 2022/09/27 03:48:18 - mmengine - INFO - Epoch(train) [106][150/293] lr: 5.000000e-04 eta: 4:15:08 time: 0.563228 data_time: 0.148148 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.773667 loss: 0.000668 2022/09/27 03:48:46 - mmengine - INFO - Epoch(train) [106][200/293] lr: 5.000000e-04 eta: 4:14:47 time: 0.569984 data_time: 0.102213 memory: 3840 loss_kpt: 0.000677 acc_pose: 0.780700 loss: 0.000677 2022/09/27 03:49:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:49:15 - mmengine - INFO - Epoch(train) [106][250/293] lr: 5.000000e-04 eta: 4:14:25 time: 0.571850 data_time: 0.115802 memory: 3840 loss_kpt: 0.000683 acc_pose: 0.781112 loss: 0.000683 2022/09/27 03:49:39 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:50:09 - mmengine - INFO - Epoch(train) [107][50/293] lr: 5.000000e-04 eta: 4:13:22 time: 0.593674 data_time: 0.136191 memory: 3840 loss_kpt: 0.000678 acc_pose: 0.819940 loss: 0.000678 2022/09/27 03:50:38 - mmengine - INFO - Epoch(train) [107][100/293] lr: 5.000000e-04 eta: 4:13:01 time: 0.573273 data_time: 0.079038 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.845710 loss: 0.000681 2022/09/27 03:51:05 - mmengine - INFO - Epoch(train) [107][150/293] lr: 5.000000e-04 eta: 4:12:38 time: 0.538666 data_time: 0.072139 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.803157 loss: 0.000681 2022/09/27 03:51:34 - mmengine - INFO - Epoch(train) [107][200/293] lr: 5.000000e-04 eta: 4:12:16 time: 0.581244 data_time: 0.088632 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.802617 loss: 0.000689 2022/09/27 03:52:01 - mmengine - INFO - Epoch(train) [107][250/293] lr: 5.000000e-04 eta: 4:11:54 time: 0.552498 data_time: 0.069122 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.787787 loss: 0.000679 2022/09/27 03:52:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:52:55 - mmengine - INFO - Epoch(train) [108][50/293] lr: 5.000000e-04 eta: 4:10:51 time: 0.591568 data_time: 0.096918 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.848984 loss: 0.000686 2022/09/27 03:53:23 - mmengine - INFO - Epoch(train) [108][100/293] lr: 5.000000e-04 eta: 4:10:29 time: 0.563237 data_time: 0.076939 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.817120 loss: 0.000692 2022/09/27 03:53:51 - mmengine - INFO - Epoch(train) [108][150/293] lr: 5.000000e-04 eta: 4:10:07 time: 0.559166 data_time: 0.069677 memory: 3840 loss_kpt: 0.000678 acc_pose: 0.803101 loss: 0.000678 2022/09/27 03:54:20 - mmengine - INFO - Epoch(train) [108][200/293] lr: 5.000000e-04 eta: 4:09:46 time: 0.576873 data_time: 0.074968 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.816937 loss: 0.000686 2022/09/27 03:54:49 - mmengine - INFO - Epoch(train) [108][250/293] lr: 5.000000e-04 eta: 4:09:25 time: 0.584604 data_time: 0.080473 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.818972 loss: 0.000689 2022/09/27 03:55:14 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:55:43 - mmengine - INFO - Epoch(train) [109][50/293] lr: 5.000000e-04 eta: 4:08:22 time: 0.590659 data_time: 0.098868 memory: 3840 loss_kpt: 0.000673 acc_pose: 0.822536 loss: 0.000673 2022/09/27 03:56:12 - mmengine - INFO - Epoch(train) [109][100/293] lr: 5.000000e-04 eta: 4:08:01 time: 0.579330 data_time: 0.195992 memory: 3840 loss_kpt: 0.000695 acc_pose: 0.795600 loss: 0.000695 2022/09/27 03:56:40 - mmengine - INFO - Epoch(train) [109][150/293] lr: 5.000000e-04 eta: 4:07:39 time: 0.558482 data_time: 0.076660 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.844085 loss: 0.000672 2022/09/27 03:57:07 - mmengine - INFO - Epoch(train) [109][200/293] lr: 5.000000e-04 eta: 4:07:16 time: 0.540718 data_time: 0.091151 memory: 3840 loss_kpt: 0.000693 acc_pose: 0.850494 loss: 0.000693 2022/09/27 03:57:36 - mmengine - INFO - Epoch(train) [109][250/293] lr: 5.000000e-04 eta: 4:06:54 time: 0.572319 data_time: 0.086329 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.763866 loss: 0.000686 2022/09/27 03:58:00 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:58:30 - mmengine - INFO - Epoch(train) [110][50/293] lr: 5.000000e-04 eta: 4:05:52 time: 0.589167 data_time: 0.162360 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.792490 loss: 0.000667 2022/09/27 03:58:38 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 03:58:59 - mmengine - INFO - Epoch(train) [110][100/293] lr: 5.000000e-04 eta: 4:05:31 time: 0.576932 data_time: 0.088300 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.769124 loss: 0.000682 2022/09/27 03:59:27 - mmengine - INFO - Epoch(train) [110][150/293] lr: 5.000000e-04 eta: 4:05:08 time: 0.560270 data_time: 0.079311 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.863128 loss: 0.000672 2022/09/27 03:59:56 - mmengine - INFO - Epoch(train) [110][200/293] lr: 5.000000e-04 eta: 4:04:47 time: 0.580977 data_time: 0.105396 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.779816 loss: 0.000682 2022/09/27 04:00:25 - mmengine - INFO - Epoch(train) [110][250/293] lr: 5.000000e-04 eta: 4:04:26 time: 0.576652 data_time: 0.109872 memory: 3840 loss_kpt: 0.000680 acc_pose: 0.824282 loss: 0.000680 2022/09/27 04:00:48 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:00:48 - mmengine - INFO - Saving checkpoint at 110 epochs 2022/09/27 04:01:09 - mmengine - INFO - Epoch(val) [110][50/407] eta: 0:01:52 time: 0.315490 data_time: 0.178040 memory: 3840 2022/09/27 04:01:25 - mmengine - INFO - Epoch(val) [110][100/407] eta: 0:01:35 time: 0.312189 data_time: 0.185524 memory: 1017 2022/09/27 04:01:41 - mmengine - INFO - Epoch(val) [110][150/407] eta: 0:01:21 time: 0.316096 data_time: 0.177414 memory: 1017 2022/09/27 04:01:56 - mmengine - INFO - Epoch(val) [110][200/407] eta: 0:01:04 time: 0.309318 data_time: 0.173163 memory: 1017 2022/09/27 04:02:12 - mmengine - INFO - Epoch(val) [110][250/407] eta: 0:00:50 time: 0.320038 data_time: 0.186241 memory: 1017 2022/09/27 04:02:27 - mmengine - INFO - Epoch(val) [110][300/407] eta: 0:00:32 time: 0.303106 data_time: 0.171843 memory: 1017 2022/09/27 04:02:43 - mmengine - INFO - Epoch(val) [110][350/407] eta: 0:00:17 time: 0.311966 data_time: 0.178384 memory: 1017 2022/09/27 04:02:56 - mmengine - INFO - Epoch(val) [110][400/407] eta: 0:00:01 time: 0.255075 data_time: 0.156766 memory: 1017 2022/09/27 04:03:29 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 04:03:42 - mmengine - INFO - Epoch(val) [110][407/407] coco/AP: 0.696918 coco/AP .5: 0.892348 coco/AP .75: 0.774451 coco/AP (M): 0.660100 coco/AP (L): 0.763874 coco/AR: 0.753999 coco/AR .5: 0.930730 coco/AR .75: 0.822890 coco/AR (M): 0.710844 coco/AR (L): 0.815496 2022/09/27 04:03:42 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_100.pth is removed 2022/09/27 04:03:44 - mmengine - INFO - The best checkpoint with 0.6969 coco/AP at 110 epoch is saved to best_coco/AP_epoch_110.pth. 2022/09/27 04:04:12 - mmengine - INFO - Epoch(train) [111][50/293] lr: 5.000000e-04 eta: 4:03:22 time: 0.554930 data_time: 0.163225 memory: 3840 loss_kpt: 0.000676 acc_pose: 0.797127 loss: 0.000676 2022/09/27 04:04:40 - mmengine - INFO - Epoch(train) [111][100/293] lr: 5.000000e-04 eta: 4:03:00 time: 0.565381 data_time: 0.095789 memory: 3840 loss_kpt: 0.000680 acc_pose: 0.803267 loss: 0.000680 2022/09/27 04:05:08 - mmengine - INFO - Epoch(train) [111][150/293] lr: 5.000000e-04 eta: 4:02:37 time: 0.542891 data_time: 0.064466 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.840039 loss: 0.000686 2022/09/27 04:05:35 - mmengine - INFO - Epoch(train) [111][200/293] lr: 5.000000e-04 eta: 4:02:14 time: 0.539566 data_time: 0.087604 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.845318 loss: 0.000674 2022/09/27 04:06:03 - mmengine - INFO - Epoch(train) [111][250/293] lr: 5.000000e-04 eta: 4:01:52 time: 0.565884 data_time: 0.067350 memory: 3840 loss_kpt: 0.000688 acc_pose: 0.817720 loss: 0.000688 2022/09/27 04:06:26 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:06:55 - mmengine - INFO - Epoch(train) [112][50/293] lr: 5.000000e-04 eta: 4:00:50 time: 0.584498 data_time: 0.175260 memory: 3840 loss_kpt: 0.000688 acc_pose: 0.796634 loss: 0.000688 2022/09/27 04:07:23 - mmengine - INFO - Epoch(train) [112][100/293] lr: 5.000000e-04 eta: 4:00:28 time: 0.548139 data_time: 0.184737 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.746684 loss: 0.000679 2022/09/27 04:07:50 - mmengine - INFO - Epoch(train) [112][150/293] lr: 5.000000e-04 eta: 4:00:05 time: 0.544228 data_time: 0.122636 memory: 3840 loss_kpt: 0.000689 acc_pose: 0.810959 loss: 0.000689 2022/09/27 04:08:18 - mmengine - INFO - Epoch(train) [112][200/293] lr: 5.000000e-04 eta: 3:59:42 time: 0.555245 data_time: 0.088976 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.785846 loss: 0.000679 2022/09/27 04:08:46 - mmengine - INFO - Epoch(train) [112][250/293] lr: 5.000000e-04 eta: 3:59:20 time: 0.561231 data_time: 0.098677 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.835205 loss: 0.000666 2022/09/27 04:09:09 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:09:39 - mmengine - INFO - Epoch(train) [113][50/293] lr: 5.000000e-04 eta: 3:58:19 time: 0.593001 data_time: 0.095641 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.831514 loss: 0.000681 2022/09/27 04:10:06 - mmengine - INFO - Epoch(train) [113][100/293] lr: 5.000000e-04 eta: 3:57:56 time: 0.540327 data_time: 0.091076 memory: 3840 loss_kpt: 0.000677 acc_pose: 0.775592 loss: 0.000677 2022/09/27 04:10:33 - mmengine - INFO - Epoch(train) [113][150/293] lr: 5.000000e-04 eta: 3:57:33 time: 0.547345 data_time: 0.064665 memory: 3840 loss_kpt: 0.000675 acc_pose: 0.807125 loss: 0.000675 2022/09/27 04:10:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:11:02 - mmengine - INFO - Epoch(train) [113][200/293] lr: 5.000000e-04 eta: 3:57:11 time: 0.571254 data_time: 0.097189 memory: 3840 loss_kpt: 0.000684 acc_pose: 0.779162 loss: 0.000684 2022/09/27 04:11:29 - mmengine - INFO - Epoch(train) [113][250/293] lr: 5.000000e-04 eta: 3:56:48 time: 0.549686 data_time: 0.081579 memory: 3840 loss_kpt: 0.000677 acc_pose: 0.802797 loss: 0.000677 2022/09/27 04:11:52 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:12:21 - mmengine - INFO - Epoch(train) [114][50/293] lr: 5.000000e-04 eta: 3:55:47 time: 0.589229 data_time: 0.083783 memory: 3840 loss_kpt: 0.000686 acc_pose: 0.811509 loss: 0.000686 2022/09/27 04:12:48 - mmengine - INFO - Epoch(train) [114][100/293] lr: 5.000000e-04 eta: 3:55:24 time: 0.538817 data_time: 0.075552 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.796633 loss: 0.000674 2022/09/27 04:13:15 - mmengine - INFO - Epoch(train) [114][150/293] lr: 5.000000e-04 eta: 3:55:01 time: 0.542961 data_time: 0.074331 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.755054 loss: 0.000671 2022/09/27 04:13:43 - mmengine - INFO - Epoch(train) [114][200/293] lr: 5.000000e-04 eta: 3:54:39 time: 0.560070 data_time: 0.134364 memory: 3840 loss_kpt: 0.000683 acc_pose: 0.804587 loss: 0.000683 2022/09/27 04:14:12 - mmengine - INFO - Epoch(train) [114][250/293] lr: 5.000000e-04 eta: 3:54:17 time: 0.569362 data_time: 0.189115 memory: 3840 loss_kpt: 0.000683 acc_pose: 0.867523 loss: 0.000683 2022/09/27 04:14:36 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:15:04 - mmengine - INFO - Epoch(train) [115][50/293] lr: 5.000000e-04 eta: 3:53:15 time: 0.575214 data_time: 0.119940 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.865014 loss: 0.000682 2022/09/27 04:15:31 - mmengine - INFO - Epoch(train) [115][100/293] lr: 5.000000e-04 eta: 3:52:52 time: 0.537076 data_time: 0.142457 memory: 3840 loss_kpt: 0.000678 acc_pose: 0.800691 loss: 0.000678 2022/09/27 04:15:59 - mmengine - INFO - Epoch(train) [115][150/293] lr: 5.000000e-04 eta: 3:52:30 time: 0.565398 data_time: 0.083360 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.817686 loss: 0.000679 2022/09/27 04:16:27 - mmengine - INFO - Epoch(train) [115][200/293] lr: 5.000000e-04 eta: 3:52:07 time: 0.556358 data_time: 0.068980 memory: 3840 loss_kpt: 0.000697 acc_pose: 0.787417 loss: 0.000697 2022/09/27 04:16:56 - mmengine - INFO - Epoch(train) [115][250/293] lr: 5.000000e-04 eta: 3:51:45 time: 0.567140 data_time: 0.071329 memory: 3840 loss_kpt: 0.000684 acc_pose: 0.794643 loss: 0.000684 2022/09/27 04:17:20 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:17:48 - mmengine - INFO - Epoch(train) [116][50/293] lr: 5.000000e-04 eta: 3:50:44 time: 0.571293 data_time: 0.156096 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.814546 loss: 0.000682 2022/09/27 04:18:16 - mmengine - INFO - Epoch(train) [116][100/293] lr: 5.000000e-04 eta: 3:50:22 time: 0.557358 data_time: 0.158443 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.853559 loss: 0.000670 2022/09/27 04:18:44 - mmengine - INFO - Epoch(train) [116][150/293] lr: 5.000000e-04 eta: 3:49:59 time: 0.547828 data_time: 0.096402 memory: 3840 loss_kpt: 0.000675 acc_pose: 0.807286 loss: 0.000675 2022/09/27 04:19:11 - mmengine - INFO - Epoch(train) [116][200/293] lr: 5.000000e-04 eta: 3:49:36 time: 0.553490 data_time: 0.067733 memory: 3840 loss_kpt: 0.000687 acc_pose: 0.805279 loss: 0.000687 2022/09/27 04:19:41 - mmengine - INFO - Epoch(train) [116][250/293] lr: 5.000000e-04 eta: 3:49:15 time: 0.591870 data_time: 0.157786 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.830102 loss: 0.000668 2022/09/27 04:20:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:20:12 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:20:33 - mmengine - INFO - Epoch(train) [117][50/293] lr: 5.000000e-04 eta: 3:48:15 time: 0.579316 data_time: 0.093094 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.840577 loss: 0.000669 2022/09/27 04:21:01 - mmengine - INFO - Epoch(train) [117][100/293] lr: 5.000000e-04 eta: 3:47:52 time: 0.557635 data_time: 0.172134 memory: 3840 loss_kpt: 0.000685 acc_pose: 0.813817 loss: 0.000685 2022/09/27 04:21:29 - mmengine - INFO - Epoch(train) [117][150/293] lr: 5.000000e-04 eta: 3:47:29 time: 0.553582 data_time: 0.085376 memory: 3840 loss_kpt: 0.000675 acc_pose: 0.791009 loss: 0.000675 2022/09/27 04:21:56 - mmengine - INFO - Epoch(train) [117][200/293] lr: 5.000000e-04 eta: 3:47:06 time: 0.546958 data_time: 0.109463 memory: 3840 loss_kpt: 0.000675 acc_pose: 0.809171 loss: 0.000675 2022/09/27 04:22:24 - mmengine - INFO - Epoch(train) [117][250/293] lr: 5.000000e-04 eta: 3:46:44 time: 0.557350 data_time: 0.086433 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.818286 loss: 0.000679 2022/09/27 04:22:48 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:23:18 - mmengine - INFO - Epoch(train) [118][50/293] lr: 5.000000e-04 eta: 3:45:44 time: 0.587134 data_time: 0.088557 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.834632 loss: 0.000671 2022/09/27 04:23:46 - mmengine - INFO - Epoch(train) [118][100/293] lr: 5.000000e-04 eta: 3:45:22 time: 0.559922 data_time: 0.107653 memory: 3840 loss_kpt: 0.000687 acc_pose: 0.763282 loss: 0.000687 2022/09/27 04:24:14 - mmengine - INFO - Epoch(train) [118][150/293] lr: 5.000000e-04 eta: 3:45:00 time: 0.571367 data_time: 0.080826 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.816062 loss: 0.000672 2022/09/27 04:24:41 - mmengine - INFO - Epoch(train) [118][200/293] lr: 5.000000e-04 eta: 3:44:36 time: 0.533692 data_time: 0.161543 memory: 3840 loss_kpt: 0.000692 acc_pose: 0.817321 loss: 0.000692 2022/09/27 04:25:09 - mmengine - INFO - Epoch(train) [118][250/293] lr: 5.000000e-04 eta: 3:44:14 time: 0.558849 data_time: 0.100731 memory: 3840 loss_kpt: 0.000680 acc_pose: 0.818062 loss: 0.000680 2022/09/27 04:25:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:26:01 - mmengine - INFO - Epoch(train) [119][50/293] lr: 5.000000e-04 eta: 3:43:13 time: 0.564486 data_time: 0.092290 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.827933 loss: 0.000674 2022/09/27 04:26:30 - mmengine - INFO - Epoch(train) [119][100/293] lr: 5.000000e-04 eta: 3:42:51 time: 0.569432 data_time: 0.082388 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.797890 loss: 0.000669 2022/09/27 04:26:57 - mmengine - INFO - Epoch(train) [119][150/293] lr: 5.000000e-04 eta: 3:42:28 time: 0.552123 data_time: 0.117485 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.791508 loss: 0.000672 2022/09/27 04:27:26 - mmengine - INFO - Epoch(train) [119][200/293] lr: 5.000000e-04 eta: 3:42:06 time: 0.570518 data_time: 0.131226 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.779014 loss: 0.000679 2022/09/27 04:27:54 - mmengine - INFO - Epoch(train) [119][250/293] lr: 5.000000e-04 eta: 3:41:44 time: 0.565561 data_time: 0.071883 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.812569 loss: 0.000657 2022/09/27 04:28:18 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:28:46 - mmengine - INFO - Epoch(train) [120][50/293] lr: 5.000000e-04 eta: 3:40:44 time: 0.562557 data_time: 0.090560 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.825085 loss: 0.000662 2022/09/27 04:29:14 - mmengine - INFO - Epoch(train) [120][100/293] lr: 5.000000e-04 eta: 3:40:21 time: 0.560657 data_time: 0.113186 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.776388 loss: 0.000668 2022/09/27 04:29:32 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:29:42 - mmengine - INFO - Epoch(train) [120][150/293] lr: 5.000000e-04 eta: 3:39:59 time: 0.561489 data_time: 0.128007 memory: 3840 loss_kpt: 0.000696 acc_pose: 0.819156 loss: 0.000696 2022/09/27 04:30:11 - mmengine - INFO - Epoch(train) [120][200/293] lr: 5.000000e-04 eta: 3:39:37 time: 0.576305 data_time: 0.089832 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.787079 loss: 0.000679 2022/09/27 04:30:38 - mmengine - INFO - Epoch(train) [120][250/293] lr: 5.000000e-04 eta: 3:39:13 time: 0.542792 data_time: 0.094180 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.769910 loss: 0.000667 2022/09/27 04:31:02 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:31:02 - mmengine - INFO - Saving checkpoint at 120 epochs 2022/09/27 04:31:22 - mmengine - INFO - Epoch(val) [120][50/407] eta: 0:01:49 time: 0.307420 data_time: 0.164168 memory: 3840 2022/09/27 04:31:37 - mmengine - INFO - Epoch(val) [120][100/407] eta: 0:01:31 time: 0.299518 data_time: 0.153897 memory: 1017 2022/09/27 04:31:53 - mmengine - INFO - Epoch(val) [120][150/407] eta: 0:01:19 time: 0.309933 data_time: 0.179020 memory: 1017 2022/09/27 04:32:08 - mmengine - INFO - Epoch(val) [120][200/407] eta: 0:01:04 time: 0.309542 data_time: 0.180479 memory: 1017 2022/09/27 04:32:23 - mmengine - INFO - Epoch(val) [120][250/407] eta: 0:00:46 time: 0.297495 data_time: 0.181453 memory: 1017 2022/09/27 04:32:38 - mmengine - INFO - Epoch(val) [120][300/407] eta: 0:00:32 time: 0.305123 data_time: 0.174391 memory: 1017 2022/09/27 04:32:53 - mmengine - INFO - Epoch(val) [120][350/407] eta: 0:00:16 time: 0.289064 data_time: 0.177813 memory: 1017 2022/09/27 04:33:05 - mmengine - INFO - Epoch(val) [120][400/407] eta: 0:00:01 time: 0.252002 data_time: 0.137510 memory: 1017 2022/09/27 04:33:39 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 04:33:52 - mmengine - INFO - Epoch(val) [120][407/407] coco/AP: 0.694857 coco/AP .5: 0.889077 coco/AP .75: 0.772125 coco/AP (M): 0.656249 coco/AP (L): 0.763607 coco/AR: 0.752377 coco/AR .5: 0.929471 coco/AR .75: 0.821788 coco/AR (M): 0.706392 coco/AR (L): 0.817503 2022/09/27 04:34:21 - mmengine - INFO - Epoch(train) [121][50/293] lr: 5.000000e-04 eta: 3:38:14 time: 0.575818 data_time: 0.086837 memory: 3840 loss_kpt: 0.000645 acc_pose: 0.792038 loss: 0.000645 2022/09/27 04:34:50 - mmengine - INFO - Epoch(train) [121][100/293] lr: 5.000000e-04 eta: 3:37:52 time: 0.573137 data_time: 0.121784 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.830226 loss: 0.000674 2022/09/27 04:35:18 - mmengine - INFO - Epoch(train) [121][150/293] lr: 5.000000e-04 eta: 3:37:30 time: 0.569191 data_time: 0.108592 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.785271 loss: 0.000669 2022/09/27 04:35:46 - mmengine - INFO - Epoch(train) [121][200/293] lr: 5.000000e-04 eta: 3:37:07 time: 0.550222 data_time: 0.118282 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.790747 loss: 0.000679 2022/09/27 04:36:13 - mmengine - INFO - Epoch(train) [121][250/293] lr: 5.000000e-04 eta: 3:36:44 time: 0.544202 data_time: 0.172688 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.792354 loss: 0.000681 2022/09/27 04:36:36 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:37:05 - mmengine - INFO - Epoch(train) [122][50/293] lr: 5.000000e-04 eta: 3:35:44 time: 0.574801 data_time: 0.094938 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.814882 loss: 0.000666 2022/09/27 04:37:33 - mmengine - INFO - Epoch(train) [122][100/293] lr: 5.000000e-04 eta: 3:35:22 time: 0.558711 data_time: 0.081749 memory: 3840 loss_kpt: 0.000684 acc_pose: 0.786564 loss: 0.000684 2022/09/27 04:38:01 - mmengine - INFO - Epoch(train) [122][150/293] lr: 5.000000e-04 eta: 3:35:00 time: 0.567875 data_time: 0.106477 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.777084 loss: 0.000668 2022/09/27 04:38:29 - mmengine - INFO - Epoch(train) [122][200/293] lr: 5.000000e-04 eta: 3:34:37 time: 0.553914 data_time: 0.081839 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.775129 loss: 0.000674 2022/09/27 04:38:56 - mmengine - INFO - Epoch(train) [122][250/293] lr: 5.000000e-04 eta: 3:34:13 time: 0.538233 data_time: 0.079073 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.825845 loss: 0.000661 2022/09/27 04:39:19 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:39:47 - mmengine - INFO - Epoch(train) [123][50/293] lr: 5.000000e-04 eta: 3:33:14 time: 0.571862 data_time: 0.099840 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.842989 loss: 0.000671 2022/09/27 04:40:16 - mmengine - INFO - Epoch(train) [123][100/293] lr: 5.000000e-04 eta: 3:32:52 time: 0.570705 data_time: 0.088389 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.825215 loss: 0.000660 2022/09/27 04:40:43 - mmengine - INFO - Epoch(train) [123][150/293] lr: 5.000000e-04 eta: 3:32:29 time: 0.555715 data_time: 0.070049 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.804129 loss: 0.000679 2022/09/27 04:41:13 - mmengine - INFO - Epoch(train) [123][200/293] lr: 5.000000e-04 eta: 3:32:07 time: 0.581351 data_time: 0.074567 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.789639 loss: 0.000674 2022/09/27 04:41:40 - mmengine - INFO - Epoch(train) [123][250/293] lr: 5.000000e-04 eta: 3:31:45 time: 0.558031 data_time: 0.079317 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.816902 loss: 0.000668 2022/09/27 04:41:43 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:42:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:42:32 - mmengine - INFO - Epoch(train) [124][50/293] lr: 5.000000e-04 eta: 3:30:46 time: 0.562230 data_time: 0.095346 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.795464 loss: 0.000651 2022/09/27 04:43:00 - mmengine - INFO - Epoch(train) [124][100/293] lr: 5.000000e-04 eta: 3:30:23 time: 0.555552 data_time: 0.081847 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.770769 loss: 0.000662 2022/09/27 04:43:27 - mmengine - INFO - Epoch(train) [124][150/293] lr: 5.000000e-04 eta: 3:30:00 time: 0.550008 data_time: 0.130495 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.776940 loss: 0.000660 2022/09/27 04:43:54 - mmengine - INFO - Epoch(train) [124][200/293] lr: 5.000000e-04 eta: 3:29:36 time: 0.540725 data_time: 0.101227 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.798657 loss: 0.000665 2022/09/27 04:44:23 - mmengine - INFO - Epoch(train) [124][250/293] lr: 5.000000e-04 eta: 3:29:14 time: 0.574733 data_time: 0.076167 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.837298 loss: 0.000662 2022/09/27 04:44:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:45:15 - mmengine - INFO - Epoch(train) [125][50/293] lr: 5.000000e-04 eta: 3:28:16 time: 0.571509 data_time: 0.099409 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.788916 loss: 0.000672 2022/09/27 04:45:42 - mmengine - INFO - Epoch(train) [125][100/293] lr: 5.000000e-04 eta: 3:27:53 time: 0.558282 data_time: 0.081965 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.844119 loss: 0.000681 2022/09/27 04:46:10 - mmengine - INFO - Epoch(train) [125][150/293] lr: 5.000000e-04 eta: 3:27:30 time: 0.559383 data_time: 0.106964 memory: 3840 loss_kpt: 0.000674 acc_pose: 0.824905 loss: 0.000674 2022/09/27 04:46:38 - mmengine - INFO - Epoch(train) [125][200/293] lr: 5.000000e-04 eta: 3:27:07 time: 0.548808 data_time: 0.155512 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.836253 loss: 0.000662 2022/09/27 04:47:06 - mmengine - INFO - Epoch(train) [125][250/293] lr: 5.000000e-04 eta: 3:26:44 time: 0.554432 data_time: 0.078137 memory: 3840 loss_kpt: 0.000694 acc_pose: 0.844716 loss: 0.000694 2022/09/27 04:47:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:47:58 - mmengine - INFO - Epoch(train) [126][50/293] lr: 5.000000e-04 eta: 3:25:47 time: 0.589148 data_time: 0.086235 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.790222 loss: 0.000671 2022/09/27 04:48:27 - mmengine - INFO - Epoch(train) [126][100/293] lr: 5.000000e-04 eta: 3:25:25 time: 0.578376 data_time: 0.090771 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.786800 loss: 0.000672 2022/09/27 04:48:55 - mmengine - INFO - Epoch(train) [126][150/293] lr: 5.000000e-04 eta: 3:25:02 time: 0.564000 data_time: 0.078831 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.848231 loss: 0.000667 2022/09/27 04:49:22 - mmengine - INFO - Epoch(train) [126][200/293] lr: 5.000000e-04 eta: 3:24:39 time: 0.535749 data_time: 0.126169 memory: 3840 loss_kpt: 0.000676 acc_pose: 0.798666 loss: 0.000676 2022/09/27 04:49:51 - mmengine - INFO - Epoch(train) [126][250/293] lr: 5.000000e-04 eta: 3:24:16 time: 0.568321 data_time: 0.108940 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.817514 loss: 0.000667 2022/09/27 04:50:15 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:50:44 - mmengine - INFO - Epoch(train) [127][50/293] lr: 5.000000e-04 eta: 3:23:19 time: 0.590081 data_time: 0.121554 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.821848 loss: 0.000682 2022/09/27 04:51:03 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:51:12 - mmengine - INFO - Epoch(train) [127][100/293] lr: 5.000000e-04 eta: 3:22:56 time: 0.548827 data_time: 0.073549 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.801189 loss: 0.000671 2022/09/27 04:51:40 - mmengine - INFO - Epoch(train) [127][150/293] lr: 5.000000e-04 eta: 3:22:33 time: 0.557923 data_time: 0.141934 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.796034 loss: 0.000665 2022/09/27 04:52:08 - mmengine - INFO - Epoch(train) [127][200/293] lr: 5.000000e-04 eta: 3:22:10 time: 0.559588 data_time: 0.158189 memory: 3840 loss_kpt: 0.000683 acc_pose: 0.845535 loss: 0.000683 2022/09/27 04:52:34 - mmengine - INFO - Epoch(train) [127][250/293] lr: 5.000000e-04 eta: 3:21:46 time: 0.535137 data_time: 0.137879 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.796274 loss: 0.000672 2022/09/27 04:52:58 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:53:26 - mmengine - INFO - Epoch(train) [128][50/293] lr: 5.000000e-04 eta: 3:20:48 time: 0.559928 data_time: 0.122676 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.794093 loss: 0.000670 2022/09/27 04:53:52 - mmengine - INFO - Epoch(train) [128][100/293] lr: 5.000000e-04 eta: 3:20:25 time: 0.533272 data_time: 0.077571 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.795888 loss: 0.000657 2022/09/27 04:54:19 - mmengine - INFO - Epoch(train) [128][150/293] lr: 5.000000e-04 eta: 3:20:01 time: 0.542545 data_time: 0.077257 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.843749 loss: 0.000660 2022/09/27 04:54:47 - mmengine - INFO - Epoch(train) [128][200/293] lr: 5.000000e-04 eta: 3:19:38 time: 0.555964 data_time: 0.074758 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.843751 loss: 0.000670 2022/09/27 04:55:16 - mmengine - INFO - Epoch(train) [128][250/293] lr: 5.000000e-04 eta: 3:19:16 time: 0.583668 data_time: 0.089296 memory: 3840 loss_kpt: 0.000687 acc_pose: 0.770828 loss: 0.000687 2022/09/27 04:55:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:56:10 - mmengine - INFO - Epoch(train) [129][50/293] lr: 5.000000e-04 eta: 3:18:19 time: 0.588212 data_time: 0.093518 memory: 3840 loss_kpt: 0.000673 acc_pose: 0.816515 loss: 0.000673 2022/09/27 04:56:37 - mmengine - INFO - Epoch(train) [129][100/293] lr: 5.000000e-04 eta: 3:17:56 time: 0.542876 data_time: 0.153250 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.812612 loss: 0.000679 2022/09/27 04:57:06 - mmengine - INFO - Epoch(train) [129][150/293] lr: 5.000000e-04 eta: 3:17:34 time: 0.574692 data_time: 0.151458 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.823879 loss: 0.000666 2022/09/27 04:57:33 - mmengine - INFO - Epoch(train) [129][200/293] lr: 5.000000e-04 eta: 3:17:10 time: 0.541245 data_time: 0.099232 memory: 3840 loss_kpt: 0.000683 acc_pose: 0.839411 loss: 0.000683 2022/09/27 04:58:00 - mmengine - INFO - Epoch(train) [129][250/293] lr: 5.000000e-04 eta: 3:16:47 time: 0.543349 data_time: 0.103326 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.825922 loss: 0.000667 2022/09/27 04:58:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 04:58:52 - mmengine - INFO - Epoch(train) [130][50/293] lr: 5.000000e-04 eta: 3:15:50 time: 0.589260 data_time: 0.159728 memory: 3840 loss_kpt: 0.000673 acc_pose: 0.848376 loss: 0.000673 2022/09/27 04:59:20 - mmengine - INFO - Epoch(train) [130][100/293] lr: 5.000000e-04 eta: 3:15:27 time: 0.555109 data_time: 0.087920 memory: 3840 loss_kpt: 0.000678 acc_pose: 0.756782 loss: 0.000678 2022/09/27 04:59:48 - mmengine - INFO - Epoch(train) [130][150/293] lr: 5.000000e-04 eta: 3:15:05 time: 0.566774 data_time: 0.090220 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.783687 loss: 0.000670 2022/09/27 05:00:16 - mmengine - INFO - Epoch(train) [130][200/293] lr: 5.000000e-04 eta: 3:14:42 time: 0.552599 data_time: 0.072076 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.731948 loss: 0.000669 2022/09/27 05:00:17 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:00:44 - mmengine - INFO - Epoch(train) [130][250/293] lr: 5.000000e-04 eta: 3:14:19 time: 0.563328 data_time: 0.103468 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.809028 loss: 0.000667 2022/09/27 05:01:08 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:01:08 - mmengine - INFO - Saving checkpoint at 130 epochs 2022/09/27 05:01:29 - mmengine - INFO - Epoch(val) [130][50/407] eta: 0:01:51 time: 0.313410 data_time: 0.186302 memory: 3840 2022/09/27 05:01:44 - mmengine - INFO - Epoch(val) [130][100/407] eta: 0:01:32 time: 0.301500 data_time: 0.172612 memory: 1017 2022/09/27 05:02:00 - mmengine - INFO - Epoch(val) [130][150/407] eta: 0:01:20 time: 0.311728 data_time: 0.160243 memory: 1017 2022/09/27 05:02:14 - mmengine - INFO - Epoch(val) [130][200/407] eta: 0:00:59 time: 0.285068 data_time: 0.145369 memory: 1017 2022/09/27 05:02:29 - mmengine - INFO - Epoch(val) [130][250/407] eta: 0:00:48 time: 0.311286 data_time: 0.185707 memory: 1017 2022/09/27 05:02:44 - mmengine - INFO - Epoch(val) [130][300/407] eta: 0:00:32 time: 0.301541 data_time: 0.180059 memory: 1017 2022/09/27 05:03:00 - mmengine - INFO - Epoch(val) [130][350/407] eta: 0:00:17 time: 0.307303 data_time: 0.204677 memory: 1017 2022/09/27 05:03:13 - mmengine - INFO - Epoch(val) [130][400/407] eta: 0:00:01 time: 0.255282 data_time: 0.149596 memory: 1017 2022/09/27 05:03:46 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 05:03:59 - mmengine - INFO - Epoch(val) [130][407/407] coco/AP: 0.699408 coco/AP .5: 0.891672 coco/AP .75: 0.777542 coco/AP (M): 0.662041 coco/AP (L): 0.764900 coco/AR: 0.756392 coco/AR .5: 0.930730 coco/AR .75: 0.826669 coco/AR (M): 0.713084 coco/AR (L): 0.817726 2022/09/27 05:03:59 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_110.pth is removed 2022/09/27 05:04:01 - mmengine - INFO - The best checkpoint with 0.6994 coco/AP at 130 epoch is saved to best_coco/AP_epoch_130.pth. 2022/09/27 05:04:29 - mmengine - INFO - Epoch(train) [131][50/293] lr: 5.000000e-04 eta: 3:13:21 time: 0.553153 data_time: 0.131272 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.813766 loss: 0.000661 2022/09/27 05:04:57 - mmengine - INFO - Epoch(train) [131][100/293] lr: 5.000000e-04 eta: 3:12:59 time: 0.564383 data_time: 0.077339 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.849291 loss: 0.000656 2022/09/27 05:05:24 - mmengine - INFO - Epoch(train) [131][150/293] lr: 5.000000e-04 eta: 3:12:35 time: 0.533919 data_time: 0.088569 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.809123 loss: 0.000668 2022/09/27 05:05:51 - mmengine - INFO - Epoch(train) [131][200/293] lr: 5.000000e-04 eta: 3:12:12 time: 0.549511 data_time: 0.079864 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.801334 loss: 0.000665 2022/09/27 05:06:18 - mmengine - INFO - Epoch(train) [131][250/293] lr: 5.000000e-04 eta: 3:11:48 time: 0.536556 data_time: 0.120089 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.854117 loss: 0.000659 2022/09/27 05:06:42 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:07:11 - mmengine - INFO - Epoch(train) [132][50/293] lr: 5.000000e-04 eta: 3:10:52 time: 0.573956 data_time: 0.089976 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.845940 loss: 0.000658 2022/09/27 05:07:38 - mmengine - INFO - Epoch(train) [132][100/293] lr: 5.000000e-04 eta: 3:10:28 time: 0.531847 data_time: 0.090110 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.806972 loss: 0.000658 2022/09/27 05:08:06 - mmengine - INFO - Epoch(train) [132][150/293] lr: 5.000000e-04 eta: 3:10:05 time: 0.557436 data_time: 0.089643 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.788189 loss: 0.000656 2022/09/27 05:08:34 - mmengine - INFO - Epoch(train) [132][200/293] lr: 5.000000e-04 eta: 3:09:42 time: 0.565791 data_time: 0.117120 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.800069 loss: 0.000666 2022/09/27 05:09:01 - mmengine - INFO - Epoch(train) [132][250/293] lr: 5.000000e-04 eta: 3:09:19 time: 0.551107 data_time: 0.110146 memory: 3840 loss_kpt: 0.000679 acc_pose: 0.819798 loss: 0.000679 2022/09/27 05:09:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:09:55 - mmengine - INFO - Epoch(train) [133][50/293] lr: 5.000000e-04 eta: 3:08:23 time: 0.594788 data_time: 0.113940 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.819103 loss: 0.000671 2022/09/27 05:10:22 - mmengine - INFO - Epoch(train) [133][100/293] lr: 5.000000e-04 eta: 3:08:00 time: 0.548885 data_time: 0.085703 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.805463 loss: 0.000658 2022/09/27 05:10:50 - mmengine - INFO - Epoch(train) [133][150/293] lr: 5.000000e-04 eta: 3:07:37 time: 0.544818 data_time: 0.132284 memory: 3840 loss_kpt: 0.000655 acc_pose: 0.851864 loss: 0.000655 2022/09/27 05:11:17 - mmengine - INFO - Epoch(train) [133][200/293] lr: 5.000000e-04 eta: 3:07:13 time: 0.549481 data_time: 0.161853 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.833641 loss: 0.000667 2022/09/27 05:11:45 - mmengine - INFO - Epoch(train) [133][250/293] lr: 5.000000e-04 eta: 3:06:50 time: 0.553805 data_time: 0.071097 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.827183 loss: 0.000663 2022/09/27 05:12:08 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:12:27 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:12:37 - mmengine - INFO - Epoch(train) [134][50/293] lr: 5.000000e-04 eta: 3:05:54 time: 0.577844 data_time: 0.095494 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.823434 loss: 0.000671 2022/09/27 05:13:06 - mmengine - INFO - Epoch(train) [134][100/293] lr: 5.000000e-04 eta: 3:05:32 time: 0.566351 data_time: 0.082704 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.840667 loss: 0.000664 2022/09/27 05:13:33 - mmengine - INFO - Epoch(train) [134][150/293] lr: 5.000000e-04 eta: 3:05:08 time: 0.537275 data_time: 0.082870 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.813335 loss: 0.000671 2022/09/27 05:14:00 - mmengine - INFO - Epoch(train) [134][200/293] lr: 5.000000e-04 eta: 3:04:45 time: 0.543536 data_time: 0.125651 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.873775 loss: 0.000670 2022/09/27 05:14:28 - mmengine - INFO - Epoch(train) [134][250/293] lr: 5.000000e-04 eta: 3:04:22 time: 0.561070 data_time: 0.087629 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.841060 loss: 0.000670 2022/09/27 05:14:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:15:21 - mmengine - INFO - Epoch(train) [135][50/293] lr: 5.000000e-04 eta: 3:03:25 time: 0.560518 data_time: 0.083363 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.830535 loss: 0.000664 2022/09/27 05:15:48 - mmengine - INFO - Epoch(train) [135][100/293] lr: 5.000000e-04 eta: 3:03:02 time: 0.551936 data_time: 0.098369 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.808546 loss: 0.000663 2022/09/27 05:16:16 - mmengine - INFO - Epoch(train) [135][150/293] lr: 5.000000e-04 eta: 3:02:39 time: 0.551485 data_time: 0.085544 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.814337 loss: 0.000664 2022/09/27 05:16:43 - mmengine - INFO - Epoch(train) [135][200/293] lr: 5.000000e-04 eta: 3:02:16 time: 0.544442 data_time: 0.079830 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.818801 loss: 0.000667 2022/09/27 05:17:12 - mmengine - INFO - Epoch(train) [135][250/293] lr: 5.000000e-04 eta: 3:01:53 time: 0.570382 data_time: 0.110513 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.816208 loss: 0.000666 2022/09/27 05:17:34 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:18:03 - mmengine - INFO - Epoch(train) [136][50/293] lr: 5.000000e-04 eta: 3:00:57 time: 0.573364 data_time: 0.093349 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.795625 loss: 0.000671 2022/09/27 05:18:29 - mmengine - INFO - Epoch(train) [136][100/293] lr: 5.000000e-04 eta: 3:00:33 time: 0.525248 data_time: 0.068404 memory: 3840 loss_kpt: 0.000682 acc_pose: 0.820007 loss: 0.000682 2022/09/27 05:18:57 - mmengine - INFO - Epoch(train) [136][150/293] lr: 5.000000e-04 eta: 3:00:10 time: 0.561790 data_time: 0.083933 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.814139 loss: 0.000669 2022/09/27 05:19:24 - mmengine - INFO - Epoch(train) [136][200/293] lr: 5.000000e-04 eta: 2:59:47 time: 0.545134 data_time: 0.081580 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.826477 loss: 0.000666 2022/09/27 05:19:52 - mmengine - INFO - Epoch(train) [136][250/293] lr: 5.000000e-04 eta: 2:59:24 time: 0.554429 data_time: 0.080280 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.825276 loss: 0.000672 2022/09/27 05:20:15 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:20:45 - mmengine - INFO - Epoch(train) [137][50/293] lr: 5.000000e-04 eta: 2:58:29 time: 0.589017 data_time: 0.106175 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.882243 loss: 0.000660 2022/09/27 05:21:13 - mmengine - INFO - Epoch(train) [137][100/293] lr: 5.000000e-04 eta: 2:58:06 time: 0.561031 data_time: 0.076067 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.819404 loss: 0.000669 2022/09/27 05:21:40 - mmengine - INFO - Epoch(train) [137][150/293] lr: 5.000000e-04 eta: 2:57:42 time: 0.537991 data_time: 0.075794 memory: 3840 loss_kpt: 0.000678 acc_pose: 0.832528 loss: 0.000678 2022/09/27 05:21:41 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:22:08 - mmengine - INFO - Epoch(train) [137][200/293] lr: 5.000000e-04 eta: 2:57:19 time: 0.555516 data_time: 0.078362 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.849609 loss: 0.000666 2022/09/27 05:22:36 - mmengine - INFO - Epoch(train) [137][250/293] lr: 5.000000e-04 eta: 2:56:56 time: 0.571737 data_time: 0.082789 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.838728 loss: 0.000661 2022/09/27 05:23:00 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:23:29 - mmengine - INFO - Epoch(train) [138][50/293] lr: 5.000000e-04 eta: 2:56:01 time: 0.588343 data_time: 0.157075 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.798035 loss: 0.000670 2022/09/27 05:23:58 - mmengine - INFO - Epoch(train) [138][100/293] lr: 5.000000e-04 eta: 2:55:39 time: 0.572453 data_time: 0.093434 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.805834 loss: 0.000672 2022/09/27 05:24:27 - mmengine - INFO - Epoch(train) [138][150/293] lr: 5.000000e-04 eta: 2:55:16 time: 0.576704 data_time: 0.124878 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.792997 loss: 0.000656 2022/09/27 05:24:54 - mmengine - INFO - Epoch(train) [138][200/293] lr: 5.000000e-04 eta: 2:54:53 time: 0.551138 data_time: 0.086657 memory: 3840 loss_kpt: 0.000677 acc_pose: 0.808023 loss: 0.000677 2022/09/27 05:25:22 - mmengine - INFO - Epoch(train) [138][250/293] lr: 5.000000e-04 eta: 2:54:29 time: 0.545191 data_time: 0.116073 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.773666 loss: 0.000669 2022/09/27 05:25:45 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:26:14 - mmengine - INFO - Epoch(train) [139][50/293] lr: 5.000000e-04 eta: 2:53:34 time: 0.566037 data_time: 0.234281 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.824572 loss: 0.000665 2022/09/27 05:26:40 - mmengine - INFO - Epoch(train) [139][100/293] lr: 5.000000e-04 eta: 2:53:10 time: 0.535359 data_time: 0.129538 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.797411 loss: 0.000664 2022/09/27 05:27:07 - mmengine - INFO - Epoch(train) [139][150/293] lr: 5.000000e-04 eta: 2:52:47 time: 0.543886 data_time: 0.161822 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.798789 loss: 0.000659 2022/09/27 05:27:35 - mmengine - INFO - Epoch(train) [139][200/293] lr: 5.000000e-04 eta: 2:52:23 time: 0.543616 data_time: 0.136855 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.807875 loss: 0.000670 2022/09/27 05:28:02 - mmengine - INFO - Epoch(train) [139][250/293] lr: 5.000000e-04 eta: 2:52:00 time: 0.550260 data_time: 0.081988 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.774909 loss: 0.000667 2022/09/27 05:28:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:28:55 - mmengine - INFO - Epoch(train) [140][50/293] lr: 5.000000e-04 eta: 2:51:05 time: 0.584445 data_time: 0.180862 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.798695 loss: 0.000660 2022/09/27 05:29:22 - mmengine - INFO - Epoch(train) [140][100/293] lr: 5.000000e-04 eta: 2:50:42 time: 0.550492 data_time: 0.116981 memory: 3840 loss_kpt: 0.000673 acc_pose: 0.818591 loss: 0.000673 2022/09/27 05:29:48 - mmengine - INFO - Epoch(train) [140][150/293] lr: 5.000000e-04 eta: 2:50:18 time: 0.525052 data_time: 0.072325 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.779935 loss: 0.000668 2022/09/27 05:30:16 - mmengine - INFO - Epoch(train) [140][200/293] lr: 5.000000e-04 eta: 2:49:55 time: 0.561671 data_time: 0.079952 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.817871 loss: 0.000664 2022/09/27 05:30:44 - mmengine - INFO - Epoch(train) [140][250/293] lr: 5.000000e-04 eta: 2:49:32 time: 0.549718 data_time: 0.075582 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.864650 loss: 0.000650 2022/09/27 05:30:56 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:31:07 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:31:07 - mmengine - INFO - Saving checkpoint at 140 epochs 2022/09/27 05:31:28 - mmengine - INFO - Epoch(val) [140][50/407] eta: 0:01:54 time: 0.319978 data_time: 0.178160 memory: 3840 2022/09/27 05:31:42 - mmengine - INFO - Epoch(val) [140][100/407] eta: 0:01:29 time: 0.291987 data_time: 0.156226 memory: 1017 2022/09/27 05:31:57 - mmengine - INFO - Epoch(val) [140][150/407] eta: 0:01:18 time: 0.305796 data_time: 0.182193 memory: 1017 2022/09/27 05:32:13 - mmengine - INFO - Epoch(val) [140][200/407] eta: 0:01:04 time: 0.310176 data_time: 0.172173 memory: 1017 2022/09/27 05:32:28 - mmengine - INFO - Epoch(val) [140][250/407] eta: 0:00:45 time: 0.292706 data_time: 0.167350 memory: 1017 2022/09/27 05:32:43 - mmengine - INFO - Epoch(val) [140][300/407] eta: 0:00:31 time: 0.298086 data_time: 0.179845 memory: 1017 2022/09/27 05:32:58 - mmengine - INFO - Epoch(val) [140][350/407] eta: 0:00:17 time: 0.315572 data_time: 0.187076 memory: 1017 2022/09/27 05:33:11 - mmengine - INFO - Epoch(val) [140][400/407] eta: 0:00:01 time: 0.250542 data_time: 0.155170 memory: 1017 2022/09/27 05:33:44 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 05:33:57 - mmengine - INFO - Epoch(val) [140][407/407] coco/AP: 0.696872 coco/AP .5: 0.889420 coco/AP .75: 0.775124 coco/AP (M): 0.658505 coco/AP (L): 0.764877 coco/AR: 0.753904 coco/AR .5: 0.929786 coco/AR .75: 0.822733 coco/AR (M): 0.707539 coco/AR (L): 0.819844 2022/09/27 05:34:25 - mmengine - INFO - Epoch(train) [141][50/293] lr: 5.000000e-04 eta: 2:48:36 time: 0.557555 data_time: 0.118584 memory: 3840 loss_kpt: 0.000670 acc_pose: 0.795529 loss: 0.000670 2022/09/27 05:34:52 - mmengine - INFO - Epoch(train) [141][100/293] lr: 5.000000e-04 eta: 2:48:13 time: 0.539647 data_time: 0.107505 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.792520 loss: 0.000649 2022/09/27 05:35:19 - mmengine - INFO - Epoch(train) [141][150/293] lr: 5.000000e-04 eta: 2:47:50 time: 0.549476 data_time: 0.088636 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.831066 loss: 0.000661 2022/09/27 05:35:48 - mmengine - INFO - Epoch(train) [141][200/293] lr: 5.000000e-04 eta: 2:47:27 time: 0.570544 data_time: 0.087588 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.789395 loss: 0.000652 2022/09/27 05:36:16 - mmengine - INFO - Epoch(train) [141][250/293] lr: 5.000000e-04 eta: 2:47:04 time: 0.569823 data_time: 0.080789 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.849234 loss: 0.000658 2022/09/27 05:36:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:37:10 - mmengine - INFO - Epoch(train) [142][50/293] lr: 5.000000e-04 eta: 2:46:10 time: 0.590203 data_time: 0.118287 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.800128 loss: 0.000660 2022/09/27 05:37:38 - mmengine - INFO - Epoch(train) [142][100/293] lr: 5.000000e-04 eta: 2:45:47 time: 0.566678 data_time: 0.099218 memory: 3840 loss_kpt: 0.000673 acc_pose: 0.787998 loss: 0.000673 2022/09/27 05:38:06 - mmengine - INFO - Epoch(train) [142][150/293] lr: 5.000000e-04 eta: 2:45:24 time: 0.556364 data_time: 0.105764 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.763027 loss: 0.000657 2022/09/27 05:38:33 - mmengine - INFO - Epoch(train) [142][200/293] lr: 5.000000e-04 eta: 2:45:00 time: 0.548787 data_time: 0.065963 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.785630 loss: 0.000659 2022/09/27 05:39:02 - mmengine - INFO - Epoch(train) [142][250/293] lr: 5.000000e-04 eta: 2:44:37 time: 0.572253 data_time: 0.084641 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.764796 loss: 0.000668 2022/09/27 05:39:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:39:55 - mmengine - INFO - Epoch(train) [143][50/293] lr: 5.000000e-04 eta: 2:43:43 time: 0.581987 data_time: 0.090583 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.859350 loss: 0.000664 2022/09/27 05:40:22 - mmengine - INFO - Epoch(train) [143][100/293] lr: 5.000000e-04 eta: 2:43:20 time: 0.545212 data_time: 0.086110 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.813007 loss: 0.000656 2022/09/27 05:40:50 - mmengine - INFO - Epoch(train) [143][150/293] lr: 5.000000e-04 eta: 2:42:57 time: 0.567731 data_time: 0.116642 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.771700 loss: 0.000662 2022/09/27 05:41:17 - mmengine - INFO - Epoch(train) [143][200/293] lr: 5.000000e-04 eta: 2:42:33 time: 0.541425 data_time: 0.113066 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.827151 loss: 0.000665 2022/09/27 05:41:45 - mmengine - INFO - Epoch(train) [143][250/293] lr: 5.000000e-04 eta: 2:42:10 time: 0.548743 data_time: 0.078832 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.820184 loss: 0.000663 2022/09/27 05:42:08 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:42:37 - mmengine - INFO - Epoch(train) [144][50/293] lr: 5.000000e-04 eta: 2:41:16 time: 0.576412 data_time: 0.134366 memory: 3840 loss_kpt: 0.000676 acc_pose: 0.841160 loss: 0.000676 2022/09/27 05:43:04 - mmengine - INFO - Epoch(train) [144][100/293] lr: 5.000000e-04 eta: 2:40:52 time: 0.540557 data_time: 0.096182 memory: 3840 loss_kpt: 0.000655 acc_pose: 0.817100 loss: 0.000655 2022/09/27 05:43:05 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:43:32 - mmengine - INFO - Epoch(train) [144][150/293] lr: 5.000000e-04 eta: 2:40:29 time: 0.549397 data_time: 0.094225 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.798850 loss: 0.000664 2022/09/27 05:44:00 - mmengine - INFO - Epoch(train) [144][200/293] lr: 5.000000e-04 eta: 2:40:06 time: 0.569429 data_time: 0.085863 memory: 3840 loss_kpt: 0.000680 acc_pose: 0.778104 loss: 0.000680 2022/09/27 05:44:28 - mmengine - INFO - Epoch(train) [144][250/293] lr: 5.000000e-04 eta: 2:39:42 time: 0.554226 data_time: 0.079748 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.807604 loss: 0.000657 2022/09/27 05:44:52 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:45:22 - mmengine - INFO - Epoch(train) [145][50/293] lr: 5.000000e-04 eta: 2:38:49 time: 0.582537 data_time: 0.085983 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.764243 loss: 0.000667 2022/09/27 05:45:49 - mmengine - INFO - Epoch(train) [145][100/293] lr: 5.000000e-04 eta: 2:38:25 time: 0.554091 data_time: 0.094738 memory: 3840 loss_kpt: 0.000655 acc_pose: 0.839992 loss: 0.000655 2022/09/27 05:46:19 - mmengine - INFO - Epoch(train) [145][150/293] lr: 5.000000e-04 eta: 2:38:03 time: 0.593327 data_time: 0.082070 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.820611 loss: 0.000659 2022/09/27 05:46:46 - mmengine - INFO - Epoch(train) [145][200/293] lr: 5.000000e-04 eta: 2:37:39 time: 0.541861 data_time: 0.078423 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.829200 loss: 0.000665 2022/09/27 05:47:14 - mmengine - INFO - Epoch(train) [145][250/293] lr: 5.000000e-04 eta: 2:37:16 time: 0.558876 data_time: 0.075959 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.859734 loss: 0.000651 2022/09/27 05:47:38 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:48:07 - mmengine - INFO - Epoch(train) [146][50/293] lr: 5.000000e-04 eta: 2:36:22 time: 0.570428 data_time: 0.220794 memory: 3840 loss_kpt: 0.000648 acc_pose: 0.849846 loss: 0.000648 2022/09/27 05:48:34 - mmengine - INFO - Epoch(train) [146][100/293] lr: 5.000000e-04 eta: 2:35:59 time: 0.544848 data_time: 0.179559 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.819330 loss: 0.000650 2022/09/27 05:49:01 - mmengine - INFO - Epoch(train) [146][150/293] lr: 5.000000e-04 eta: 2:35:35 time: 0.540767 data_time: 0.148201 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.840188 loss: 0.000651 2022/09/27 05:49:28 - mmengine - INFO - Epoch(train) [146][200/293] lr: 5.000000e-04 eta: 2:35:11 time: 0.551253 data_time: 0.181500 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.824352 loss: 0.000662 2022/09/27 05:49:56 - mmengine - INFO - Epoch(train) [146][250/293] lr: 5.000000e-04 eta: 2:34:48 time: 0.545116 data_time: 0.096864 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.812046 loss: 0.000647 2022/09/27 05:50:19 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:50:48 - mmengine - INFO - Epoch(train) [147][50/293] lr: 5.000000e-04 eta: 2:33:54 time: 0.576621 data_time: 0.109932 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.816591 loss: 0.000662 2022/09/27 05:51:16 - mmengine - INFO - Epoch(train) [147][100/293] lr: 5.000000e-04 eta: 2:33:31 time: 0.551302 data_time: 0.082080 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.800632 loss: 0.000656 2022/09/27 05:51:43 - mmengine - INFO - Epoch(train) [147][150/293] lr: 5.000000e-04 eta: 2:33:07 time: 0.545685 data_time: 0.087378 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.748163 loss: 0.000656 2022/09/27 05:52:12 - mmengine - INFO - Epoch(train) [147][200/293] lr: 5.000000e-04 eta: 2:32:44 time: 0.575361 data_time: 0.082199 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.810935 loss: 0.000661 2022/09/27 05:52:24 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:52:39 - mmengine - INFO - Epoch(train) [147][250/293] lr: 5.000000e-04 eta: 2:32:21 time: 0.548494 data_time: 0.112966 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.813330 loss: 0.000660 2022/09/27 05:53:03 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:53:32 - mmengine - INFO - Epoch(train) [148][50/293] lr: 5.000000e-04 eta: 2:31:28 time: 0.583172 data_time: 0.114927 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.818629 loss: 0.000662 2022/09/27 05:53:59 - mmengine - INFO - Epoch(train) [148][100/293] lr: 5.000000e-04 eta: 2:31:04 time: 0.540702 data_time: 0.072548 memory: 3840 loss_kpt: 0.000672 acc_pose: 0.837048 loss: 0.000672 2022/09/27 05:54:26 - mmengine - INFO - Epoch(train) [148][150/293] lr: 5.000000e-04 eta: 2:30:40 time: 0.537678 data_time: 0.102433 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.805822 loss: 0.000656 2022/09/27 05:54:54 - mmengine - INFO - Epoch(train) [148][200/293] lr: 5.000000e-04 eta: 2:30:17 time: 0.550786 data_time: 0.092324 memory: 3840 loss_kpt: 0.000648 acc_pose: 0.865448 loss: 0.000648 2022/09/27 05:55:21 - mmengine - INFO - Epoch(train) [148][250/293] lr: 5.000000e-04 eta: 2:29:53 time: 0.542170 data_time: 0.086244 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.790533 loss: 0.000666 2022/09/27 05:55:45 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:56:14 - mmengine - INFO - Epoch(train) [149][50/293] lr: 5.000000e-04 eta: 2:29:01 time: 0.596753 data_time: 0.087499 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.833894 loss: 0.000656 2022/09/27 05:56:42 - mmengine - INFO - Epoch(train) [149][100/293] lr: 5.000000e-04 eta: 2:28:37 time: 0.557343 data_time: 0.135241 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.820355 loss: 0.000657 2022/09/27 05:57:10 - mmengine - INFO - Epoch(train) [149][150/293] lr: 5.000000e-04 eta: 2:28:14 time: 0.554003 data_time: 0.086881 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.830210 loss: 0.000651 2022/09/27 05:57:38 - mmengine - INFO - Epoch(train) [149][200/293] lr: 5.000000e-04 eta: 2:27:50 time: 0.555738 data_time: 0.078763 memory: 3840 loss_kpt: 0.000667 acc_pose: 0.777677 loss: 0.000667 2022/09/27 05:58:06 - mmengine - INFO - Epoch(train) [149][250/293] lr: 5.000000e-04 eta: 2:27:27 time: 0.567342 data_time: 0.080150 memory: 3840 loss_kpt: 0.000681 acc_pose: 0.845364 loss: 0.000681 2022/09/27 05:58:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 05:58:58 - mmengine - INFO - Epoch(train) [150][50/293] lr: 5.000000e-04 eta: 2:26:34 time: 0.578833 data_time: 0.088535 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.819149 loss: 0.000663 2022/09/27 05:59:26 - mmengine - INFO - Epoch(train) [150][100/293] lr: 5.000000e-04 eta: 2:26:11 time: 0.562208 data_time: 0.070604 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.801367 loss: 0.000661 2022/09/27 05:59:54 - mmengine - INFO - Epoch(train) [150][150/293] lr: 5.000000e-04 eta: 2:25:48 time: 0.553669 data_time: 0.077826 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.831333 loss: 0.000657 2022/09/27 06:00:22 - mmengine - INFO - Epoch(train) [150][200/293] lr: 5.000000e-04 eta: 2:25:24 time: 0.551048 data_time: 0.157922 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.814358 loss: 0.000669 2022/09/27 06:00:49 - mmengine - INFO - Epoch(train) [150][250/293] lr: 5.000000e-04 eta: 2:25:01 time: 0.555661 data_time: 0.150485 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.755626 loss: 0.000663 2022/09/27 06:01:12 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:01:12 - mmengine - INFO - Saving checkpoint at 150 epochs 2022/09/27 06:01:32 - mmengine - INFO - Epoch(val) [150][50/407] eta: 0:01:53 time: 0.317666 data_time: 0.182266 memory: 3840 2022/09/27 06:01:47 - mmengine - INFO - Epoch(val) [150][100/407] eta: 0:01:34 time: 0.307024 data_time: 0.174657 memory: 1017 2022/09/27 06:02:03 - mmengine - INFO - Epoch(val) [150][150/407] eta: 0:01:18 time: 0.307180 data_time: 0.183519 memory: 1017 2022/09/27 06:02:18 - mmengine - INFO - Epoch(val) [150][200/407] eta: 0:01:01 time: 0.298790 data_time: 0.166776 memory: 1017 2022/09/27 06:02:33 - mmengine - INFO - Epoch(val) [150][250/407] eta: 0:00:46 time: 0.294880 data_time: 0.187095 memory: 1017 2022/09/27 06:02:47 - mmengine - INFO - Epoch(val) [150][300/407] eta: 0:00:30 time: 0.285397 data_time: 0.159028 memory: 1017 2022/09/27 06:03:02 - mmengine - INFO - Epoch(val) [150][350/407] eta: 0:00:17 time: 0.301409 data_time: 0.175539 memory: 1017 2022/09/27 06:03:15 - mmengine - INFO - Epoch(val) [150][400/407] eta: 0:00:01 time: 0.260523 data_time: 0.156116 memory: 1017 2022/09/27 06:03:49 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 06:04:02 - mmengine - INFO - Epoch(val) [150][407/407] coco/AP: 0.703858 coco/AP .5: 0.895271 coco/AP .75: 0.782478 coco/AP (M): 0.665691 coco/AP (L): 0.771194 coco/AR: 0.758800 coco/AR .5: 0.931360 coco/AR .75: 0.829503 coco/AR (M): 0.713330 coco/AR (L): 0.823671 2022/09/27 06:04:02 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_130.pth is removed 2022/09/27 06:04:04 - mmengine - INFO - The best checkpoint with 0.7039 coco/AP at 150 epoch is saved to best_coco/AP_epoch_150.pth. 2022/09/27 06:04:32 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:04:32 - mmengine - INFO - Epoch(train) [151][50/293] lr: 5.000000e-04 eta: 2:24:07 time: 0.550627 data_time: 0.180678 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.793580 loss: 0.000658 2022/09/27 06:04:59 - mmengine - INFO - Epoch(train) [151][100/293] lr: 5.000000e-04 eta: 2:23:44 time: 0.550458 data_time: 0.097048 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.812202 loss: 0.000652 2022/09/27 06:05:27 - mmengine - INFO - Epoch(train) [151][150/293] lr: 5.000000e-04 eta: 2:23:20 time: 0.543234 data_time: 0.100920 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.801529 loss: 0.000671 2022/09/27 06:05:54 - mmengine - INFO - Epoch(train) [151][200/293] lr: 5.000000e-04 eta: 2:22:57 time: 0.546352 data_time: 0.166039 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.871892 loss: 0.000657 2022/09/27 06:06:20 - mmengine - INFO - Epoch(train) [151][250/293] lr: 5.000000e-04 eta: 2:22:33 time: 0.526909 data_time: 0.133094 memory: 3840 loss_kpt: 0.000653 acc_pose: 0.796038 loss: 0.000653 2022/09/27 06:06:44 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:07:14 - mmengine - INFO - Epoch(train) [152][50/293] lr: 5.000000e-04 eta: 2:21:40 time: 0.595376 data_time: 0.086143 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.845982 loss: 0.000649 2022/09/27 06:07:43 - mmengine - INFO - Epoch(train) [152][100/293] lr: 5.000000e-04 eta: 2:21:17 time: 0.576267 data_time: 0.088683 memory: 3840 loss_kpt: 0.000644 acc_pose: 0.827127 loss: 0.000644 2022/09/27 06:08:10 - mmengine - INFO - Epoch(train) [152][150/293] lr: 5.000000e-04 eta: 2:20:54 time: 0.543480 data_time: 0.080501 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.794818 loss: 0.000662 2022/09/27 06:08:38 - mmengine - INFO - Epoch(train) [152][200/293] lr: 5.000000e-04 eta: 2:20:30 time: 0.556396 data_time: 0.127803 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.831373 loss: 0.000656 2022/09/27 06:09:06 - mmengine - INFO - Epoch(train) [152][250/293] lr: 5.000000e-04 eta: 2:20:07 time: 0.559583 data_time: 0.101857 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.850015 loss: 0.000651 2022/09/27 06:09:30 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:10:00 - mmengine - INFO - Epoch(train) [153][50/293] lr: 5.000000e-04 eta: 2:19:14 time: 0.582345 data_time: 0.079366 memory: 3840 loss_kpt: 0.000640 acc_pose: 0.868928 loss: 0.000640 2022/09/27 06:10:27 - mmengine - INFO - Epoch(train) [153][100/293] lr: 5.000000e-04 eta: 2:18:51 time: 0.557019 data_time: 0.073323 memory: 3840 loss_kpt: 0.000643 acc_pose: 0.854786 loss: 0.000643 2022/09/27 06:10:56 - mmengine - INFO - Epoch(train) [153][150/293] lr: 5.000000e-04 eta: 2:18:28 time: 0.575064 data_time: 0.077237 memory: 3840 loss_kpt: 0.000645 acc_pose: 0.833892 loss: 0.000645 2022/09/27 06:11:24 - mmengine - INFO - Epoch(train) [153][200/293] lr: 5.000000e-04 eta: 2:18:05 time: 0.562331 data_time: 0.086753 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.834553 loss: 0.000649 2022/09/27 06:11:52 - mmengine - INFO - Epoch(train) [153][250/293] lr: 5.000000e-04 eta: 2:17:41 time: 0.553062 data_time: 0.107274 memory: 3840 loss_kpt: 0.000655 acc_pose: 0.817171 loss: 0.000655 2022/09/27 06:12:17 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:12:46 - mmengine - INFO - Epoch(train) [154][50/293] lr: 5.000000e-04 eta: 2:16:49 time: 0.584487 data_time: 0.096156 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.849221 loss: 0.000658 2022/09/27 06:13:14 - mmengine - INFO - Epoch(train) [154][100/293] lr: 5.000000e-04 eta: 2:16:25 time: 0.557367 data_time: 0.084768 memory: 3840 loss_kpt: 0.000653 acc_pose: 0.789955 loss: 0.000653 2022/09/27 06:13:42 - mmengine - INFO - Epoch(train) [154][150/293] lr: 5.000000e-04 eta: 2:16:02 time: 0.554814 data_time: 0.068547 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.755350 loss: 0.000664 2022/09/27 06:13:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:14:09 - mmengine - INFO - Epoch(train) [154][200/293] lr: 5.000000e-04 eta: 2:15:38 time: 0.553145 data_time: 0.070052 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.837150 loss: 0.000664 2022/09/27 06:14:38 - mmengine - INFO - Epoch(train) [154][250/293] lr: 5.000000e-04 eta: 2:15:15 time: 0.579576 data_time: 0.103765 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.830134 loss: 0.000656 2022/09/27 06:15:01 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:15:30 - mmengine - INFO - Epoch(train) [155][50/293] lr: 5.000000e-04 eta: 2:14:23 time: 0.576137 data_time: 0.116858 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.823215 loss: 0.000651 2022/09/27 06:15:58 - mmengine - INFO - Epoch(train) [155][100/293] lr: 5.000000e-04 eta: 2:14:00 time: 0.566659 data_time: 0.081915 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.837286 loss: 0.000663 2022/09/27 06:16:26 - mmengine - INFO - Epoch(train) [155][150/293] lr: 5.000000e-04 eta: 2:13:36 time: 0.554254 data_time: 0.084607 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.749878 loss: 0.000657 2022/09/27 06:16:54 - mmengine - INFO - Epoch(train) [155][200/293] lr: 5.000000e-04 eta: 2:13:13 time: 0.566720 data_time: 0.066678 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.796859 loss: 0.000661 2022/09/27 06:17:22 - mmengine - INFO - Epoch(train) [155][250/293] lr: 5.000000e-04 eta: 2:12:49 time: 0.550316 data_time: 0.066895 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.818792 loss: 0.000650 2022/09/27 06:17:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:18:14 - mmengine - INFO - Epoch(train) [156][50/293] lr: 5.000000e-04 eta: 2:11:57 time: 0.560588 data_time: 0.090405 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.842508 loss: 0.000647 2022/09/27 06:18:42 - mmengine - INFO - Epoch(train) [156][100/293] lr: 5.000000e-04 eta: 2:11:34 time: 0.560976 data_time: 0.106596 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.841904 loss: 0.000660 2022/09/27 06:19:10 - mmengine - INFO - Epoch(train) [156][150/293] lr: 5.000000e-04 eta: 2:11:10 time: 0.565527 data_time: 0.110213 memory: 3840 loss_kpt: 0.000671 acc_pose: 0.813011 loss: 0.000671 2022/09/27 06:19:38 - mmengine - INFO - Epoch(train) [156][200/293] lr: 5.000000e-04 eta: 2:10:47 time: 0.550039 data_time: 0.088364 memory: 3840 loss_kpt: 0.000654 acc_pose: 0.873072 loss: 0.000654 2022/09/27 06:20:05 - mmengine - INFO - Epoch(train) [156][250/293] lr: 5.000000e-04 eta: 2:10:23 time: 0.551854 data_time: 0.091744 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.813686 loss: 0.000656 2022/09/27 06:20:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:20:58 - mmengine - INFO - Epoch(train) [157][50/293] lr: 5.000000e-04 eta: 2:09:31 time: 0.581769 data_time: 0.106990 memory: 3840 loss_kpt: 0.000641 acc_pose: 0.807105 loss: 0.000641 2022/09/27 06:21:26 - mmengine - INFO - Epoch(train) [157][100/293] lr: 5.000000e-04 eta: 2:09:08 time: 0.559478 data_time: 0.083404 memory: 3840 loss_kpt: 0.000653 acc_pose: 0.827992 loss: 0.000653 2022/09/27 06:21:53 - mmengine - INFO - Epoch(train) [157][150/293] lr: 5.000000e-04 eta: 2:08:44 time: 0.531226 data_time: 0.067526 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.809528 loss: 0.000652 2022/09/27 06:22:20 - mmengine - INFO - Epoch(train) [157][200/293] lr: 5.000000e-04 eta: 2:08:20 time: 0.542693 data_time: 0.094133 memory: 3840 loss_kpt: 0.000668 acc_pose: 0.844291 loss: 0.000668 2022/09/27 06:22:47 - mmengine - INFO - Epoch(train) [157][250/293] lr: 5.000000e-04 eta: 2:07:56 time: 0.533390 data_time: 0.113079 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.853353 loss: 0.000651 2022/09/27 06:23:11 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:23:11 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:23:40 - mmengine - INFO - Epoch(train) [158][50/293] lr: 5.000000e-04 eta: 2:07:05 time: 0.578596 data_time: 0.098620 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.838018 loss: 0.000660 2022/09/27 06:24:07 - mmengine - INFO - Epoch(train) [158][100/293] lr: 5.000000e-04 eta: 2:06:41 time: 0.536100 data_time: 0.078122 memory: 3840 loss_kpt: 0.000663 acc_pose: 0.808384 loss: 0.000663 2022/09/27 06:24:33 - mmengine - INFO - Epoch(train) [158][150/293] lr: 5.000000e-04 eta: 2:06:17 time: 0.529658 data_time: 0.078362 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.820232 loss: 0.000647 2022/09/27 06:25:03 - mmengine - INFO - Epoch(train) [158][200/293] lr: 5.000000e-04 eta: 2:05:54 time: 0.588554 data_time: 0.153174 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.820395 loss: 0.000652 2022/09/27 06:25:31 - mmengine - INFO - Epoch(train) [158][250/293] lr: 5.000000e-04 eta: 2:05:30 time: 0.569574 data_time: 0.132973 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.828361 loss: 0.000652 2022/09/27 06:25:55 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:26:25 - mmengine - INFO - Epoch(train) [159][50/293] lr: 5.000000e-04 eta: 2:04:39 time: 0.584673 data_time: 0.105391 memory: 3840 loss_kpt: 0.000669 acc_pose: 0.782179 loss: 0.000669 2022/09/27 06:26:51 - mmengine - INFO - Epoch(train) [159][100/293] lr: 5.000000e-04 eta: 2:04:15 time: 0.534122 data_time: 0.161617 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.763623 loss: 0.000650 2022/09/27 06:27:20 - mmengine - INFO - Epoch(train) [159][150/293] lr: 5.000000e-04 eta: 2:03:52 time: 0.565047 data_time: 0.090596 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.807430 loss: 0.000652 2022/09/27 06:27:47 - mmengine - INFO - Epoch(train) [159][200/293] lr: 5.000000e-04 eta: 2:03:28 time: 0.557015 data_time: 0.080229 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.833122 loss: 0.000652 2022/09/27 06:28:16 - mmengine - INFO - Epoch(train) [159][250/293] lr: 5.000000e-04 eta: 2:03:05 time: 0.565451 data_time: 0.132268 memory: 3840 loss_kpt: 0.000646 acc_pose: 0.751276 loss: 0.000646 2022/09/27 06:28:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:29:10 - mmengine - INFO - Epoch(train) [160][50/293] lr: 5.000000e-04 eta: 2:02:14 time: 0.596881 data_time: 0.093392 memory: 3840 loss_kpt: 0.000653 acc_pose: 0.815467 loss: 0.000653 2022/09/27 06:29:37 - mmengine - INFO - Epoch(train) [160][100/293] lr: 5.000000e-04 eta: 2:01:50 time: 0.547982 data_time: 0.068093 memory: 3840 loss_kpt: 0.000631 acc_pose: 0.801286 loss: 0.000631 2022/09/27 06:30:06 - mmengine - INFO - Epoch(train) [160][150/293] lr: 5.000000e-04 eta: 2:01:27 time: 0.571693 data_time: 0.117656 memory: 3840 loss_kpt: 0.000657 acc_pose: 0.850374 loss: 0.000657 2022/09/27 06:30:33 - mmengine - INFO - Epoch(train) [160][200/293] lr: 5.000000e-04 eta: 2:01:03 time: 0.549287 data_time: 0.149393 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.818166 loss: 0.000656 2022/09/27 06:31:01 - mmengine - INFO - Epoch(train) [160][250/293] lr: 5.000000e-04 eta: 2:00:39 time: 0.549941 data_time: 0.106929 memory: 3840 loss_kpt: 0.000639 acc_pose: 0.859580 loss: 0.000639 2022/09/27 06:31:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:31:25 - mmengine - INFO - Saving checkpoint at 160 epochs 2022/09/27 06:31:46 - mmengine - INFO - Epoch(val) [160][50/407] eta: 0:01:51 time: 0.313226 data_time: 0.184170 memory: 3840 2022/09/27 06:32:01 - mmengine - INFO - Epoch(val) [160][100/407] eta: 0:01:34 time: 0.308254 data_time: 0.180975 memory: 1017 2022/09/27 06:32:17 - mmengine - INFO - Epoch(val) [160][150/407] eta: 0:01:18 time: 0.305742 data_time: 0.184319 memory: 1017 2022/09/27 06:32:32 - mmengine - INFO - Epoch(val) [160][200/407] eta: 0:01:03 time: 0.306421 data_time: 0.180625 memory: 1017 2022/09/27 06:32:47 - mmengine - INFO - Epoch(val) [160][250/407] eta: 0:00:48 time: 0.306530 data_time: 0.178643 memory: 1017 2022/09/27 06:33:03 - mmengine - INFO - Epoch(val) [160][300/407] eta: 0:00:32 time: 0.305554 data_time: 0.187074 memory: 1017 2022/09/27 06:33:18 - mmengine - INFO - Epoch(val) [160][350/407] eta: 0:00:17 time: 0.310291 data_time: 0.180912 memory: 1017 2022/09/27 06:33:31 - mmengine - INFO - Epoch(val) [160][400/407] eta: 0:00:01 time: 0.252865 data_time: 0.143759 memory: 1017 2022/09/27 06:34:04 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 06:34:17 - mmengine - INFO - Epoch(val) [160][407/407] coco/AP: 0.701062 coco/AP .5: 0.889232 coco/AP .75: 0.778317 coco/AP (M): 0.664196 coco/AP (L): 0.769811 coco/AR: 0.758234 coco/AR .5: 0.929943 coco/AR .75: 0.826669 coco/AR (M): 0.713794 coco/AR (L): 0.822074 2022/09/27 06:34:47 - mmengine - INFO - Epoch(train) [161][50/293] lr: 5.000000e-04 eta: 1:59:48 time: 0.597282 data_time: 0.121768 memory: 3840 loss_kpt: 0.000666 acc_pose: 0.814852 loss: 0.000666 2022/09/27 06:35:16 - mmengine - INFO - Epoch(train) [161][100/293] lr: 5.000000e-04 eta: 1:59:25 time: 0.574805 data_time: 0.124252 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.814367 loss: 0.000664 2022/09/27 06:35:27 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:35:44 - mmengine - INFO - Epoch(train) [161][150/293] lr: 5.000000e-04 eta: 1:59:01 time: 0.566813 data_time: 0.123332 memory: 3840 loss_kpt: 0.000665 acc_pose: 0.806663 loss: 0.000665 2022/09/27 06:36:12 - mmengine - INFO - Epoch(train) [161][200/293] lr: 5.000000e-04 eta: 1:58:38 time: 0.547920 data_time: 0.079807 memory: 3840 loss_kpt: 0.000653 acc_pose: 0.852272 loss: 0.000653 2022/09/27 06:36:39 - mmengine - INFO - Epoch(train) [161][250/293] lr: 5.000000e-04 eta: 1:58:14 time: 0.541023 data_time: 0.068907 memory: 3840 loss_kpt: 0.000640 acc_pose: 0.852279 loss: 0.000640 2022/09/27 06:37:01 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:37:30 - mmengine - INFO - Epoch(train) [162][50/293] lr: 5.000000e-04 eta: 1:57:23 time: 0.571497 data_time: 0.092891 memory: 3840 loss_kpt: 0.000644 acc_pose: 0.828331 loss: 0.000644 2022/09/27 06:37:58 - mmengine - INFO - Epoch(train) [162][100/293] lr: 5.000000e-04 eta: 1:56:59 time: 0.562170 data_time: 0.152089 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.794809 loss: 0.000659 2022/09/27 06:38:24 - mmengine - INFO - Epoch(train) [162][150/293] lr: 5.000000e-04 eta: 1:56:35 time: 0.522930 data_time: 0.163829 memory: 3840 loss_kpt: 0.000648 acc_pose: 0.871293 loss: 0.000648 2022/09/27 06:38:52 - mmengine - INFO - Epoch(train) [162][200/293] lr: 5.000000e-04 eta: 1:56:11 time: 0.557781 data_time: 0.085519 memory: 3840 loss_kpt: 0.000654 acc_pose: 0.836848 loss: 0.000654 2022/09/27 06:39:20 - mmengine - INFO - Epoch(train) [162][250/293] lr: 5.000000e-04 eta: 1:55:48 time: 0.559455 data_time: 0.117516 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.793161 loss: 0.000652 2022/09/27 06:39:44 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:40:13 - mmengine - INFO - Epoch(train) [163][50/293] lr: 5.000000e-04 eta: 1:54:57 time: 0.567234 data_time: 0.098528 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.856020 loss: 0.000649 2022/09/27 06:40:41 - mmengine - INFO - Epoch(train) [163][100/293] lr: 5.000000e-04 eta: 1:54:33 time: 0.558979 data_time: 0.083063 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.853201 loss: 0.000650 2022/09/27 06:41:08 - mmengine - INFO - Epoch(train) [163][150/293] lr: 5.000000e-04 eta: 1:54:09 time: 0.546810 data_time: 0.077084 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.790519 loss: 0.000659 2022/09/27 06:41:36 - mmengine - INFO - Epoch(train) [163][200/293] lr: 5.000000e-04 eta: 1:53:46 time: 0.551220 data_time: 0.074263 memory: 3840 loss_kpt: 0.000645 acc_pose: 0.867152 loss: 0.000645 2022/09/27 06:42:04 - mmengine - INFO - Epoch(train) [163][250/293] lr: 5.000000e-04 eta: 1:53:22 time: 0.558365 data_time: 0.129630 memory: 3840 loss_kpt: 0.000642 acc_pose: 0.808863 loss: 0.000642 2022/09/27 06:42:27 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:42:55 - mmengine - INFO - Epoch(train) [164][50/293] lr: 5.000000e-04 eta: 1:52:31 time: 0.568147 data_time: 0.091685 memory: 3840 loss_kpt: 0.000638 acc_pose: 0.853537 loss: 0.000638 2022/09/27 06:43:23 - mmengine - INFO - Epoch(train) [164][100/293] lr: 5.000000e-04 eta: 1:52:08 time: 0.558753 data_time: 0.084361 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.857121 loss: 0.000649 2022/09/27 06:43:51 - mmengine - INFO - Epoch(train) [164][150/293] lr: 5.000000e-04 eta: 1:51:44 time: 0.557912 data_time: 0.072938 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.848433 loss: 0.000650 2022/09/27 06:44:19 - mmengine - INFO - Epoch(train) [164][200/293] lr: 5.000000e-04 eta: 1:51:20 time: 0.560136 data_time: 0.074250 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.798416 loss: 0.000651 2022/09/27 06:44:41 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:44:46 - mmengine - INFO - Epoch(train) [164][250/293] lr: 5.000000e-04 eta: 1:50:56 time: 0.541794 data_time: 0.083395 memory: 3840 loss_kpt: 0.000644 acc_pose: 0.774679 loss: 0.000644 2022/09/27 06:45:09 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:45:39 - mmengine - INFO - Epoch(train) [165][50/293] lr: 5.000000e-04 eta: 1:50:06 time: 0.591498 data_time: 0.117823 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.812308 loss: 0.000661 2022/09/27 06:46:08 - mmengine - INFO - Epoch(train) [165][100/293] lr: 5.000000e-04 eta: 1:49:43 time: 0.581923 data_time: 0.075839 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.786519 loss: 0.000647 2022/09/27 06:46:37 - mmengine - INFO - Epoch(train) [165][150/293] lr: 5.000000e-04 eta: 1:49:19 time: 0.581765 data_time: 0.105012 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.827851 loss: 0.000656 2022/09/27 06:47:04 - mmengine - INFO - Epoch(train) [165][200/293] lr: 5.000000e-04 eta: 1:48:55 time: 0.538264 data_time: 0.080839 memory: 3840 loss_kpt: 0.000658 acc_pose: 0.849194 loss: 0.000658 2022/09/27 06:47:33 - mmengine - INFO - Epoch(train) [165][250/293] lr: 5.000000e-04 eta: 1:48:32 time: 0.569637 data_time: 0.189935 memory: 3840 loss_kpt: 0.000662 acc_pose: 0.819579 loss: 0.000662 2022/09/27 06:47:56 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:48:25 - mmengine - INFO - Epoch(train) [166][50/293] lr: 5.000000e-04 eta: 1:47:41 time: 0.578802 data_time: 0.127925 memory: 3840 loss_kpt: 0.000642 acc_pose: 0.841679 loss: 0.000642 2022/09/27 06:48:54 - mmengine - INFO - Epoch(train) [166][100/293] lr: 5.000000e-04 eta: 1:47:18 time: 0.566057 data_time: 0.083619 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.852784 loss: 0.000656 2022/09/27 06:49:22 - mmengine - INFO - Epoch(train) [166][150/293] lr: 5.000000e-04 eta: 1:46:54 time: 0.573062 data_time: 0.078372 memory: 3840 loss_kpt: 0.000648 acc_pose: 0.853023 loss: 0.000648 2022/09/27 06:49:50 - mmengine - INFO - Epoch(train) [166][200/293] lr: 5.000000e-04 eta: 1:46:31 time: 0.557033 data_time: 0.076191 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.839202 loss: 0.000652 2022/09/27 06:50:17 - mmengine - INFO - Epoch(train) [166][250/293] lr: 5.000000e-04 eta: 1:46:07 time: 0.547077 data_time: 0.067806 memory: 3840 loss_kpt: 0.000651 acc_pose: 0.789709 loss: 0.000651 2022/09/27 06:50:42 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:51:10 - mmengine - INFO - Epoch(train) [167][50/293] lr: 5.000000e-04 eta: 1:45:16 time: 0.574272 data_time: 0.122644 memory: 3840 loss_kpt: 0.000659 acc_pose: 0.848060 loss: 0.000659 2022/09/27 06:51:38 - mmengine - INFO - Epoch(train) [167][100/293] lr: 5.000000e-04 eta: 1:44:53 time: 0.552767 data_time: 0.073208 memory: 3840 loss_kpt: 0.000632 acc_pose: 0.828432 loss: 0.000632 2022/09/27 06:52:07 - mmengine - INFO - Epoch(train) [167][150/293] lr: 5.000000e-04 eta: 1:44:29 time: 0.580134 data_time: 0.080310 memory: 3840 loss_kpt: 0.000646 acc_pose: 0.834295 loss: 0.000646 2022/09/27 06:52:35 - mmengine - INFO - Epoch(train) [167][200/293] lr: 5.000000e-04 eta: 1:44:05 time: 0.559966 data_time: 0.086138 memory: 3840 loss_kpt: 0.000646 acc_pose: 0.843843 loss: 0.000646 2022/09/27 06:53:02 - mmengine - INFO - Epoch(train) [167][250/293] lr: 5.000000e-04 eta: 1:43:41 time: 0.545786 data_time: 0.106652 memory: 3840 loss_kpt: 0.000655 acc_pose: 0.839654 loss: 0.000655 2022/09/27 06:53:25 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:53:54 - mmengine - INFO - Epoch(train) [168][50/293] lr: 5.000000e-04 eta: 1:42:51 time: 0.568935 data_time: 0.085340 memory: 3840 loss_kpt: 0.000661 acc_pose: 0.797178 loss: 0.000661 2022/09/27 06:54:04 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:54:22 - mmengine - INFO - Epoch(train) [168][100/293] lr: 5.000000e-04 eta: 1:42:28 time: 0.559103 data_time: 0.074230 memory: 3840 loss_kpt: 0.000640 acc_pose: 0.829254 loss: 0.000640 2022/09/27 06:54:49 - mmengine - INFO - Epoch(train) [168][150/293] lr: 5.000000e-04 eta: 1:42:04 time: 0.550526 data_time: 0.079657 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.821549 loss: 0.000647 2022/09/27 06:55:17 - mmengine - INFO - Epoch(train) [168][200/293] lr: 5.000000e-04 eta: 1:41:40 time: 0.558876 data_time: 0.098971 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.776278 loss: 0.000656 2022/09/27 06:55:46 - mmengine - INFO - Epoch(train) [168][250/293] lr: 5.000000e-04 eta: 1:41:16 time: 0.570106 data_time: 0.079996 memory: 3840 loss_kpt: 0.000664 acc_pose: 0.839264 loss: 0.000664 2022/09/27 06:56:09 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:56:37 - mmengine - INFO - Epoch(train) [169][50/293] lr: 5.000000e-04 eta: 1:40:26 time: 0.574454 data_time: 0.184968 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.846716 loss: 0.000649 2022/09/27 06:57:05 - mmengine - INFO - Epoch(train) [169][100/293] lr: 5.000000e-04 eta: 1:40:02 time: 0.551220 data_time: 0.120077 memory: 3840 loss_kpt: 0.000649 acc_pose: 0.817000 loss: 0.000649 2022/09/27 06:57:33 - mmengine - INFO - Epoch(train) [169][150/293] lr: 5.000000e-04 eta: 1:39:39 time: 0.557070 data_time: 0.100002 memory: 3840 loss_kpt: 0.000652 acc_pose: 0.879647 loss: 0.000652 2022/09/27 06:58:01 - mmengine - INFO - Epoch(train) [169][200/293] lr: 5.000000e-04 eta: 1:39:15 time: 0.571450 data_time: 0.081212 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.843820 loss: 0.000647 2022/09/27 06:58:29 - mmengine - INFO - Epoch(train) [169][250/293] lr: 5.000000e-04 eta: 1:38:51 time: 0.550578 data_time: 0.094420 memory: 3840 loss_kpt: 0.000650 acc_pose: 0.789577 loss: 0.000650 2022/09/27 06:58:53 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 06:59:23 - mmengine - INFO - Epoch(train) [170][50/293] lr: 5.000000e-04 eta: 1:38:02 time: 0.601878 data_time: 0.092709 memory: 3840 loss_kpt: 0.000653 acc_pose: 0.837097 loss: 0.000653 2022/09/27 06:59:49 - mmengine - INFO - Epoch(train) [170][100/293] lr: 5.000000e-04 eta: 1:37:38 time: 0.533645 data_time: 0.085587 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.821313 loss: 0.000660 2022/09/27 07:00:16 - mmengine - INFO - Epoch(train) [170][150/293] lr: 5.000000e-04 eta: 1:37:13 time: 0.534201 data_time: 0.151455 memory: 3840 loss_kpt: 0.000647 acc_pose: 0.818944 loss: 0.000647 2022/09/27 07:00:44 - mmengine - INFO - Epoch(train) [170][200/293] lr: 5.000000e-04 eta: 1:36:50 time: 0.558171 data_time: 0.092275 memory: 3840 loss_kpt: 0.000660 acc_pose: 0.823759 loss: 0.000660 2022/09/27 07:01:12 - mmengine - INFO - Epoch(train) [170][250/293] lr: 5.000000e-04 eta: 1:36:26 time: 0.550423 data_time: 0.095348 memory: 3840 loss_kpt: 0.000656 acc_pose: 0.824891 loss: 0.000656 2022/09/27 07:01:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:01:37 - mmengine - INFO - Saving checkpoint at 170 epochs 2022/09/27 07:01:58 - mmengine - INFO - Epoch(val) [170][50/407] eta: 0:01:50 time: 0.310337 data_time: 0.185555 memory: 3840 2022/09/27 07:02:13 - mmengine - INFO - Epoch(val) [170][100/407] eta: 0:01:32 time: 0.301648 data_time: 0.186082 memory: 1017 2022/09/27 07:02:29 - mmengine - INFO - Epoch(val) [170][150/407] eta: 0:01:20 time: 0.313260 data_time: 0.178010 memory: 1017 2022/09/27 07:02:43 - mmengine - INFO - Epoch(val) [170][200/407] eta: 0:01:00 time: 0.294501 data_time: 0.164192 memory: 1017 2022/09/27 07:02:58 - mmengine - INFO - Epoch(val) [170][250/407] eta: 0:00:47 time: 0.301081 data_time: 0.176115 memory: 1017 2022/09/27 07:03:15 - mmengine - INFO - Epoch(val) [170][300/407] eta: 0:00:35 time: 0.335956 data_time: 0.214185 memory: 1017 2022/09/27 07:03:31 - mmengine - INFO - Epoch(val) [170][350/407] eta: 0:00:18 time: 0.321607 data_time: 0.194179 memory: 1017 2022/09/27 07:03:41 - mmengine - INFO - Epoch(val) [170][400/407] eta: 0:00:01 time: 0.203085 data_time: 0.126345 memory: 1017 2022/09/27 07:04:15 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 07:04:28 - mmengine - INFO - Epoch(val) [170][407/407] coco/AP: 0.701232 coco/AP .5: 0.888535 coco/AP .75: 0.780773 coco/AP (M): 0.663728 coco/AP (L): 0.769382 coco/AR: 0.756958 coco/AR .5: 0.928526 coco/AR .75: 0.828558 coco/AR (M): 0.712620 coco/AR (L): 0.820661 2022/09/27 07:04:57 - mmengine - INFO - Epoch(train) [171][50/293] lr: 5.000000e-05 eta: 1:35:36 time: 0.577674 data_time: 0.198693 memory: 3840 loss_kpt: 0.000636 acc_pose: 0.818763 loss: 0.000636 2022/09/27 07:05:25 - mmengine - INFO - Epoch(train) [171][100/293] lr: 5.000000e-05 eta: 1:35:12 time: 0.559931 data_time: 0.189504 memory: 3840 loss_kpt: 0.000641 acc_pose: 0.816400 loss: 0.000641 2022/09/27 07:05:53 - mmengine - INFO - Epoch(train) [171][150/293] lr: 5.000000e-05 eta: 1:34:49 time: 0.571276 data_time: 0.119540 memory: 3840 loss_kpt: 0.000633 acc_pose: 0.789412 loss: 0.000633 2022/09/27 07:06:15 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:06:21 - mmengine - INFO - Epoch(train) [171][200/293] lr: 5.000000e-05 eta: 1:34:25 time: 0.541792 data_time: 0.131999 memory: 3840 loss_kpt: 0.000638 acc_pose: 0.811416 loss: 0.000638 2022/09/27 07:06:49 - mmengine - INFO - Epoch(train) [171][250/293] lr: 5.000000e-05 eta: 1:34:01 time: 0.563426 data_time: 0.157503 memory: 3840 loss_kpt: 0.000628 acc_pose: 0.795802 loss: 0.000628 2022/09/27 07:07:12 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:07:41 - mmengine - INFO - Epoch(train) [172][50/293] lr: 5.000000e-05 eta: 1:33:11 time: 0.566991 data_time: 0.088803 memory: 3840 loss_kpt: 0.000627 acc_pose: 0.806032 loss: 0.000627 2022/09/27 07:08:08 - mmengine - INFO - Epoch(train) [172][100/293] lr: 5.000000e-05 eta: 1:32:47 time: 0.555874 data_time: 0.081645 memory: 3840 loss_kpt: 0.000626 acc_pose: 0.836489 loss: 0.000626 2022/09/27 07:08:35 - mmengine - INFO - Epoch(train) [172][150/293] lr: 5.000000e-05 eta: 1:32:23 time: 0.539969 data_time: 0.075461 memory: 3840 loss_kpt: 0.000622 acc_pose: 0.820209 loss: 0.000622 2022/09/27 07:09:03 - mmengine - INFO - Epoch(train) [172][200/293] lr: 5.000000e-05 eta: 1:31:59 time: 0.554686 data_time: 0.087472 memory: 3840 loss_kpt: 0.000629 acc_pose: 0.779590 loss: 0.000629 2022/09/27 07:09:31 - mmengine - INFO - Epoch(train) [172][250/293] lr: 5.000000e-05 eta: 1:31:36 time: 0.554295 data_time: 0.076924 memory: 3840 loss_kpt: 0.000628 acc_pose: 0.836939 loss: 0.000628 2022/09/27 07:09:55 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:10:24 - mmengine - INFO - Epoch(train) [173][50/293] lr: 5.000000e-05 eta: 1:30:46 time: 0.574585 data_time: 0.088722 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.863342 loss: 0.000607 2022/09/27 07:10:52 - mmengine - INFO - Epoch(train) [173][100/293] lr: 5.000000e-05 eta: 1:30:22 time: 0.561080 data_time: 0.075113 memory: 3840 loss_kpt: 0.000632 acc_pose: 0.782545 loss: 0.000632 2022/09/27 07:11:20 - mmengine - INFO - Epoch(train) [173][150/293] lr: 5.000000e-05 eta: 1:29:58 time: 0.555303 data_time: 0.089027 memory: 3840 loss_kpt: 0.000623 acc_pose: 0.851096 loss: 0.000623 2022/09/27 07:11:48 - mmengine - INFO - Epoch(train) [173][200/293] lr: 5.000000e-05 eta: 1:29:35 time: 0.556802 data_time: 0.074454 memory: 3840 loss_kpt: 0.000619 acc_pose: 0.840387 loss: 0.000619 2022/09/27 07:12:16 - mmengine - INFO - Epoch(train) [173][250/293] lr: 5.000000e-05 eta: 1:29:11 time: 0.567995 data_time: 0.081787 memory: 3840 loss_kpt: 0.000618 acc_pose: 0.864042 loss: 0.000618 2022/09/27 07:12:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:13:09 - mmengine - INFO - Epoch(train) [174][50/293] lr: 5.000000e-05 eta: 1:28:21 time: 0.566916 data_time: 0.167428 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.845104 loss: 0.000609 2022/09/27 07:13:36 - mmengine - INFO - Epoch(train) [174][100/293] lr: 5.000000e-05 eta: 1:27:57 time: 0.542467 data_time: 0.094590 memory: 3840 loss_kpt: 0.000622 acc_pose: 0.864263 loss: 0.000622 2022/09/27 07:14:04 - mmengine - INFO - Epoch(train) [174][150/293] lr: 5.000000e-05 eta: 1:27:34 time: 0.564089 data_time: 0.100276 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.866918 loss: 0.000617 2022/09/27 07:14:32 - mmengine - INFO - Epoch(train) [174][200/293] lr: 5.000000e-05 eta: 1:27:10 time: 0.549499 data_time: 0.074368 memory: 3840 loss_kpt: 0.000623 acc_pose: 0.828786 loss: 0.000623 2022/09/27 07:14:59 - mmengine - INFO - Epoch(train) [174][250/293] lr: 5.000000e-05 eta: 1:26:45 time: 0.541776 data_time: 0.069495 memory: 3840 loss_kpt: 0.000614 acc_pose: 0.836674 loss: 0.000614 2022/09/27 07:15:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:15:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:15:52 - mmengine - INFO - Epoch(train) [175][50/293] lr: 5.000000e-05 eta: 1:25:56 time: 0.583703 data_time: 0.090443 memory: 3840 loss_kpt: 0.000618 acc_pose: 0.853803 loss: 0.000618 2022/09/27 07:16:18 - mmengine - INFO - Epoch(train) [175][100/293] lr: 5.000000e-05 eta: 1:25:32 time: 0.532336 data_time: 0.071983 memory: 3840 loss_kpt: 0.000625 acc_pose: 0.845222 loss: 0.000625 2022/09/27 07:16:46 - mmengine - INFO - Epoch(train) [175][150/293] lr: 5.000000e-05 eta: 1:25:08 time: 0.560270 data_time: 0.085222 memory: 3840 loss_kpt: 0.000620 acc_pose: 0.830873 loss: 0.000620 2022/09/27 07:17:13 - mmengine - INFO - Epoch(train) [175][200/293] lr: 5.000000e-05 eta: 1:24:44 time: 0.538946 data_time: 0.073781 memory: 3840 loss_kpt: 0.000624 acc_pose: 0.841857 loss: 0.000624 2022/09/27 07:17:41 - mmengine - INFO - Epoch(train) [175][250/293] lr: 5.000000e-05 eta: 1:24:20 time: 0.564675 data_time: 0.077406 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.833694 loss: 0.000612 2022/09/27 07:18:05 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:18:35 - mmengine - INFO - Epoch(train) [176][50/293] lr: 5.000000e-05 eta: 1:23:32 time: 0.588011 data_time: 0.125536 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.867226 loss: 0.000611 2022/09/27 07:19:04 - mmengine - INFO - Epoch(train) [176][100/293] lr: 5.000000e-05 eta: 1:23:08 time: 0.575532 data_time: 0.092551 memory: 3840 loss_kpt: 0.000624 acc_pose: 0.817674 loss: 0.000624 2022/09/27 07:19:30 - mmengine - INFO - Epoch(train) [176][150/293] lr: 5.000000e-05 eta: 1:22:44 time: 0.538788 data_time: 0.094475 memory: 3840 loss_kpt: 0.000615 acc_pose: 0.851274 loss: 0.000615 2022/09/27 07:19:58 - mmengine - INFO - Epoch(train) [176][200/293] lr: 5.000000e-05 eta: 1:22:20 time: 0.559901 data_time: 0.105976 memory: 3840 loss_kpt: 0.000626 acc_pose: 0.850995 loss: 0.000626 2022/09/27 07:20:26 - mmengine - INFO - Epoch(train) [176][250/293] lr: 5.000000e-05 eta: 1:21:56 time: 0.559392 data_time: 0.096279 memory: 3840 loss_kpt: 0.000624 acc_pose: 0.838358 loss: 0.000624 2022/09/27 07:20:50 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:21:20 - mmengine - INFO - Epoch(train) [177][50/293] lr: 5.000000e-05 eta: 1:21:07 time: 0.595304 data_time: 0.091665 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.843910 loss: 0.000612 2022/09/27 07:21:48 - mmengine - INFO - Epoch(train) [177][100/293] lr: 5.000000e-05 eta: 1:20:43 time: 0.557246 data_time: 0.097180 memory: 3840 loss_kpt: 0.000632 acc_pose: 0.881064 loss: 0.000632 2022/09/27 07:22:16 - mmengine - INFO - Epoch(train) [177][150/293] lr: 5.000000e-05 eta: 1:20:19 time: 0.552179 data_time: 0.069861 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.887726 loss: 0.000613 2022/09/27 07:22:44 - mmengine - INFO - Epoch(train) [177][200/293] lr: 5.000000e-05 eta: 1:19:55 time: 0.567133 data_time: 0.078350 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.802055 loss: 0.000611 2022/09/27 07:23:10 - mmengine - INFO - Epoch(train) [177][250/293] lr: 5.000000e-05 eta: 1:19:31 time: 0.524008 data_time: 0.115609 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.831009 loss: 0.000610 2022/09/27 07:23:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:24:02 - mmengine - INFO - Epoch(train) [178][50/293] lr: 5.000000e-05 eta: 1:18:42 time: 0.579907 data_time: 0.159891 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.814333 loss: 0.000609 2022/09/27 07:24:30 - mmengine - INFO - Epoch(train) [178][100/293] lr: 5.000000e-05 eta: 1:18:18 time: 0.551947 data_time: 0.073202 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.846890 loss: 0.000617 2022/09/27 07:24:51 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:24:56 - mmengine - INFO - Epoch(train) [178][150/293] lr: 5.000000e-05 eta: 1:17:54 time: 0.527784 data_time: 0.137816 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.803745 loss: 0.000617 2022/09/27 07:25:24 - mmengine - INFO - Epoch(train) [178][200/293] lr: 5.000000e-05 eta: 1:17:30 time: 0.541146 data_time: 0.139053 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.856102 loss: 0.000612 2022/09/27 07:25:52 - mmengine - INFO - Epoch(train) [178][250/293] lr: 5.000000e-05 eta: 1:17:06 time: 0.561401 data_time: 0.090160 memory: 3840 loss_kpt: 0.000622 acc_pose: 0.817384 loss: 0.000622 2022/09/27 07:26:15 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:26:43 - mmengine - INFO - Epoch(train) [179][50/293] lr: 5.000000e-05 eta: 1:16:17 time: 0.576830 data_time: 0.105241 memory: 3840 loss_kpt: 0.000620 acc_pose: 0.862798 loss: 0.000620 2022/09/27 07:27:11 - mmengine - INFO - Epoch(train) [179][100/293] lr: 5.000000e-05 eta: 1:15:53 time: 0.549703 data_time: 0.090492 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.871258 loss: 0.000612 2022/09/27 07:27:39 - mmengine - INFO - Epoch(train) [179][150/293] lr: 5.000000e-05 eta: 1:15:29 time: 0.552512 data_time: 0.086728 memory: 3840 loss_kpt: 0.000618 acc_pose: 0.819823 loss: 0.000618 2022/09/27 07:28:07 - mmengine - INFO - Epoch(train) [179][200/293] lr: 5.000000e-05 eta: 1:15:06 time: 0.567077 data_time: 0.176633 memory: 3840 loss_kpt: 0.000621 acc_pose: 0.813582 loss: 0.000621 2022/09/27 07:28:35 - mmengine - INFO - Epoch(train) [179][250/293] lr: 5.000000e-05 eta: 1:14:42 time: 0.558356 data_time: 0.077990 memory: 3840 loss_kpt: 0.000623 acc_pose: 0.840871 loss: 0.000623 2022/09/27 07:28:58 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:29:28 - mmengine - INFO - Epoch(train) [180][50/293] lr: 5.000000e-05 eta: 1:13:53 time: 0.588191 data_time: 0.146875 memory: 3840 loss_kpt: 0.000624 acc_pose: 0.847418 loss: 0.000624 2022/09/27 07:29:56 - mmengine - INFO - Epoch(train) [180][100/293] lr: 5.000000e-05 eta: 1:13:29 time: 0.569675 data_time: 0.093704 memory: 3840 loss_kpt: 0.000606 acc_pose: 0.843551 loss: 0.000606 2022/09/27 07:30:24 - mmengine - INFO - Epoch(train) [180][150/293] lr: 5.000000e-05 eta: 1:13:05 time: 0.560115 data_time: 0.069931 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.852556 loss: 0.000604 2022/09/27 07:30:52 - mmengine - INFO - Epoch(train) [180][200/293] lr: 5.000000e-05 eta: 1:12:41 time: 0.549009 data_time: 0.098921 memory: 3840 loss_kpt: 0.000616 acc_pose: 0.819296 loss: 0.000616 2022/09/27 07:31:20 - mmengine - INFO - Epoch(train) [180][250/293] lr: 5.000000e-05 eta: 1:12:17 time: 0.559042 data_time: 0.108078 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.853903 loss: 0.000617 2022/09/27 07:31:43 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:31:43 - mmengine - INFO - Saving checkpoint at 180 epochs 2022/09/27 07:32:05 - mmengine - INFO - Epoch(val) [180][50/407] eta: 0:01:52 time: 0.315605 data_time: 0.166059 memory: 3840 2022/09/27 07:32:20 - mmengine - INFO - Epoch(val) [180][100/407] eta: 0:01:34 time: 0.308532 data_time: 0.183214 memory: 1017 2022/09/27 07:32:36 - mmengine - INFO - Epoch(val) [180][150/407] eta: 0:01:19 time: 0.308258 data_time: 0.166678 memory: 1017 2022/09/27 07:32:51 - mmengine - INFO - Epoch(val) [180][200/407] eta: 0:01:02 time: 0.301086 data_time: 0.167748 memory: 1017 2022/09/27 07:33:06 - mmengine - INFO - Epoch(val) [180][250/407] eta: 0:00:47 time: 0.301911 data_time: 0.173951 memory: 1017 2022/09/27 07:33:21 - mmengine - INFO - Epoch(val) [180][300/407] eta: 0:00:32 time: 0.303388 data_time: 0.177835 memory: 1017 2022/09/27 07:33:36 - mmengine - INFO - Epoch(val) [180][350/407] eta: 0:00:17 time: 0.310356 data_time: 0.179736 memory: 1017 2022/09/27 07:33:47 - mmengine - INFO - Epoch(val) [180][400/407] eta: 0:00:01 time: 0.203497 data_time: 0.109612 memory: 1017 2022/09/27 07:34:20 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 07:34:33 - mmengine - INFO - Epoch(val) [180][407/407] coco/AP: 0.714631 coco/AP .5: 0.897514 coco/AP .75: 0.795277 coco/AP (M): 0.678170 coco/AP (L): 0.782473 coco/AR: 0.771111 coco/AR .5: 0.937657 coco/AR .75: 0.841152 coco/AR (M): 0.727615 coco/AR (L): 0.833854 2022/09/27 07:34:34 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_150.pth is removed 2022/09/27 07:34:36 - mmengine - INFO - The best checkpoint with 0.7146 coco/AP at 180 epoch is saved to best_coco/AP_epoch_180.pth. 2022/09/27 07:35:04 - mmengine - INFO - Epoch(train) [181][50/293] lr: 5.000000e-05 eta: 1:11:29 time: 0.553514 data_time: 0.200195 memory: 3840 loss_kpt: 0.000614 acc_pose: 0.860938 loss: 0.000614 2022/09/27 07:35:32 - mmengine - INFO - Epoch(train) [181][100/293] lr: 5.000000e-05 eta: 1:11:05 time: 0.562481 data_time: 0.173929 memory: 3840 loss_kpt: 0.000614 acc_pose: 0.815288 loss: 0.000614 2022/09/27 07:35:59 - mmengine - INFO - Epoch(train) [181][150/293] lr: 5.000000e-05 eta: 1:10:41 time: 0.544476 data_time: 0.166896 memory: 3840 loss_kpt: 0.000625 acc_pose: 0.802729 loss: 0.000625 2022/09/27 07:36:27 - mmengine - INFO - Epoch(train) [181][200/293] lr: 5.000000e-05 eta: 1:10:17 time: 0.561291 data_time: 0.090175 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.831527 loss: 0.000610 2022/09/27 07:36:55 - mmengine - INFO - Epoch(train) [181][250/293] lr: 5.000000e-05 eta: 1:09:53 time: 0.565156 data_time: 0.092575 memory: 3840 loss_kpt: 0.000619 acc_pose: 0.859650 loss: 0.000619 2022/09/27 07:37:00 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:37:19 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:37:49 - mmengine - INFO - Epoch(train) [182][50/293] lr: 5.000000e-05 eta: 1:09:04 time: 0.602814 data_time: 0.106926 memory: 3840 loss_kpt: 0.000622 acc_pose: 0.818360 loss: 0.000622 2022/09/27 07:38:16 - mmengine - INFO - Epoch(train) [182][100/293] lr: 5.000000e-05 eta: 1:08:40 time: 0.544650 data_time: 0.083665 memory: 3840 loss_kpt: 0.000618 acc_pose: 0.841136 loss: 0.000618 2022/09/27 07:38:45 - mmengine - INFO - Epoch(train) [182][150/293] lr: 5.000000e-05 eta: 1:08:16 time: 0.568836 data_time: 0.092259 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.846300 loss: 0.000617 2022/09/27 07:39:13 - mmengine - INFO - Epoch(train) [182][200/293] lr: 5.000000e-05 eta: 1:07:52 time: 0.567233 data_time: 0.139167 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.840452 loss: 0.000603 2022/09/27 07:39:40 - mmengine - INFO - Epoch(train) [182][250/293] lr: 5.000000e-05 eta: 1:07:28 time: 0.545480 data_time: 0.115351 memory: 3840 loss_kpt: 0.000619 acc_pose: 0.806254 loss: 0.000619 2022/09/27 07:40:05 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:40:34 - mmengine - INFO - Epoch(train) [183][50/293] lr: 5.000000e-05 eta: 1:06:40 time: 0.581552 data_time: 0.088775 memory: 3840 loss_kpt: 0.000627 acc_pose: 0.824523 loss: 0.000627 2022/09/27 07:41:01 - mmengine - INFO - Epoch(train) [183][100/293] lr: 5.000000e-05 eta: 1:06:16 time: 0.541673 data_time: 0.080233 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.829709 loss: 0.000613 2022/09/27 07:41:29 - mmengine - INFO - Epoch(train) [183][150/293] lr: 5.000000e-05 eta: 1:05:52 time: 0.565419 data_time: 0.072691 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.812741 loss: 0.000609 2022/09/27 07:41:57 - mmengine - INFO - Epoch(train) [183][200/293] lr: 5.000000e-05 eta: 1:05:28 time: 0.556170 data_time: 0.083773 memory: 3840 loss_kpt: 0.000622 acc_pose: 0.827244 loss: 0.000622 2022/09/27 07:42:25 - mmengine - INFO - Epoch(train) [183][250/293] lr: 5.000000e-05 eta: 1:05:04 time: 0.551688 data_time: 0.081702 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.861918 loss: 0.000609 2022/09/27 07:42:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:43:18 - mmengine - INFO - Epoch(train) [184][50/293] lr: 5.000000e-05 eta: 1:04:16 time: 0.595697 data_time: 0.144200 memory: 3840 loss_kpt: 0.000620 acc_pose: 0.792271 loss: 0.000620 2022/09/27 07:43:46 - mmengine - INFO - Epoch(train) [184][100/293] lr: 5.000000e-05 eta: 1:03:52 time: 0.559318 data_time: 0.103899 memory: 3840 loss_kpt: 0.000616 acc_pose: 0.888847 loss: 0.000616 2022/09/27 07:44:14 - mmengine - INFO - Epoch(train) [184][150/293] lr: 5.000000e-05 eta: 1:03:28 time: 0.551027 data_time: 0.079562 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.824042 loss: 0.000607 2022/09/27 07:44:42 - mmengine - INFO - Epoch(train) [184][200/293] lr: 5.000000e-05 eta: 1:03:04 time: 0.566772 data_time: 0.082571 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.834381 loss: 0.000605 2022/09/27 07:45:10 - mmengine - INFO - Epoch(train) [184][250/293] lr: 5.000000e-05 eta: 1:02:40 time: 0.545641 data_time: 0.089252 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.844182 loss: 0.000617 2022/09/27 07:45:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:46:02 - mmengine - INFO - Epoch(train) [185][50/293] lr: 5.000000e-05 eta: 1:01:52 time: 0.571871 data_time: 0.113061 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.843228 loss: 0.000617 2022/09/27 07:46:22 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:46:29 - mmengine - INFO - Epoch(train) [185][100/293] lr: 5.000000e-05 eta: 1:01:27 time: 0.539219 data_time: 0.083825 memory: 3840 loss_kpt: 0.000622 acc_pose: 0.847342 loss: 0.000622 2022/09/27 07:46:58 - mmengine - INFO - Epoch(train) [185][150/293] lr: 5.000000e-05 eta: 1:01:04 time: 0.586936 data_time: 0.084690 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.851278 loss: 0.000617 2022/09/27 07:47:27 - mmengine - INFO - Epoch(train) [185][200/293] lr: 5.000000e-05 eta: 1:00:40 time: 0.563538 data_time: 0.086195 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.857595 loss: 0.000612 2022/09/27 07:47:54 - mmengine - INFO - Epoch(train) [185][250/293] lr: 5.000000e-05 eta: 1:00:15 time: 0.545118 data_time: 0.083619 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.842415 loss: 0.000610 2022/09/27 07:48:18 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:48:47 - mmengine - INFO - Epoch(train) [186][50/293] lr: 5.000000e-05 eta: 0:59:28 time: 0.587948 data_time: 0.173009 memory: 3840 loss_kpt: 0.000596 acc_pose: 0.894187 loss: 0.000596 2022/09/27 07:49:15 - mmengine - INFO - Epoch(train) [186][100/293] lr: 5.000000e-05 eta: 0:59:04 time: 0.559801 data_time: 0.160821 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.877545 loss: 0.000617 2022/09/27 07:49:43 - mmengine - INFO - Epoch(train) [186][150/293] lr: 5.000000e-05 eta: 0:58:39 time: 0.558170 data_time: 0.101809 memory: 3840 loss_kpt: 0.000602 acc_pose: 0.868971 loss: 0.000602 2022/09/27 07:50:10 - mmengine - INFO - Epoch(train) [186][200/293] lr: 5.000000e-05 eta: 0:58:15 time: 0.545783 data_time: 0.088743 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.830132 loss: 0.000600 2022/09/27 07:50:37 - mmengine - INFO - Epoch(train) [186][250/293] lr: 5.000000e-05 eta: 0:57:51 time: 0.532905 data_time: 0.083216 memory: 3840 loss_kpt: 0.000621 acc_pose: 0.854152 loss: 0.000621 2022/09/27 07:51:01 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:51:29 - mmengine - INFO - Epoch(train) [187][50/293] lr: 5.000000e-05 eta: 0:57:03 time: 0.567430 data_time: 0.102597 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.860798 loss: 0.000601 2022/09/27 07:51:57 - mmengine - INFO - Epoch(train) [187][100/293] lr: 5.000000e-05 eta: 0:56:39 time: 0.552370 data_time: 0.129485 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.837396 loss: 0.000601 2022/09/27 07:52:23 - mmengine - INFO - Epoch(train) [187][150/293] lr: 5.000000e-05 eta: 0:56:15 time: 0.532669 data_time: 0.072106 memory: 3840 loss_kpt: 0.000619 acc_pose: 0.836537 loss: 0.000619 2022/09/27 07:52:51 - mmengine - INFO - Epoch(train) [187][200/293] lr: 5.000000e-05 eta: 0:55:51 time: 0.561960 data_time: 0.093697 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.860469 loss: 0.000607 2022/09/27 07:53:18 - mmengine - INFO - Epoch(train) [187][250/293] lr: 5.000000e-05 eta: 0:55:27 time: 0.541189 data_time: 0.096078 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.838295 loss: 0.000608 2022/09/27 07:53:42 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:54:10 - mmengine - INFO - Epoch(train) [188][50/293] lr: 5.000000e-05 eta: 0:54:39 time: 0.577012 data_time: 0.096569 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.846745 loss: 0.000612 2022/09/27 07:54:38 - mmengine - INFO - Epoch(train) [188][100/293] lr: 5.000000e-05 eta: 0:54:15 time: 0.542583 data_time: 0.080455 memory: 3840 loss_kpt: 0.000596 acc_pose: 0.871354 loss: 0.000596 2022/09/27 07:55:05 - mmengine - INFO - Epoch(train) [188][150/293] lr: 5.000000e-05 eta: 0:53:50 time: 0.543166 data_time: 0.078798 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.811588 loss: 0.000611 2022/09/27 07:55:32 - mmengine - INFO - Epoch(train) [188][200/293] lr: 5.000000e-05 eta: 0:53:26 time: 0.547943 data_time: 0.076018 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.881891 loss: 0.000611 2022/09/27 07:55:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:56:00 - mmengine - INFO - Epoch(train) [188][250/293] lr: 5.000000e-05 eta: 0:53:02 time: 0.563638 data_time: 0.084425 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.872267 loss: 0.000613 2022/09/27 07:56:23 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:56:51 - mmengine - INFO - Epoch(train) [189][50/293] lr: 5.000000e-05 eta: 0:52:15 time: 0.559200 data_time: 0.129689 memory: 3840 loss_kpt: 0.000621 acc_pose: 0.857162 loss: 0.000621 2022/09/27 07:57:18 - mmengine - INFO - Epoch(train) [189][100/293] lr: 5.000000e-05 eta: 0:51:50 time: 0.540517 data_time: 0.089645 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.836833 loss: 0.000600 2022/09/27 07:57:45 - mmengine - INFO - Epoch(train) [189][150/293] lr: 5.000000e-05 eta: 0:51:26 time: 0.538981 data_time: 0.079524 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.811208 loss: 0.000611 2022/09/27 07:58:12 - mmengine - INFO - Epoch(train) [189][200/293] lr: 5.000000e-05 eta: 0:51:02 time: 0.541748 data_time: 0.078776 memory: 3840 loss_kpt: 0.000616 acc_pose: 0.808840 loss: 0.000616 2022/09/27 07:58:39 - mmengine - INFO - Epoch(train) [189][250/293] lr: 5.000000e-05 eta: 0:50:38 time: 0.530676 data_time: 0.079565 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.828971 loss: 0.000601 2022/09/27 07:59:03 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 07:59:32 - mmengine - INFO - Epoch(train) [190][50/293] lr: 5.000000e-05 eta: 0:49:50 time: 0.582524 data_time: 0.097753 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.876353 loss: 0.000600 2022/09/27 07:59:59 - mmengine - INFO - Epoch(train) [190][100/293] lr: 5.000000e-05 eta: 0:49:26 time: 0.544527 data_time: 0.071548 memory: 3840 loss_kpt: 0.000606 acc_pose: 0.809557 loss: 0.000606 2022/09/27 08:00:26 - mmengine - INFO - Epoch(train) [190][150/293] lr: 5.000000e-05 eta: 0:49:02 time: 0.530223 data_time: 0.090835 memory: 3840 loss_kpt: 0.000606 acc_pose: 0.809416 loss: 0.000606 2022/09/27 08:00:54 - mmengine - INFO - Epoch(train) [190][200/293] lr: 5.000000e-05 eta: 0:48:38 time: 0.563498 data_time: 0.150207 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.796107 loss: 0.000610 2022/09/27 08:01:21 - mmengine - INFO - Epoch(train) [190][250/293] lr: 5.000000e-05 eta: 0:48:13 time: 0.550604 data_time: 0.158120 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.848575 loss: 0.000605 2022/09/27 08:01:45 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:01:45 - mmengine - INFO - Saving checkpoint at 190 epochs 2022/09/27 08:02:07 - mmengine - INFO - Epoch(val) [190][50/407] eta: 0:01:48 time: 0.302801 data_time: 0.186833 memory: 3840 2022/09/27 08:02:21 - mmengine - INFO - Epoch(val) [190][100/407] eta: 0:01:29 time: 0.290257 data_time: 0.160501 memory: 1017 2022/09/27 08:02:38 - mmengine - INFO - Epoch(val) [190][150/407] eta: 0:01:25 time: 0.333324 data_time: 0.197885 memory: 1017 2022/09/27 08:02:52 - mmengine - INFO - Epoch(val) [190][200/407] eta: 0:00:59 time: 0.289492 data_time: 0.159856 memory: 1017 2022/09/27 08:03:07 - mmengine - INFO - Epoch(val) [190][250/407] eta: 0:00:44 time: 0.285829 data_time: 0.165932 memory: 1017 2022/09/27 08:03:22 - mmengine - INFO - Epoch(val) [190][300/407] eta: 0:00:32 time: 0.305277 data_time: 0.196564 memory: 1017 2022/09/27 08:03:38 - mmengine - INFO - Epoch(val) [190][350/407] eta: 0:00:18 time: 0.321625 data_time: 0.191259 memory: 1017 2022/09/27 08:03:49 - mmengine - INFO - Epoch(val) [190][400/407] eta: 0:00:01 time: 0.228666 data_time: 0.147951 memory: 1017 2022/09/27 08:04:23 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 08:04:36 - mmengine - INFO - Epoch(val) [190][407/407] coco/AP: 0.714611 coco/AP .5: 0.896278 coco/AP .75: 0.792208 coco/AP (M): 0.677912 coco/AP (L): 0.782548 coco/AR: 0.770828 coco/AR .5: 0.934666 coco/AR .75: 0.839736 coco/AR (M): 0.727588 coco/AR (L): 0.833185 2022/09/27 08:05:04 - mmengine - INFO - Epoch(train) [191][50/293] lr: 5.000000e-05 eta: 0:47:26 time: 0.553065 data_time: 0.084838 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.841219 loss: 0.000603 2022/09/27 08:05:32 - mmengine - INFO - Epoch(train) [191][100/293] lr: 5.000000e-05 eta: 0:47:02 time: 0.569263 data_time: 0.089386 memory: 3840 loss_kpt: 0.000614 acc_pose: 0.826499 loss: 0.000614 2022/09/27 08:06:01 - mmengine - INFO - Epoch(train) [191][150/293] lr: 5.000000e-05 eta: 0:46:38 time: 0.567699 data_time: 0.118801 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.831222 loss: 0.000613 2022/09/27 08:06:28 - mmengine - INFO - Epoch(train) [191][200/293] lr: 5.000000e-05 eta: 0:46:14 time: 0.555766 data_time: 0.083047 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.812910 loss: 0.000601 2022/09/27 08:06:56 - mmengine - INFO - Epoch(train) [191][250/293] lr: 5.000000e-05 eta: 0:45:49 time: 0.542619 data_time: 0.070180 memory: 3840 loss_kpt: 0.000606 acc_pose: 0.859568 loss: 0.000606 2022/09/27 08:07:18 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:07:40 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:07:47 - mmengine - INFO - Epoch(train) [192][50/293] lr: 5.000000e-05 eta: 0:45:02 time: 0.563478 data_time: 0.094046 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.823622 loss: 0.000610 2022/09/27 08:08:15 - mmengine - INFO - Epoch(train) [192][100/293] lr: 5.000000e-05 eta: 0:44:38 time: 0.557101 data_time: 0.112819 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.841189 loss: 0.000608 2022/09/27 08:08:43 - mmengine - INFO - Epoch(train) [192][150/293] lr: 5.000000e-05 eta: 0:44:14 time: 0.563681 data_time: 0.103355 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.807953 loss: 0.000608 2022/09/27 08:09:10 - mmengine - INFO - Epoch(train) [192][200/293] lr: 5.000000e-05 eta: 0:43:49 time: 0.545826 data_time: 0.079370 memory: 3840 loss_kpt: 0.000614 acc_pose: 0.864271 loss: 0.000614 2022/09/27 08:09:38 - mmengine - INFO - Epoch(train) [192][250/293] lr: 5.000000e-05 eta: 0:43:25 time: 0.560189 data_time: 0.071979 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.858251 loss: 0.000604 2022/09/27 08:10:02 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:10:31 - mmengine - INFO - Epoch(train) [193][50/293] lr: 5.000000e-05 eta: 0:42:38 time: 0.581962 data_time: 0.149608 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.812817 loss: 0.000605 2022/09/27 08:10:58 - mmengine - INFO - Epoch(train) [193][100/293] lr: 5.000000e-05 eta: 0:42:14 time: 0.545242 data_time: 0.136293 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.857922 loss: 0.000600 2022/09/27 08:11:26 - mmengine - INFO - Epoch(train) [193][150/293] lr: 5.000000e-05 eta: 0:41:50 time: 0.568477 data_time: 0.106795 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.797737 loss: 0.000610 2022/09/27 08:11:54 - mmengine - INFO - Epoch(train) [193][200/293] lr: 5.000000e-05 eta: 0:41:26 time: 0.555903 data_time: 0.100027 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.811876 loss: 0.000608 2022/09/27 08:12:22 - mmengine - INFO - Epoch(train) [193][250/293] lr: 5.000000e-05 eta: 0:41:01 time: 0.561879 data_time: 0.141577 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.851416 loss: 0.000607 2022/09/27 08:12:46 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:13:15 - mmengine - INFO - Epoch(train) [194][50/293] lr: 5.000000e-05 eta: 0:40:14 time: 0.579817 data_time: 0.094204 memory: 3840 loss_kpt: 0.000602 acc_pose: 0.802304 loss: 0.000602 2022/09/27 08:13:42 - mmengine - INFO - Epoch(train) [194][100/293] lr: 5.000000e-05 eta: 0:39:50 time: 0.537702 data_time: 0.149655 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.805494 loss: 0.000608 2022/09/27 08:14:10 - mmengine - INFO - Epoch(train) [194][150/293] lr: 5.000000e-05 eta: 0:39:26 time: 0.548497 data_time: 0.088387 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.871872 loss: 0.000607 2022/09/27 08:14:37 - mmengine - INFO - Epoch(train) [194][200/293] lr: 5.000000e-05 eta: 0:39:02 time: 0.550211 data_time: 0.096877 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.818424 loss: 0.000604 2022/09/27 08:15:05 - mmengine - INFO - Epoch(train) [194][250/293] lr: 5.000000e-05 eta: 0:38:37 time: 0.552153 data_time: 0.080131 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.853664 loss: 0.000604 2022/09/27 08:15:29 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:15:58 - mmengine - INFO - Epoch(train) [195][50/293] lr: 5.000000e-05 eta: 0:37:51 time: 0.583455 data_time: 0.103676 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.837298 loss: 0.000609 2022/09/27 08:16:26 - mmengine - INFO - Epoch(train) [195][100/293] lr: 5.000000e-05 eta: 0:37:26 time: 0.549202 data_time: 0.112321 memory: 3840 loss_kpt: 0.000602 acc_pose: 0.800372 loss: 0.000602 2022/09/27 08:16:54 - mmengine - INFO - Epoch(train) [195][150/293] lr: 5.000000e-05 eta: 0:37:02 time: 0.555822 data_time: 0.129261 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.851867 loss: 0.000607 2022/09/27 08:16:58 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:17:22 - mmengine - INFO - Epoch(train) [195][200/293] lr: 5.000000e-05 eta: 0:36:38 time: 0.573622 data_time: 0.093308 memory: 3840 loss_kpt: 0.000599 acc_pose: 0.810976 loss: 0.000599 2022/09/27 08:17:51 - mmengine - INFO - Epoch(train) [195][250/293] lr: 5.000000e-05 eta: 0:36:14 time: 0.569669 data_time: 0.066437 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.857001 loss: 0.000604 2022/09/27 08:18:15 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:18:44 - mmengine - INFO - Epoch(train) [196][50/293] lr: 5.000000e-05 eta: 0:35:27 time: 0.577485 data_time: 0.182190 memory: 3840 loss_kpt: 0.000617 acc_pose: 0.847505 loss: 0.000617 2022/09/27 08:19:11 - mmengine - INFO - Epoch(train) [196][100/293] lr: 5.000000e-05 eta: 0:35:03 time: 0.541229 data_time: 0.124458 memory: 3840 loss_kpt: 0.000599 acc_pose: 0.870263 loss: 0.000599 2022/09/27 08:19:39 - mmengine - INFO - Epoch(train) [196][150/293] lr: 5.000000e-05 eta: 0:34:38 time: 0.561396 data_time: 0.087463 memory: 3840 loss_kpt: 0.000599 acc_pose: 0.832601 loss: 0.000599 2022/09/27 08:20:06 - mmengine - INFO - Epoch(train) [196][200/293] lr: 5.000000e-05 eta: 0:34:14 time: 0.538377 data_time: 0.084096 memory: 3840 loss_kpt: 0.000597 acc_pose: 0.813390 loss: 0.000597 2022/09/27 08:20:34 - mmengine - INFO - Epoch(train) [196][250/293] lr: 5.000000e-05 eta: 0:33:50 time: 0.558625 data_time: 0.084750 memory: 3840 loss_kpt: 0.000594 acc_pose: 0.862537 loss: 0.000594 2022/09/27 08:20:59 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:21:28 - mmengine - INFO - Epoch(train) [197][50/293] lr: 5.000000e-05 eta: 0:33:03 time: 0.596703 data_time: 0.115691 memory: 3840 loss_kpt: 0.000602 acc_pose: 0.859899 loss: 0.000602 2022/09/27 08:21:55 - mmengine - INFO - Epoch(train) [197][100/293] lr: 5.000000e-05 eta: 0:32:39 time: 0.531901 data_time: 0.069962 memory: 3840 loss_kpt: 0.000598 acc_pose: 0.824190 loss: 0.000598 2022/09/27 08:22:23 - mmengine - INFO - Epoch(train) [197][150/293] lr: 5.000000e-05 eta: 0:32:15 time: 0.563977 data_time: 0.168725 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.900475 loss: 0.000601 2022/09/27 08:22:51 - mmengine - INFO - Epoch(train) [197][200/293] lr: 5.000000e-05 eta: 0:31:50 time: 0.548181 data_time: 0.143698 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.821876 loss: 0.000604 2022/09/27 08:23:20 - mmengine - INFO - Epoch(train) [197][250/293] lr: 5.000000e-05 eta: 0:31:26 time: 0.579102 data_time: 0.087474 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.839181 loss: 0.000600 2022/09/27 08:23:43 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:24:13 - mmengine - INFO - Epoch(train) [198][50/293] lr: 5.000000e-05 eta: 0:30:40 time: 0.605443 data_time: 0.120067 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.840502 loss: 0.000604 2022/09/27 08:24:45 - mmengine - INFO - Epoch(train) [198][100/293] lr: 5.000000e-05 eta: 0:30:16 time: 0.640412 data_time: 0.113110 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.807774 loss: 0.000613 2022/09/27 08:25:21 - mmengine - INFO - Epoch(train) [198][150/293] lr: 5.000000e-05 eta: 0:29:52 time: 0.705026 data_time: 0.092483 memory: 3840 loss_kpt: 0.000595 acc_pose: 0.870361 loss: 0.000595 2022/09/27 08:25:51 - mmengine - INFO - Epoch(train) [198][200/293] lr: 5.000000e-05 eta: 0:29:28 time: 0.610137 data_time: 0.090480 memory: 3840 loss_kpt: 0.000610 acc_pose: 0.798457 loss: 0.000610 2022/09/27 08:26:20 - mmengine - INFO - Epoch(train) [198][250/293] lr: 5.000000e-05 eta: 0:29:03 time: 0.578358 data_time: 0.084985 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.815373 loss: 0.000600 2022/09/27 08:26:37 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:26:45 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:27:14 - mmengine - INFO - Epoch(train) [199][50/293] lr: 5.000000e-05 eta: 0:28:17 time: 0.591959 data_time: 0.165350 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.834920 loss: 0.000607 2022/09/27 08:27:42 - mmengine - INFO - Epoch(train) [199][100/293] lr: 5.000000e-05 eta: 0:27:53 time: 0.561344 data_time: 0.093394 memory: 3840 loss_kpt: 0.000599 acc_pose: 0.833263 loss: 0.000599 2022/09/27 08:28:11 - mmengine - INFO - Epoch(train) [199][150/293] lr: 5.000000e-05 eta: 0:27:28 time: 0.571408 data_time: 0.087992 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.847513 loss: 0.000612 2022/09/27 08:28:39 - mmengine - INFO - Epoch(train) [199][200/293] lr: 5.000000e-05 eta: 0:27:04 time: 0.560213 data_time: 0.099974 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.839609 loss: 0.000603 2022/09/27 08:29:07 - mmengine - INFO - Epoch(train) [199][250/293] lr: 5.000000e-05 eta: 0:26:40 time: 0.570853 data_time: 0.091043 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.876789 loss: 0.000600 2022/09/27 08:29:32 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:30:00 - mmengine - INFO - Epoch(train) [200][50/293] lr: 5.000000e-05 eta: 0:25:53 time: 0.573594 data_time: 0.103132 memory: 3840 loss_kpt: 0.000598 acc_pose: 0.893074 loss: 0.000598 2022/09/27 08:30:28 - mmengine - INFO - Epoch(train) [200][100/293] lr: 5.000000e-05 eta: 0:25:29 time: 0.552784 data_time: 0.102169 memory: 3840 loss_kpt: 0.000600 acc_pose: 0.834683 loss: 0.000600 2022/09/27 08:30:56 - mmengine - INFO - Epoch(train) [200][150/293] lr: 5.000000e-05 eta: 0:25:05 time: 0.554259 data_time: 0.089682 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.839401 loss: 0.000603 2022/09/27 08:31:24 - mmengine - INFO - Epoch(train) [200][200/293] lr: 5.000000e-05 eta: 0:24:40 time: 0.558995 data_time: 0.079938 memory: 3840 loss_kpt: 0.000616 acc_pose: 0.803695 loss: 0.000616 2022/09/27 08:31:51 - mmengine - INFO - Epoch(train) [200][250/293] lr: 5.000000e-05 eta: 0:24:16 time: 0.545803 data_time: 0.089229 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.858489 loss: 0.000607 2022/09/27 08:32:14 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:32:14 - mmengine - INFO - Saving checkpoint at 200 epochs 2022/09/27 08:32:37 - mmengine - INFO - Epoch(val) [200][50/407] eta: 0:01:55 time: 0.323144 data_time: 0.169461 memory: 3840 2022/09/27 08:32:52 - mmengine - INFO - Epoch(val) [200][100/407] eta: 0:01:35 time: 0.309976 data_time: 0.157622 memory: 1017 2022/09/27 08:33:08 - mmengine - INFO - Epoch(val) [200][150/407] eta: 0:01:20 time: 0.314861 data_time: 0.158573 memory: 1017 2022/09/27 08:33:24 - mmengine - INFO - Epoch(val) [200][200/407] eta: 0:01:06 time: 0.321734 data_time: 0.178431 memory: 1017 2022/09/27 08:33:39 - mmengine - INFO - Epoch(val) [200][250/407] eta: 0:00:47 time: 0.304878 data_time: 0.166975 memory: 1017 2022/09/27 08:33:54 - mmengine - INFO - Epoch(val) [200][300/407] eta: 0:00:32 time: 0.302478 data_time: 0.154488 memory: 1017 2022/09/27 08:34:10 - mmengine - INFO - Epoch(val) [200][350/407] eta: 0:00:17 time: 0.309813 data_time: 0.171094 memory: 1017 2022/09/27 08:34:20 - mmengine - INFO - Epoch(val) [200][400/407] eta: 0:00:01 time: 0.198291 data_time: 0.116029 memory: 1017 2022/09/27 08:34:53 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 08:35:06 - mmengine - INFO - Epoch(val) [200][407/407] coco/AP: 0.716109 coco/AP .5: 0.898340 coco/AP .75: 0.797615 coco/AP (M): 0.679939 coco/AP (L): 0.783225 coco/AR: 0.771773 coco/AR .5: 0.936713 coco/AR .75: 0.843514 coco/AR (M): 0.728216 coco/AR (L): 0.834485 2022/09/27 08:35:06 - mmengine - INFO - The previous best checkpoint /mnt/lustre/liqikai/work_dirs/20220927/res50_fp16/best_coco/AP_epoch_180.pth is removed 2022/09/27 08:35:09 - mmengine - INFO - The best checkpoint with 0.7161 coco/AP at 200 epoch is saved to best_coco/AP_epoch_200.pth. 2022/09/27 08:35:36 - mmengine - INFO - Epoch(train) [201][50/293] lr: 5.000000e-06 eta: 0:23:30 time: 0.541448 data_time: 0.181202 memory: 3840 loss_kpt: 0.000606 acc_pose: 0.855811 loss: 0.000606 2022/09/27 08:36:03 - mmengine - INFO - Epoch(train) [201][100/293] lr: 5.000000e-06 eta: 0:23:05 time: 0.552870 data_time: 0.108496 memory: 3840 loss_kpt: 0.000591 acc_pose: 0.847546 loss: 0.000591 2022/09/27 08:36:32 - mmengine - INFO - Epoch(train) [201][150/293] lr: 5.000000e-06 eta: 0:22:41 time: 0.576380 data_time: 0.102877 memory: 3840 loss_kpt: 0.000593 acc_pose: 0.852945 loss: 0.000593 2022/09/27 08:36:59 - mmengine - INFO - Epoch(train) [201][200/293] lr: 5.000000e-06 eta: 0:22:17 time: 0.535737 data_time: 0.095631 memory: 3840 loss_kpt: 0.000586 acc_pose: 0.845123 loss: 0.000586 2022/09/27 08:37:25 - mmengine - INFO - Epoch(train) [201][250/293] lr: 5.000000e-06 eta: 0:21:52 time: 0.519041 data_time: 0.072440 memory: 3840 loss_kpt: 0.000598 acc_pose: 0.821630 loss: 0.000598 2022/09/27 08:37:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:38:17 - mmengine - INFO - Epoch(train) [202][50/293] lr: 5.000000e-06 eta: 0:21:06 time: 0.566635 data_time: 0.096161 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.870616 loss: 0.000607 2022/09/27 08:38:45 - mmengine - INFO - Epoch(train) [202][100/293] lr: 5.000000e-06 eta: 0:20:42 time: 0.562607 data_time: 0.179558 memory: 3840 loss_kpt: 0.000596 acc_pose: 0.848010 loss: 0.000596 2022/09/27 08:38:49 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:39:14 - mmengine - INFO - Epoch(train) [202][150/293] lr: 5.000000e-06 eta: 0:20:17 time: 0.566489 data_time: 0.170232 memory: 3840 loss_kpt: 0.000604 acc_pose: 0.837032 loss: 0.000604 2022/09/27 08:39:41 - mmengine - INFO - Epoch(train) [202][200/293] lr: 5.000000e-06 eta: 0:19:53 time: 0.544745 data_time: 0.083713 memory: 3840 loss_kpt: 0.000606 acc_pose: 0.820268 loss: 0.000606 2022/09/27 08:40:08 - mmengine - INFO - Epoch(train) [202][250/293] lr: 5.000000e-06 eta: 0:19:29 time: 0.548743 data_time: 0.074974 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.800180 loss: 0.000608 2022/09/27 08:40:31 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:41:00 - mmengine - INFO - Epoch(train) [203][50/293] lr: 5.000000e-06 eta: 0:18:42 time: 0.564421 data_time: 0.112550 memory: 3840 loss_kpt: 0.000598 acc_pose: 0.856551 loss: 0.000598 2022/09/27 08:41:27 - mmengine - INFO - Epoch(train) [203][100/293] lr: 5.000000e-06 eta: 0:18:18 time: 0.540820 data_time: 0.179672 memory: 3840 loss_kpt: 0.000597 acc_pose: 0.837626 loss: 0.000597 2022/09/27 08:41:54 - mmengine - INFO - Epoch(train) [203][150/293] lr: 5.000000e-06 eta: 0:17:54 time: 0.551235 data_time: 0.090510 memory: 3840 loss_kpt: 0.000585 acc_pose: 0.859896 loss: 0.000585 2022/09/27 08:42:21 - mmengine - INFO - Epoch(train) [203][200/293] lr: 5.000000e-06 eta: 0:17:29 time: 0.542813 data_time: 0.072021 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.831857 loss: 0.000603 2022/09/27 08:42:49 - mmengine - INFO - Epoch(train) [203][250/293] lr: 5.000000e-06 eta: 0:17:05 time: 0.548555 data_time: 0.117424 memory: 3840 loss_kpt: 0.000595 acc_pose: 0.861259 loss: 0.000595 2022/09/27 08:43:13 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:43:42 - mmengine - INFO - Epoch(train) [204][50/293] lr: 5.000000e-06 eta: 0:16:19 time: 0.582571 data_time: 0.112517 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.863260 loss: 0.000609 2022/09/27 08:44:11 - mmengine - INFO - Epoch(train) [204][100/293] lr: 5.000000e-06 eta: 0:15:54 time: 0.580903 data_time: 0.085794 memory: 3840 loss_kpt: 0.000614 acc_pose: 0.826246 loss: 0.000614 2022/09/27 08:44:39 - mmengine - INFO - Epoch(train) [204][150/293] lr: 5.000000e-06 eta: 0:15:30 time: 0.561785 data_time: 0.079748 memory: 3840 loss_kpt: 0.000596 acc_pose: 0.877617 loss: 0.000596 2022/09/27 08:45:07 - mmengine - INFO - Epoch(train) [204][200/293] lr: 5.000000e-06 eta: 0:15:06 time: 0.550493 data_time: 0.072545 memory: 3840 loss_kpt: 0.000596 acc_pose: 0.851224 loss: 0.000596 2022/09/27 08:45:34 - mmengine - INFO - Epoch(train) [204][250/293] lr: 5.000000e-06 eta: 0:14:41 time: 0.548925 data_time: 0.078743 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.864665 loss: 0.000601 2022/09/27 08:45:58 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:46:27 - mmengine - INFO - Epoch(train) [205][50/293] lr: 5.000000e-06 eta: 0:13:55 time: 0.587645 data_time: 0.113332 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.832047 loss: 0.000605 2022/09/27 08:46:55 - mmengine - INFO - Epoch(train) [205][100/293] lr: 5.000000e-06 eta: 0:13:31 time: 0.550062 data_time: 0.119120 memory: 3840 loss_kpt: 0.000597 acc_pose: 0.798039 loss: 0.000597 2022/09/27 08:47:22 - mmengine - INFO - Epoch(train) [205][150/293] lr: 5.000000e-06 eta: 0:13:07 time: 0.551676 data_time: 0.080891 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.839232 loss: 0.000609 2022/09/27 08:47:50 - mmengine - INFO - Epoch(train) [205][200/293] lr: 5.000000e-06 eta: 0:12:42 time: 0.560791 data_time: 0.075381 memory: 3840 loss_kpt: 0.000594 acc_pose: 0.854845 loss: 0.000594 2022/09/27 08:48:06 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:48:19 - mmengine - INFO - Epoch(train) [205][250/293] lr: 5.000000e-06 eta: 0:12:18 time: 0.562339 data_time: 0.079871 memory: 3840 loss_kpt: 0.000596 acc_pose: 0.830137 loss: 0.000596 2022/09/27 08:48:42 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:49:11 - mmengine - INFO - Epoch(train) [206][50/293] lr: 5.000000e-06 eta: 0:11:32 time: 0.566058 data_time: 0.195641 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.876507 loss: 0.000603 2022/09/27 08:49:38 - mmengine - INFO - Epoch(train) [206][100/293] lr: 5.000000e-06 eta: 0:11:07 time: 0.554432 data_time: 0.224509 memory: 3840 loss_kpt: 0.000588 acc_pose: 0.842667 loss: 0.000588 2022/09/27 08:50:06 - mmengine - INFO - Epoch(train) [206][150/293] lr: 5.000000e-06 eta: 0:10:43 time: 0.549137 data_time: 0.156237 memory: 3840 loss_kpt: 0.000607 acc_pose: 0.785174 loss: 0.000607 2022/09/27 08:50:33 - mmengine - INFO - Epoch(train) [206][200/293] lr: 5.000000e-06 eta: 0:10:19 time: 0.543994 data_time: 0.076531 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.806266 loss: 0.000605 2022/09/27 08:51:01 - mmengine - INFO - Epoch(train) [206][250/293] lr: 5.000000e-06 eta: 0:09:54 time: 0.558713 data_time: 0.087302 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.835109 loss: 0.000609 2022/09/27 08:51:24 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:51:53 - mmengine - INFO - Epoch(train) [207][50/293] lr: 5.000000e-06 eta: 0:09:08 time: 0.582362 data_time: 0.109018 memory: 3840 loss_kpt: 0.000597 acc_pose: 0.846636 loss: 0.000597 2022/09/27 08:52:21 - mmengine - INFO - Epoch(train) [207][100/293] lr: 5.000000e-06 eta: 0:08:44 time: 0.547329 data_time: 0.134035 memory: 3840 loss_kpt: 0.000591 acc_pose: 0.865679 loss: 0.000591 2022/09/27 08:52:48 - mmengine - INFO - Epoch(train) [207][150/293] lr: 5.000000e-06 eta: 0:08:20 time: 0.545619 data_time: 0.137942 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.831657 loss: 0.000611 2022/09/27 08:53:16 - mmengine - INFO - Epoch(train) [207][200/293] lr: 5.000000e-06 eta: 0:07:55 time: 0.561142 data_time: 0.084853 memory: 3840 loss_kpt: 0.000590 acc_pose: 0.855063 loss: 0.000590 2022/09/27 08:53:44 - mmengine - INFO - Epoch(train) [207][250/293] lr: 5.000000e-06 eta: 0:07:31 time: 0.555167 data_time: 0.078534 memory: 3840 loss_kpt: 0.000601 acc_pose: 0.858209 loss: 0.000601 2022/09/27 08:54:08 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:54:37 - mmengine - INFO - Epoch(train) [208][50/293] lr: 5.000000e-06 eta: 0:06:45 time: 0.590526 data_time: 0.188258 memory: 3840 loss_kpt: 0.000612 acc_pose: 0.841139 loss: 0.000612 2022/09/27 08:55:06 - mmengine - INFO - Epoch(train) [208][100/293] lr: 5.000000e-06 eta: 0:06:21 time: 0.571867 data_time: 0.108271 memory: 3840 loss_kpt: 0.000597 acc_pose: 0.833170 loss: 0.000597 2022/09/27 08:55:33 - mmengine - INFO - Epoch(train) [208][150/293] lr: 5.000000e-06 eta: 0:05:56 time: 0.551642 data_time: 0.098140 memory: 3840 loss_kpt: 0.000595 acc_pose: 0.845318 loss: 0.000595 2022/09/27 08:56:01 - mmengine - INFO - Epoch(train) [208][200/293] lr: 5.000000e-06 eta: 0:05:32 time: 0.555551 data_time: 0.159338 memory: 3840 loss_kpt: 0.000603 acc_pose: 0.828606 loss: 0.000603 2022/09/27 08:56:29 - mmengine - INFO - Epoch(train) [208][250/293] lr: 5.000000e-06 eta: 0:05:07 time: 0.557456 data_time: 0.158303 memory: 3840 loss_kpt: 0.000609 acc_pose: 0.890595 loss: 0.000609 2022/09/27 08:56:52 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:57:21 - mmengine - INFO - Epoch(train) [209][50/293] lr: 5.000000e-06 eta: 0:04:22 time: 0.572791 data_time: 0.164460 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.836705 loss: 0.000605 2022/09/27 08:57:24 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 08:57:48 - mmengine - INFO - Epoch(train) [209][100/293] lr: 5.000000e-06 eta: 0:03:57 time: 0.545998 data_time: 0.098225 memory: 3840 loss_kpt: 0.000602 acc_pose: 0.864584 loss: 0.000602 2022/09/27 08:58:15 - mmengine - INFO - Epoch(train) [209][150/293] lr: 5.000000e-06 eta: 0:03:33 time: 0.544589 data_time: 0.093473 memory: 3840 loss_kpt: 0.000597 acc_pose: 0.863351 loss: 0.000597 2022/09/27 08:58:43 - mmengine - INFO - Epoch(train) [209][200/293] lr: 5.000000e-06 eta: 0:03:08 time: 0.543476 data_time: 0.078690 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.837591 loss: 0.000613 2022/09/27 08:59:10 - mmengine - INFO - Epoch(train) [209][250/293] lr: 5.000000e-06 eta: 0:02:44 time: 0.549047 data_time: 0.078931 memory: 3840 loss_kpt: 0.000608 acc_pose: 0.839333 loss: 0.000608 2022/09/27 08:59:33 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 09:00:02 - mmengine - INFO - Epoch(train) [210][50/293] lr: 5.000000e-06 eta: 0:01:58 time: 0.564445 data_time: 0.126413 memory: 3840 loss_kpt: 0.000611 acc_pose: 0.810274 loss: 0.000611 2022/09/27 09:00:28 - mmengine - INFO - Epoch(train) [210][100/293] lr: 5.000000e-06 eta: 0:01:34 time: 0.526928 data_time: 0.077389 memory: 3840 loss_kpt: 0.000613 acc_pose: 0.843488 loss: 0.000613 2022/09/27 09:00:54 - mmengine - INFO - Epoch(train) [210][150/293] lr: 5.000000e-06 eta: 0:01:09 time: 0.529982 data_time: 0.083869 memory: 3840 loss_kpt: 0.000594 acc_pose: 0.830777 loss: 0.000594 2022/09/27 09:01:22 - mmengine - INFO - Epoch(train) [210][200/293] lr: 5.000000e-06 eta: 0:00:45 time: 0.553377 data_time: 0.083391 memory: 3840 loss_kpt: 0.000605 acc_pose: 0.883317 loss: 0.000605 2022/09/27 09:01:50 - mmengine - INFO - Epoch(train) [210][250/293] lr: 5.000000e-06 eta: 0:00:21 time: 0.554424 data_time: 0.101992 memory: 3840 loss_kpt: 0.000594 acc_pose: 0.861527 loss: 0.000594 2022/09/27 09:02:14 - mmengine - INFO - Exp name: td-hm_res50_fp16-8xb64-210e_coco-256x192_20220926_221722 2022/09/27 09:02:14 - mmengine - INFO - Saving checkpoint at 210 epochs 2022/09/27 09:02:36 - mmengine - INFO - Epoch(val) [210][50/407] eta: 0:01:55 time: 0.322589 data_time: 0.170817 memory: 3840 2022/09/27 09:02:51 - mmengine - INFO - Epoch(val) [210][100/407] eta: 0:01:32 time: 0.300741 data_time: 0.141454 memory: 1017 2022/09/27 09:03:06 - mmengine - INFO - Epoch(val) [210][150/407] eta: 0:01:16 time: 0.299503 data_time: 0.158172 memory: 1017 2022/09/27 09:03:21 - mmengine - INFO - Epoch(val) [210][200/407] eta: 0:01:00 time: 0.294509 data_time: 0.148546 memory: 1017 2022/09/27 09:03:35 - mmengine - INFO - Epoch(val) [210][250/407] eta: 0:00:46 time: 0.295309 data_time: 0.161073 memory: 1017 2022/09/27 09:03:50 - mmengine - INFO - Epoch(val) [210][300/407] eta: 0:00:30 time: 0.288456 data_time: 0.153712 memory: 1017 2022/09/27 09:04:05 - mmengine - INFO - Epoch(val) [210][350/407] eta: 0:00:16 time: 0.295489 data_time: 0.175064 memory: 1017 2022/09/27 09:04:16 - mmengine - INFO - Epoch(val) [210][400/407] eta: 0:00:01 time: 0.237673 data_time: 0.138811 memory: 1017 2022/09/27 09:04:50 - mmengine - INFO - Evaluating CocoMetric... 2022/09/27 09:05:03 - mmengine - INFO - Epoch(val) [210][407/407] coco/AP: 0.715939 coco/AP .5: 0.896934 coco/AP .75: 0.797521 coco/AP (M): 0.678300 coco/AP (L): 0.784367 coco/AR: 0.771993 coco/AR .5: 0.935139 coco/AR .75: 0.843829 coco/AR (M): 0.728435 coco/AR (L): 0.835080