2020-08-04 11:14:11,177 - mmaction - INFO - Environment info: ------------------------------------------------------------ sys.platform: linux Python: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) [GCC 7.3.0] CUDA available: True CUDA_HOME: /mnt/lustre/share/cuda-9.0/ NVCC: Cuda compilation tools, release 9.0, V9.0.176 GPU 0: GeForce GTX 1080 Ti GCC: gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4) PyTorch: 1.1.0+cuda90_cudnn7.6.3_lms PyTorch compiling details: PyTorch built with: - GCC 5.3 - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications - Intel(R) MKL-DNN v0.18.1 (Git Hash 7de7e5d02bf687f971e7668963649728356e0c20) - OpenMP 201307 (a.k.a. OpenMP 4.0) - NNPACK is enabled - CUDA Runtime 9.0 - NVCC architecture flags: -gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_50,code=compute_50 - CuDNN 7.6.3 - Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=True, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=True, USE_NNPACK=True, USE_OPENMP=ON, TorchVision: 0.2.2 OpenCV: 4.2.0 MMCV: 1.0.3 MMAction2: 0.5.0+7457e5f ------------------------------------------------------------ 2020-08-04 11:14:11,178 - mmaction - INFO - Distributed training: False 2020-08-04 11:14:11,178 - mmaction - INFO - Config: /mnt/lustre/duanhaodong/mmaction2/configs/kennyloc/bsn/bsn_pem.py # model settings model = dict( type='PEM', pem_feat_dim=32, pem_hidden_dim=256, pem_u_ratio_m=1, pem_u_ratio_l=2, pem_high_temporal_iou_threshold=0.6, pem_low_temporal_iou_threshold=2.2, soft_nms_alpha=0.75, soft_nms_low_threshold=0.65, soft_nms_high_threshold=0.9, post_process_top_k=100) # model training and testing settings train_cfg = None test_cfg = dict(average_clips='score') # dataset settings dataset_type = 'ActivityNetDataset' data_root = 'data/ActivityNet/feat_kenny_fine/' data_root_val = 'data/ActivityNet/feat_kenny_fine/' ann_file_train = 'data/ActivityNet/anet_anno_train.json' ann_file_val = 'data/ActivityNet/anet_anno_val_kenny.json' ann_file_test = 'data/ActivityNet/anet_anno_val_kenny.json' work_dir = 'work_dirs/bsn_feat/' pgm_proposals_dir = f'{work_dir}/pgm_proposals/' pgm_features_dir = f'{work_dir}/pgm_features/' test_pipeline = [ dict( type='LoadProposals', top_k=1000, pgm_proposals_dir=pgm_proposals_dir, pgm_features_dir=pgm_features_dir), dict( type='Collect', keys=['bsp_feature', 'tmin', 'tmax', 'tmin_score', 'tmax_score'], meta_name='video_meta', meta_keys=[ 'video_name', 'duration_second', 'duration_frame', 'annotations', 'feature_frame' ]), dict(type='ToTensor', keys=['bsp_feature']) ] train_pipeline = [ dict( type='LoadProposals', top_k=500, pgm_proposals_dir=pgm_proposals_dir, pgm_features_dir=pgm_features_dir), dict( type='Collect', keys=['bsp_feature', 'reference_temporal_iou'], meta_name='video_meta', meta_keys=[]), dict(type='ToTensor', keys=['bsp_feature', 'reference_temporal_iou']), dict( type='ToDataContainer', fields=(dict(key='bsp_feature', stack=False), dict(key='reference_temporal_iou', stack=False))) ] val_pipeline = [ dict( type='LoadProposals', top_k=1000, pgm_proposals_dir=pgm_proposals_dir, pgm_features_dir=pgm_features_dir), dict( type='Collect', keys=['bsp_feature', 'tmin', 'tmax', 'tmin_score', 'tmax_score'], meta_name='video_meta', meta_keys=[ 'video_name', 'duration_second', 'duration_frame', 'annotations', 'feature_frame' ]), dict(type='ToTensor', keys=['bsp_feature']) ] data = dict( videos_per_gpu=16, workers_per_gpu=8, train_dataloader=dict(drop_last=True), val_dataloader=dict(videos_per_gpu=1), test=dict( type=dataset_type, ann_file=ann_file_test, pipeline=test_pipeline, data_prefix=data_root_val), val=dict( type=dataset_type, ann_file=ann_file_val, pipeline=val_pipeline, data_prefix=data_root_val), train=dict( type=dataset_type, ann_file=ann_file_train, pipeline=train_pipeline, data_prefix=data_root)) # optimizer optimizer = dict( type='Adam', lr=0.01, weight_decay=0.00001) # this lr is used for 1 gpus optimizer_config = dict(grad_clip=None) # learning policy lr_config = dict(policy='step', step=10) total_epochs = 20 checkpoint_config = dict(interval=1, filename_tmpl='pem_epoch_{}.pth') evaluation = dict(interval=1, metrics=['AR@AN']) log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')]) # runtime settings dist_params = dict(backend='nccl') log_level = 'INFO' load_from = None resume_from = None workflow = [('train', 1)] output_config = dict(out=f'{work_dir}/results.json', output_format='json') 2020-08-04 11:14:16,235 - mmaction - INFO - Start running, host: duanhaodong@SH-IDC1-10-5-30-137, work_dir: /mnt/lustre/duanhaodong/mmaction2/work_dirs/bsn_feat 2020-08-04 11:14:16,235 - mmaction - INFO - workflow: [('train', 1)], max: 20 epochs 2020-08-04 11:14:31,896 - mmaction - INFO - Epoch [1][50/603] lr: 1.000e-02, eta: 1:02:39, time: 0.313, data_time: 0.303, memory: 25, temporal_iou_loss: 0.0402, loss: 0.0402 2020-08-04 11:14:42,932 - mmaction - INFO - Epoch [1][100/603] lr: 1.000e-02, eta: 0:53:12, time: 0.221, data_time: 0.215, memory: 25, temporal_iou_loss: 0.0307, loss: 0.0307 2020-08-04 11:14:53,708 - mmaction - INFO - Epoch [1][150/603] lr: 1.000e-02, eta: 0:49:34, time: 0.216, data_time: 0.210, memory: 25, temporal_iou_loss: 0.0274, loss: 0.0274 2020-08-04 11:15:04,931 - mmaction - INFO - Epoch [1][200/603] lr: 1.000e-02, eta: 0:48:07, time: 0.224, data_time: 0.218, memory: 25, temporal_iou_loss: 0.0271, loss: 0.0271 2020-08-04 11:15:18,633 - mmaction - INFO - Epoch [1][250/603] lr: 1.000e-02, eta: 0:49:07, time: 0.274, data_time: 0.268, memory: 25, temporal_iou_loss: 0.0272, loss: 0.0272 2020-08-04 11:15:32,783 - mmaction - INFO - Epoch [1][300/603] lr: 1.000e-02, eta: 0:50:00, time: 0.283, data_time: 0.277, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 11:15:45,153 - mmaction - INFO - Epoch [1][350/603] lr: 1.000e-02, eta: 0:49:34, time: 0.247, data_time: 0.241, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:15:58,816 - mmaction - INFO - Epoch [1][400/603] lr: 1.000e-02, eta: 0:49:49, time: 0.273, data_time: 0.267, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:16:11,664 - mmaction - INFO - Epoch [1][450/603] lr: 1.000e-02, eta: 0:49:37, time: 0.257, data_time: 0.251, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:16:25,265 - mmaction - INFO - Epoch [1][500/603] lr: 1.000e-02, eta: 0:49:42, time: 0.272, data_time: 0.266, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:16:39,179 - mmaction - INFO - Epoch [1][550/603] lr: 1.000e-02, eta: 0:49:51, time: 0.278, data_time: 0.272, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:16:51,109 - mmaction - INFO - Epoch [1][600/603] lr: 1.000e-02, eta: 0:49:17, time: 0.239, data_time: 0.233, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:16:52,161 - mmaction - INFO - Saving checkpoint at 1 epochs 2020-08-04 11:17:06,930 - mmaction - INFO - Epoch [2][50/603] lr: 1.000e-02, eta: 0:49:19, time: 0.291, data_time: 0.285, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:17:18,439 - mmaction - INFO - Epoch [2][100/603] lr: 1.000e-02, eta: 0:48:43, time: 0.230, data_time: 0.224, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:17:30,566 - mmaction - INFO - Epoch [2][150/603] lr: 1.000e-02, eta: 0:48:19, time: 0.243, data_time: 0.236, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:17:46,476 - mmaction - INFO - Epoch [2][200/603] lr: 1.000e-02, eta: 0:48:49, time: 0.318, data_time: 0.312, memory: 25, temporal_iou_loss: 0.0250, loss: 0.0250 2020-08-04 11:17:59,188 - mmaction - INFO - Epoch [2][250/603] lr: 1.000e-02, eta: 0:48:32, time: 0.254, data_time: 0.248, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 11:18:10,779 - mmaction - INFO - Epoch [2][300/603] lr: 1.000e-02, eta: 0:48:02, time: 0.232, data_time: 0.226, memory: 25, temporal_iou_loss: 0.0265, loss: 0.0265 2020-08-04 11:18:25,268 - mmaction - INFO - Epoch [2][350/603] lr: 1.000e-02, eta: 0:48:07, time: 0.290, data_time: 0.284, memory: 25, temporal_iou_loss: 0.0273, loss: 0.0273 2020-08-04 11:18:38,232 - mmaction - INFO - Epoch [2][400/603] lr: 1.000e-02, eta: 0:47:54, time: 0.259, data_time: 0.253, memory: 25, temporal_iou_loss: 0.0267, loss: 0.0267 2020-08-04 11:18:50,987 - mmaction - INFO - Epoch [2][450/603] lr: 1.000e-02, eta: 0:47:38, time: 0.255, data_time: 0.249, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:19:04,959 - mmaction - INFO - Epoch [2][500/603] lr: 1.000e-02, eta: 0:47:35, time: 0.279, data_time: 0.273, memory: 25, temporal_iou_loss: 0.0269, loss: 0.0269 2020-08-04 11:19:18,614 - mmaction - INFO - Epoch [2][550/603] lr: 1.000e-02, eta: 0:47:28, time: 0.273, data_time: 0.267, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:19:29,710 - mmaction - INFO - Epoch [2][600/603] lr: 1.000e-02, eta: 0:46:57, time: 0.222, data_time: 0.216, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:19:30,869 - mmaction - INFO - Saving checkpoint at 2 epochs 2020-08-04 11:19:46,463 - mmaction - INFO - Epoch [3][50/603] lr: 1.000e-02, eta: 0:46:58, time: 0.308, data_time: 0.302, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:19:57,836 - mmaction - INFO - Epoch [3][100/603] lr: 1.000e-02, eta: 0:46:31, time: 0.228, data_time: 0.222, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 11:20:11,083 - mmaction - INFO - Epoch [3][150/603] lr: 1.000e-02, eta: 0:46:20, time: 0.265, data_time: 0.258, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:20:26,256 - mmaction - INFO - Epoch [3][200/603] lr: 1.000e-02, eta: 0:46:24, time: 0.303, data_time: 0.297, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:20:43,859 - mmaction - INFO - Epoch [3][250/603] lr: 1.000e-02, eta: 0:46:44, time: 0.352, data_time: 0.346, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:21:00,417 - mmaction - INFO - Epoch [3][300/603] lr: 1.000e-02, eta: 0:46:54, time: 0.331, data_time: 0.325, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:21:16,617 - mmaction - INFO - Epoch [3][350/603] lr: 1.000e-02, eta: 0:47:00, time: 0.324, data_time: 0.318, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:21:32,198 - mmaction - INFO - Epoch [3][400/603] lr: 1.000e-02, eta: 0:47:00, time: 0.312, data_time: 0.305, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:21:50,292 - mmaction - INFO - Epoch [3][450/603] lr: 1.000e-02, eta: 0:47:16, time: 0.362, data_time: 0.356, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 11:22:05,700 - mmaction - INFO - Epoch [3][500/603] lr: 1.000e-02, eta: 0:47:13, time: 0.308, data_time: 0.302, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:22:22,979 - mmaction - INFO - Epoch [3][550/603] lr: 1.000e-02, eta: 0:47:20, time: 0.346, data_time: 0.339, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:22:37,828 - mmaction - INFO - Epoch [3][600/603] lr: 1.000e-02, eta: 0:47:13, time: 0.297, data_time: 0.291, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:22:38,167 - mmaction - INFO - Saving checkpoint at 3 epochs 2020-08-04 11:22:56,124 - mmaction - INFO - Epoch [4][50/603] lr: 1.000e-02, eta: 0:47:05, time: 0.318, data_time: 0.311, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:23:10,694 - mmaction - INFO - Epoch [4][100/603] lr: 1.000e-02, eta: 0:46:55, time: 0.292, data_time: 0.286, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 11:23:23,069 - mmaction - INFO - Epoch [4][150/603] lr: 1.000e-02, eta: 0:46:33, time: 0.248, data_time: 0.241, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:23:38,106 - mmaction - INFO - Epoch [4][200/603] lr: 1.000e-02, eta: 0:46:25, time: 0.301, data_time: 0.295, memory: 25, temporal_iou_loss: 0.0265, loss: 0.0265 2020-08-04 11:23:56,037 - mmaction - INFO - Epoch [4][250/603] lr: 1.000e-02, eta: 0:46:31, time: 0.359, data_time: 0.353, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:24:12,050 - mmaction - INFO - Epoch [4][300/603] lr: 1.000e-02, eta: 0:46:27, time: 0.320, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:24:27,962 - mmaction - INFO - Epoch [4][350/603] lr: 1.000e-02, eta: 0:46:22, time: 0.318, data_time: 0.312, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:24:43,749 - mmaction - INFO - Epoch [4][400/603] lr: 1.000e-02, eta: 0:46:15, time: 0.316, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:25:01,975 - mmaction - INFO - Epoch [4][450/603] lr: 1.000e-02, eta: 0:46:19, time: 0.365, data_time: 0.358, memory: 25, temporal_iou_loss: 0.0265, loss: 0.0265 2020-08-04 11:25:17,857 - mmaction - INFO - Epoch [4][500/603] lr: 1.000e-02, eta: 0:46:12, time: 0.318, data_time: 0.312, memory: 25, temporal_iou_loss: 0.0247, loss: 0.0247 2020-08-04 11:25:33,402 - mmaction - INFO - Epoch [4][550/603] lr: 1.000e-02, eta: 0:46:04, time: 0.311, data_time: 0.305, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:25:48,062 - mmaction - INFO - Epoch [4][600/603] lr: 1.000e-02, eta: 0:45:51, time: 0.293, data_time: 0.287, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:25:49,579 - mmaction - INFO - Saving checkpoint at 4 epochs 2020-08-04 11:26:06,380 - mmaction - INFO - Epoch [5][50/603] lr: 1.000e-02, eta: 0:45:36, time: 0.303, data_time: 0.297, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:26:20,805 - mmaction - INFO - Epoch [5][100/603] lr: 1.000e-02, eta: 0:45:22, time: 0.289, data_time: 0.282, memory: 25, temporal_iou_loss: 0.0252, loss: 0.0252 2020-08-04 11:26:36,080 - mmaction - INFO - Epoch [5][150/603] lr: 1.000e-02, eta: 0:45:12, time: 0.305, data_time: 0.299, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:26:52,076 - mmaction - INFO - Epoch [5][200/603] lr: 1.000e-02, eta: 0:45:04, time: 0.320, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:27:08,655 - mmaction - INFO - Epoch [5][250/603] lr: 1.000e-02, eta: 0:44:58, time: 0.332, data_time: 0.326, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:27:26,293 - mmaction - INFO - Epoch [5][300/603] lr: 1.000e-02, eta: 0:44:55, time: 0.353, data_time: 0.347, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:27:41,920 - mmaction - INFO - Epoch [5][350/603] lr: 1.000e-02, eta: 0:44:44, time: 0.313, data_time: 0.307, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:27:57,607 - mmaction - INFO - Epoch [5][400/603] lr: 1.000e-02, eta: 0:44:34, time: 0.314, data_time: 0.307, memory: 25, temporal_iou_loss: 0.0267, loss: 0.0267 2020-08-04 11:28:14,332 - mmaction - INFO - Epoch [5][450/603] lr: 1.000e-02, eta: 0:44:27, time: 0.334, data_time: 0.325, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:28:33,617 - mmaction - INFO - Epoch [5][500/603] lr: 1.000e-02, eta: 0:44:27, time: 0.386, data_time: 0.379, memory: 25, temporal_iou_loss: 0.0265, loss: 0.0265 2020-08-04 11:28:51,009 - mmaction - INFO - Epoch [5][550/603] lr: 1.000e-02, eta: 0:44:21, time: 0.348, data_time: 0.337, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:29:06,149 - mmaction - INFO - Epoch [5][600/603] lr: 1.000e-02, eta: 0:44:08, time: 0.303, data_time: 0.296, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:29:07,582 - mmaction - INFO - Saving checkpoint at 5 epochs 2020-08-04 11:29:26,334 - mmaction - INFO - Epoch [6][50/603] lr: 1.000e-02, eta: 0:44:02, time: 0.371, data_time: 0.364, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:29:42,372 - mmaction - INFO - Epoch [6][100/603] lr: 1.000e-02, eta: 0:43:51, time: 0.321, data_time: 0.311, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 11:29:58,287 - mmaction - INFO - Epoch [6][150/603] lr: 1.000e-02, eta: 0:43:39, time: 0.318, data_time: 0.307, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:30:17,188 - mmaction - INFO - Epoch [6][200/603] lr: 1.000e-02, eta: 0:43:36, time: 0.378, data_time: 0.367, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:30:33,917 - mmaction - INFO - Epoch [6][250/603] lr: 1.000e-02, eta: 0:43:27, time: 0.335, data_time: 0.323, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:30:49,219 - mmaction - INFO - Epoch [6][300/603] lr: 1.000e-02, eta: 0:43:13, time: 0.306, data_time: 0.300, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:31:02,975 - mmaction - INFO - Epoch [6][350/603] lr: 1.000e-02, eta: 0:42:55, time: 0.275, data_time: 0.269, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:31:21,509 - mmaction - INFO - Epoch [6][400/603] lr: 1.000e-02, eta: 0:42:50, time: 0.371, data_time: 0.361, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:31:39,522 - mmaction - INFO - Epoch [6][450/603] lr: 1.000e-02, eta: 0:42:43, time: 0.360, data_time: 0.350, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:31:58,090 - mmaction - INFO - Epoch [6][500/603] lr: 1.000e-02, eta: 0:42:37, time: 0.371, data_time: 0.360, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:32:15,691 - mmaction - INFO - Epoch [6][550/603] lr: 1.000e-02, eta: 0:42:28, time: 0.352, data_time: 0.343, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:32:30,561 - mmaction - INFO - Epoch [6][600/603] lr: 1.000e-02, eta: 0:42:13, time: 0.297, data_time: 0.289, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:32:32,495 - mmaction - INFO - Saving checkpoint at 6 epochs 2020-08-04 11:32:48,652 - mmaction - INFO - Epoch [7][50/603] lr: 1.000e-02, eta: 0:41:57, time: 0.319, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0266, loss: 0.0266 2020-08-04 11:33:06,433 - mmaction - INFO - Epoch [7][100/603] lr: 1.000e-02, eta: 0:41:48, time: 0.356, data_time: 0.344, memory: 25, temporal_iou_loss: 0.0266, loss: 0.0266 2020-08-04 11:33:22,228 - mmaction - INFO - Epoch [7][150/603] lr: 1.000e-02, eta: 0:41:35, time: 0.316, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:33:38,540 - mmaction - INFO - Epoch [7][200/603] lr: 1.000e-02, eta: 0:41:23, time: 0.326, data_time: 0.319, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:33:55,495 - mmaction - INFO - Epoch [7][250/603] lr: 1.000e-02, eta: 0:41:12, time: 0.339, data_time: 0.333, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:34:12,077 - mmaction - INFO - Epoch [7][300/603] lr: 1.000e-02, eta: 0:41:00, time: 0.332, data_time: 0.326, memory: 25, temporal_iou_loss: 0.0270, loss: 0.0270 2020-08-04 11:34:27,969 - mmaction - INFO - Epoch [7][350/603] lr: 1.000e-02, eta: 0:40:46, time: 0.318, data_time: 0.312, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:34:44,117 - mmaction - INFO - Epoch [7][400/603] lr: 1.000e-02, eta: 0:40:33, time: 0.323, data_time: 0.317, memory: 25, temporal_iou_loss: 0.0250, loss: 0.0250 2020-08-04 11:35:00,328 - mmaction - INFO - Epoch [7][450/603] lr: 1.000e-02, eta: 0:40:20, time: 0.324, data_time: 0.318, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:35:18,322 - mmaction - INFO - Epoch [7][500/603] lr: 1.000e-02, eta: 0:40:11, time: 0.360, data_time: 0.354, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:35:34,053 - mmaction - INFO - Epoch [7][550/603] lr: 1.000e-02, eta: 0:39:56, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:35:48,880 - mmaction - INFO - Epoch [7][600/603] lr: 1.000e-02, eta: 0:39:41, time: 0.297, data_time: 0.290, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:35:50,060 - mmaction - INFO - Saving checkpoint at 7 epochs 2020-08-04 11:36:05,162 - mmaction - INFO - Epoch [8][50/603] lr: 1.000e-02, eta: 0:39:22, time: 0.298, data_time: 0.291, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:36:19,668 - mmaction - INFO - Epoch [8][100/603] lr: 1.000e-02, eta: 0:39:06, time: 0.290, data_time: 0.284, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:36:34,794 - mmaction - INFO - Epoch [8][150/603] lr: 1.000e-02, eta: 0:38:51, time: 0.302, data_time: 0.296, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 11:36:50,972 - mmaction - INFO - Epoch [8][200/603] lr: 1.000e-02, eta: 0:38:37, time: 0.324, data_time: 0.317, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:37:06,419 - mmaction - INFO - Epoch [8][250/603] lr: 1.000e-02, eta: 0:38:23, time: 0.309, data_time: 0.303, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:37:22,181 - mmaction - INFO - Epoch [8][300/603] lr: 1.000e-02, eta: 0:38:08, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:37:39,986 - mmaction - INFO - Epoch [8][350/603] lr: 1.000e-02, eta: 0:37:58, time: 0.356, data_time: 0.350, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:37:55,461 - mmaction - INFO - Epoch [8][400/603] lr: 1.000e-02, eta: 0:37:43, time: 0.309, data_time: 0.303, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:38:11,732 - mmaction - INFO - Epoch [8][450/603] lr: 1.000e-02, eta: 0:37:29, time: 0.325, data_time: 0.319, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:38:27,487 - mmaction - INFO - Epoch [8][500/603] lr: 1.000e-02, eta: 0:37:15, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:38:44,269 - mmaction - INFO - Epoch [8][550/603] lr: 1.000e-02, eta: 0:37:02, time: 0.336, data_time: 0.329, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:39:00,420 - mmaction - INFO - Epoch [8][600/603] lr: 1.000e-02, eta: 0:36:48, time: 0.323, data_time: 0.317, memory: 25, temporal_iou_loss: 0.0252, loss: 0.0252 2020-08-04 11:39:00,598 - mmaction - INFO - Saving checkpoint at 8 epochs 2020-08-04 11:39:21,002 - mmaction - INFO - Epoch [9][50/603] lr: 1.000e-02, eta: 0:36:38, time: 0.407, data_time: 0.400, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:39:34,952 - mmaction - INFO - Epoch [9][100/603] lr: 1.000e-02, eta: 0:36:21, time: 0.279, data_time: 0.273, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:39:51,302 - mmaction - INFO - Epoch [9][150/603] lr: 1.000e-02, eta: 0:36:07, time: 0.327, data_time: 0.321, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:40:06,620 - mmaction - INFO - Epoch [9][200/603] lr: 1.000e-02, eta: 0:35:52, time: 0.306, data_time: 0.300, memory: 25, temporal_iou_loss: 0.0268, loss: 0.0268 2020-08-04 11:40:22,195 - mmaction - INFO - Epoch [9][250/603] lr: 1.000e-02, eta: 0:35:37, time: 0.311, data_time: 0.305, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:40:38,224 - mmaction - INFO - Epoch [9][300/603] lr: 1.000e-02, eta: 0:35:23, time: 0.321, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:40:56,049 - mmaction - INFO - Epoch [9][350/603] lr: 1.000e-02, eta: 0:35:11, time: 0.356, data_time: 0.350, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:41:12,711 - mmaction - INFO - Epoch [9][400/603] lr: 1.000e-02, eta: 0:34:57, time: 0.333, data_time: 0.327, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:41:28,820 - mmaction - INFO - Epoch [9][450/603] lr: 1.000e-02, eta: 0:34:43, time: 0.322, data_time: 0.316, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:41:44,651 - mmaction - INFO - Epoch [9][500/603] lr: 1.000e-02, eta: 0:34:28, time: 0.317, data_time: 0.311, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:42:03,888 - mmaction - INFO - Epoch [9][550/603] lr: 1.000e-02, eta: 0:34:17, time: 0.385, data_time: 0.379, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:42:18,189 - mmaction - INFO - Epoch [9][600/603] lr: 1.000e-02, eta: 0:34:01, time: 0.286, data_time: 0.280, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:42:18,349 - mmaction - INFO - Saving checkpoint at 9 epochs 2020-08-04 11:42:34,152 - mmaction - INFO - Epoch [10][50/603] lr: 1.000e-02, eta: 0:33:43, time: 0.311, data_time: 0.304, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:42:46,587 - mmaction - INFO - Epoch [10][100/603] lr: 1.000e-02, eta: 0:33:25, time: 0.249, data_time: 0.243, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:43:02,744 - mmaction - INFO - Epoch [10][150/603] lr: 1.000e-02, eta: 0:33:10, time: 0.323, data_time: 0.317, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 11:43:19,493 - mmaction - INFO - Epoch [10][200/603] lr: 1.000e-02, eta: 0:32:56, time: 0.335, data_time: 0.329, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:43:35,486 - mmaction - INFO - Epoch [10][250/603] lr: 1.000e-02, eta: 0:32:42, time: 0.320, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:43:51,612 - mmaction - INFO - Epoch [10][300/603] lr: 1.000e-02, eta: 0:32:27, time: 0.322, data_time: 0.316, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:44:08,220 - mmaction - INFO - Epoch [10][350/603] lr: 1.000e-02, eta: 0:32:13, time: 0.332, data_time: 0.326, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:44:24,821 - mmaction - INFO - Epoch [10][400/603] lr: 1.000e-02, eta: 0:31:59, time: 0.332, data_time: 0.326, memory: 25, temporal_iou_loss: 0.0268, loss: 0.0268 2020-08-04 11:44:41,726 - mmaction - INFO - Epoch [10][450/603] lr: 1.000e-02, eta: 0:31:45, time: 0.338, data_time: 0.332, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:44:57,009 - mmaction - INFO - Epoch [10][500/603] lr: 1.000e-02, eta: 0:31:30, time: 0.306, data_time: 0.300, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:45:16,005 - mmaction - INFO - Epoch [10][550/603] lr: 1.000e-02, eta: 0:31:18, time: 0.380, data_time: 0.374, memory: 25, temporal_iou_loss: 0.0267, loss: 0.0267 2020-08-04 11:45:30,016 - mmaction - INFO - Epoch [10][600/603] lr: 1.000e-02, eta: 0:31:01, time: 0.280, data_time: 0.274, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:45:30,186 - mmaction - INFO - Saving checkpoint at 10 epochs 2020-08-04 11:45:46,427 - mmaction - INFO - Epoch [11][50/603] lr: 1.000e-03, eta: 0:30:44, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:45:58,947 - mmaction - INFO - Epoch [11][100/603] lr: 1.000e-03, eta: 0:30:26, time: 0.251, data_time: 0.244, memory: 25, temporal_iou_loss: 0.0250, loss: 0.0250 2020-08-04 11:46:17,050 - mmaction - INFO - Epoch [11][150/603] lr: 1.000e-03, eta: 0:30:13, time: 0.362, data_time: 0.356, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 11:46:32,477 - mmaction - INFO - Epoch [11][200/603] lr: 1.000e-03, eta: 0:29:58, time: 0.309, data_time: 0.303, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:46:48,973 - mmaction - INFO - Epoch [11][250/603] lr: 1.000e-03, eta: 0:29:43, time: 0.330, data_time: 0.324, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:47:04,364 - mmaction - INFO - Epoch [11][300/603] lr: 1.000e-03, eta: 0:29:28, time: 0.308, data_time: 0.302, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:47:22,813 - mmaction - INFO - Epoch [11][350/603] lr: 1.000e-03, eta: 0:29:15, time: 0.369, data_time: 0.363, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:47:38,790 - mmaction - INFO - Epoch [11][400/603] lr: 1.000e-03, eta: 0:29:00, time: 0.320, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:47:54,351 - mmaction - INFO - Epoch [11][450/603] lr: 1.000e-03, eta: 0:28:45, time: 0.311, data_time: 0.305, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:48:09,890 - mmaction - INFO - Epoch [11][500/603] lr: 1.000e-03, eta: 0:28:29, time: 0.311, data_time: 0.305, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 11:48:26,495 - mmaction - INFO - Epoch [11][550/603] lr: 1.000e-03, eta: 0:28:15, time: 0.332, data_time: 0.326, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:48:42,625 - mmaction - INFO - Epoch [11][600/603] lr: 1.000e-03, eta: 0:28:00, time: 0.323, data_time: 0.317, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:48:42,766 - mmaction - INFO - Saving checkpoint at 11 epochs 2020-08-04 11:49:00,092 - mmaction - INFO - Epoch [12][50/603] lr: 1.000e-03, eta: 0:27:44, time: 0.343, data_time: 0.336, memory: 25, temporal_iou_loss: 0.0252, loss: 0.0252 2020-08-04 11:49:13,462 - mmaction - INFO - Epoch [12][100/603] lr: 1.000e-03, eta: 0:27:27, time: 0.268, data_time: 0.261, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:49:28,789 - mmaction - INFO - Epoch [12][150/603] lr: 1.000e-03, eta: 0:27:11, time: 0.307, data_time: 0.301, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:49:43,962 - mmaction - INFO - Epoch [12][200/603] lr: 1.000e-03, eta: 0:26:56, time: 0.303, data_time: 0.297, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:50:02,511 - mmaction - INFO - Epoch [12][250/603] lr: 1.000e-03, eta: 0:26:43, time: 0.371, data_time: 0.365, memory: 25, temporal_iou_loss: 0.0249, loss: 0.0249 2020-08-04 11:50:17,968 - mmaction - INFO - Epoch [12][300/603] lr: 1.000e-03, eta: 0:26:27, time: 0.309, data_time: 0.303, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 11:50:33,251 - mmaction - INFO - Epoch [12][350/603] lr: 1.000e-03, eta: 0:26:11, time: 0.306, data_time: 0.300, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:50:50,282 - mmaction - INFO - Epoch [12][400/603] lr: 1.000e-03, eta: 0:25:57, time: 0.341, data_time: 0.335, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:51:06,138 - mmaction - INFO - Epoch [12][450/603] lr: 1.000e-03, eta: 0:25:42, time: 0.317, data_time: 0.311, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:51:24,256 - mmaction - INFO - Epoch [12][500/603] lr: 1.000e-03, eta: 0:25:28, time: 0.362, data_time: 0.356, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:51:40,362 - mmaction - INFO - Epoch [12][550/603] lr: 1.000e-03, eta: 0:25:13, time: 0.322, data_time: 0.316, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:51:55,746 - mmaction - INFO - Epoch [12][600/603] lr: 1.000e-03, eta: 0:24:57, time: 0.308, data_time: 0.301, memory: 25, temporal_iou_loss: 0.0252, loss: 0.0252 2020-08-04 11:51:56,868 - mmaction - INFO - Saving checkpoint at 12 epochs 2020-08-04 11:52:13,145 - mmaction - INFO - Epoch [13][50/603] lr: 1.000e-03, eta: 0:24:41, time: 0.322, data_time: 0.315, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:52:26,558 - mmaction - INFO - Epoch [13][100/603] lr: 1.000e-03, eta: 0:24:24, time: 0.268, data_time: 0.262, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:52:39,816 - mmaction - INFO - Epoch [13][150/603] lr: 1.000e-03, eta: 0:24:07, time: 0.265, data_time: 0.259, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:52:56,868 - mmaction - INFO - Epoch [13][200/603] lr: 1.000e-03, eta: 0:23:52, time: 0.341, data_time: 0.335, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 11:53:13,471 - mmaction - INFO - Epoch [13][250/603] lr: 1.000e-03, eta: 0:23:38, time: 0.332, data_time: 0.326, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 11:53:29,907 - mmaction - INFO - Epoch [13][300/603] lr: 1.000e-03, eta: 0:23:23, time: 0.329, data_time: 0.322, memory: 25, temporal_iou_loss: 0.0246, loss: 0.0246 2020-08-04 11:53:45,299 - mmaction - INFO - Epoch [13][350/603] lr: 1.000e-03, eta: 0:23:07, time: 0.308, data_time: 0.302, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 11:54:01,757 - mmaction - INFO - Epoch [13][400/603] lr: 1.000e-03, eta: 0:22:52, time: 0.329, data_time: 0.323, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:54:17,812 - mmaction - INFO - Epoch [13][450/603] lr: 1.000e-03, eta: 0:22:37, time: 0.321, data_time: 0.315, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:54:35,159 - mmaction - INFO - Epoch [13][500/603] lr: 1.000e-03, eta: 0:22:22, time: 0.347, data_time: 0.341, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:54:50,917 - mmaction - INFO - Epoch [13][550/603] lr: 1.000e-03, eta: 0:22:07, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:55:06,944 - mmaction - INFO - Epoch [13][600/603] lr: 1.000e-03, eta: 0:21:52, time: 0.321, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:55:08,582 - mmaction - INFO - Saving checkpoint at 13 epochs 2020-08-04 11:55:24,291 - mmaction - INFO - Epoch [14][50/603] lr: 1.000e-03, eta: 0:21:35, time: 0.307, data_time: 0.301, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:55:38,179 - mmaction - INFO - Epoch [14][100/603] lr: 1.000e-03, eta: 0:21:18, time: 0.278, data_time: 0.272, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:55:50,915 - mmaction - INFO - Epoch [14][150/603] lr: 1.000e-03, eta: 0:21:01, time: 0.255, data_time: 0.249, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:56:06,395 - mmaction - INFO - Epoch [14][200/603] lr: 1.000e-03, eta: 0:20:46, time: 0.310, data_time: 0.304, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:56:26,419 - mmaction - INFO - Epoch [14][250/603] lr: 1.000e-03, eta: 0:20:33, time: 0.400, data_time: 0.394, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 11:56:42,161 - mmaction - INFO - Epoch [14][300/603] lr: 1.000e-03, eta: 0:20:17, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 11:56:57,164 - mmaction - INFO - Epoch [14][350/603] lr: 1.000e-03, eta: 0:20:01, time: 0.300, data_time: 0.294, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:57:13,240 - mmaction - INFO - Epoch [14][400/603] lr: 1.000e-03, eta: 0:19:46, time: 0.322, data_time: 0.316, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 11:57:30,505 - mmaction - INFO - Epoch [14][450/603] lr: 1.000e-03, eta: 0:19:31, time: 0.345, data_time: 0.340, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:57:46,593 - mmaction - INFO - Epoch [14][500/603] lr: 1.000e-03, eta: 0:19:16, time: 0.322, data_time: 0.316, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 11:58:02,610 - mmaction - INFO - Epoch [14][550/603] lr: 1.000e-03, eta: 0:19:01, time: 0.320, data_time: 0.314, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 11:58:17,751 - mmaction - INFO - Epoch [14][600/603] lr: 1.000e-03, eta: 0:18:45, time: 0.303, data_time: 0.297, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:58:18,439 - mmaction - INFO - Saving checkpoint at 14 epochs 2020-08-04 11:58:34,943 - mmaction - INFO - Epoch [15][50/603] lr: 1.000e-03, eta: 0:18:28, time: 0.295, data_time: 0.288, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 11:58:49,336 - mmaction - INFO - Epoch [15][100/603] lr: 1.000e-03, eta: 0:18:12, time: 0.288, data_time: 0.282, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 11:59:05,148 - mmaction - INFO - Epoch [15][150/603] lr: 1.000e-03, eta: 0:17:56, time: 0.316, data_time: 0.310, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 11:59:20,708 - mmaction - INFO - Epoch [15][200/603] lr: 1.000e-03, eta: 0:17:41, time: 0.311, data_time: 0.305, memory: 25, temporal_iou_loss: 0.0249, loss: 0.0249 2020-08-04 11:59:38,460 - mmaction - INFO - Epoch [15][250/603] lr: 1.000e-03, eta: 0:17:26, time: 0.355, data_time: 0.349, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 11:59:54,530 - mmaction - INFO - Epoch [15][300/603] lr: 1.000e-03, eta: 0:17:11, time: 0.321, data_time: 0.315, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 12:00:09,987 - mmaction - INFO - Epoch [15][350/603] lr: 1.000e-03, eta: 0:16:55, time: 0.309, data_time: 0.300, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:00:21,969 - mmaction - INFO - Epoch [15][400/603] lr: 1.000e-03, eta: 0:16:39, time: 0.240, data_time: 0.234, memory: 25, temporal_iou_loss: 0.0250, loss: 0.0250 2020-08-04 12:00:37,689 - mmaction - INFO - Epoch [15][450/603] lr: 1.000e-03, eta: 0:16:23, time: 0.314, data_time: 0.308, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 12:00:57,272 - mmaction - INFO - Epoch [15][500/603] lr: 1.000e-03, eta: 0:16:09, time: 0.392, data_time: 0.386, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:01:14,918 - mmaction - INFO - Epoch [15][550/603] lr: 1.000e-03, eta: 0:15:54, time: 0.353, data_time: 0.347, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 12:01:28,723 - mmaction - INFO - Epoch [15][600/603] lr: 1.000e-03, eta: 0:15:38, time: 0.276, data_time: 0.270, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:01:28,874 - mmaction - INFO - Saving checkpoint at 15 epochs 2020-08-04 12:01:44,895 - mmaction - INFO - Epoch [16][50/603] lr: 1.000e-03, eta: 0:15:21, time: 0.319, data_time: 0.312, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:02:00,839 - mmaction - INFO - Epoch [16][100/603] lr: 1.000e-03, eta: 0:15:06, time: 0.319, data_time: 0.313, memory: 25, temporal_iou_loss: 0.0244, loss: 0.0244 2020-08-04 12:02:16,807 - mmaction - INFO - Epoch [16][150/603] lr: 1.000e-03, eta: 0:14:51, time: 0.319, data_time: 0.313, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:02:32,497 - mmaction - INFO - Epoch [16][200/603] lr: 1.000e-03, eta: 0:14:35, time: 0.314, data_time: 0.308, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:02:48,840 - mmaction - INFO - Epoch [16][250/603] lr: 1.000e-03, eta: 0:14:20, time: 0.327, data_time: 0.320, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 12:03:06,182 - mmaction - INFO - Epoch [16][300/603] lr: 1.000e-03, eta: 0:14:05, time: 0.347, data_time: 0.341, memory: 25, temporal_iou_loss: 0.0252, loss: 0.0252 2020-08-04 12:03:21,701 - mmaction - INFO - Epoch [16][350/603] lr: 1.000e-03, eta: 0:13:49, time: 0.310, data_time: 0.304, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:03:37,579 - mmaction - INFO - Epoch [16][400/603] lr: 1.000e-03, eta: 0:13:34, time: 0.318, data_time: 0.312, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:03:54,712 - mmaction - INFO - Epoch [16][450/603] lr: 1.000e-03, eta: 0:13:18, time: 0.343, data_time: 0.337, memory: 25, temporal_iou_loss: 0.0249, loss: 0.0249 2020-08-04 12:04:11,917 - mmaction - INFO - Epoch [16][500/603] lr: 1.000e-03, eta: 0:13:03, time: 0.344, data_time: 0.338, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 12:04:25,726 - mmaction - INFO - Epoch [16][550/603] lr: 1.000e-03, eta: 0:12:47, time: 0.276, data_time: 0.270, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:04:40,190 - mmaction - INFO - Epoch [16][600/603] lr: 1.000e-03, eta: 0:12:31, time: 0.289, data_time: 0.283, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 12:04:40,632 - mmaction - INFO - Saving checkpoint at 16 epochs 2020-08-04 12:04:56,259 - mmaction - INFO - Epoch [17][50/603] lr: 1.000e-03, eta: 0:12:14, time: 0.294, data_time: 0.288, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:05:20,000 - mmaction - INFO - Epoch [17][100/603] lr: 1.000e-03, eta: 0:12:01, time: 0.475, data_time: 0.469, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:05:51,037 - mmaction - INFO - Epoch [17][150/603] lr: 1.000e-03, eta: 0:11:49, time: 0.621, data_time: 0.614, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:06:23,441 - mmaction - INFO - Epoch [17][200/603] lr: 1.000e-03, eta: 0:11:37, time: 0.648, data_time: 0.642, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 12:06:56,183 - mmaction - INFO - Epoch [17][250/603] lr: 1.000e-03, eta: 0:11:25, time: 0.655, data_time: 0.649, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 12:07:25,034 - mmaction - INFO - Epoch [17][300/603] lr: 1.000e-03, eta: 0:11:12, time: 0.577, data_time: 0.570, memory: 25, temporal_iou_loss: 0.0251, loss: 0.0251 2020-08-04 12:07:52,550 - mmaction - INFO - Epoch [17][350/603] lr: 1.000e-03, eta: 0:10:58, time: 0.550, data_time: 0.543, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:08:24,544 - mmaction - INFO - Epoch [17][400/603] lr: 1.000e-03, eta: 0:10:45, time: 0.640, data_time: 0.633, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 12:08:52,093 - mmaction - INFO - Epoch [17][450/603] lr: 1.000e-03, eta: 0:10:32, time: 0.551, data_time: 0.545, memory: 25, temporal_iou_loss: 0.0262, loss: 0.0262 2020-08-04 12:09:17,958 - mmaction - INFO - Epoch [17][500/603] lr: 1.000e-03, eta: 0:10:17, time: 0.517, data_time: 0.511, memory: 25, temporal_iou_loss: 0.0248, loss: 0.0248 2020-08-04 12:09:47,953 - mmaction - INFO - Epoch [17][550/603] lr: 1.000e-03, eta: 0:10:04, time: 0.600, data_time: 0.594, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 12:10:38,934 - mmaction - INFO - Epoch [17][600/603] lr: 1.000e-03, eta: 0:09:54, time: 1.020, data_time: 1.013, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:10:39,246 - mmaction - INFO - Saving checkpoint at 17 epochs 2020-08-04 12:11:34,780 - mmaction - INFO - Epoch [18][50/603] lr: 1.000e-03, eta: 0:09:43, time: 1.104, data_time: 1.098, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 12:12:02,540 - mmaction - INFO - Epoch [18][100/603] lr: 1.000e-03, eta: 0:09:28, time: 0.555, data_time: 0.549, memory: 25, temporal_iou_loss: 0.0249, loss: 0.0249 2020-08-04 12:12:36,456 - mmaction - INFO - Epoch [18][150/603] lr: 1.000e-03, eta: 0:09:14, time: 0.678, data_time: 0.672, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:12:52,724 - mmaction - INFO - Epoch [18][200/603] lr: 1.000e-03, eta: 0:08:57, time: 0.325, data_time: 0.319, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 12:13:09,866 - mmaction - INFO - Epoch [18][250/603] lr: 1.000e-03, eta: 0:08:41, time: 0.343, data_time: 0.337, memory: 25, temporal_iou_loss: 0.0252, loss: 0.0252 2020-08-04 12:13:27,005 - mmaction - INFO - Epoch [18][300/603] lr: 1.000e-03, eta: 0:08:24, time: 0.343, data_time: 0.337, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 12:13:42,193 - mmaction - INFO - Epoch [18][350/603] lr: 1.000e-03, eta: 0:08:07, time: 0.304, data_time: 0.298, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:13:57,883 - mmaction - INFO - Epoch [18][400/603] lr: 1.000e-03, eta: 0:07:50, time: 0.314, data_time: 0.308, memory: 25, temporal_iou_loss: 0.0263, loss: 0.0263 2020-08-04 12:14:16,059 - mmaction - INFO - Epoch [18][450/603] lr: 1.000e-03, eta: 0:07:34, time: 0.364, data_time: 0.357, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 12:14:31,054 - mmaction - INFO - Epoch [18][500/603] lr: 1.000e-03, eta: 0:07:17, time: 0.300, data_time: 0.294, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:14:46,800 - mmaction - INFO - Epoch [18][550/603] lr: 1.000e-03, eta: 0:07:00, time: 0.315, data_time: 0.309, memory: 25, temporal_iou_loss: 0.0260, loss: 0.0260 2020-08-04 12:15:02,045 - mmaction - INFO - Epoch [18][600/603] lr: 1.000e-03, eta: 0:06:43, time: 0.305, data_time: 0.299, memory: 25, temporal_iou_loss: 0.0258, loss: 0.0258 2020-08-04 12:15:03,618 - mmaction - INFO - Saving checkpoint at 18 epochs 2020-08-04 12:15:20,557 - mmaction - INFO - Epoch [19][50/603] lr: 1.000e-03, eta: 0:06:25, time: 0.296, data_time: 0.290, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 12:15:31,022 - mmaction - INFO - Epoch [19][100/603] lr: 1.000e-03, eta: 0:06:08, time: 0.209, data_time: 0.203, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 12:15:42,020 - mmaction - INFO - Epoch [19][150/603] lr: 1.000e-03, eta: 0:05:51, time: 0.220, data_time: 0.214, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:15:53,189 - mmaction - INFO - Epoch [19][200/603] lr: 1.000e-03, eta: 0:05:34, time: 0.223, data_time: 0.217, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 12:16:05,393 - mmaction - INFO - Epoch [19][250/603] lr: 1.000e-03, eta: 0:05:17, time: 0.244, data_time: 0.238, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:16:15,859 - mmaction - INFO - Epoch [19][300/603] lr: 1.000e-03, eta: 0:04:59, time: 0.209, data_time: 0.203, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 12:16:27,001 - mmaction - INFO - Epoch [19][350/603] lr: 1.000e-03, eta: 0:04:43, time: 0.223, data_time: 0.217, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:16:37,779 - mmaction - INFO - Epoch [19][400/603] lr: 1.000e-03, eta: 0:04:26, time: 0.216, data_time: 0.210, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:16:49,088 - mmaction - INFO - Epoch [19][450/603] lr: 1.000e-03, eta: 0:04:09, time: 0.226, data_time: 0.220, memory: 25, temporal_iou_loss: 0.0248, loss: 0.0248 2020-08-04 12:17:01,027 - mmaction - INFO - Epoch [19][500/603] lr: 1.000e-03, eta: 0:03:52, time: 0.239, data_time: 0.233, memory: 25, temporal_iou_loss: 0.0261, loss: 0.0261 2020-08-04 12:17:11,130 - mmaction - INFO - Epoch [19][550/603] lr: 1.000e-03, eta: 0:03:35, time: 0.202, data_time: 0.196, memory: 25, temporal_iou_loss: 0.0255, loss: 0.0255 2020-08-04 12:17:22,876 - mmaction - INFO - Epoch [19][600/603] lr: 1.000e-03, eta: 0:03:18, time: 0.235, data_time: 0.228, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 12:17:23,027 - mmaction - INFO - Saving checkpoint at 19 epochs 2020-08-04 12:17:37,762 - mmaction - INFO - Epoch [20][50/603] lr: 1.000e-03, eta: 0:03:01, time: 0.291, data_time: 0.285, memory: 25, temporal_iou_loss: 0.0253, loss: 0.0253 2020-08-04 12:17:49,214 - mmaction - INFO - Epoch [20][100/603] lr: 1.000e-03, eta: 0:02:44, time: 0.229, data_time: 0.223, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:18:00,123 - mmaction - INFO - Epoch [20][150/603] lr: 1.000e-03, eta: 0:02:28, time: 0.218, data_time: 0.212, memory: 25, temporal_iou_loss: 0.0259, loss: 0.0259 2020-08-04 12:18:11,398 - mmaction - INFO - Epoch [20][200/603] lr: 1.000e-03, eta: 0:02:11, time: 0.226, data_time: 0.220, memory: 25, temporal_iou_loss: 0.0247, loss: 0.0247 2020-08-04 12:18:22,156 - mmaction - INFO - Epoch [20][250/603] lr: 1.000e-03, eta: 0:01:55, time: 0.215, data_time: 0.209, memory: 25, temporal_iou_loss: 0.0264, loss: 0.0264 2020-08-04 12:18:34,442 - mmaction - INFO - Epoch [20][300/603] lr: 1.000e-03, eta: 0:01:38, time: 0.246, data_time: 0.240, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 12:18:44,820 - mmaction - INFO - Epoch [20][350/603] lr: 1.000e-03, eta: 0:01:22, time: 0.208, data_time: 0.202, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:18:55,178 - mmaction - INFO - Epoch [20][400/603] lr: 1.000e-03, eta: 0:01:05, time: 0.207, data_time: 0.201, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 12:19:06,857 - mmaction - INFO - Epoch [20][450/603] lr: 1.000e-03, eta: 0:00:49, time: 0.234, data_time: 0.227, memory: 25, temporal_iou_loss: 0.0254, loss: 0.0254 2020-08-04 12:19:18,343 - mmaction - INFO - Epoch [20][500/603] lr: 1.000e-03, eta: 0:00:33, time: 0.230, data_time: 0.224, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:19:29,281 - mmaction - INFO - Epoch [20][550/603] lr: 1.000e-03, eta: 0:00:17, time: 0.219, data_time: 0.212, memory: 25, temporal_iou_loss: 0.0257, loss: 0.0257 2020-08-04 12:19:39,945 - mmaction - INFO - Epoch [20][600/603] lr: 1.000e-03, eta: 0:00:00, time: 0.213, data_time: 0.207, memory: 25, temporal_iou_loss: 0.0256, loss: 0.0256 2020-08-04 12:19:40,221 - mmaction - INFO - Saving checkpoint at 20 epochs