Nameexact_genome_1512568488_44_6909_0
Workunit2605877
Created7 Dec 2017, 22:45:13 UTC
Sent7 Dec 2017, 22:51:49 UTC
Report deadline17 Dec 2017, 14:41:00 UTC
Received8 Dec 2017, 14:32:13 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64142
Run time4 hours 32 min 48 sec
CPU time3 hours 53 min 49 sec
Validate stateValid
Credit2,414.79
Device peak FLOPS3.08 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size156.82 MB
Peak swap size159.77 MB
Peak disk usage1.89 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
000 (97.71%), error:       997.03961, best error:       832.99609 on epoch:    11, epoch:   14/100, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 131.9635314941s, input_fired_time: 1.9376714230, output_fired_time: 1.6402813196, propagate_forward_time: 48.3600120544, propagate_backward_time: 76.9514770508, weight_update_time: 0.1281032711, other_time: 3.0740966797
epoch time: 10.2152633667s, input_fired_time: 0.2318590730, output_fired_time: 0.0000000000, propagate_forward_time: 9.4292554855, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5541486740
validation[          , genome  6909] predictions:    9729/  10000 (97.29%), best:    9771/10000 (97.71%), error:       856.80096, best error:       832.99609 on epoch:    11, epoch:   15/100, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 131.3224182129s, input_fired_time: 1.9251194000, output_fired_time: 1.6274092197, propagate_forward_time: 47.5754508972, propagate_backward_time: 77.1281585693, weight_update_time: 0.1277059317, other_time: 3.0662689209
epoch time: 10.1826457977s, input_fired_time: 0.2340965271, output_fired_time: 0.0000000000, propagate_forward_time: 9.4001617432, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5483875275
validation[          , genome  6909] predictions:    9698/  10000 (96.98%), best:    9771/10000 (97.71%), error:       977.68048, best error:       832.99609 on epoch:    11, epoch:   16/100, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 128.0767059326s, input_fired_time: 1.9111151695, output_fired_time: 1.6089701653, propagate_forward_time: 46.6949882507, propagate_backward_time: 74.8386306763, weight_update_time: 0.1251658946, other_time: 3.0230026245
epoch time: 9.8715057373s, input_fired_time: 0.2270062715, output_fired_time: 0.0000000000, propagate_forward_time: 9.1000623703, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5444374084
validation[          , genome  6909] predictions:    9752/  10000 (97.52%), best:    9752/10000 (97.52%), error:       796.98737, best error:       796.98737 on epoch:    17, epoch:   17/100, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 123.1324539185s, input_fired_time: 1.8643867970, output_fired_time: 1.5533795357, propagate_forward_time: 44.4086952209, propagate_backward_time: 72.3717269897, weight_update_time: 0.1207468063, other_time: 2.9342575073
epoch time: 9.3051118851s, input_fired_time: 0.2196508050, output_fired_time: 0.0000000000, propagate_forward_time: 8.5618343353, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5236263275
validation[          , genome  6909] predictions:    9762/  10000 (97.62%), best:    9762/10000 (97.62%), error:       795.64697, best error:       795.64697 on epoch:    18, epoch:   18/100, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 122.9012222290s, input_fired_time: 1.8662449121, output_fired_time: 1.5484607220, propagate_forward_time: 44.5718688965, propagate_backward_time: 71.9706344604, weight_update_time: 0.1211436018, other_time: 2.9440155029
epoch time: 9.7759227753s, input_fired_time: 0.2248829752, output_fired_time: 0.0000000000, propagate_forward_time: 9.0114326477, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5396070480
validation[          , genome  6909] predictions:    9756/  10000 (97.56%), best:    9762/10000 (97.62%), error:       859.82916, best error:       795.64697 on epoch:    18, epoch:   19/100, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 125.6525115967s, input_fired_time: 1.8896952868, output_fired_time: 1.5862413645, propagate_forward_time: 45.7399444580, propagate_backward_time: 73.4601364136, weight_update_time: 0.1227912307, other_time: 2.9764938354
epoch time: 9.8948316574s, input_fired_time: 0.2264707237, output_fired_time: 0.0000000000, propagate_forward_time: 9.1239557266, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5444049835
validation[          , genome  6909] predictions:    9670/  10000 (96.70%), best:    9762/10000 (97.62%), error:      1050.08228, best error:       795.64697 on epoch:    18, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 124.5536117554s, input_fired_time: 1.8800014257, output_fired_time: 1.5713853836, propagate_forward_time: 45.0992126465, propagate_backward_time: 73.0304183960, weight_update_time: 0.1227554157, other_time: 2.9725875854
epoch time: 8.9164552689s, input_fired_time: 0.2113076746, output_fired_time: 0.0000000000, propagate_forward_time: 8.1925964355, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5125513077
validation[          , genome  6909] predictions:    9718/  10000 (97.18%), best:    9762/10000 (97.62%), error:       935.18677, best error:       795.64697 on epoch:    18, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 127.0843353271s, input_fired_time: 1.8464146852, output_fired_time: 1.5373119116, propagate_forward_time: 45.8028144836, propagate_backward_time: 74.9913558960, weight_update_time: 0.1196234822, other_time: 2.9064407349
epoch time: 16.0064201355s, input_fired_time: 0.2625069320, output_fired_time: 0.0000000000, propagate_forward_time: 15.1244411469, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6194725037
validation[          , genome  6909] predictions:    9680/  10000 (96.80%), best:    9762/10000 (97.62%), error:       948.37085, best error:       795.64697 on epoch:    18, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 188.3959503174s, input_fired_time: 2.7707204819, output_fired_time: 2.5058534145, propagate_forward_time: 68.6804199219, propagate_backward_time: 110.7050704956, weight_update_time: 0.1355590373, other_time: 3.7338790894
epoch time: 14.1012735367s, input_fired_time: 0.2404479682, output_fired_time: 0.0000000000, propagate_forward_time: 13.0398244858, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8210010529
validation[          , genome  6909] predictions:    9758/  10000 (97.58%), best:    9762/10000 (97.62%), error:       803.74115, best error:       795.64697 on epoch:    18, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 167.5748748779s, input_fired_time: 2.6418266296, output_fired_time: 1.8314540386, propagate_forward_time: 60.8659706116, propagate_backward_time: 97.7463607788, weight_update_time: 0.1233841702, other_time: 4.4892654419
epoch time: 11.6835746765s, input_fired_time: 0.2080758512, output_fired_time: 0.0000000000, propagate_forward_time: 10.9708709717, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5046281815
validation[          , genome  6909] predictions:    9750/  10000 (97.50%), best:    9762/10000 (97.62%), error:       799.31195, best error:       795.64697 on epoch:    18, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 175.8409881592s, input_fired_time: 2.6944360733, output_fired_time: 2.6696164608, propagate_forward_time: 64.6558685303, propagate_backward_time: 102.2331848145, weight_update_time: 0.1282103062, other_time: 3.5878753662
epoch time: 12.6032152176s, input_fired_time: 0.4691794813, output_fired_time: 0.0000000000, propagate_forward_time: 11.6065073013, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5275287628
validation[          , genome  6909] predictions:    9728/  10000 (97.28%), best:    9762/10000 (97.62%), error:       936.19568, best error:       795.64697 on epoch:    18, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 172.4110412598s, input_fired_time: 2.4215345383, output_fired_time: 2.1188166142, propagate_forward_time: 64.2888870239, propagate_backward_time: 99.5292587280, weight_update_time: 0.1255197227, other_time: 4.0525360107
epoch time: 13.2407808304s, input_fired_time: 0.2320234478, output_fired_time: 0.0000000000, propagate_forward_time: 12.2096738815, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7990837097
validation[          , genome  6909] predictions:    9744/  10000 (97.44%), best:    9762/10000 (97.62%), error:       860.84851, best error:       795.64697 on epoch:    18, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 170.9725646973s, input_fired_time: 2.1686942577, output_fired_time: 2.6099298000, propagate_forward_time: 60.9026184082, propagate_backward_time: 100.7589340210, weight_update_time: 0.1254649013, other_time: 4.5323867798
epoch time: 13.4389219284s, input_fired_time: 0.2289263159, output_fired_time: 0.0000000000, propagate_forward_time: 12.6634788513, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5465164185
validation[          , genome  6909] predictions:    9760/  10000 (97.60%), best:    9762/10000 (97.62%), error:       839.49597, best error:       795.64697 on epoch:    18, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 175.9425811768s, input_fired_time: 3.4477665424, output_fired_time: 1.9025330544, propagate_forward_time: 63.0424537659, propagate_backward_time: 103.4586257935, weight_update_time: 0.3784333766, other_time: 4.0912094116
epoch time: 11.3219537735s, input_fired_time: 0.2070083022, output_fired_time: 0.0000000000, propagate_forward_time: 10.1149168015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0000286102
validation[          , genome  6909] predictions:    9735/  10000 (97.35%), best:    9762/10000 (97.62%), error:       891.19904, best error:       795.64697 on epoch:    18, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 175.8317413330s, input_fired_time: 2.6976408958, output_fired_time: 2.1448483467, propagate_forward_time: 62.6490631104, propagate_backward_time: 104.2560195923, weight_update_time: 0.1278687865, other_time: 4.0841598511
epoch time: 14.8882064819s, input_fired_time: 0.2525421679, output_fired_time: 0.0000000000, propagate_forward_time: 14.0472326279, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5884313583
validation[          , genome  6909] predictions:    9763/  10000 (97.63%), best:    9762/10000 (97.62%), error:       808.48828, best error:       795.64697 on epoch:    18, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 149.4336395264s, input_fired_time: 3.3933277130, output_fired_time: 2.3377518654, propagate_forward_time: 53.6684722900, propagate_backward_time: 86.0531234741, weight_update_time: 0.1231869981, other_time: 3.9809646606
epoch time: 9.3689661026s, input_fired_time: 0.2209016085, output_fired_time: 0.0000000000, propagate_forward_time: 8.6271505356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5209140778
validation[          , genome  6909] predictions:    9719/  10000 (97.19%), best:    9762/10000 (97.62%), error:       909.56567, best error:       795.64697 on epoch:    18, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 118.3344116211s, input_fired_time: 1.8369388580, output_fired_time: 1.4880272150, propagate_forward_time: 42.6570014954, propagate_backward_time: 69.4836120605, weight_update_time: 0.1178196296, other_time: 2.8688354492
epoch time: 9.8123550415s, input_fired_time: 0.2280385196, output_fired_time: 0.0000000000, propagate_forward_time: 9.0458869934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5384292603
validation[          , genome  6909] predictions:    9770/  10000 (97.70%), best:    9770/10000 (97.70%), error:       745.93927, best error:       745.93927 on epoch:    31, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 119.7333145142s, input_fired_time: 1.8512732983, output_fired_time: 1.5118149519, propagate_forward_time: 43.3964920044, propagate_backward_time: 70.0954818726, weight_update_time: 0.1186721325, other_time: 2.8782501221
epoch time: 9.2315006256s, input_fired_time: 0.2169596106, output_fired_time: 0.0000000000, propagate_forward_time: 8.4981136322, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5164270401
validation[          , genome  6909] predictions:    9728/  10000 (97.28%), best:    9770/10000 (97.70%), error:       910.73456, best error:       745.93927 on epoch:    31, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 116.9250488281s, input_fired_time: 1.8230323792, output_fired_time: 1.4802879095, propagate_forward_time: 41.9231262207, propagate_backward_time: 68.8746032715, weight_update_time: 0.1156825647, other_time: 2.8240051270
epoch time: 8.5250358582s, input_fired_time: 0.2064579278, output_fired_time: 0.0000000000, propagate_forward_time: 7.8224706650, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4961071014
validation[          , genome  6909] predictions:    9761/  10000 (97.61%), best:    9770/10000 (97.70%), error:       780.90057, best error:       745.93927 on epoch:    31, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 115.6396484375s, input_fired_time: 1.8150377274, output_fired_time: 1.4683187008, propagate_forward_time: 41.4499549866, propagate_backward_time: 68.0976715088, weight_update_time: 0.1150072441, other_time: 2.8086700439
epoch time: 10.7053632736s, input_fired_time: 0.2417275310, output_fired_time: 0.0000000000, propagate_forward_time: 9.8963460922, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5672893524
validation[          , genome  6909] predictions:    9743/  10000 (97.43%), best:    9770/10000 (97.70%), error:       843.50201, best error:       745.93927 on epoch:    31, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 119.6316070557s, input_fired_time: 1.8489364386, output_fired_time: 1.5117503405, propagate_forward_time: 43.2130393982, propagate_backward_time: 70.1830215454, weight_update_time: 0.1184309945, other_time: 2.8748550415
epoch time: 10.0476093292s, input_fired_time: 0.2314775884, output_fired_time: 0.0000000000, propagate_forward_time: 9.2717838287, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5443477631
validation[          , genome  6909] predictions:    9680/  10000 (96.80%), best:    9770/10000 (97.70%), error:      1089.43738, best error:       745.93927 on epoch:    31, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 117.7198944092s, input_fired_time: 1.8392028809, output_fired_time: 1.4959744215, propagate_forward_time: 42.3161430359, propagate_backward_time: 69.2254104614, weight_update_time: 0.1164203286, other_time: 2.8431625366
epoch time: 9.9379472733s, input_fired_time: 0.2305237949, output_fired_time: 0.0000000000, propagate_forward_time: 9.1662855148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5411376953
validation[          , genome  6909] predictions:    9763/  10000 (97.63%), best:    9770/10000 (97.70%), error:       760.28217, best error:       745.93927 on epoch:    31, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 117.4426574707s, input_fired_time: 1.8334903717, output_fired_time: 1.4856412411, propagate_forward_time: 42.3796882629, propagate_backward_time: 68.8988189697, weight_update_time: 0.1165043041, other_time: 2.8450164795
epoch time: 9.0434474945s, input_fired_time: 0.2188591957, output_fired_time: 0.0000000000, propagate_forward_time: 8.3130903244, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5114984512
validation[          , genome  6909] predictions:    9761/  10000 (97.61%), best:    9770/10000 (97.70%), error:       781.91565, best error:       745.93927 on epoch:    31, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 117.9106597900s, input_fired_time: 1.8364584446, output_fired_time: 1.4873512983, propagate_forward_time: 42.5000839233, propagate_backward_time: 69.2472991943, weight_update_time: 0.1164320484, other_time: 2.8394699097
epoch time: 8.8050680161s, input_fired_time: 0.2128769308, output_fired_time: 0.0000000000, propagate_forward_time: 8.0890016556, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5031890869
validation[          , genome  6909] predictions:    9777/  10000 (97.77%), best:    9770/10000 (97.70%), error:       773.06165, best error:       745.93927 on epoch:    31, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 115.3677444458s, input_fired_time: 1.8194557428, output_fired_time: 1.4527209997, propagate_forward_time: 41.5005378723, propagate_backward_time: 67.7726593018, weight_update_time: 0.1157803312, other_time: 2.8223724365
epoch time: 9.6668596268s, input_fired_time: 0.2268439978, output_fired_time: 0.0000000000, propagate_forward_time: 8.9106769562, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5293388367
validation[          , genome  6909] predictions:    9774/  10000 (97.74%), best:    9774/10000 (97.74%), error:       734.23248, best error:       734.23248 on epoch:    39, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 117.9104309082s, input_fired_time: 1.8407753706, output_fired_time: 1.4977879524, propagate_forward_time: 42.6347999573, propagate_backward_time: 69.1015396118, weight_update_time: 0.1164531037, other_time: 2.8355331421
epoch time: 9.5736751556s, input_fired_time: 0.2220986784, output_fired_time: 0.0000000000, propagate_forward_time: 8.8216266632, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5299501419
validation[          , genome  6909] predictions:    9774/  10000 (97.74%), best:    9774/10000 (97.74%), error:       759.85638, best error:       734.23248 on epoch:    39, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 118.4132766724s, input_fired_time: 1.8423262835, output_fired_time: 1.4840787649, propagate_forward_time: 42.8530235291, propagate_backward_time: 69.3692245483, weight_update_time: 0.1177018881, other_time: 2.8646316528
epoch time: 9.5714769363s, input_fired_time: 0.2253909260, output_fired_time: 0.0000000000, propagate_forward_time: 8.8177394867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5283460617
validation[          , genome  6909] predictions:    9762/  10000 (97.62%), best:    9774/10000 (97.74%), error:       753.01483, best error:       734.23248 on epoch:    39, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 120.3687667847s, input_fired_time: 1.8577557802, output_fired_time: 1.5199606419, propagate_forward_time: 43.5876007080, propagate_backward_time: 70.5084762573, weight_update_time: 0.1194043308, other_time: 2.8949737549
epoch time: 9.8817949295s, input_fired_time: 0.2311294377, output_fired_time: 0.0000000000, propagate_forward_time: 9.1105098724, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5401554108
validation[          , genome  6909] predictions:    9793/  10000 (97.93%), best:    9793/10000 (97.93%), error:       699.09589, best error:       699.09589 on epoch:    42, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 120.4763259888s, input_fired_time: 1.8620089293, output_fired_time: 1.5282045603, propagate_forward_time: 43.7891082764, propagate_backward_time: 70.4204711914, weight_update_time: 0.1183700636, other_time: 2.8765335083
epoch time: 8.6227960587s, input_fired_time: 0.2103199512, output_fired_time: 0.0000000000, propagate_forward_time: 7.9181594849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4943170547
validation[          , genome  6909] predictions:    9752/  10000 (97.52%), best:    9793/10000 (97.93%), error:       798.38104, best error:       699.09589 on epoch:    42, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 119.0286483765s, input_fired_time: 1.8405680656, output_fired_time: 1.4975314140, propagate_forward_time: 42.8478164673, propagate_backward_time: 69.9703826904, weight_update_time: 0.1184868217, other_time: 2.8723526001
epoch time: 8.2033596039s, input_fired_time: 0.2043735087, output_fired_time: 0.0000000000, propagate_forward_time: 7.5190563202, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4799299240
validation[          , genome  6909] predictions:    9727/  10000 (97.27%), best:    9793/10000 (97.93%), error:       890.22784, best error:       699.09589 on epoch:    42, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 114.5274963379s, input_fired_time: 1.8066726923, output_fired_time: 1.4480102062, propagate_forward_time: 41.2014503479, propagate_backward_time: 67.2663497925, weight_update_time: 0.1146390960, other_time: 2.8050155640
epoch time: 9.2735424042s, input_fired_time: 0.2259090841, output_fired_time: 0.0000000000, propagate_forward_time: 8.5313673019, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5162658691
validation[          , genome  6909] predictions:    9748/  10000 (97.48%), best:    9793/10000 (97.93%), error:       821.33331, best error:       699.09589 on epoch:    42, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 115.2328567505s, input_fired_time: 1.8128052950, output_fired_time: 1.4592216015, propagate_forward_time: 41.5133705139, propagate_backward_time: 67.6370239258, weight_update_time: 0.1149329990, other_time: 2.8104400635
epoch time: 10.4339113235s, input_fired_time: 0.2481045872, output_fired_time: 0.0000000000, propagate_forward_time: 9.6262245178, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5595817566
validation[          , genome  6909] predictions:    9731/  10000 (97.31%), best:    9793/10000 (97.93%), error:       923.08746, best error:       699.09589 on epoch:    42, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 112.6376800537s, input_fired_time: 1.8037798405, output_fired_time: 1.4256416559, propagate_forward_time: 40.5315399170, propagate_backward_time: 66.0960693359, weight_update_time: 0.1135689765, other_time: 2.7806472778
epoch time: 8.3133392334s, input_fired_time: 0.2071721107, output_fired_time: 0.0000000000, propagate_forward_time: 7.6198582649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4863085747
validation[          , genome  6909] predictions:    9734/  10000 (97.34%), best:    9793/10000 (97.93%), error:       856.43677, best error:       699.09589 on epoch:    42, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 116.2524948120s, input_fired_time: 1.8272912502, output_fired_time: 1.4735553265, propagate_forward_time: 41.8365516663, propagate_backward_time: 68.2792129517, weight_update_time: 0.1157926619, other_time: 2.8358840942
epoch time: 8.9463062286s, input_fired_time: 0.2243069857, output_fired_time: 0.0000000000, propagate_forward_time: 8.2157697678, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5062294006
validation[          , genome  6909] predictions:    9785/  10000 (97.85%), best:    9793/10000 (97.93%), error:       724.11377, best error:       699.09589 on epoch:    42, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 112.2165374756s, input_fired_time: 1.7958285809, output_fired_time: 1.4278318882, propagate_forward_time: 40.2165145874, propagate_backward_time: 66.0151443481, weight_update_time: 0.1125357747, other_time: 2.7612152100
epoch time: 7.8470616341s, input_fired_time: 0.1979196072, output_fired_time: 0.0000000000, propagate_forward_time: 7.1787743568, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4703679085
validation[          , genome  6909] predictions:    9755/  10000 (97.55%), best:    9793/10000 (97.93%), error:       811.82947, best error:       699.09589 on epoch:    42, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 113.0306549072s, input_fired_time: 1.8029218912, output_fired_time: 1.4348583221, propagate_forward_time: 40.4954643250, propagate_backward_time: 66.5216827393, weight_update_time: 0.1130052805, other_time: 2.7757263184
epoch time: 8.5218620300s, input_fired_time: 0.2117914706, output_fired_time: 0.0000000000, propagate_forward_time: 7.8141536713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4959173203
validation[          , genome  6909] predictions:    9663/  10000 (96.63%), best:    9793/10000 (97.93%), error:      1079.01306, best error:       699.09589 on epoch:    42, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 115.8356781006s, input_fired_time: 1.8309056759, output_fired_time: 1.4732785225, propagate_forward_time: 41.7041664124, propagate_backward_time: 68.0179977417, weight_update_time: 0.1150517464, other_time: 2.8093338013
epoch time: 9.6284828186s, input_fired_time: 0.2283600569, output_fired_time: 0.0000000000, propagate_forward_time: 8.8694419861, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5306806564
validation[          , genome  6909] predictions:    9768/  10000 (97.68%), best:    9793/10000 (97.93%), error:       826.40839, best error:       699.09589 on epoch:    42, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 108.8488082886s, input_fired_time: 1.7685639858, output_fired_time: 1.3860545158, propagate_forward_time: 38.7951583862, propagate_backward_time: 64.1819076538, weight_update_time: 0.1102276817, other_time: 2.7171249390
epoch time: 9.4090318680s, input_fired_time: 0.2246951014, output_fired_time: 0.0000000000, propagate_forward_time: 8.6592693329, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5250673294
validation[          , genome  6909] predictions:    9750/  10000 (97.50%), best:    9793/10000 (97.93%), error:       847.61896, best error:       699.09589 on epoch:    42, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 118.3794326782s, input_fired_time: 1.8488372564, output_fired_time: 1.4960515499, propagate_forward_time: 42.5173683167, propagate_backward_time: 69.6505279541, weight_update_time: 0.1177233830, other_time: 2.8666534424
epoch time: 8.6952953339s, input_fired_time: 0.2209138870, output_fired_time: 0.0000000000, propagate_forward_time: 7.9722027779, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5021786690
validation[          , genome  6909] predictions:    9709/  10000 (97.09%), best:    9793/10000 (97.93%), error:       962.39337, best error:       699.09589 on epoch:    42, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 124.4288558960s, input_fired_time: 1.9027410746, output_fired_time: 1.5577930212, propagate_forward_time: 45.2691688538, propagate_backward_time: 72.7487716675, weight_update_time: 0.1222182810, other_time: 2.9503860474
epoch time: 9.9526863098s, input_fired_time: 0.2285478562, output_fired_time: 0.0000000000, propagate_forward_time: 9.1796598434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5444784164
validation[          , genome  6909] predictions:    9649/  10000 (96.49%), best:    9793/10000 (97.93%), error:      1165.44788, best error:       699.09589 on epoch:    42, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 121.4263763428s, input_fired_time: 1.8707240820, output_fired_time: 1.5347826481, propagate_forward_time: 43.9243087769, propagate_backward_time: 71.1929168701, weight_update_time: 0.1196434274, other_time: 2.9036407471
epoch time: 9.4681634903s, input_fired_time: 0.2285950780, output_fired_time: 0.0000000000, propagate_forward_time: 8.7089920044, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5305767059
validation[          , genome  6909] predictions:    9758/  10000 (97.58%), best:    9793/10000 (97.93%), error:       795.31580, best error:       699.09589 on epoch:    42, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 129.4295196533s, input_fired_time: 2.6344068050, output_fired_time: 1.5406450033, propagate_forward_time: 46.8367424011, propagate_backward_time: 75.4869079590, weight_update_time: 0.1207841709, other_time: 2.9308166504
epoch time: 14.2501888275s, input_fired_time: 0.2450561225, output_fired_time: 0.0000000000, propagate_forward_time: 12.9324254990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0727071762
validation[          , genome  6909] predictions:    9743/  10000 (97.43%), best:    9793/10000 (97.93%), error:       815.03632, best error:       699.09589 on epoch:    42, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 173.5524902344s, input_fired_time: 2.4531869888, output_fired_time: 1.8821396828, propagate_forward_time: 61.5512313843, propagate_backward_time: 103.5972824097, weight_update_time: 0.1272277981, other_time: 4.0686492920
epoch time: 13.1820001602s, input_fired_time: 0.2309226841, output_fired_time: 0.0000000000, propagate_forward_time: 12.1532020569, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7978754044
validation[          , genome  6909] predictions:    9743/  10000 (97.43%), best:    9793/10000 (97.93%), error:       869.75769, best error:       699.09589 on epoch:    42, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 186.7963867188s, input_fired_time: 2.2807168961, output_fired_time: 2.9891250134, propagate_forward_time: 69.5445251465, propagate_backward_time: 108.0125198364, weight_update_time: 0.1349420846, other_time: 3.9695053101
epoch time: 12.0648727417s, input_fired_time: 0.4744662941, output_fired_time: 0.0000000000, propagate_forward_time: 11.0715703964, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5188360214
validation[          , genome  6909] predictions:    9678/  10000 (96.78%), best:    9793/10000 (97.93%), error:      1072.03601, best error:       699.09589 on epoch:    42, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 187.1855316162s, input_fired_time: 2.2810063362, output_fired_time: 2.4973862171, propagate_forward_time: 71.1645202637, propagate_backward_time: 107.0156097412, weight_update_time: 0.1354055256, other_time: 4.2270050049
epoch time: 13.3874721527s, input_fired_time: 0.2351011336, output_fired_time: 0.0000000000, propagate_forward_time: 12.3478507996, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8045206070
validation[          , genome  6909] predictions:    9778/  10000 (97.78%), best:    9793/10000 (97.93%), error:       745.34436, best error:       699.09589 on epoch:    42, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 180.7029266357s, input_fired_time: 2.2424523830, output_fired_time: 1.6907413006, propagate_forward_time: 66.1377334595, propagate_backward_time: 105.2385482788, weight_update_time: 0.6310588717, other_time: 5.3934631348
epoch time: 16.4641017914s, input_fired_time: 0.5272219181, output_fired_time: 0.0000000000, propagate_forward_time: 15.3034515381, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6334285736
validation[          , genome  6909] predictions:    9761/  10000 (97.61%), best:    9793/10000 (97.93%), error:       841.89899, best error:       699.09589 on epoch:    42, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 183.7519531250s, input_fired_time: 3.5273234844, output_fired_time: 1.7186337709, propagate_forward_time: 68.6539611816, propagate_backward_time: 106.1758041382, weight_update_time: 0.1326263249, other_time: 3.6762313843
epoch time: 13.2531089783s, input_fired_time: 0.2334319502, output_fired_time: 0.0000000000, propagate_forward_time: 12.4699048996, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5497722626
validation[          , genome  6909] predictions:    9433/  10000 (94.33%), best:    9793/10000 (97.93%), error:      1845.62488, best error:       699.09589 on epoch:    42, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

resetting weights to those on epoch: 42
epoch time: 181.1667938232s, input_fired_time: 2.9926502705, output_fired_time: 1.9535983801, propagate_forward_time: 66.9559631348, propagate_backward_time: 104.8685379028, weight_update_time: 0.1312444210, other_time: 4.3960494995
epoch time: 14.4748983383s, input_fired_time: 0.2436050773, output_fired_time: 0.0000000000, propagate_forward_time: 13.6578216553, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5734720230
validation[          , genome  6909] predictions:    9811/  10000 (98.11%), best:    9811/10000 (98.11%), error:       647.87897, best error:       647.87897 on epoch:    62, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 179.5584869385s, input_fired_time: 2.7276442051, output_fired_time: 1.6838836670, propagate_forward_time: 66.6867828369, propagate_backward_time: 104.0755844116, weight_update_time: 0.1306684166, other_time: 4.3845901489
epoch time: 12.6206808090s, input_fired_time: 0.2234381586, output_fired_time: 0.0000000000, propagate_forward_time: 11.3687562943, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0284862518
validation[          , genome  6909] predictions:    9797/  10000 (97.97%), best:    9811/10000 (98.11%), error:       692.54510, best error:       647.87897 on epoch:    62, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 140.9810180664s, input_fired_time: 2.8266184330, output_fired_time: 1.7287170887, propagate_forward_time: 50.1828994751, propagate_backward_time: 83.1548614502, weight_update_time: 0.1161218956, other_time: 3.0879287720
epoch time: 8.5623512268s, input_fired_time: 0.2111632228, output_fired_time: 0.0000000000, propagate_forward_time: 7.8560724258, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4951152802
validation[          , genome  6909] predictions:    9796/  10000 (97.96%), best:    9811/10000 (98.11%), error:       683.93884, best error:       647.87897 on epoch:    62, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 107.8888168335s, input_fired_time: 1.7526156902, output_fired_time: 1.3766341209, propagate_forward_time: 38.3213348389, propagate_backward_time: 63.7416229248, weight_update_time: 0.1094514653, other_time: 2.6966094971
epoch time: 7.7102165222s, input_fired_time: 0.1989175230, output_fired_time: 0.0000000000, propagate_forward_time: 7.0460205078, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4652786255
validation[          , genome  6909] predictions:    9782/  10000 (97.82%), best:    9811/10000 (98.11%), error:       691.05566, best error:       647.87897 on epoch:    62, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 102.3670501709s, input_fired_time: 1.7133798599, output_fired_time: 1.3166081905, propagate_forward_time: 36.2670860291, propagate_backward_time: 60.4493026733, weight_update_time: 0.1052309275, other_time: 2.6206779480
epoch time: 7.1133689880s, input_fired_time: 0.1897316277, output_fired_time: 0.0000000000, propagate_forward_time: 6.4787144661, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4449229240
validation[          , genome  6909] predictions:    9750/  10000 (97.50%), best:    9811/10000 (98.11%), error:       815.64929, best error:       647.87897 on epoch:    62, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 99.8506927490s, input_fired_time: 1.6881700754, output_fired_time: 1.2856770754, propagate_forward_time: 35.1205863953, propagate_backward_time: 59.1879539490, weight_update_time: 0.1027932838, other_time: 2.5683059692
epoch time: 7.1162343025s, input_fired_time: 0.1953729391, output_fired_time: 0.0000000000, propagate_forward_time: 6.4753351212, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4455261230
validation[          , genome  6909] predictions:    9222/  10000 (92.22%), best:    9811/10000 (98.11%), error:      2462.99634, best error:       647.87897 on epoch:    62, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

resetting weights to those on epoch: 62
epoch time: 100.6106262207s, input_fired_time: 1.6854227781, output_fired_time: 1.2932943106, propagate_forward_time: 35.4323501587, propagate_backward_time: 59.6171531677, weight_update_time: 0.1031496972, other_time: 2.5824012756
epoch time: 9.3972787857s, input_fired_time: 0.2222850472, output_fired_time: 0.0000000000, propagate_forward_time: 8.6521539688, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5228395462
validation[          , genome  6909] predictions:    9792/  10000 (97.92%), best:    9811/10000 (98.11%), error:       676.72772, best error:       647.87897 on epoch:    62, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 142.8066711426s, input_fired_time: 2.1016943455, output_fired_time: 1.5059078932, propagate_forward_time: 53.0275268555, propagate_backward_time: 82.7751846313, weight_update_time: 0.1190760508, other_time: 3.3963546753
epoch time: 13.5083217621s, input_fired_time: 0.2358701676, output_fired_time: 0.0000000000, propagate_forward_time: 12.4630327225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8094186783
validation[          , genome  6909] predictions:    9763/  10000 (97.63%), best:    9811/10000 (98.11%), error:       836.32141, best error:       647.87897 on epoch:    62, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 173.2717437744s, input_fired_time: 2.6871657372, output_fired_time: 2.3732473850, propagate_forward_time: 63.8461723328, propagate_backward_time: 100.0541534424, weight_update_time: 0.3765923381, other_time: 4.3110046387
epoch time: 13.2934474945s, input_fired_time: 0.2357006520, output_fired_time: 0.0000000000, propagate_forward_time: 12.0090923309, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0486545563
validation[          , genome  6909] predictions:    9766/  10000 (97.66%), best:    9811/10000 (98.11%), error:       764.67480, best error:       647.87897 on epoch:    62, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 158.9820404053s, input_fired_time: 3.3490746021, output_fired_time: 1.7575829029, propagate_forward_time: 60.3465270996, propagate_backward_time: 89.6346893311, weight_update_time: 0.1182307974, other_time: 3.8941650391
epoch time: 13.5085449219s, input_fired_time: 0.2329154015, output_fired_time: 0.0000000000, propagate_forward_time: 12.7252235413, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5504055023
validation[          , genome  6909] predictions:    9760/  10000 (97.60%), best:    9811/10000 (98.11%), error:       767.51306, best error:       647.87897 on epoch:    62, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 158.0068664551s, input_fired_time: 2.8499405384, output_fired_time: 1.5016552210, propagate_forward_time: 58.7606811523, propagate_backward_time: 91.5125350952, weight_update_time: 0.1175744012, other_time: 3.3820571899
epoch time: 11.7909650803s, input_fired_time: 0.2224320024, output_fired_time: 0.0000000000, propagate_forward_time: 11.0633544922, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5051784515
validation[          , genome  6909] predictions:    9783/  10000 (97.83%), best:    9811/10000 (98.11%), error:       760.46826, best error:       647.87897 on epoch:    62, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 175.3604125977s, input_fired_time: 2.7086002827, output_fired_time: 1.6418819427, propagate_forward_time: 63.6664962769, propagate_backward_time: 103.5113601685, weight_update_time: 0.3777294755, other_time: 3.8320770264
epoch time: 13.5225925446s, input_fired_time: 0.2341150343, output_fired_time: 0.0000000000, propagate_forward_time: 12.4877185822, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8007593155
validation[          , genome  6909] predictions:    9752/  10000 (97.52%), best:    9811/10000 (98.11%), error:       796.47980, best error:       647.87897 on epoch:    62, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 173.3966064453s, input_fired_time: 2.6912295818, output_fired_time: 2.1266350746, propagate_forward_time: 62.2531585693, propagate_backward_time: 102.2632751465, weight_update_time: 0.1267060190, other_time: 4.0623168945
epoch time: 12.2380590439s, input_fired_time: 0.2225902379, output_fired_time: 0.0000000000, propagate_forward_time: 11.4946613312, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5208072662
validation[          , genome  6909] predictions:    9788/  10000 (97.88%), best:    9811/10000 (98.11%), error:       689.21356, best error:       647.87897 on epoch:    62, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 175.6513061523s, input_fired_time: 2.4575242996, output_fired_time: 2.1471543312, propagate_forward_time: 63.7778205872, propagate_backward_time: 103.6860504150, weight_update_time: 0.1280628741, other_time: 3.5827636719
epoch time: 13.2696914673s, input_fired_time: 0.2364845872, output_fired_time: 0.0000000000, propagate_forward_time: 12.2332296371, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7999773026
validation[          , genome  6909] predictions:    9711/  10000 (97.11%), best:    9811/10000 (98.11%), error:       951.22998, best error:       647.87897 on epoch:    62, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 175.9082489014s, input_fired_time: 2.2155027390, output_fired_time: 2.4076769352, propagate_forward_time: 62.2845191956, propagate_backward_time: 105.1635360718, weight_update_time: 0.1282394230, other_time: 3.8370132446
epoch time: 14.1400070190s, input_fired_time: 0.4948384166, output_fired_time: 0.0000000000, propagate_forward_time: 13.0712003708, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5739679337
validation[          , genome  6909] predictions:    9784/  10000 (97.84%), best:    9811/10000 (98.11%), error:       734.24469, best error:       647.87897 on epoch:    62, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 178.0468750000s, input_fired_time: 2.2282483578, output_fired_time: 2.4213879108, propagate_forward_time: 65.5170135498, propagate_backward_time: 103.2689666748, weight_update_time: 0.1293428242, other_time: 4.6112670898
epoch time: 15.2096672058s, input_fired_time: 0.2553039193, output_fired_time: 0.0000000000, propagate_forward_time: 13.8538522720, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1005105972
validation[          , genome  6909] predictions:    9791/  10000 (97.91%), best:    9811/10000 (98.11%), error:       685.23962, best error:       647.87897 on epoch:    62, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 186.1159820557s, input_fired_time: 2.2863690853, output_fired_time: 1.9918358326, propagate_forward_time: 68.6885833740, propagate_backward_time: 108.9406051636, weight_update_time: 0.1343394369, other_time: 4.2085952759
epoch time: 14.3201465607s, input_fired_time: 0.2479213178, output_fired_time: 0.0000000000, propagate_forward_time: 13.4961280823, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5760974884
validation[          , genome  6909] predictions:    9803/  10000 (98.03%), best:    9811/10000 (98.11%), error:       670.62854, best error:       647.87897 on epoch:    62, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 188.4190673828s, input_fired_time: 2.5512466431, output_fired_time: 1.7614610195, propagate_forward_time: 70.2375259399, propagate_backward_time: 109.3915634155, weight_update_time: 0.1352759749, other_time: 4.4772796631
epoch time: 12.7234582901s, input_fired_time: 0.2299318910, output_fired_time: 0.0000000000, propagate_forward_time: 11.7006473541, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7928791046
validation[          , genome  6909] predictions:    9791/  10000 (97.91%), best:    9811/10000 (98.11%), error:       707.59497, best error:       647.87897 on epoch:    62, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 185.9223937988s, input_fired_time: 3.2868022919, output_fired_time: 1.7415239811, propagate_forward_time: 66.1361694336, propagate_backward_time: 110.2988357544, weight_update_time: 0.1340973079, other_time: 4.4590530396
epoch time: 14.7110671997s, input_fired_time: 0.2554733753, output_fired_time: 0.0000000000, propagate_forward_time: 13.8602800369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5953140259
validation[          , genome  6909] predictions:    9590/  10000 (95.90%), best:    9811/10000 (98.11%), error:      1278.51465, best error:       647.87897 on epoch:    62, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 196.5875549316s, input_fired_time: 2.6046714783, output_fired_time: 2.3359422684, propagate_forward_time: 70.6331634521, propagate_backward_time: 117.1729812622, weight_update_time: 0.1407974362, other_time: 3.8407974243
epoch time: 15.1118469238s, input_fired_time: 0.5053514838, output_fired_time: 0.0000000000, propagate_forward_time: 14.0127029419, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5937929153
validation[          , genome  6909] predictions:    9799/  10000 (97.99%), best:    9811/10000 (98.11%), error:       700.43433, best error:       647.87897 on epoch:    62, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 196.6293182373s, input_fired_time: 2.6016085148, output_fired_time: 2.0818917751, propagate_forward_time: 75.3846054077, propagate_backward_time: 112.7233505249, weight_update_time: 0.1406136453, other_time: 3.8378601074
epoch time: 14.6955604553s, input_fired_time: 0.2558840215, output_fired_time: 0.0000000000, propagate_forward_time: 13.0974922180, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3421840668
validation[          , genome  6909] predictions:    9723/  10000 (97.23%), best:    9811/10000 (98.11%), error:       897.81018, best error:       647.87897 on epoch:    62, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 195.7059478760s, input_fired_time: 2.8437752724, output_fired_time: 3.0754759312, propagate_forward_time: 72.5268554688, propagate_backward_time: 112.9354934692, weight_update_time: 0.1403028369, other_time: 4.3243484497
epoch time: 14.5048036575s, input_fired_time: 0.2502883673, output_fired_time: 0.0000000000, propagate_forward_time: 13.6737270355, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5807886124
validation[          , genome  6909] predictions:    9766/  10000 (97.66%), best:    9811/10000 (98.11%), error:       754.82050, best error:       647.87897 on epoch:    62, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 193.0804290771s, input_fired_time: 2.8362584114, output_fired_time: 2.0496125221, propagate_forward_time: 73.5281066895, propagate_backward_time: 110.3781356812, weight_update_time: 0.3882900774, other_time: 4.2883224487
epoch time: 13.7102708817s, input_fired_time: 0.2408479005, output_fired_time: 0.0000000000, propagate_forward_time: 12.6527032852, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8167200089
validation[          , genome  6909] predictions:    9648/  10000 (96.48%), best:    9811/10000 (98.11%), error:      1193.87317, best error:       647.87897 on epoch:    62, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 186.2091674805s, input_fired_time: 2.7942783833, output_fired_time: 2.2431755066, propagate_forward_time: 68.6575164795, propagate_backward_time: 107.2776336670, weight_update_time: 0.1345811784, other_time: 5.2365570068
epoch time: 13.9443264008s, input_fired_time: 0.4904049337, output_fired_time: 0.0000000000, propagate_forward_time: 12.6322927475, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8216285706
validation[          , genome  6909] predictions:    9800/  10000 (98.00%), best:    9811/10000 (98.11%), error:       676.65198, best error:       647.87897 on epoch:    62, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 187.9698181152s, input_fired_time: 2.0445067883, output_fired_time: 2.2527110577, propagate_forward_time: 67.5551605225, propagate_backward_time: 112.1205902100, weight_update_time: 0.1357623190, other_time: 3.9968414307
epoch time: 13.4441757202s, input_fired_time: 0.2348553091, output_fired_time: 0.0000000000, propagate_forward_time: 12.6448688507, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5644512177
validation[          , genome  6909] predictions:    9753/  10000 (97.53%), best:    9811/10000 (98.11%), error:       884.81671, best error:       647.87897 on epoch:    62, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 175.5144042969s, input_fired_time: 2.7211520672, output_fired_time: 2.1576552391, propagate_forward_time: 63.6073760986, propagate_backward_time: 103.1921615601, weight_update_time: 0.1276726574, other_time: 3.8360671997
epoch time: 12.5650186539s, input_fired_time: 0.2283221781, output_fired_time: 0.0000000000, propagate_forward_time: 11.8011493683, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5355472565
validation[          , genome  6909] predictions:    9768/  10000 (97.68%), best:    9811/10000 (98.11%), error:       758.19482, best error:       647.87897 on epoch:    62, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 171.4370727539s, input_fired_time: 2.9328303337, output_fired_time: 1.8618739843, propagate_forward_time: 61.5193862915, propagate_backward_time: 101.3153686523, weight_update_time: 0.1258094609, other_time: 3.8076095581
epoch time: 13.8927640915s, input_fired_time: 0.2406610847, output_fired_time: 0.0000000000, propagate_forward_time: 13.0888290405, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5632743835
validation[          , genome  6909] predictions:    9762/  10000 (97.62%), best:    9811/10000 (98.11%), error:       769.46851, best error:       647.87897 on epoch:    62, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 169.4211883545s, input_fired_time: 2.1795656681, output_fired_time: 1.8508709669, propagate_forward_time: 61.4817276001, propagate_backward_time: 99.6484603882, weight_update_time: 0.1246892139, other_time: 4.2605590820
epoch time: 13.0916662216s, input_fired_time: 0.4837385118, output_fired_time: 0.0000000000, propagate_forward_time: 12.0644330978, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5434942245
validation[          , genome  6909] predictions:    9777/  10000 (97.77%), best:    9811/10000 (98.11%), error:       724.61420, best error:       647.87897 on epoch:    62, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 171.2490844727s, input_fired_time: 2.4608280659, output_fired_time: 2.3648908138, propagate_forward_time: 59.9455451965, propagate_backward_time: 102.4388198853, weight_update_time: 0.1255007088, other_time: 4.0390090942
epoch time: 15.3960494995s, input_fired_time: 0.7577308416, output_fired_time: 0.0000000000, propagate_forward_time: 14.0388898849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5994291306
validation[          , genome  6909] predictions:    9747/  10000 (97.47%), best:    9811/10000 (98.11%), error:       785.30109, best error:       647.87897 on epoch:    62, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 166.9090881348s, input_fired_time: 2.4126522541, output_fired_time: 2.0796046257, propagate_forward_time: 59.3136024475, propagate_backward_time: 99.3576889038, weight_update_time: 0.1233503148, other_time: 3.7455368042
epoch time: 12.0777568817s, input_fired_time: 0.2212267071, output_fired_time: 0.0000000000, propagate_forward_time: 10.8395900726, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.0169401169
validation[          , genome  6909] predictions:    9797/  10000 (97.97%), best:    9811/10000 (98.11%), error:       698.11096, best error:       647.87897 on epoch:    62, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 174.0623626709s, input_fired_time: 2.2131628990, output_fired_time: 2.1363747120, propagate_forward_time: 63.0304908752, propagate_backward_time: 103.1157379150, weight_update_time: 0.1273261011, other_time: 3.5666046143
epoch time: 11.9570693970s, input_fired_time: 0.4709250629, output_fired_time: 0.0000000000, propagate_forward_time: 10.7217378616, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7644062042
validation[          , genome  6909] predictions:    9793/  10000 (97.93%), best:    9811/10000 (98.11%), error:       675.49738, best error:       647.87897 on epoch:    62, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 170.8767700195s, input_fired_time: 3.4415674210, output_fired_time: 2.1066801548, propagate_forward_time: 62.1814689636, propagate_backward_time: 99.6070938110, weight_update_time: 0.1257890165, other_time: 3.5399551392
epoch time: 15.6943178177s, input_fired_time: 0.2678055465, output_fired_time: 0.0000000000, propagate_forward_time: 14.8079185486, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6185941696
validation[          , genome  6909] predictions:    9789/  10000 (97.89%), best:    9811/10000 (98.11%), error:       690.53979, best error:       647.87897 on epoch:    62, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 171.0336151123s, input_fired_time: 2.1940214634, output_fired_time: 2.8613049984, propagate_forward_time: 61.8699226379, propagate_backward_time: 100.3148803711, weight_update_time: 0.1261325777, other_time: 3.7935028076
epoch time: 12.1240921021s, input_fired_time: 0.2206425071, output_fired_time: 0.0000000000, propagate_forward_time: 11.1347532272, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7686967850
validation[          , genome  6909] predictions:    9709/  10000 (97.09%), best:    9811/10000 (98.11%), error:      1029.94556, best error:       647.87897 on epoch:    62, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 168.0670013428s, input_fired_time: 2.1756200790, output_fired_time: 1.8313006163, propagate_forward_time: 60.9134216309, propagate_backward_time: 99.1459503174, weight_update_time: 0.6242298484, other_time: 4.0007171631
epoch time: 14.4543113708s, input_fired_time: 0.2462552488, output_fired_time: 0.0000000000, propagate_forward_time: 13.3855857849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8224706650
validation[          , genome  6909] predictions:    9785/  10000 (97.85%), best:    9811/10000 (98.11%), error:       722.67133, best error:       647.87897 on epoch:    62, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 148.0438232422s, input_fired_time: 2.6237688065, output_fired_time: 1.7723317146, propagate_forward_time: 52.4181404114, propagate_backward_time: 87.8388595581, weight_update_time: 0.1182625443, other_time: 3.3907241821
epoch time: 7.9057226181s, input_fired_time: 0.2051000595, output_fired_time: 0.0000000000, propagate_forward_time: 7.2299351692, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4706873894
validation[          , genome  6909] predictions:    9777/  10000 (97.77%), best:    9811/10000 (98.11%), error:       721.93054, best error:       647.87897 on epoch:    62, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 114.3896636963s, input_fired_time: 1.8240040541, output_fired_time: 1.4462027550, propagate_forward_time: 41.1677360535, propagate_backward_time: 67.1574859619, weight_update_time: 0.1140149459, other_time: 2.7942352295
epoch time: 9.3214893341s, input_fired_time: 0.2256596237, output_fired_time: 0.0000000000, propagate_forward_time: 8.5749874115, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5208425522
validation[          , genome  6909] predictions:    9716/  10000 (97.16%), best:    9811/10000 (98.11%), error:       983.87378, best error:       647.87897 on epoch:    62, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 107.9770278931s, input_fired_time: 1.7696179152, output_fired_time: 1.3825162649, propagate_forward_time: 38.4394416809, propagate_backward_time: 63.6954689026, weight_update_time: 0.1084002480, other_time: 2.6899833679
epoch time: 8.6367492676s, input_fired_time: 0.2148189843, output_fired_time: 0.0000000000, propagate_forward_time: 7.9273662567, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4945640564
validation[          , genome  6909] predictions:    9761/  10000 (97.61%), best:    9811/10000 (98.11%), error:       736.60516, best error:       647.87897 on epoch:    62, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 109.9761810303s, input_fired_time: 1.7832444906, output_fired_time: 1.3985561132, propagate_forward_time: 39.2435646057, propagate_backward_time: 64.8198013306, weight_update_time: 0.1111477166, other_time: 2.7310104370
epoch time: 8.9543943405s, input_fired_time: 0.2188991308, output_fired_time: 0.0000000000, propagate_forward_time: 8.2283906937, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5071048737
validation[          , genome  6909] predictions:    9802/  10000 (98.02%), best:    9811/10000 (98.11%), error:       709.67249, best error:       647.87897 on epoch:    62, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 116.4966201782s, input_fired_time: 1.8442676067, output_fired_time: 1.4769248962, propagate_forward_time: 41.9980392456, propagate_backward_time: 68.3572463989, weight_update_time: 0.1155166626, other_time: 2.8201446533
epoch time: 8.1991147995s, input_fired_time: 0.2084864378, output_fired_time: 0.0000000000, propagate_forward_time: 7.5108175278, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4798107147
validation[          , genome  6909] predictions:    9805/  10000 (98.05%), best:    9811/10000 (98.11%), error:       683.70313, best error:       647.87897 on epoch:    62, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 45.3717689514s, input_fired_time: 1.0887842178, output_fired_time: 0.0000000000, propagate_forward_time: 41.6549758911, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6280097961
best training[          , genome  6909] predictions:   49628/  50000 (99.26%), best:    9811/10000 (98.11%), error:      1481.08301, best error:       647.87897 on epoch:    62, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 7.1087222099s, input_fired_time: 0.1850534230, output_fired_time: 0.0000000000, propagate_forward_time: 6.4770450592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4466238022
   testing[          , genome  6909] predictions:    9815/  10000 (98.15%), best:    9811/10000 (98.11%), error:       649.11890, best error:       647.87897 on epoch:    62, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
14:42:03 (8824): called boinc_finish(0)

</stderr_txt>
]]>