Nameexact_genome_1512568488_44_7138_1
Workunit2606986
Created8 Dec 2017, 0:03:04 UTC
Sent8 Dec 2017, 0:12:43 UTC
Report deadline14 Dec 2017, 19:24:56 UTC
Received8 Dec 2017, 17:03:46 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64142
Run time4 hours 36 min 16 sec
CPU time3 hours 24 min 18 sec
Validate stateValid
Credit1,700.04
Device peak FLOPS3.08 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size157.35 MB
Peak swap size160.29 MB
Peak disk usage1.92 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
ons:    9640/  10000 (96.40%), best:    9724/10000 (97.24%), error:      1204.45691, best error:       970.58423 on epoch:     9, epoch:   14/100, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 118.0014801025s, input_fired_time: 1.8210376501, output_fired_time: 1.7292509079, propagate_forward_time: 42.9911384583, propagate_backward_time: 68.2560729980, weight_update_time: 0.1247609183, other_time: 3.2039794922
epoch time: 10.4634752274s, input_fired_time: 0.2375063747, output_fired_time: 0.0000000000, propagate_forward_time: 9.5022850037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7236843109
validation[          , genome  7138] predictions:    9660/  10000 (96.60%), best:    9724/10000 (97.24%), error:      1157.65491, best error:       970.58423 on epoch:     9, epoch:   15/100, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 116.4798736572s, input_fired_time: 1.9253839254, output_fired_time: 1.5811895132, propagate_forward_time: 42.6011886597, propagate_backward_time: 67.0466537476, weight_update_time: 0.1714542955, other_time: 3.3254547119
epoch time: 10.0226736069s, input_fired_time: 0.2633219361, output_fired_time: 0.0000000000, propagate_forward_time: 9.1534233093, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6059284210
validation[          , genome  7138] predictions:    9459/  10000 (94.59%), best:    9724/10000 (97.24%), error:      1741.40027, best error:       970.58423 on epoch:     9, epoch:   16/100, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 114.5382919312s, input_fired_time: 1.8595836163, output_fired_time: 1.5080084801, propagate_forward_time: 41.9398574829, propagate_backward_time: 65.9043197632, weight_update_time: 0.1262820661, other_time: 3.3265228271
epoch time: 10.9018716812s, input_fired_time: 0.2403645813, output_fired_time: 0.0000000000, propagate_forward_time: 9.9816360474, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6798706055
validation[          , genome  7138] predictions:    9719/  10000 (97.19%), best:    9724/10000 (97.24%), error:       995.68744, best error:       970.58423 on epoch:     9, epoch:   17/100, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 111.1684341431s, input_fired_time: 1.6215209961, output_fired_time: 1.5033905506, propagate_forward_time: 40.8252449036, propagate_backward_time: 64.1255645752, weight_update_time: 0.1263516694, other_time: 3.0927124023
epoch time: 8.9956798553s, input_fired_time: 0.2397680879, output_fired_time: 0.0000000000, propagate_forward_time: 8.0787067413, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6772050858
validation[          , genome  7138] predictions:    9731/  10000 (97.31%), best:    9731/10000 (97.31%), error:       920.42090, best error:       920.42090 on epoch:    18, epoch:   18/100, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 111.1934890747s, input_fired_time: 1.6435046196, output_fired_time: 1.5393717289, propagate_forward_time: 40.8256340027, propagate_backward_time: 64.0931243896, weight_update_time: 0.1286976635, other_time: 3.0918579102
epoch time: 9.4651069641s, input_fired_time: 0.2142240107, output_fired_time: 0.0000000000, propagate_forward_time: 8.7273664474, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5235166550
validation[          , genome  7138] predictions:    9630/  10000 (96.30%), best:    9731/10000 (97.31%), error:      1203.92957, best error:       920.42090 on epoch:    18, epoch:   19/100, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 111.8379821777s, input_fired_time: 1.6188117266, output_fired_time: 1.5140737295, propagate_forward_time: 40.9122924805, propagate_backward_time: 64.6411132813, weight_update_time: 0.1259856075, other_time: 3.1516876221
epoch time: 9.4273176193s, input_fired_time: 0.1996282637, output_fired_time: 0.0000000000, propagate_forward_time: 8.6718606949, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5558290482
validation[          , genome  7138] predictions:    9689/  10000 (96.89%), best:    9731/10000 (97.31%), error:      1021.36383, best error:       920.42090 on epoch:    18, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 112.1649703979s, input_fired_time: 1.6578638554, output_fired_time: 1.5283155441, propagate_forward_time: 41.0796165466, propagate_backward_time: 64.7699584961, weight_update_time: 0.1250312626, other_time: 3.1292114258
epoch time: 9.2862167358s, input_fired_time: 0.2060206234, output_fired_time: 0.0000000000, propagate_forward_time: 8.5113372803, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5688591003
validation[          , genome  7138] predictions:    9705/  10000 (97.05%), best:    9731/10000 (97.31%), error:       930.50824, best error:       920.42090 on epoch:    18, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 110.0438461304s, input_fired_time: 1.6696915627, output_fired_time: 1.4533933401, propagate_forward_time: 40.5320510864, propagate_backward_time: 63.2029724121, weight_update_time: 0.1261612475, other_time: 3.1857376099
epoch time: 9.7223882675s, input_fired_time: 0.2186155170, output_fired_time: 0.0000000000, propagate_forward_time: 8.9038772583, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5998954773
validation[          , genome  7138] predictions:    9651/  10000 (96.51%), best:    9731/10000 (97.31%), error:      1142.30237, best error:       920.42090 on epoch:    18, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 110.9983901978s, input_fired_time: 1.8111226559, output_fired_time: 1.4928690195, propagate_forward_time: 40.4284439087, propagate_backward_time: 64.1644515991, weight_update_time: 0.1253430396, other_time: 3.1015090942
epoch time: 10.5116348267s, input_fired_time: 0.2381429821, output_fired_time: 0.0000000000, propagate_forward_time: 9.6211643219, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6523275375
validation[          , genome  7138] predictions:    9718/  10000 (97.18%), best:    9731/10000 (97.31%), error:       940.27496, best error:       920.42090 on epoch:    18, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 107.8352890015s, input_fired_time: 1.6279535294, output_fired_time: 1.4495747089, propagate_forward_time: 39.4493675232, propagate_backward_time: 62.2153968811, weight_update_time: 0.1259687543, other_time: 3.0929985046
epoch time: 9.3625411987s, input_fired_time: 0.2500567436, output_fired_time: 0.0000000000, propagate_forward_time: 8.4977970123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6146869659
validation[          , genome  7138] predictions:    9613/  10000 (96.13%), best:    9731/10000 (97.31%), error:      1289.85645, best error:       920.42090 on epoch:    18, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 108.3104324341s, input_fired_time: 1.6575604677, output_fired_time: 1.4582899809, propagate_forward_time: 39.6522903442, propagate_backward_time: 62.4593391418, weight_update_time: 0.1233738959, other_time: 3.0829505920
epoch time: 8.7938451767s, input_fired_time: 0.2061988711, output_fired_time: 0.0000000000, propagate_forward_time: 8.0265007019, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5611457825
validation[          , genome  7138] predictions:    9718/  10000 (97.18%), best:    9731/10000 (97.31%), error:       930.73969, best error:       920.42090 on epoch:    18, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 111.5425872803s, input_fired_time: 1.7375106812, output_fired_time: 1.5198572874, propagate_forward_time: 40.8060035706, propagate_backward_time: 64.2901306152, weight_update_time: 0.1242417172, other_time: 3.1890869141
epoch time: 8.5838871002s, input_fired_time: 0.1981664598, output_fired_time: 0.0000000000, propagate_forward_time: 7.8372015953, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5485186577
validation[          , genome  7138] predictions:    9622/  10000 (96.22%), best:    9731/10000 (97.31%), error:      1217.71338, best error:       920.42090 on epoch:    18, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 113.7347946167s, input_fired_time: 1.6467592716, output_fired_time: 1.5348151922, propagate_forward_time: 41.8752822876, propagate_backward_time: 65.5685348511, weight_update_time: 0.1276378483, other_time: 3.1094055176
epoch time: 8.9160118103s, input_fired_time: 0.2016174048, output_fired_time: 0.0000000000, propagate_forward_time: 8.1539678574, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5604267120
validation[          , genome  7138] predictions:    9716/  10000 (97.16%), best:    9731/10000 (97.31%), error:       971.83923, best error:       920.42090 on epoch:    18, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 112.7402343750s, input_fired_time: 1.6564159393, output_fired_time: 1.5022840500, propagate_forward_time: 41.5505142212, propagate_backward_time: 64.8339080811, weight_update_time: 0.1292737275, other_time: 3.1971130371
epoch time: 9.9014663696s, input_fired_time: 0.2713410854, output_fired_time: 0.0000000000, propagate_forward_time: 8.9831743240, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6469507217
validation[          , genome  7138] predictions:    9719/  10000 (97.19%), best:    9731/10000 (97.31%), error:       937.48456, best error:       920.42090 on epoch:    18, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 126.1968765259s, input_fired_time: 2.2448055744, output_fired_time: 1.6614757776, propagate_forward_time: 46.5204200745, propagate_backward_time: 72.2085800171, weight_update_time: 0.1297068447, other_time: 3.5615921021
epoch time: 15.2283868790s, input_fired_time: 0.7963242531, output_fired_time: 0.0000000000, propagate_forward_time: 13.5379800797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8940820694
validation[          , genome  7138] predictions:    9492/  10000 (94.92%), best:    9731/10000 (97.31%), error:      1588.19934, best error:       920.42090 on epoch:    18, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 245.4530944824s, input_fired_time: 3.7605092525, output_fired_time: 3.7135555744, propagate_forward_time: 90.4972305298, propagate_backward_time: 140.8866729736, weight_update_time: 0.1391236484, other_time: 6.5951080322
epoch time: 24.5825233459s, input_fired_time: 0.2671677470, output_fired_time: 0.0000000000, propagate_forward_time: 22.6436290741, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6717262268
validation[          , genome  7138] predictions:    9664/  10000 (96.64%), best:    9731/10000 (97.31%), error:      1107.29883, best error:       920.42090 on epoch:    18, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 255.5231933594s, input_fired_time: 3.3900716305, output_fired_time: 3.9040060043, propagate_forward_time: 94.8062820435, propagate_backward_time: 147.0030670166, weight_update_time: 0.2486524433, other_time: 6.4197540283
epoch time: 20.5232582092s, input_fired_time: 0.5182090402, output_fired_time: 0.0000000000, propagate_forward_time: 18.8027706146, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2022781372
validation[          , genome  7138] predictions:    9469/  10000 (94.69%), best:    9731/10000 (97.31%), error:      1735.49048, best error:       920.42090 on epoch:    18, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 213.6879425049s, input_fired_time: 2.2974455357, output_fired_time: 2.6826319695, propagate_forward_time: 80.7741088867, propagate_backward_time: 122.7973175049, weight_update_time: 0.1391702443, other_time: 5.1364440918
epoch time: 15.7574691772s, input_fired_time: 0.4750472009, output_fired_time: 0.0000000000, propagate_forward_time: 14.3126926422, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9697294235
validation[          , genome  7138] predictions:    9720/  10000 (97.20%), best:    9720/10000 (97.20%), error:       913.07281, best error:       913.07281 on epoch:    32, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 206.3852539063s, input_fired_time: 3.0280964375, output_fired_time: 2.2790851593, propagate_forward_time: 75.9078063965, propagate_backward_time: 120.5050277710, weight_update_time: 0.1386730820, other_time: 4.6652450562
epoch time: 14.9971885681s, input_fired_time: 0.2223654836, output_fired_time: 0.0000000000, propagate_forward_time: 13.5574684143, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2173547745
validation[          , genome  7138] predictions:    9717/  10000 (97.17%), best:    9720/10000 (97.20%), error:       967.99323, best error:       913.07281 on epoch:    32, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 205.7018585205s, input_fired_time: 3.0746839046, output_fired_time: 3.0163922310, propagate_forward_time: 74.8438110352, propagate_backward_time: 118.8392715454, weight_update_time: 0.4356633723, other_time: 5.9277114868
epoch time: 16.9983921051s, input_fired_time: 0.2831033766, output_fired_time: 0.0000000000, propagate_forward_time: 15.3880262375, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3272619247
validation[          , genome  7138] predictions:    9760/  10000 (97.60%), best:    9760/10000 (97.60%), error:       847.03644, best error:       847.03644 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 280.8214721680s, input_fired_time: 3.8189125061, output_fired_time: 3.2961778641, propagate_forward_time: 107.4900970459, propagate_backward_time: 160.2200775146, weight_update_time: 0.4516495168, other_time: 5.9962158203
epoch time: 23.0832519531s, input_fired_time: 0.9226875901, output_fired_time: 0.0000000000, propagate_forward_time: 21.3649616241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7956027985
validation[          , genome  7138] predictions:    9654/  10000 (96.54%), best:    9760/10000 (97.60%), error:      1155.80017, best error:       847.03644 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 276.1433410645s, input_fired_time: 4.7846398354, output_fired_time: 4.1735420227, propagate_forward_time: 98.2749328613, propagate_backward_time: 162.2938842773, weight_update_time: 0.2012250870, other_time: 6.6163330078
epoch time: 24.4209232330s, input_fired_time: 0.4764865041, output_fired_time: 0.0000000000, propagate_forward_time: 22.3010807037, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6433563232
validation[          , genome  7138] predictions:    9590/  10000 (95.90%), best:    9760/10000 (97.60%), error:      1299.35815, best error:       847.03644 on epoch:    34, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 289.5355224609s, input_fired_time: 4.8115606308, output_fired_time: 6.8455772400, propagate_forward_time: 107.1920242310, propagate_backward_time: 161.4030761719, weight_update_time: 0.2325697392, other_time: 9.2832946777
epoch time: 18.4456405640s, input_fired_time: 0.2208483815, output_fired_time: 0.0000000000, propagate_forward_time: 17.0086650848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2161273956
validation[          , genome  7138] predictions:    9728/  10000 (97.28%), best:    9760/10000 (97.60%), error:       887.32227, best error:       847.03644 on epoch:    34, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 253.9506530762s, input_fired_time: 3.9256656170, output_fired_time: 2.7835211754, propagate_forward_time: 93.7277832031, propagate_backward_time: 147.8016357422, weight_update_time: 0.1857892126, other_time: 5.7120513916
epoch time: 21.2192859650s, input_fired_time: 0.6895234585, output_fired_time: 0.0000000000, propagate_forward_time: 18.2190589905, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.3107032776
validation[          , genome  7138] predictions:    9746/  10000 (97.46%), best:    9760/10000 (97.60%), error:       889.47443, best error:       847.03644 on epoch:    34, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 226.8737487793s, input_fired_time: 2.3142297268, output_fired_time: 2.9541194439, propagate_forward_time: 86.6697692871, propagate_backward_time: 128.0864105225, weight_update_time: 0.1391594559, other_time: 6.8492279053
epoch time: 27.9118614197s, input_fired_time: 0.3144464791, output_fired_time: 0.0000000000, propagate_forward_time: 26.8014774323, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7959365845
validation[          , genome  7138] predictions:    9586/  10000 (95.86%), best:    9760/10000 (97.60%), error:      1414.27283, best error:       847.03644 on epoch:    34, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 108.6385879517s, input_fired_time: 1.5819370747, output_fired_time: 1.5023417473, propagate_forward_time: 39.8734169006, propagate_backward_time: 62.5590934753, weight_update_time: 0.1214830801, other_time: 3.1217994690
epoch time: 9.1464376450s, input_fired_time: 0.2020315677, output_fired_time: 0.0000000000, propagate_forward_time: 8.3931608200, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5512456894
validation[          , genome  7138] predictions:    9671/  10000 (96.71%), best:    9760/10000 (97.60%), error:      1030.56628, best error:       847.03644 on epoch:    34, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 107.2477951050s, input_fired_time: 1.5807405710, output_fired_time: 1.3813369274, propagate_forward_time: 39.2989196777, propagate_backward_time: 62.0659980774, weight_update_time: 0.1203493774, other_time: 2.9207954407
epoch time: 8.3080940247s, input_fired_time: 0.1978940815, output_fired_time: 0.0000000000, propagate_forward_time: 7.5735092163, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5366907120
validation[          , genome  7138] predictions:    9745/  10000 (97.45%), best:    9745/10000 (97.45%), error:       846.61603, best error:       846.61603 on epoch:    41, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 110.1124343872s, input_fired_time: 1.6536931992, output_fired_time: 1.3905689716, propagate_forward_time: 40.3428916931, propagate_backward_time: 63.4781799316, weight_update_time: 0.1200048849, other_time: 3.2471008301
epoch time: 8.6225948334s, input_fired_time: 0.1914647520, output_fired_time: 0.0000000000, propagate_forward_time: 7.8845658302, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5465645790
validation[          , genome  7138] predictions:    9552/  10000 (95.52%), best:    9745/10000 (97.45%), error:      1432.45691, best error:       846.61603 on epoch:    41, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 111.2831420898s, input_fired_time: 1.7460579872, output_fired_time: 1.7530485392, propagate_forward_time: 40.2932167053, propagate_backward_time: 64.2638092041, weight_update_time: 0.1195752770, other_time: 3.2270050049
epoch time: 8.6636934280s, input_fired_time: 0.1881081164, output_fired_time: 0.0000000000, propagate_forward_time: 7.9495902061, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5259947777
validation[          , genome  7138] predictions:    9622/  10000 (96.22%), best:    9745/10000 (97.45%), error:      1235.30237, best error:       846.61603 on epoch:    41, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 104.7464218140s, input_fired_time: 1.6858385801, output_fired_time: 1.4039523602, propagate_forward_time: 38.2719345093, propagate_backward_time: 60.4402503967, weight_update_time: 0.1213391423, other_time: 2.9444465637
epoch time: 8.6596040726s, input_fired_time: 0.2010245323, output_fired_time: 0.0000000000, propagate_forward_time: 7.9020152092, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5565648079
validation[          , genome  7138] predictions:    9660/  10000 (96.60%), best:    9745/10000 (97.45%), error:      1142.39038, best error:       846.61603 on epoch:    41, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 110.9575881958s, input_fired_time: 1.6265088320, output_fired_time: 1.4503792524, propagate_forward_time: 40.7677421570, propagate_backward_time: 63.8498497009, weight_update_time: 0.1248648465, other_time: 3.2631111145
epoch time: 9.4171924591s, input_fired_time: 0.2980346382, output_fired_time: 0.0000000000, propagate_forward_time: 8.3798580170, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7392997742
validation[          , genome  7138] predictions:    9677/  10000 (96.77%), best:    9745/10000 (97.45%), error:      1085.04797, best error:       846.61603 on epoch:    41, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 117.8805847168s, input_fired_time: 1.6436357498, output_fired_time: 1.8079146147, propagate_forward_time: 43.4321212769, propagate_backward_time: 67.6413803101, weight_update_time: 0.2209901959, other_time: 3.3555297852
epoch time: 9.2705764771s, input_fired_time: 0.2044657320, output_fired_time: 0.0000000000, propagate_forward_time: 8.5057001114, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5604104996
validation[          , genome  7138] predictions:    9679/  10000 (96.79%), best:    9745/10000 (97.45%), error:      1114.05188, best error:       846.61603 on epoch:    41, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 117.2014770508s, input_fired_time: 1.6753263474, output_fired_time: 1.5154385567, propagate_forward_time: 43.2282981873, propagate_backward_time: 67.4325027466, weight_update_time: 0.1303448081, other_time: 3.3499069214
epoch time: 8.6577796936s, input_fired_time: 0.1925630271, output_fired_time: 0.0000000000, propagate_forward_time: 7.9306626320, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5345540047
validation[          , genome  7138] predictions:    9663/  10000 (96.63%), best:    9745/10000 (97.45%), error:      1067.84888, best error:       846.61603 on epoch:    41, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 115.2164764404s, input_fired_time: 1.7904223204, output_fired_time: 1.5921204090, propagate_forward_time: 42.4068603516, propagate_backward_time: 66.3456268311, weight_update_time: 0.1278011948, other_time: 3.0814514160
epoch time: 9.3957290649s, input_fired_time: 0.1965992302, output_fired_time: 0.0000000000, propagate_forward_time: 8.5500259399, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6491041183
validation[          , genome  7138] predictions:    9705/  10000 (97.05%), best:    9745/10000 (97.45%), error:       960.43567, best error:       846.61603 on epoch:    41, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 114.5025863647s, input_fired_time: 1.7470945120, output_fired_time: 1.6786215305, propagate_forward_time: 42.0215072632, propagate_backward_time: 65.7724075317, weight_update_time: 0.1265789866, other_time: 3.2829589844
epoch time: 10.4835748672s, input_fired_time: 0.3226830661, output_fired_time: 0.0000000000, propagate_forward_time: 9.5812263489, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5796651840
validation[          , genome  7138] predictions:    9723/  10000 (97.23%), best:    9745/10000 (97.45%), error:       896.73016, best error:       846.61603 on epoch:    41, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 111.0970687866s, input_fired_time: 1.6365844011, output_fired_time: 1.4666113853, propagate_forward_time: 40.7691078186, propagate_backward_time: 64.1872177124, weight_update_time: 0.1262478381, other_time: 3.0375442505
epoch time: 9.2600164413s, input_fired_time: 0.1987460405, output_fired_time: 0.0000000000, propagate_forward_time: 8.2845115662, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7767591476
validation[          , genome  7138] predictions:    9616/  10000 (96.16%), best:    9745/10000 (97.45%), error:      1307.89172, best error:       846.61603 on epoch:    41, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 113.6434020996s, input_fired_time: 1.7436411381, output_fired_time: 1.5393359661, propagate_forward_time: 41.7954101563, propagate_backward_time: 65.3709335327, weight_update_time: 0.1275661737, other_time: 3.1940765381
epoch time: 8.8321170807s, input_fired_time: 0.1878193021, output_fired_time: 0.0000000000, propagate_forward_time: 7.9978809357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6464166641
validation[          , genome  7138] predictions:    9673/  10000 (96.73%), best:    9745/10000 (97.45%), error:      1094.84619, best error:       846.61603 on epoch:    41, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 118.7983932495s, input_fired_time: 1.8641138077, output_fired_time: 1.4797428846, propagate_forward_time: 43.4307289124, propagate_backward_time: 68.7170257568, weight_update_time: 0.1274330169, other_time: 3.3067779541
epoch time: 8.4556875229s, input_fired_time: 0.1910694838, output_fired_time: 0.0000000000, propagate_forward_time: 7.7269968987, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5376210213
validation[          , genome  7138] predictions:    9602/  10000 (96.02%), best:    9745/10000 (97.45%), error:      1291.65466, best error:       846.61603 on epoch:    41, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 115.7367858887s, input_fired_time: 1.6360784769, output_fired_time: 1.6191304922, propagate_forward_time: 42.7177581787, propagate_backward_time: 66.5795288086, weight_update_time: 0.2368366122, other_time: 3.1842880249
epoch time: 10.0054750443s, input_fired_time: 0.2184680551, output_fired_time: 0.0000000000, propagate_forward_time: 9.1915712357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5954360962
validation[          , genome  7138] predictions:    9674/  10000 (96.74%), best:    9745/10000 (97.45%), error:      1111.94434, best error:       846.61603 on epoch:    41, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 162.5808868408s, input_fired_time: 2.0573654175, output_fired_time: 2.2595891953, propagate_forward_time: 60.7478981018, propagate_backward_time: 93.3016052246, weight_update_time: 0.1368016601, other_time: 4.2144470215
epoch time: 14.7449493408s, input_fired_time: 0.2229174078, output_fired_time: 0.0000000000, propagate_forward_time: 13.9158048630, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6062269211
validation[          , genome  7138] predictions:    9747/  10000 (97.47%), best:    9747/10000 (97.47%), error:       830.94232, best error:       830.94232 on epoch:    54, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 211.7895507813s, input_fired_time: 3.1804177761, output_fired_time: 2.7663092613, propagate_forward_time: 77.2151412964, propagate_backward_time: 124.1878128052, weight_update_time: 0.1387437135, other_time: 4.4398727417
epoch time: 16.6940498352s, input_fired_time: 0.3173152804, output_fired_time: 0.0000000000, propagate_forward_time: 15.7704868317, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6062469482
validation[          , genome  7138] predictions:    9718/  10000 (97.18%), best:    9747/10000 (97.47%), error:       915.02924, best error:       830.94232 on epoch:    54, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 214.4048156738s, input_fired_time: 2.9454836845, output_fired_time: 3.1885066032, propagate_forward_time: 78.2235260010, propagate_backward_time: 124.2591629028, weight_update_time: 0.1387564987, other_time: 5.7881393433
epoch time: 21.7678184509s, input_fired_time: 0.5209881663, output_fired_time: 0.0000000000, propagate_forward_time: 19.5137386322, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7330913544
validation[          , genome  7138] predictions:    9257/  10000 (92.57%), best:    9747/10000 (97.47%), error:      2328.49585, best error:       830.94232 on epoch:    54, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

resetting weights to those on epoch: 54
epoch time: 255.3741455078s, input_fired_time: 3.7037537098, output_fired_time: 3.4232716560, propagate_forward_time: 93.3110046387, propagate_backward_time: 146.6819763184, weight_update_time: 0.2796983421, other_time: 8.2541351318
epoch time: 19.6176624298s, input_fired_time: 0.2216916382, output_fired_time: 0.0000000000, propagate_forward_time: 18.1513519287, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2446193695
validation[          , genome  7138] predictions:    9743/  10000 (97.43%), best:    9747/10000 (97.47%), error:       843.62286, best error:       830.94232 on epoch:    54, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 205.8881835938s, input_fired_time: 2.5169227123, output_fired_time: 3.0483338833, propagate_forward_time: 75.8303756714, propagate_backward_time: 118.0251235962, weight_update_time: 0.5293301940, other_time: 6.4674224854
epoch time: 19.8283271790s, input_fired_time: 0.2263376713, output_fired_time: 0.0000000000, propagate_forward_time: 18.9922084808, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6097812653
validation[          , genome  7138] predictions:    9655/  10000 (96.55%), best:    9747/10000 (97.47%), error:      1187.93042, best error:       830.94232 on epoch:    54, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 389.2068786621s, input_fired_time: 5.7053828239, output_fired_time: 4.9357094765, propagate_forward_time: 145.2677917480, propagate_backward_time: 221.7380828857, weight_update_time: 0.1387815922, other_time: 11.5599212646
epoch time: 17.9095821381s, input_fired_time: 0.2202263027, output_fired_time: 0.0000000000, propagate_forward_time: 17.0838680267, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6054878235
validation[          , genome  7138] predictions:    9700/  10000 (97.00%), best:    9747/10000 (97.47%), error:       963.56879, best error:       830.94232 on epoch:    54, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 370.3662414551s, input_fired_time: 4.9150462151, output_fired_time: 3.9829919338, propagate_forward_time: 135.1593017578, propagate_backward_time: 214.2125244141, weight_update_time: 0.2475394905, other_time: 12.0963745117
epoch time: 9.1792173386s, input_fired_time: 0.2001715153, output_fired_time: 0.0000000000, propagate_forward_time: 8.4330615997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5459842682
validation[          , genome  7138] predictions:    9740/  10000 (97.40%), best:    9740/10000 (97.40%), error:       822.70245, best error:       822.70245 on epoch:    60, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 111.9949798584s, input_fired_time: 1.7294398546, output_fired_time: 1.5638737679, propagate_forward_time: 41.1155204773, propagate_backward_time: 64.2485885620, weight_update_time: 0.1226069033, other_time: 3.3375625610
epoch time: 8.7208271027s, input_fired_time: 0.2027217746, output_fired_time: 0.0000000000, propagate_forward_time: 7.9640216827, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5540838242
validation[          , genome  7138] predictions:    9728/  10000 (97.28%), best:    9740/10000 (97.40%), error:       911.80609, best error:       822.70245 on epoch:    60, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 108.0489959717s, input_fired_time: 1.7269713879, output_fired_time: 1.4105308056, propagate_forward_time: 39.7859611511, propagate_backward_time: 62.1598930359, weight_update_time: 0.1225469187, other_time: 2.9656410217
epoch time: 10.3153371811s, input_fired_time: 0.3281834722, output_fired_time: 0.0000000000, propagate_forward_time: 9.4663381577, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5208158493
validation[          , genome  7138] predictions:    9748/  10000 (97.48%), best:    9740/10000 (97.40%), error:       849.19800, best error:       822.70245 on epoch:    60, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 111.9177169800s, input_fired_time: 1.7389171124, output_fired_time: 1.4548764229, propagate_forward_time: 40.9007644653, propagate_backward_time: 64.8107070923, weight_update_time: 0.1243902370, other_time: 3.0124511719
epoch time: 10.0325870514s, input_fired_time: 0.2101832777, output_fired_time: 0.0000000000, propagate_forward_time: 9.1362504959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6861534119
validation[          , genome  7138] predictions:    9621/  10000 (96.21%), best:    9740/10000 (97.40%), error:      1195.32410, best error:       822.70245 on epoch:    60, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 111.7779922485s, input_fired_time: 1.7169606686, output_fired_time: 1.5004954338, propagate_forward_time: 40.8678283691, propagate_backward_time: 64.7340698242, weight_update_time: 0.1221424863, other_time: 2.9586410522
epoch time: 8.5700740814s, input_fired_time: 0.1946249157, output_fired_time: 0.0000000000, propagate_forward_time: 7.8409986496, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5344505310
validation[          , genome  7138] predictions:    9727/  10000 (97.27%), best:    9740/10000 (97.40%), error:       902.00171, best error:       822.70245 on epoch:    60, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 114.3966674805s, input_fired_time: 1.6201748848, output_fired_time: 1.6991251707, propagate_forward_time: 42.2421836853, propagate_backward_time: 65.7329864502, weight_update_time: 0.1247430220, other_time: 3.1021881104
epoch time: 9.0163412094s, input_fired_time: 0.1979611814, output_fired_time: 0.0000000000, propagate_forward_time: 8.2809267044, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5374536514
validation[          , genome  7138] predictions:    9710/  10000 (97.10%), best:    9740/10000 (97.40%), error:       937.42065, best error:       822.70245 on epoch:    60, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 112.0308456421s, input_fired_time: 1.6132223606, output_fired_time: 1.4302204847, propagate_forward_time: 40.7897758484, propagate_backward_time: 64.6558761597, weight_update_time: 0.1232529953, other_time: 3.5417556763
epoch time: 8.9005584717s, input_fired_time: 0.2012492269, output_fired_time: 0.0000000000, propagate_forward_time: 8.1484413147, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5508680344
validation[          , genome  7138] predictions:    9756/  10000 (97.56%), best:    9756/10000 (97.56%), error:       795.33893, best error:       795.33893 on epoch:    66, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 113.9811630249s, input_fired_time: 1.6171414852, output_fired_time: 1.5536218882, propagate_forward_time: 42.1451225281, propagate_backward_time: 65.6387176514, weight_update_time: 0.1249150187, other_time: 3.0265655518
epoch time: 10.6791248322s, input_fired_time: 0.3118925095, output_fired_time: 0.0000000000, propagate_forward_time: 9.6528081894, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7144241333
validation[          , genome  7138] predictions:    9601/  10000 (96.01%), best:    9756/10000 (97.56%), error:      1309.24561, best error:       795.33893 on epoch:    66, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 113.6917572021s, input_fired_time: 1.5950673819, output_fired_time: 1.6106332541, propagate_forward_time: 41.8782844543, propagate_backward_time: 65.1921539307, weight_update_time: 0.1241551563, other_time: 3.4156188965
epoch time: 8.3017997742s, input_fired_time: 0.1915052384, output_fired_time: 0.0000000000, propagate_forward_time: 7.5686984062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5415959358
validation[          , genome  7138] predictions:    9618/  10000 (96.18%), best:    9756/10000 (97.56%), error:      1243.77087, best error:       795.33893 on epoch:    66, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 108.3529968262s, input_fired_time: 1.7178369761, output_fired_time: 1.4320050478, propagate_forward_time: 39.6592712402, propagate_backward_time: 62.5383529663, weight_update_time: 0.1240634844, other_time: 3.0055313110
epoch time: 9.1916990280s, input_fired_time: 0.3037521839, output_fired_time: 0.0000000000, propagate_forward_time: 8.3303461075, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5576009750
validation[          , genome  7138] predictions:    9725/  10000 (97.25%), best:    9756/10000 (97.56%), error:       952.16473, best error:       795.33893 on epoch:    66, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 111.4227600098s, input_fired_time: 1.7927809954, output_fired_time: 1.4057962894, propagate_forward_time: 40.9087104797, propagate_backward_time: 64.2007369995, weight_update_time: 0.1209715456, other_time: 3.1147384644
epoch time: 9.5736398697s, input_fired_time: 0.2821848989, output_fired_time: 0.0000000000, propagate_forward_time: 8.5578756332, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7335796356
validation[          , genome  7138] predictions:    9686/  10000 (96.86%), best:    9756/10000 (97.56%), error:      1061.89600, best error:       795.33893 on epoch:    66, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 107.0303192139s, input_fired_time: 1.5984580517, output_fired_time: 1.5315920115, propagate_forward_time: 38.8920669556, propagate_backward_time: 61.8522758484, weight_update_time: 0.1225361601, other_time: 3.1559257507
epoch time: 8.3483848572s, input_fired_time: 0.1972780973, output_fired_time: 0.0000000000, propagate_forward_time: 7.6100745201, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5410323143
validation[          , genome  7138] predictions:    9703/  10000 (97.03%), best:    9756/10000 (97.56%), error:       987.53699, best error:       795.33893 on epoch:    66, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 113.5323486328s, input_fired_time: 1.5813440084, output_fired_time: 1.3923566341, propagate_forward_time: 41.8004417419, propagate_backward_time: 65.3437957764, weight_update_time: 0.1215291470, other_time: 3.4144134521
epoch time: 9.6302728653s, input_fired_time: 0.2063801289, output_fired_time: 0.0000000000, propagate_forward_time: 8.7653226852, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6585702896
validation[          , genome  7138] predictions:    9729/  10000 (97.29%), best:    9756/10000 (97.56%), error:       924.47644, best error:       795.33893 on epoch:    66, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 108.0510025024s, input_fired_time: 1.6185312271, output_fired_time: 1.4331020117, propagate_forward_time: 39.5141258240, propagate_backward_time: 62.3181228638, weight_update_time: 0.1231439859, other_time: 3.1671218872
epoch time: 9.0207223892s, input_fired_time: 0.1990471035, output_fired_time: 0.0000000000, propagate_forward_time: 8.2753143311, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5463609695
validation[          , genome  7138] predictions:    9744/  10000 (97.44%), best:    9756/10000 (97.56%), error:       898.51581, best error:       795.33893 on epoch:    66, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 112.3514099121s, input_fired_time: 1.7228678465, output_fired_time: 1.6091649532, propagate_forward_time: 41.1162528992, propagate_backward_time: 64.8109817505, weight_update_time: 0.1227528155, other_time: 3.0921401978
epoch time: 8.8780679703s, input_fired_time: 0.1927120537, output_fired_time: 0.0000000000, propagate_forward_time: 8.0476932526, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6376628876
validation[          , genome  7138] predictions:    9633/  10000 (96.33%), best:    9756/10000 (97.56%), error:      1201.38635, best error:       795.33893 on epoch:    66, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 104.6149749756s, input_fired_time: 1.6778869629, output_fired_time: 1.4039425850, propagate_forward_time: 37.9935951233, propagate_backward_time: 60.5094604492, weight_update_time: 0.1207149848, other_time: 3.0300865173
epoch time: 10.7171373367s, input_fired_time: 0.1941435784, output_fired_time: 0.0000000000, propagate_forward_time: 9.9835853577, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5394086838
validation[          , genome  7138] predictions:    9751/  10000 (97.51%), best:    9756/10000 (97.56%), error:       856.08685, best error:       795.33893 on epoch:    66, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 110.4479980469s, input_fired_time: 1.5767717361, output_fired_time: 1.4968670607, propagate_forward_time: 40.5391159058, propagate_backward_time: 63.5681037903, weight_update_time: 0.1220631450, other_time: 3.2671394348
epoch time: 9.5477628708s, input_fired_time: 0.3793762028, output_fired_time: 0.0000000000, propagate_forward_time: 8.6294317245, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5389547348
validation[          , genome  7138] predictions:    9675/  10000 (96.75%), best:    9756/10000 (97.56%), error:      1108.13525, best error:       795.33893 on epoch:    66, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 109.1580200195s, input_fired_time: 1.6686656475, output_fired_time: 1.4342292547, propagate_forward_time: 39.8431549072, propagate_backward_time: 63.1986160278, weight_update_time: 0.1399108171, other_time: 3.0133590698
epoch time: 8.4212760925s, input_fired_time: 0.1951620132, output_fired_time: 0.0000000000, propagate_forward_time: 7.6782274246, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5478868484
validation[          , genome  7138] predictions:    9587/  10000 (95.87%), best:    9756/10000 (97.56%), error:      1374.61523, best error:       795.33893 on epoch:    66, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 111.8723678589s, input_fired_time: 1.7107386589, output_fired_time: 1.5385375023, propagate_forward_time: 40.8331718445, propagate_backward_time: 64.5824737549, weight_update_time: 0.1265810579, other_time: 3.2074432373
epoch time: 8.9421977997s, input_fired_time: 0.2637878358, output_fired_time: 0.0000000000, propagate_forward_time: 8.1171684265, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5612411499
validation[          , genome  7138] predictions:    9651/  10000 (96.51%), best:    9756/10000 (97.56%), error:      1188.41870, best error:       795.33893 on epoch:    66, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 111.1912078857s, input_fired_time: 1.7193710804, output_fired_time: 1.4815081358, propagate_forward_time: 40.5121879578, propagate_backward_time: 64.3911514282, weight_update_time: 0.1239217445, other_time: 3.0869979858
epoch time: 8.8839807510s, input_fired_time: 0.1905769557, output_fired_time: 0.0000000000, propagate_forward_time: 8.0767507553, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6166534424
validation[          , genome  7138] predictions:    9639/  10000 (96.39%), best:    9756/10000 (97.56%), error:      1211.16943, best error:       795.33893 on epoch:    66, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 109.1528472900s, input_fired_time: 1.6001409292, output_fired_time: 1.4986042976, propagate_forward_time: 40.1857223511, propagate_backward_time: 62.8721160889, weight_update_time: 0.1232786477, other_time: 2.9962615967
epoch time: 8.3025217056s, input_fired_time: 0.1866513640, output_fired_time: 0.0000000000, propagate_forward_time: 7.5402131081, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5756573677
validation[          , genome  7138] predictions:    9551/  10000 (95.51%), best:    9756/10000 (97.56%), error:      1464.56616, best error:       795.33893 on epoch:    66, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 171.1648864746s, input_fired_time: 2.5769507885, output_fired_time: 1.7934753895, propagate_forward_time: 64.3971633911, propagate_backward_time: 97.8615798950, weight_update_time: 0.1852201819, other_time: 4.5357208252
epoch time: 16.2628135681s, input_fired_time: 0.4685638249, output_fired_time: 0.0000000000, propagate_forward_time: 14.9743576050, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8198919296
validation[          , genome  7138] predictions:    9598/  10000 (95.98%), best:    9756/10000 (97.56%), error:      1294.49280, best error:       795.33893 on epoch:    66, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 173.8650512695s, input_fired_time: 1.9446692467, output_fired_time: 2.3263554573, propagate_forward_time: 64.5972518921, propagate_backward_time: 99.6522140503, weight_update_time: 0.1853948236, other_time: 5.3445587158
epoch time: 14.5961685181s, input_fired_time: 0.2642692924, output_fired_time: 0.0000000000, propagate_forward_time: 13.1149282455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2169713974
validation[          , genome  7138] predictions:    9607/  10000 (96.07%), best:    9756/10000 (97.56%), error:      1332.17615, best error:       795.33893 on epoch:    66, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 178.2741394043s, input_fired_time: 2.4032571316, output_fired_time: 2.7487783432, propagate_forward_time: 67.1911315918, propagate_backward_time: 101.2503662109, weight_update_time: 0.1387370527, other_time: 4.6806030273
epoch time: 13.9845952988s, input_fired_time: 0.2175305635, output_fired_time: 0.0000000000, propagate_forward_time: 12.8644819260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9025831223
validation[          , genome  7138] predictions:    9613/  10000 (96.13%), best:    9756/10000 (97.56%), error:      1294.91760, best error:       795.33893 on epoch:    66, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 181.0287170410s, input_fired_time: 2.4648642540, output_fired_time: 1.7515707016, propagate_forward_time: 67.9113540649, propagate_backward_time: 104.1324234009, weight_update_time: 0.4514113963, other_time: 4.7685089111
epoch time: 14.5105953217s, input_fired_time: 0.4680701792, output_fired_time: 0.0000000000, propagate_forward_time: 13.3903875351, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6521377563
validation[          , genome  7138] predictions:    9568/  10000 (95.68%), best:    9756/10000 (97.56%), error:      1455.47339, best error:       795.33893 on epoch:    66, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 178.5543365479s, input_fired_time: 2.6777839661, output_fired_time: 2.0647153854, propagate_forward_time: 66.4496154785, propagate_backward_time: 102.7468490601, weight_update_time: 0.1687176973, other_time: 4.6153793335
epoch time: 13.3351116180s, input_fired_time: 0.2627962530, output_fired_time: 0.0000000000, propagate_forward_time: 11.9352807999, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1370344162
validation[          , genome  7138] predictions:    9628/  10000 (96.28%), best:    9756/10000 (97.56%), error:      1227.71521, best error:       795.33893 on epoch:    66, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 166.4969482422s, input_fired_time: 2.7382874489, output_fired_time: 1.8735954762, propagate_forward_time: 60.8807449341, propagate_backward_time: 97.1039276123, weight_update_time: 0.1387109309, other_time: 3.9003982544
epoch time: 13.8770084381s, input_fired_time: 0.2181956768, output_fired_time: 0.0000000000, propagate_forward_time: 12.7551259995, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9036865234
validation[          , genome  7138] predictions:    9657/  10000 (96.57%), best:    9756/10000 (97.56%), error:      1156.51599, best error:       795.33893 on epoch:    66, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 166.9720611572s, input_fired_time: 2.8825256824, output_fired_time: 1.9596096277, propagate_forward_time: 61.9385452271, propagate_backward_time: 95.4623336792, weight_update_time: 0.1389029175, other_time: 4.7290496826
epoch time: 13.7835788727s, input_fired_time: 0.2182299793, output_fired_time: 0.0000000000, propagate_forward_time: 12.9587764740, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6065721512
validation[          , genome  7138] predictions:    9651/  10000 (96.51%), best:    9756/10000 (97.56%), error:      1161.40161, best error:       795.33893 on epoch:    66, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 166.9407043457s, input_fired_time: 1.8258155584, output_fired_time: 2.7502326965, propagate_forward_time: 61.7985382080, propagate_backward_time: 95.3435668945, weight_update_time: 0.4046626687, other_time: 5.2225494385
epoch time: 14.1261644363s, input_fired_time: 0.2186288387, output_fired_time: 0.0000000000, propagate_forward_time: 13.2449436188, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6625919342
validation[          , genome  7138] predictions:    9594/  10000 (95.94%), best:    9756/10000 (97.56%), error:      1362.56848, best error:       795.33893 on epoch:    66, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 173.0277709961s, input_fired_time: 2.0567152500, output_fired_time: 2.4067244530, propagate_forward_time: 65.2390365601, propagate_backward_time: 98.3118896484, weight_update_time: 0.1388537884, other_time: 5.0134048462
epoch time: 14.3822546005s, input_fired_time: 0.2178342193, output_fired_time: 0.0000000000, propagate_forward_time: 13.5088605881, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6555595398
validation[          , genome  7138] predictions:    9499/  10000 (94.99%), best:    9756/10000 (97.56%), error:      1660.33740, best error:       795.33893 on epoch:    66, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 170.3224029541s, input_fired_time: 2.6233937740, output_fired_time: 2.0497362614, propagate_forward_time: 62.2238235474, propagate_backward_time: 98.5719833374, weight_update_time: 0.1387902647, other_time: 4.8534545898
epoch time: 14.8281412125s, input_fired_time: 0.2973628640, output_fired_time: 0.0000000000, propagate_forward_time: 13.7158641815, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8149137497
validation[          , genome  7138] predictions:    9564/  10000 (95.64%), best:    9756/10000 (97.56%), error:      1402.56213, best error:       795.33893 on epoch:    66, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 176.7618560791s, input_fired_time: 2.6887171268, output_fired_time: 2.3648326397, propagate_forward_time: 64.4011764526, propagate_backward_time: 101.7963485718, weight_update_time: 0.4202044010, other_time: 5.5107727051
epoch time: 13.9575557709s, input_fired_time: 0.2191823274, output_fired_time: 0.0000000000, propagate_forward_time: 12.8792676926, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8591060638
validation[          , genome  7138] predictions:    9638/  10000 (96.38%), best:    9756/10000 (97.56%), error:      1203.42651, best error:       795.33893 on epoch:    66, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 179.2205810547s, input_fired_time: 2.4366166592, output_fired_time: 2.7508828640, propagate_forward_time: 66.9253005981, propagate_backward_time: 102.5340805054, weight_update_time: 0.1702670455, other_time: 4.5736999512
epoch time: 15.1693477631s, input_fired_time: 0.4694835544, output_fired_time: 0.0000000000, propagate_forward_time: 13.9372892380, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7625751495
validation[          , genome  7138] predictions:    9648/  10000 (96.48%), best:    9756/10000 (97.56%), error:      1209.17358, best error:       795.33893 on epoch:    66, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 172.1299438477s, input_fired_time: 2.8638739586, output_fired_time: 2.6706447601, propagate_forward_time: 64.9348220825, propagate_backward_time: 96.5806884766, weight_update_time: 0.1388466209, other_time: 5.0799179077
epoch time: 13.0875749588s, input_fired_time: 0.2341821194, output_fired_time: 0.0000000000, propagate_forward_time: 12.2456607819, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6077318192
validation[          , genome  7138] predictions:    9648/  10000 (96.48%), best:    9756/10000 (97.56%), error:      1182.96680, best error:       795.33893 on epoch:    66, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 166.7194671631s, input_fired_time: 2.0370895863, output_fired_time: 1.7654591799, propagate_forward_time: 61.3631782532, propagate_backward_time: 96.3645477295, weight_update_time: 0.1386945248, other_time: 5.1891937256
epoch time: 13.9327468872s, input_fired_time: 0.2482939065, output_fired_time: 0.0000000000, propagate_forward_time: 13.0326032639, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6518497467
validation[          , genome  7138] predictions:    9668/  10000 (96.68%), best:    9756/10000 (97.56%), error:      1097.90479, best error:       795.33893 on epoch:    66, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 166.1854553223s, input_fired_time: 2.2560958862, output_fired_time: 2.2842459679, propagate_forward_time: 61.5535621643, propagate_backward_time: 94.8457031250, weight_update_time: 0.1548576951, other_time: 5.2458496094
epoch time: 14.1363658905s, input_fired_time: 0.2165229619, output_fired_time: 0.0000000000, propagate_forward_time: 12.9671421051, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9527006149
validation[          , genome  7138] predictions:    9668/  10000 (96.68%), best:    9756/10000 (97.56%), error:      1109.53479, best error:       795.33893 on epoch:    66, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 167.0348968506s, input_fired_time: 1.8099668026, output_fired_time: 2.1400461197, propagate_forward_time: 61.9839019775, propagate_backward_time: 96.8610229492, weight_update_time: 0.1859353632, other_time: 4.2399597168
epoch time: 13.7758321762s, input_fired_time: 0.2497036010, output_fired_time: 0.0000000000, propagate_forward_time: 12.6672515869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8588771820
validation[          , genome  7138] predictions:    9673/  10000 (96.73%), best:    9756/10000 (97.56%), error:      1107.25525, best error:       795.33893 on epoch:    66, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 173.9514160156s, input_fired_time: 2.3605020046, output_fired_time: 2.0504624844, propagate_forward_time: 64.7228469849, propagate_backward_time: 99.6389770508, weight_update_time: 0.1386842728, other_time: 5.1786270142
epoch time: 20.7568550110s, input_fired_time: 1.1222040653, output_fired_time: 0.0000000000, propagate_forward_time: 18.4022617340, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2323894501
validation[          , genome  7138] predictions:    9677/  10000 (96.77%), best:    9756/10000 (97.56%), error:      1060.37305, best error:       795.33893 on epoch:    66, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 204.5454406738s, input_fired_time: 3.5178236961, output_fired_time: 2.4397838116, propagate_forward_time: 77.9487762451, propagate_backward_time: 115.6371307373, weight_update_time: 0.1860411912, other_time: 5.0019226074
epoch time: 15.3407497406s, input_fired_time: 0.2610967159, output_fired_time: 0.0000000000, propagate_forward_time: 14.3751935959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7044591904
validation[          , genome  7138] predictions:    9670/  10000 (96.70%), best:    9756/10000 (97.56%), error:      1091.12231, best error:       795.33893 on epoch:    66, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 183.4748687744s, input_fired_time: 2.8885164261, output_fired_time: 1.9618847370, propagate_forward_time: 68.0364837646, propagate_backward_time: 105.6138153076, weight_update_time: 0.1414127499, other_time: 4.9741668701
epoch time: 15.9389190674s, input_fired_time: 0.5891129971, output_fired_time: 0.0000000000, propagate_forward_time: 14.4459171295, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9038887024
validation[          , genome  7138] predictions:    9593/  10000 (95.93%), best:    9756/10000 (97.56%), error:      1335.61841, best error:       795.33893 on epoch:    66, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 171.1151275635s, input_fired_time: 2.3079888821, output_fired_time: 1.9624173641, propagate_forward_time: 63.8116073608, propagate_backward_time: 97.6768875122, weight_update_time: 0.2326660901, other_time: 5.3562316895
epoch time: 13.9793071747s, input_fired_time: 0.2599395514, output_fired_time: 0.0000000000, propagate_forward_time: 12.3482990265, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3710689545
validation[          , genome  7138] predictions:    9647/  10000 (96.47%), best:    9756/10000 (97.56%), error:      1182.45093, best error:       795.33893 on epoch:    66, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 67.6377182007s, input_fired_time: 2.1229979992, output_fired_time: 0.0000000000, propagate_forward_time: 61.5037612915, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0109558105
best training[          , genome  7138] predictions:   49418/  50000 (98.84%), best:    9756/10000 (97.56%), error:      2110.45459, best error:       795.33893 on epoch:    66, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 13.3200120926s, input_fired_time: 0.2697778046, output_fired_time: 0.0000000000, propagate_forward_time: 12.4424047470, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6078290939
   testing[          , genome  7138] predictions:    9771/  10000 (97.71%), best:    9756/10000 (97.56%), error:       720.01038, best error:       795.33893 on epoch:    66, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
17:26:47 (10284): called boinc_finish(0)

</stderr_txt>
]]>