Nameexact_genome_1512568488_44_5297_1
Workunit2597785
Created7 Dec 2017, 14:09:11 UTC
Sent7 Dec 2017, 14:17:22 UTC
Report deadline13 Dec 2017, 8:24:51 UTC
Received8 Dec 2017, 0:21:48 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID55108
Run time3 hours 18 min 16 sec
CPU time3 hours 11 min 5 sec
Validate stateValid
Credit1,438.80
Device peak FLOPS2.57 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size155.56 MB
Peak swap size158.71 MB
Peak disk usage1.83 MB

Stderr output

<core_client_version>7.8.2</core_client_version>
<![CDATA[
<stderr_txt>
e  5297] predictions:    9677/  10000 (96.77%), best:    9677/10000 (96.77%), error:      1097.49634, best error:      1097.49634 on epoch:    14, epoch:   14/100, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 110.6115951538s, input_fired_time: 1.3442153931, output_fired_time: 1.2428330183, propagate_forward_time: 41.9570465088, propagate_backward_time: 62.8188133240, weight_update_time: 0.1395141929, other_time: 3.2486839294
epoch time: 8.3506498337s, input_fired_time: 0.1626075953, output_fired_time: 0.0000000000, propagate_forward_time: 7.6220669746, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5659756660
validation[          , genome  5297] predictions:    9501/  10000 (95.01%), best:    9677/10000 (96.77%), error:      1560.83704, best error:      1097.49634 on epoch:    14, epoch:   15/100, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 117.1068420410s, input_fired_time: 1.3976249695, output_fired_time: 1.3197497129, propagate_forward_time: 42.4731712341, propagate_backward_time: 68.4900131226, weight_update_time: 0.1495559067, other_time: 3.4262771606
epoch time: 9.3905963898s, input_fired_time: 0.1795198768, output_fired_time: 0.0000000000, propagate_forward_time: 8.6411790848, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5698976517
validation[          , genome  5297] predictions:    9553/  10000 (95.53%), best:    9677/10000 (96.77%), error:      1463.80151, best error:      1097.49634 on epoch:    14, epoch:   16/100, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 109.4928131104s, input_fired_time: 1.3434754610, output_fired_time: 1.2433211803, propagate_forward_time: 38.9248275757, propagate_backward_time: 64.7532424927, weight_update_time: 0.1393197030, other_time: 3.2279434204
epoch time: 9.6111688614s, input_fired_time: 0.1691976339, output_fired_time: 0.0000000000, propagate_forward_time: 8.8683032990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5736675262
validation[          , genome  5297] predictions:    9625/  10000 (96.25%), best:    9677/10000 (96.77%), error:      1272.12268, best error:      1097.49634 on epoch:    14, epoch:   17/100, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 121.4240646362s, input_fired_time: 1.5030339956, output_fired_time: 1.3485797644, propagate_forward_time: 45.8936996460, propagate_backward_time: 69.1580963135, weight_update_time: 0.1628054231, other_time: 3.5206527710
epoch time: 8.4106292725s, input_fired_time: 0.1612045169, output_fired_time: 0.0000000000, propagate_forward_time: 7.6784138680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5710110664
validation[          , genome  5297] predictions:    9594/  10000 (95.94%), best:    9677/10000 (96.77%), error:      1329.13818, best error:      1097.49634 on epoch:    14, epoch:   18/100, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 133.3604431152s, input_fired_time: 1.5685844421, output_fired_time: 1.5606634617, propagate_forward_time: 49.9640960693, propagate_backward_time: 76.3305740356, weight_update_time: 0.1521501690, other_time: 3.9365158081
epoch time: 8.4005279541s, input_fired_time: 0.1652988344, output_fired_time: 0.0000000000, propagate_forward_time: 7.6679596901, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5672698021
validation[          , genome  5297] predictions:    9601/  10000 (96.01%), best:    9677/10000 (96.77%), error:      1232.98389, best error:      1097.49634 on epoch:    14, epoch:   19/100, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 110.0708618164s, input_fired_time: 1.3462724686, output_fired_time: 1.2441084385, propagate_forward_time: 40.9204139709, propagate_backward_time: 63.2847099304, weight_update_time: 0.1385866255, other_time: 3.2753486633
epoch time: 8.3484992981s, input_fired_time: 0.1633406132, output_fired_time: 0.0000000000, propagate_forward_time: 7.6306486130, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5545101166
validation[          , genome  5297] predictions:    9505/  10000 (95.05%), best:    9677/10000 (96.77%), error:      1599.19409, best error:      1097.49634 on epoch:    14, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 117.5412445068s, input_fired_time: 1.4239355326, output_fired_time: 1.3270874023, propagate_forward_time: 41.7389411926, propagate_backward_time: 69.5983276367, weight_update_time: 0.1397116482, other_time: 3.4529571533
epoch time: 8.4800033569s, input_fired_time: 0.1627428979, output_fired_time: 0.0000000000, propagate_forward_time: 7.7468848228, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5703759193
validation[          , genome  5297] predictions:    9645/  10000 (96.45%), best:    9677/10000 (96.77%), error:      1252.07886, best error:      1097.49634 on epoch:    14, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 115.7087631226s, input_fired_time: 1.4363297224, output_fired_time: 1.3557043076, propagate_forward_time: 43.0955581665, propagate_backward_time: 66.3288497925, weight_update_time: 0.1473261267, other_time: 3.4923171997
epoch time: 9.7814788818s, input_fired_time: 0.1662434936, output_fired_time: 0.0000000000, propagate_forward_time: 9.0416212082, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5736141205
validation[          , genome  5297] predictions:    9678/  10000 (96.78%), best:    9678/10000 (96.78%), error:      1079.68616, best error:      1079.68616 on epoch:    22, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 110.9795455933s, input_fired_time: 2.4461538792, output_fired_time: 1.2514107227, propagate_forward_time: 40.3361053467, propagate_backward_time: 63.6553039551, weight_update_time: 0.1411484331, other_time: 3.2905731201
epoch time: 9.6514034271s, input_fired_time: 0.1691678762, output_fired_time: 0.0000000000, propagate_forward_time: 8.8932828903, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5889530182
validation[          , genome  5297] predictions:    9594/  10000 (95.94%), best:    9678/10000 (96.78%), error:      1362.32043, best error:      1079.68616 on epoch:    22, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 109.5478286743s, input_fired_time: 1.3614937067, output_fired_time: 1.2460696697, propagate_forward_time: 42.0057678223, propagate_backward_time: 61.6820945740, weight_update_time: 0.1415055990, other_time: 3.2523994446
epoch time: 9.6795501709s, input_fired_time: 0.1680487096, output_fired_time: 0.0000000000, propagate_forward_time: 8.9277601242, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5837411880
validation[          , genome  5297] predictions:    9669/  10000 (96.69%), best:    9678/10000 (96.78%), error:      1118.39868, best error:      1079.68616 on epoch:    22, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 109.5992584229s, input_fired_time: 1.3538650274, output_fired_time: 1.2417619228, propagate_forward_time: 41.9756278992, propagate_backward_time: 61.7555084229, weight_update_time: 0.1402004212, other_time: 3.2724914551
epoch time: 14.5780696869s, input_fired_time: 0.2040007710, output_fired_time: 0.0000000000, propagate_forward_time: 13.4440383911, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9300308228
validation[          , genome  5297] predictions:    9679/  10000 (96.79%), best:    9678/10000 (96.78%), error:      1092.22388, best error:      1079.68616 on epoch:    22, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 111.8188171387s, input_fired_time: 1.3521374464, output_fired_time: 2.2581763268, propagate_forward_time: 41.2586250305, propagate_backward_time: 62.6964797974, weight_update_time: 0.1406499892, other_time: 4.2533950806
epoch time: 11.3107318878s, input_fired_time: 0.2211247087, output_fired_time: 0.0000000000, propagate_forward_time: 10.2866840363, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8029232025
validation[          , genome  5297] predictions:    9487/  10000 (94.87%), best:    9678/10000 (96.78%), error:      1722.53528, best error:      1079.68616 on epoch:    22, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 112.6015548706s, input_fired_time: 1.3812274933, output_fired_time: 1.2663922310, propagate_forward_time: 41.8747825623, propagate_backward_time: 64.7902145386, weight_update_time: 0.1391442120, other_time: 3.2889328003
epoch time: 8.4456100464s, input_fired_time: 0.1642960608, output_fired_time: 0.0000000000, propagate_forward_time: 7.7223887444, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5589251518
validation[          , genome  5297] predictions:    9658/  10000 (96.58%), best:    9678/10000 (96.78%), error:      1150.89478, best error:      1079.68616 on epoch:    22, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 110.4632568359s, input_fired_time: 1.3430334330, output_fired_time: 1.2438365221, propagate_forward_time: 40.8316688538, propagate_backward_time: 63.7630386353, weight_update_time: 0.1592394710, other_time: 3.2816848755
epoch time: 8.3154697418s, input_fired_time: 0.1681030244, output_fired_time: 0.0000000000, propagate_forward_time: 7.5720186234, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5753479004
validation[          , genome  5297] predictions:    9419/  10000 (94.19%), best:    9678/10000 (96.78%), error:      1856.14844, best error:      1079.68616 on epoch:    22, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 110.8920669556s, input_fired_time: 1.3520687819, output_fired_time: 1.2478318214, propagate_forward_time: 40.9854125977, propagate_backward_time: 64.0423583984, weight_update_time: 0.1400635540, other_time: 3.2643966675
epoch time: 8.4432373047s, input_fired_time: 0.1636367291, output_fired_time: 0.0000000000, propagate_forward_time: 7.7115387917, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5680613518
validation[          , genome  5297] predictions:    9648/  10000 (96.48%), best:    9678/10000 (96.78%), error:      1199.02930, best error:      1079.68616 on epoch:    22, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 110.9732818604s, input_fired_time: 1.3804855347, output_fired_time: 1.2526309490, propagate_forward_time: 40.2039680481, propagate_backward_time: 64.8864593506, weight_update_time: 0.1399525404, other_time: 3.2497406006
epoch time: 8.4032087326s, input_fired_time: 0.1648630053, output_fired_time: 0.0000000000, propagate_forward_time: 7.6600546837, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5782914162
validation[          , genome  5297] predictions:    9543/  10000 (95.43%), best:    9678/10000 (96.78%), error:      1543.76147, best error:      1079.68616 on epoch:    22, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 110.8113250732s, input_fired_time: 1.3512153625, output_fired_time: 1.2464247942, propagate_forward_time: 39.0421066284, propagate_backward_time: 65.9272918701, weight_update_time: 0.1400644332, other_time: 3.2442932129
epoch time: 8.4267234802s, input_fired_time: 0.1649909467, output_fired_time: 0.0000000000, propagate_forward_time: 7.6881909370, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5735411644
validation[          , genome  5297] predictions:    9568/  10000 (95.68%), best:    9678/10000 (96.78%), error:      1578.99731, best error:      1079.68616 on epoch:    22, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 110.7444534302s, input_fired_time: 1.3621957302, output_fired_time: 1.2513862848, propagate_forward_time: 41.9692611694, propagate_backward_time: 62.8415222168, weight_update_time: 0.1412764788, other_time: 3.3200836182
epoch time: 8.3404693604s, input_fired_time: 0.1619654149, output_fired_time: 0.0000000000, propagate_forward_time: 7.6141557693, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5643482208
validation[          , genome  5297] predictions:    9680/  10000 (96.80%), best:    9680/10000 (96.80%), error:      1045.40869, best error:      1045.40869 on epoch:    32, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 111.2696075439s, input_fired_time: 1.3542758226, output_fired_time: 1.2475659847, propagate_forward_time: 41.3834266663, propagate_backward_time: 64.0523071289, weight_update_time: 0.1395620555, other_time: 3.2320251465
epoch time: 8.3877716064s, input_fired_time: 0.1654733270, output_fired_time: 0.0000000000, propagate_forward_time: 7.6604609489, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5618376732
validation[          , genome  5297] predictions:    9682/  10000 (96.82%), best:    9680/10000 (96.80%), error:      1087.76868, best error:      1045.40869 on epoch:    32, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 110.8062133789s, input_fired_time: 1.3578618765, output_fired_time: 1.2634348869, propagate_forward_time: 43.0163803101, propagate_backward_time: 61.9100418091, weight_update_time: 0.1397868693, other_time: 3.2584915161
epoch time: 8.4863071442s, input_fired_time: 0.1652661264, output_fired_time: 0.0000000000, propagate_forward_time: 7.7450332642, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5760078430
validation[          , genome  5297] predictions:    9688/  10000 (96.88%), best:    9688/10000 (96.88%), error:      1042.79175, best error:      1042.79175 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 110.7281265259s, input_fired_time: 2.3577585220, output_fired_time: 1.2494864464, propagate_forward_time: 43.9216499329, propagate_backward_time: 59.9334907532, weight_update_time: 0.1398362666, other_time: 3.2657394409
epoch time: 8.3195419312s, input_fired_time: 0.1641009897, output_fired_time: 0.0000000000, propagate_forward_time: 7.5855460167, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5698952675
validation[          , genome  5297] predictions:    9670/  10000 (96.70%), best:    9688/10000 (96.88%), error:      1083.34753, best error:      1042.79175 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 112.7636413574s, input_fired_time: 1.3744047880, output_fired_time: 1.2720913887, propagate_forward_time: 41.8733634949, propagate_backward_time: 64.9686203003, weight_update_time: 0.1406254917, other_time: 3.2751541138
epoch time: 8.5130290985s, input_fired_time: 0.1692905575, output_fired_time: 0.0000000000, propagate_forward_time: 7.7555980682, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5881404877
validation[          , genome  5297] predictions:    9713/  10000 (97.13%), best:    9713/10000 (97.13%), error:       988.39417, best error:       988.39417 on epoch:    36, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 117.9903869629s, input_fired_time: 1.4476561546, output_fired_time: 1.3398873806, propagate_forward_time: 44.3093986511, propagate_backward_time: 67.3882522583, weight_update_time: 0.1488409936, other_time: 3.5051956177
epoch time: 9.6223173141s, input_fired_time: 0.1694879532, output_fired_time: 0.0000000000, propagate_forward_time: 8.8696432114, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5831861496
validation[          , genome  5297] predictions:    9633/  10000 (96.33%), best:    9713/10000 (97.13%), error:      1183.02783, best error:       988.39417 on epoch:    36, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 109.6199035645s, input_fired_time: 1.3507779837, output_fired_time: 1.2421929836, propagate_forward_time: 39.9505805969, propagate_backward_time: 63.7898864746, weight_update_time: 0.1401222199, other_time: 3.2864685059
epoch time: 9.7269468307s, input_fired_time: 0.1740836203, output_fired_time: 0.0000000000, propagate_forward_time: 8.9611034393, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5917596817
validation[          , genome  5297] predictions:    9660/  10000 (96.60%), best:    9713/10000 (97.13%), error:      1193.65967, best error:       988.39417 on epoch:    36, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 108.5184783936s, input_fired_time: 1.3798305988, output_fired_time: 1.2490987778, propagate_forward_time: 41.6521110535, propagate_backward_time: 60.9717559814, weight_update_time: 0.1393583715, other_time: 3.2656860352
epoch time: 9.6536912918s, input_fired_time: 0.1701662242, output_fired_time: 0.0000000000, propagate_forward_time: 8.9047479630, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5787773132
validation[          , genome  5297] predictions:    9653/  10000 (96.53%), best:    9713/10000 (97.13%), error:      1165.79968, best error:       988.39417 on epoch:    36, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 109.6880950928s, input_fired_time: 1.3535426855, output_fired_time: 1.2549103498, propagate_forward_time: 40.0015029907, propagate_backward_time: 62.8077087402, weight_update_time: 0.1396652609, other_time: 4.2704238892
epoch time: 9.7558498383s, input_fired_time: 0.1733496636, output_fired_time: 0.0000000000, propagate_forward_time: 8.9947576523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5877428055
validation[          , genome  5297] predictions:    9696/  10000 (96.96%), best:    9713/10000 (97.13%), error:      1045.18555, best error:       988.39417 on epoch:    36, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 110.1813049316s, input_fired_time: 1.3630198240, output_fired_time: 1.2548187971, propagate_forward_time: 42.0827178955, propagate_backward_time: 62.1625633240, weight_update_time: 0.1413601190, other_time: 3.3181800842
epoch time: 9.5950040817s, input_fired_time: 0.1724690646, output_fired_time: 0.0000000000, propagate_forward_time: 8.8299226761, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5926122665
validation[          , genome  5297] predictions:    9709/  10000 (97.09%), best:    9709/10000 (97.09%), error:       963.22363, best error:       963.22363 on epoch:    41, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 124.0869064331s, input_fired_time: 1.4701685905, output_fired_time: 1.3942116499, propagate_forward_time: 46.8718452454, propagate_backward_time: 70.6966476440, weight_update_time: 0.1489858329, other_time: 3.6540298462
epoch time: 8.4511079788s, input_fired_time: 0.1731516421, output_fired_time: 0.0000000000, propagate_forward_time: 7.7071967125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5707592964
validation[          , genome  5297] predictions:    9637/  10000 (96.37%), best:    9709/10000 (97.09%), error:      1214.19482, best error:       963.22363 on epoch:    41, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 110.3780593872s, input_fired_time: 1.3475078344, output_fired_time: 1.2489016056, propagate_forward_time: 39.8684120178, propagate_backward_time: 64.6767730713, weight_update_time: 0.1389869004, other_time: 3.2364654541
epoch time: 8.4729185104s, input_fired_time: 0.1664145440, output_fired_time: 0.0000000000, propagate_forward_time: 7.7258877754, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5806164742
validation[          , genome  5297] predictions:    9472/  10000 (94.72%), best:    9709/10000 (97.09%), error:      1701.28040, best error:       963.22363 on epoch:    41, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 110.1035842896s, input_fired_time: 1.3792653084, output_fired_time: 1.2395086288, propagate_forward_time: 41.6383247375, propagate_backward_time: 62.5872688293, weight_update_time: 0.1396984458, other_time: 3.2592124939
epoch time: 9.1853370667s, input_fired_time: 0.1629169732, output_fired_time: 0.0000000000, propagate_forward_time: 8.4217414856, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6006784439
validation[          , genome  5297] predictions:    9724/  10000 (97.24%), best:    9724/10000 (97.24%), error:       942.72833, best error:       942.72833 on epoch:    44, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 119.3531341553s, input_fired_time: 1.5509401560, output_fired_time: 1.3588997126, propagate_forward_time: 42.7048759460, propagate_backward_time: 70.1660308838, weight_update_time: 0.1656996906, other_time: 3.5723876953
epoch time: 15.3157739639s, input_fired_time: 0.2690477669, output_fired_time: 0.0000000000, propagate_forward_time: 14.1191120148, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9276142120
validation[          , genome  5297] predictions:    9368/  10000 (93.68%), best:    9724/10000 (97.24%), error:      2169.68359, best error:       942.72833 on epoch:    44, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 117.1142959595s, input_fired_time: 1.4388241768, output_fired_time: 1.3254020214, propagate_forward_time: 44.7094154358, propagate_backward_time: 66.2306594849, weight_update_time: 0.1508279741, other_time: 3.4099960327
epoch time: 8.4275112152s, input_fired_time: 0.1660103500, output_fired_time: 0.0000000000, propagate_forward_time: 7.6804485321, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5810527802
validation[          , genome  5297] predictions:    9694/  10000 (96.94%), best:    9724/10000 (97.24%), error:       981.69861, best error:       942.72833 on epoch:    44, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 111.0038986206s, input_fired_time: 1.3699697256, output_fired_time: 1.2511899471, propagate_forward_time: 40.1803398132, propagate_backward_time: 64.9308166504, weight_update_time: 0.1408258826, other_time: 3.2715759277
epoch time: 8.2835855484s, input_fired_time: 0.1641836166, output_fired_time: 0.0000000000, propagate_forward_time: 7.5607590675, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5586428642
validation[          , genome  5297] predictions:    9110/  10000 (91.10%), best:    9724/10000 (97.24%), error:      3388.95044, best error:       942.72833 on epoch:    44, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

resetting weights to those on epoch: 44
epoch time: 125.5149841309s, input_fired_time: 1.5371161699, output_fired_time: 1.3454710245, propagate_forward_time: 46.6071014404, propagate_backward_time: 72.1478958130, weight_update_time: 0.1718485206, other_time: 3.8773956299
epoch time: 10.2622957230s, input_fired_time: 0.1811164618, output_fired_time: 0.0000000000, propagate_forward_time: 9.3267536163, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7544260025
validation[          , genome  5297] predictions:    8891/  10000 (88.91%), best:    9724/10000 (97.24%), error:      3530.68848, best error:       942.72833 on epoch:    44, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

resetting weights to those on epoch: 44
epoch time: 111.0149078369s, input_fired_time: 1.3761463165, output_fired_time: 1.2490222454, propagate_forward_time: 41.1397361755, propagate_backward_time: 63.9320411682, weight_update_time: 0.1410794705, other_time: 3.3179588318
epoch time: 8.4434871674s, input_fired_time: 0.1671911031, output_fired_time: 0.0000000000, propagate_forward_time: 7.6983861923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5779094696
validation[          , genome  5297] predictions:    9614/  10000 (96.14%), best:    9724/10000 (97.24%), error:      1256.75256, best error:       942.72833 on epoch:    44, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 110.6523895264s, input_fired_time: 1.3596472740, output_fired_time: 1.2470474243, propagate_forward_time: 41.9378509521, propagate_backward_time: 62.8199310303, weight_update_time: 0.1409847885, other_time: 3.2879104614
epoch time: 8.4771642685s, input_fired_time: 0.1672257483, output_fired_time: 0.0000000000, propagate_forward_time: 7.7222480774, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5876903534
validation[          , genome  5297] predictions:    9696/  10000 (96.96%), best:    9724/10000 (97.24%), error:      1004.75885, best error:       942.72833 on epoch:    44, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 116.1142807007s, input_fired_time: 2.4488289356, output_fired_time: 1.2848542929, propagate_forward_time: 44.3175468445, propagate_backward_time: 64.6823959351, weight_update_time: 0.1402143091, other_time: 3.3806533813
epoch time: 8.4137926102s, input_fired_time: 0.1653032750, output_fired_time: 0.0000000000, propagate_forward_time: 7.6777839661, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5707054138
validation[          , genome  5297] predictions:    9676/  10000 (96.76%), best:    9724/10000 (97.24%), error:      1040.47473, best error:       942.72833 on epoch:    44, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 110.3309631348s, input_fired_time: 1.3490935564, output_fired_time: 1.2396085262, propagate_forward_time: 42.9259681702, propagate_backward_time: 61.5959587097, weight_update_time: 0.1395978332, other_time: 3.2203407288
epoch time: 8.5005788803s, input_fired_time: 0.1668733805, output_fired_time: 0.0000000000, propagate_forward_time: 7.7687554359, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5649504662
validation[          , genome  5297] predictions:    9697/  10000 (96.97%), best:    9724/10000 (97.24%), error:      1043.24438, best error:       942.72833 on epoch:    44, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 110.8960113525s, input_fired_time: 1.3559455872, output_fired_time: 1.2446506023, propagate_forward_time: 42.0537528992, propagate_backward_time: 62.9796218872, weight_update_time: 0.1397962123, other_time: 3.2620468140
epoch time: 8.3994970322s, input_fired_time: 0.1668323427, output_fired_time: 0.0000000000, propagate_forward_time: 7.6607265472, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5719385147
validation[          , genome  5297] predictions:    9731/  10000 (97.31%), best:    9731/10000 (97.31%), error:       894.26312, best error:       894.26312 on epoch:    53, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 112.0929718018s, input_fired_time: 1.3603750467, output_fired_time: 1.2604776621, propagate_forward_time: 42.3784561157, propagate_backward_time: 62.7999458313, weight_update_time: 0.1398461312, other_time: 4.2937202454
epoch time: 12.4597063065s, input_fired_time: 0.1912735403, output_fired_time: 0.0000000000, propagate_forward_time: 11.4960670471, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7723655701
validation[          , genome  5297] predictions:    9616/  10000 (96.16%), best:    9731/10000 (97.31%), error:      1281.85986, best error:       894.26312 on epoch:    53, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 115.1003265381s, input_fired_time: 1.4338563681, output_fired_time: 1.3068085909, propagate_forward_time: 41.1167488098, propagate_backward_time: 67.8788528442, weight_update_time: 0.1420729011, other_time: 3.3640670776
epoch time: 9.7068433762s, input_fired_time: 0.1722451597, output_fired_time: 0.0000000000, propagate_forward_time: 8.9363727570, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5982255936
validation[          , genome  5297] predictions:    9710/  10000 (97.10%), best:    9731/10000 (97.31%), error:       934.01587, best error:       894.26312 on epoch:    53, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 109.3899688721s, input_fired_time: 1.3554908037, output_fired_time: 2.1523509026, propagate_forward_time: 39.9324760437, propagate_backward_time: 62.7249908447, weight_update_time: 0.1399682164, other_time: 3.2246551514
epoch time: 9.7822122574s, input_fired_time: 0.1721239537, output_fired_time: 0.0000000000, propagate_forward_time: 9.0178804398, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5922079086
validation[          , genome  5297] predictions:    9714/  10000 (97.14%), best:    9731/10000 (97.31%), error:       956.49536, best error:       894.26312 on epoch:    53, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 109.2041778564s, input_fired_time: 1.3462871313, output_fired_time: 1.2443214655, propagate_forward_time: 41.7062950134, propagate_backward_time: 61.6460304260, weight_update_time: 0.1397218406, other_time: 3.2612419128
epoch time: 9.7715196609s, input_fired_time: 0.1737127155, output_fired_time: 0.0000000000, propagate_forward_time: 8.9956407547, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6021661758
validation[          , genome  5297] predictions:    8981/  10000 (89.81%), best:    9731/10000 (97.31%), error:      3785.83252, best error:       894.26312 on epoch:    53, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

resetting weights to those on epoch: 53
epoch time: 109.5580520630s, input_fired_time: 1.3506995440, output_fired_time: 1.2426747084, propagate_forward_time: 39.9098167419, propagate_backward_time: 63.7687759399, weight_update_time: 0.1396281570, other_time: 3.2860794067
epoch time: 12.8535842896s, input_fired_time: 0.2276869267, output_fired_time: 0.0000000000, propagate_forward_time: 11.8624839783, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7634134293
validation[          , genome  5297] predictions:    9667/  10000 (96.67%), best:    9731/10000 (97.31%), error:      1105.46497, best error:       894.26312 on epoch:    53, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 110.6110076904s, input_fired_time: 1.3543822765, output_fired_time: 1.2543004751, propagate_forward_time: 40.0799942017, propagate_backward_time: 64.6813201904, weight_update_time: 0.1389556229, other_time: 3.2410049438
epoch time: 8.4649839401s, input_fired_time: 0.1698549688, output_fired_time: 0.0000000000, propagate_forward_time: 7.7183604240, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5767683983
validation[          , genome  5297] predictions:    9635/  10000 (96.35%), best:    9731/10000 (97.31%), error:      1215.31433, best error:       894.26312 on epoch:    53, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 117.6294403076s, input_fired_time: 1.4387136698, output_fired_time: 1.3260104656, propagate_forward_time: 45.5579948425, propagate_backward_time: 65.8563079834, weight_update_time: 0.1510080695, other_time: 3.4504089355
epoch time: 8.3826866150s, input_fired_time: 0.1662239134, output_fired_time: 0.0000000000, propagate_forward_time: 7.6467537880, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5697093010
validation[          , genome  5297] predictions:    9691/  10000 (96.91%), best:    9731/10000 (97.31%), error:       964.94177, best error:       894.26312 on epoch:    53, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 110.2836532593s, input_fired_time: 1.3487634659, output_fired_time: 1.2402890921, propagate_forward_time: 39.8465385437, propagate_backward_time: 63.6011772156, weight_update_time: 0.1479133815, other_time: 4.2468910217
epoch time: 8.5110235214s, input_fired_time: 0.1710072756, output_fired_time: 0.0000000000, propagate_forward_time: 7.7440781593, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5959382057
validation[          , genome  5297] predictions:    9694/  10000 (96.94%), best:    9731/10000 (97.31%), error:      1014.46167, best error:       894.26312 on epoch:    53, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 125.4365005493s, input_fired_time: 1.5246833563, output_fired_time: 1.3889409304, propagate_forward_time: 48.0484046936, propagate_backward_time: 70.8071517944, weight_update_time: 0.1478415877, other_time: 3.6673202515
epoch time: 11.2097425461s, input_fired_time: 0.2342202812, output_fired_time: 0.0000000000, propagate_forward_time: 10.2132310867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7622909546
validation[          , genome  5297] predictions:    9710/  10000 (97.10%), best:    9731/10000 (97.31%), error:      1004.04517, best error:       894.26312 on epoch:    53, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 120.3449401855s, input_fired_time: 1.4221340418, output_fired_time: 1.3808673620, propagate_forward_time: 43.3066902161, propagate_backward_time: 69.7708587646, weight_update_time: 0.1471379250, other_time: 4.4643859863
epoch time: 9.8515443802s, input_fired_time: 0.1722280383, output_fired_time: 0.0000000000, propagate_forward_time: 9.0753097534, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6040067673
validation[          , genome  5297] predictions:    9553/  10000 (95.53%), best:    9731/10000 (97.31%), error:      1550.97144, best error:       894.26312 on epoch:    53, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 108.7917327881s, input_fired_time: 1.3698238134, output_fired_time: 1.2533434629, propagate_forward_time: 38.8772888184, propagate_backward_time: 64.0369338989, weight_update_time: 0.1399418712, other_time: 3.2543411255
epoch time: 9.5962324142s, input_fired_time: 0.1748675108, output_fired_time: 0.0000000000, propagate_forward_time: 8.8446407318, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5767240524
validation[          , genome  5297] predictions:    9672/  10000 (96.72%), best:    9731/10000 (97.31%), error:      1073.54626, best error:       894.26312 on epoch:    53, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 109.1492767334s, input_fired_time: 1.3570821285, output_fired_time: 2.2642512321, propagate_forward_time: 39.7272529602, propagate_backward_time: 61.5564537048, weight_update_time: 0.1391818970, other_time: 4.2442359924
epoch time: 9.7796306610s, input_fired_time: 0.1727529019, output_fired_time: 0.0000000000, propagate_forward_time: 9.0312385559, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5756387711
validation[          , genome  5297] predictions:    9700/  10000 (97.00%), best:    9731/10000 (97.31%), error:       992.03760, best error:       894.26312 on epoch:    53, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 109.7110671997s, input_fired_time: 1.3584822416, output_fired_time: 1.2408634424, propagate_forward_time: 40.0154380798, propagate_backward_time: 62.8281936646, weight_update_time: 0.1398793161, other_time: 4.2680892944
epoch time: 9.7977552414s, input_fired_time: 0.1750664413, output_fired_time: 0.0000000000, propagate_forward_time: 9.0333452225, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5893440247
validation[          , genome  5297] predictions:    9726/  10000 (97.26%), best:    9731/10000 (97.31%), error:       908.17834, best error:       894.26312 on epoch:    53, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 133.7768707275s, input_fired_time: 1.5829445124, output_fired_time: 1.6235027313, propagate_forward_time: 50.7054328918, propagate_backward_time: 76.0180587769, weight_update_time: 0.1500605047, other_time: 3.8469314575
epoch time: 9.6347217560s, input_fired_time: 0.1717613339, output_fired_time: 0.0000000000, propagate_forward_time: 8.8817186356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5812416077
validation[          , genome  5297] predictions:    9617/  10000 (96.17%), best:    9731/10000 (97.31%), error:      1220.46191, best error:       894.26312 on epoch:    53, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 116.0020523071s, input_fired_time: 1.4544277191, output_fired_time: 1.3282802105, propagate_forward_time: 43.1114044189, propagate_backward_time: 66.6967697144, weight_update_time: 0.1487089694, other_time: 3.4111709595
epoch time: 8.2859802246s, input_fired_time: 0.1673317850, output_fired_time: 0.0000000000, propagate_forward_time: 7.5561575890, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5624909401
validation[          , genome  5297] predictions:    9598/  10000 (95.98%), best:    9731/10000 (97.31%), error:      1340.30420, best error:       894.26312 on epoch:    53, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 110.6130294800s, input_fired_time: 1.3605939150, output_fired_time: 1.2458065748, propagate_forward_time: 42.9506568909, propagate_backward_time: 60.8226890564, weight_update_time: 0.1387498379, other_time: 4.2332801819
epoch time: 8.4806251526s, input_fired_time: 0.1664506644, output_fired_time: 0.0000000000, propagate_forward_time: 7.7507076263, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5634670258
validation[          , genome  5297] predictions:    9700/  10000 (97.00%), best:    9731/10000 (97.31%), error:      1037.55212, best error:       894.26312 on epoch:    53, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 110.8913116455s, input_fired_time: 2.3825535774, output_fired_time: 1.2540116310, propagate_forward_time: 41.0811805725, propagate_backward_time: 62.8895187378, weight_update_time: 0.1407344788, other_time: 3.2840499878
epoch time: 8.4855709076s, input_fired_time: 0.1696399152, output_fired_time: 0.0000000000, propagate_forward_time: 7.7321066856, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5838246346
validation[          , genome  5297] predictions:    9713/  10000 (97.13%), best:    9731/10000 (97.31%), error:       955.97687, best error:       894.26312 on epoch:    53, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 113.9526748657s, input_fired_time: 1.3948681355, output_fired_time: 1.3403501511, propagate_forward_time: 44.1498947144, propagate_backward_time: 63.7082099915, weight_update_time: 0.1575953811, other_time: 3.3593559265
epoch time: 8.4942226410s, input_fired_time: 0.1678121686, output_fired_time: 0.0000000000, propagate_forward_time: 7.7592062950, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5672039986
validation[          , genome  5297] predictions:    9644/  10000 (96.44%), best:    9731/10000 (97.31%), error:      1161.70471, best error:       894.26312 on epoch:    53, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 116.3825683594s, input_fired_time: 1.4192209244, output_fired_time: 1.3311753273, propagate_forward_time: 45.3497009277, propagate_backward_time: 64.8888092041, weight_update_time: 0.1443266273, other_time: 3.3936614990
epoch time: 8.4866485596s, input_fired_time: 0.1662942916, output_fired_time: 0.0000000000, propagate_forward_time: 7.6687750816, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6515793800
validation[          , genome  5297] predictions:    9712/  10000 (97.12%), best:    9731/10000 (97.31%), error:       979.03076, best error:       894.26312 on epoch:    53, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 110.9842910767s, input_fired_time: 1.3663395643, output_fired_time: 1.2506225109, propagate_forward_time: 42.1723022461, propagate_backward_time: 62.9358520508, weight_update_time: 0.1402424127, other_time: 3.2591705322
epoch time: 8.2386407852s, input_fired_time: 0.1646131426, output_fired_time: 0.0000000000, propagate_forward_time: 7.5116028786, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5624251366
validation[          , genome  5297] predictions:    9721/  10000 (97.21%), best:    9731/10000 (97.31%), error:       926.50867, best error:       894.26312 on epoch:    53, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 125.8345565796s, input_fired_time: 1.5261169672, output_fired_time: 1.3997710943, propagate_forward_time: 46.3793563843, propagate_backward_time: 72.9070510864, weight_update_time: 0.1774086505, other_time: 3.6222610474
epoch time: 8.4030981064s, input_fired_time: 0.1703074723, output_fired_time: 0.0000000000, propagate_forward_time: 7.6620674133, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5707235336
validation[          , genome  5297] predictions:    9662/  10000 (96.62%), best:    9731/10000 (97.31%), error:      1219.39514, best error:       894.26312 on epoch:    53, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 115.7794418335s, input_fired_time: 1.4289467335, output_fired_time: 1.3250464201, propagate_forward_time: 41.1955718994, propagate_backward_time: 68.4376525879, weight_update_time: 0.1422387958, other_time: 3.3922271729
epoch time: 8.4582090378s, input_fired_time: 0.1705004722, output_fired_time: 0.0000000000, propagate_forward_time: 7.7090401649, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5786681175
validation[          , genome  5297] predictions:    9606/  10000 (96.06%), best:    9731/10000 (97.31%), error:      1296.92529, best error:       894.26312 on epoch:    53, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 111.1850051880s, input_fired_time: 2.3915352821, output_fired_time: 1.2783919573, propagate_forward_time: 43.0748901367, propagate_backward_time: 60.0994682312, weight_update_time: 0.1537235528, other_time: 4.3407249451
epoch time: 8.3439674377s, input_fired_time: 0.1671129167, output_fired_time: 0.0000000000, propagate_forward_time: 7.6068902016, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5699639320
validation[          , genome  5297] predictions:    9704/  10000 (97.04%), best:    9731/10000 (97.31%), error:      1007.78455, best error:       894.26312 on epoch:    53, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 110.3555068970s, input_fired_time: 2.3775482178, output_fired_time: 1.2398825884, propagate_forward_time: 40.8420257568, propagate_backward_time: 62.6462287903, weight_update_time: 0.1392316669, other_time: 3.2498207092
epoch time: 8.4541931152s, input_fired_time: 0.1693119705, output_fired_time: 0.0000000000, propagate_forward_time: 7.7072501183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5776314735
validation[          , genome  5297] predictions:    9714/  10000 (97.14%), best:    9731/10000 (97.31%), error:       956.46539, best error:       894.26312 on epoch:    53, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 121.8940505981s, input_fired_time: 1.4859975576, output_fired_time: 1.3930808306, propagate_forward_time: 46.6845474243, propagate_backward_time: 68.7742767334, weight_update_time: 0.1573963314, other_time: 3.5561447144
epoch time: 9.4875192642s, input_fired_time: 0.1730187833, output_fired_time: 0.0000000000, propagate_forward_time: 7.7337918282, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5807089806
validation[          , genome  5297] predictions:    9706/  10000 (97.06%), best:    9731/10000 (97.31%), error:       945.51874, best error:       894.26312 on epoch:    53, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 109.6714096069s, input_fired_time: 1.3667101860, output_fired_time: 1.2414652109, propagate_forward_time: 41.0908966064, propagate_backward_time: 62.7193679810, weight_update_time: 0.1398548186, other_time: 3.2529754639
epoch time: 12.1920042038s, input_fired_time: 0.2356071621, output_fired_time: 0.0000000000, propagate_forward_time: 11.2215442657, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7348527908
validation[          , genome  5297] predictions:    9721/  10000 (97.21%), best:    9731/10000 (97.31%), error:       913.47589, best error:       894.26312 on epoch:    53, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 118.6830062866s, input_fired_time: 1.4649673700, output_fired_time: 1.2826730013, propagate_forward_time: 45.0906028748, propagate_backward_time: 66.3196182251, weight_update_time: 0.1387288421, other_time: 4.5251388550
epoch time: 8.5234546661s, input_fired_time: 0.1694493145, output_fired_time: 0.0000000000, propagate_forward_time: 7.7751469612, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5788588524
validation[          , genome  5297] predictions:    9676/  10000 (96.76%), best:    9731/10000 (97.31%), error:      1036.48010, best error:       894.26312 on epoch:    53, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 109.4163818359s, input_fired_time: 1.3561364412, output_fired_time: 1.2384783030, propagate_forward_time: 39.8896484375, propagate_backward_time: 63.6874465942, weight_update_time: 0.1384356171, other_time: 3.2446670532
epoch time: 8.2888717651s, input_fired_time: 0.1677418053, output_fired_time: 0.0000000000, propagate_forward_time: 7.5543227196, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5668072701
validation[          , genome  5297] predictions:    9717/  10000 (97.17%), best:    9731/10000 (97.31%), error:       981.50128, best error:       894.26312 on epoch:    53, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 110.9520645142s, input_fired_time: 1.3786947727, output_fired_time: 1.2629109621, propagate_forward_time: 39.1170768738, propagate_backward_time: 63.8952217102, weight_update_time: 0.1404798627, other_time: 5.2981681824
epoch time: 8.5328731537s, input_fired_time: 0.1696360558, output_fired_time: 0.0000000000, propagate_forward_time: 7.7995095253, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5637278557
validation[          , genome  5297] predictions:    9723/  10000 (97.23%), best:    9731/10000 (97.31%), error:       929.10223, best error:       894.26312 on epoch:    53, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 110.8576126099s, input_fired_time: 1.3673970699, output_fired_time: 1.2425209284, propagate_forward_time: 42.0506706238, propagate_backward_time: 61.9406661987, weight_update_time: 0.1401123703, other_time: 4.2563552856
epoch time: 8.1409339905s, input_fired_time: 0.1673015505, output_fired_time: 0.0000000000, propagate_forward_time: 7.4139771461, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5596551895
validation[          , genome  5297] predictions:    9710/  10000 (97.10%), best:    9731/10000 (97.31%), error:       951.68048, best error:       894.26312 on epoch:    53, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 110.8767395020s, input_fired_time: 2.3719196320, output_fired_time: 1.2495186329, propagate_forward_time: 40.9978027344, propagate_backward_time: 63.0139846802, weight_update_time: 0.1415515840, other_time: 3.2435150146
epoch time: 8.3790121078s, input_fired_time: 0.1692421436, output_fired_time: 0.0000000000, propagate_forward_time: 7.6381678581, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5716023445
validation[          , genome  5297] predictions:    9707/  10000 (97.07%), best:    9731/10000 (97.31%), error:      1012.03577, best error:       894.26312 on epoch:    53, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 110.4175491333s, input_fired_time: 1.4748882055, output_fired_time: 1.2514972687, propagate_forward_time: 41.3722686768, propagate_backward_time: 63.0650253296, weight_update_time: 0.1405668706, other_time: 3.2538681030
epoch time: 8.4250831604s, input_fired_time: 0.1709754616, output_fired_time: 0.0000000000, propagate_forward_time: 7.6844401360, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5696673393
validation[          , genome  5297] predictions:    9713/  10000 (97.13%), best:    9731/10000 (97.31%), error:       964.77838, best error:       894.26312 on epoch:    53, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 125.4010162354s, input_fired_time: 2.6054723263, output_fired_time: 1.4157036543, propagate_forward_time: 45.9166946411, propagate_backward_time: 71.6841430664, weight_update_time: 0.1571190804, other_time: 3.7790069580
epoch time: 7.9570837021s, input_fired_time: 0.1612817347, output_fired_time: 0.0000000000, propagate_forward_time: 7.2540378571, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5417642593
validation[          , genome  5297] predictions:    9693/  10000 (96.93%), best:    9731/10000 (97.31%), error:      1082.83582, best error:       894.26312 on epoch:    53, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 111.2595062256s, input_fired_time: 1.3735496998, output_fired_time: 1.2586181164, propagate_forward_time: 42.1896400452, propagate_backward_time: 63.1441841125, weight_update_time: 0.1416616589, other_time: 3.2935142517
epoch time: 8.4468917847s, input_fired_time: 0.1689748615, output_fired_time: 0.0000000000, propagate_forward_time: 7.7078442574, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5700726509
validation[          , genome  5297] predictions:    9689/  10000 (96.89%), best:    9731/10000 (97.31%), error:      1034.26953, best error:       894.26312 on epoch:    53, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 110.0541839600s, input_fired_time: 1.3611050844, output_fired_time: 1.2397807837, propagate_forward_time: 41.7100372314, propagate_backward_time: 62.4863777161, weight_update_time: 0.1394262463, other_time: 3.2568778992
epoch time: 8.5657405853s, input_fired_time: 0.1733359098, output_fired_time: 0.0000000000, propagate_forward_time: 7.8023371696, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5900673866
validation[          , genome  5297] predictions:    9714/  10000 (97.14%), best:    9731/10000 (97.31%), error:       996.11584, best error:       894.26312 on epoch:    53, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 110.3383483887s, input_fired_time: 1.3626009226, output_fired_time: 1.2413926125, propagate_forward_time: 40.9481163025, propagate_backward_time: 63.5470085144, weight_update_time: 0.1395895034, other_time: 3.2392311096
epoch time: 8.4683427811s, input_fired_time: 0.1688682437, output_fired_time: 0.0000000000, propagate_forward_time: 7.7249674797, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5745072365
validation[          , genome  5297] predictions:    9692/  10000 (96.92%), best:    9731/10000 (97.31%), error:       990.97192, best error:       894.26312 on epoch:    53, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 118.9000473022s, input_fired_time: 1.4385172129, output_fired_time: 1.3513677120, propagate_forward_time: 44.2077445984, propagate_backward_time: 68.3855819702, weight_update_time: 0.1486251205, other_time: 3.5168380737
epoch time: 8.2555389404s, input_fired_time: 0.1654209942, output_fired_time: 0.0000000000, propagate_forward_time: 7.5284137726, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5617046356
validation[          , genome  5297] predictions:    9690/  10000 (96.90%), best:    9731/10000 (97.31%), error:      1062.86780, best error:       894.26312 on epoch:    53, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 110.6477508545s, input_fired_time: 1.3657754660, output_fired_time: 1.2452702522, propagate_forward_time: 42.0149345398, propagate_backward_time: 62.7702026367, weight_update_time: 0.1397751421, other_time: 3.2515716553
epoch time: 8.4620685577s, input_fired_time: 0.1689616591, output_fired_time: 0.0000000000, propagate_forward_time: 7.7051815987, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5879254341
validation[          , genome  5297] predictions:    9730/  10000 (97.30%), best:    9731/10000 (97.31%), error:       908.76184, best error:       894.26312 on epoch:    53, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 109.5162429810s, input_fired_time: 1.3711338043, output_fired_time: 1.2484515905, propagate_forward_time: 41.1864547729, propagate_backward_time: 62.4506645203, weight_update_time: 0.1400962323, other_time: 3.2595405579
epoch time: 8.3969287872s, input_fired_time: 0.1683937311, output_fired_time: 0.0000000000, propagate_forward_time: 7.6574048996, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5711297989
validation[          , genome  5297] predictions:    9658/  10000 (96.58%), best:    9731/10000 (97.31%), error:      1111.55347, best error:       894.26312 on epoch:    53, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 111.0872344971s, input_fired_time: 1.3708158731, output_fired_time: 1.2667777538, propagate_forward_time: 43.0552825928, propagate_backward_time: 62.1511573792, weight_update_time: 0.1408630311, other_time: 3.2431983948
epoch time: 8.4010286331s, input_fired_time: 0.1688956469, output_fired_time: 0.0000000000, propagate_forward_time: 7.6704578400, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5616750717
validation[          , genome  5297] predictions:    9723/  10000 (97.23%), best:    9731/10000 (97.31%), error:       902.85498, best error:       894.26312 on epoch:    53, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 110.7344436646s, input_fired_time: 1.3694535494, output_fired_time: 1.2452809811, propagate_forward_time: 41.0559921265, propagate_backward_time: 62.8142700195, weight_update_time: 0.1397458613, other_time: 4.2494506836
epoch time: 8.1810903549s, input_fired_time: 0.1802739799, output_fired_time: 0.0000000000, propagate_forward_time: 7.4229068756, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5779094696
validation[          , genome  5297] predictions:    9708/  10000 (97.08%), best:    9731/10000 (97.31%), error:       950.07019, best error:       894.26312 on epoch:    53, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 110.7620391846s, input_fired_time: 1.3684326410, output_fired_time: 1.2494212389, propagate_forward_time: 42.9213447571, propagate_backward_time: 61.9772071838, weight_update_time: 0.1396988779, other_time: 3.2456321716
epoch time: 8.2505064011s, input_fired_time: 0.1685290784, output_fired_time: 0.0000000000, propagate_forward_time: 7.5043063164, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5776705742
validation[          , genome  5297] predictions:    9722/  10000 (97.22%), best:    9731/10000 (97.31%), error:       946.65771, best error:       894.26312 on epoch:    53, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 116.8309783936s, input_fired_time: 2.4406304359, output_fired_time: 1.2973579168, propagate_forward_time: 43.3278808594, propagate_backward_time: 66.3402709961, weight_update_time: 0.1387993544, other_time: 3.4248428345
epoch time: 8.4509744644s, input_fired_time: 0.1714011878, output_fired_time: 0.0000000000, propagate_forward_time: 7.7120971680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5674762726
validation[          , genome  5297] predictions:    9713/  10000 (97.13%), best:    9731/10000 (97.31%), error:       979.99713, best error:       894.26312 on epoch:    53, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 109.5618820190s, input_fired_time: 1.3657561541, output_fired_time: 1.2461183071, propagate_forward_time: 41.8679199219, propagate_backward_time: 61.8450584412, weight_update_time: 0.1391471624, other_time: 3.2370338440
epoch time: 8.3540935516s, input_fired_time: 0.1694918424, output_fired_time: 0.0000000000, propagate_forward_time: 7.6224989891, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5621027946
validation[          , genome  5297] predictions:    9731/  10000 (97.31%), best:    9731/10000 (97.31%), error:       924.28711, best error:       894.26312 on epoch:    53, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 111.1317749023s, input_fired_time: 1.3770394325, output_fired_time: 1.2539507151, propagate_forward_time: 42.2893447876, propagate_backward_time: 61.9475479126, weight_update_time: 0.1406241506, other_time: 4.2638931274
epoch time: 8.3577060699s, input_fired_time: 0.1719519794, output_fired_time: 0.0000000000, propagate_forward_time: 7.6202235222, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5655303001
validation[          , genome  5297] predictions:    9706/  10000 (97.06%), best:    9731/10000 (97.31%), error:      1009.43756, best error:       894.26312 on epoch:    53, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 111.7170028687s, input_fired_time: 1.3968381882, output_fired_time: 1.2534662485, propagate_forward_time: 41.4867248535, propagate_backward_time: 63.2827453613, weight_update_time: 0.1408796012, other_time: 4.2972335815
epoch time: 11.8141422272s, input_fired_time: 0.2142140418, output_fired_time: 0.0000000000, propagate_forward_time: 10.8428974152, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7570304871
validation[          , genome  5297] predictions:    9715/  10000 (97.15%), best:    9731/10000 (97.31%), error:       942.73450, best error:       894.26312 on epoch:    53, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 109.6324539185s, input_fired_time: 1.3625882864, output_fired_time: 2.2602210045, propagate_forward_time: 40.8730545044, propagate_backward_time: 61.9071006775, weight_update_time: 0.1391100138, other_time: 3.2294883728
epoch time: 9.6173534393s, input_fired_time: 0.1731893122, output_fired_time: 0.0000000000, propagate_forward_time: 7.8471474648, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5970168114
validation[          , genome  5297] predictions:    9700/  10000 (97.00%), best:    9731/10000 (97.31%), error:       971.27814, best error:       894.26312 on epoch:    53, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 44.9747314453s, input_fired_time: 0.8570769429, output_fired_time: 0.0000000000, propagate_forward_time: 41.0868263245, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0308265686
best training[          , genome  5297] predictions:   49242/  50000 (98.48%), best:    9731/10000 (97.31%), error:      2754.77588, best error:       894.26312 on epoch:    53, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 8.5181474686s, input_fired_time: 0.1673156917, output_fired_time: 0.0000000000, propagate_forward_time: 7.7840962410, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5667357445
   testing[          , genome  5297] predictions:    9756/  10000 (97.56%), best:    9731/10000 (97.31%), error:       804.47479, best error:       894.26312 on epoch:    53, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
16:13:31 (7048): called boinc_finish(0)

</stderr_txt>
]]>