Nameexact_genome_1512568488_43_7048_1
Workunit2606554
Created7 Dec 2017, 23:35:07 UTC
Sent7 Dec 2017, 23:40:38 UTC
Report deadline22 Dec 2017, 8:45:45 UTC
Received13 Dec 2017, 1:21:00 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID8241
Run time4 hours 10 min 40 sec
CPU time4 hours 10 min 40 sec
Validate stateValid
Credit3,594.64
Device peak FLOPS3.83 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size155.80 MB
Peak swap size161.11 MB
Peak disk usage2.61 MB

Stderr output

<core_client_version>7.8.2</core_client_version>
<![CDATA[
<stderr_txt>
 weight_update_time: 0.0000000000, other_time: 0.3831253052
validation[          , genome  7048] predictions:    9765/  10000 (97.65%), best:    9805/10000 (98.05%), error:       723.10840, best error:       632.24744 on epoch:    26, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 542.1864013672s, input_fired_time: 4.5041990280, output_fired_time: 13.1947097778, propagate_forward_time: 168.9797210693, propagate_backward_time: 341.1939392090, weight_update_time: 0.1616419554, other_time: 14.3138122559
epoch time: 44.2095985413s, input_fired_time: 3.1502823830, output_fired_time: 0.0000000000, propagate_forward_time: 40.6741981506, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3851165771
validation[          , genome  7048] predictions:    9793/  10000 (97.93%), best:    9805/10000 (98.05%), error:       652.41821, best error:       632.24744 on epoch:    26, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 509.9155578613s, input_fired_time: 1.5517638922, output_fired_time: 4.3115968704, propagate_forward_time: 173.3638916016, propagate_backward_time: 325.4384460449, weight_update_time: 0.1640558243, other_time: 5.2498779297
epoch time: 44.2266006470s, input_fired_time: 0.1886597425, output_fired_time: 0.0000000000, propagate_forward_time: 43.6563835144, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3815574646
validation[          , genome  7048] predictions:    9771/  10000 (97.71%), best:    9805/10000 (98.05%), error:       780.00781, best error:       632.24744 on epoch:    26, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 485.6840820313s, input_fired_time: 2.7526714802, output_fired_time: 7.2793602943, propagate_forward_time: 191.1406860352, propagate_backward_time: 270.1889953613, weight_update_time: 0.1632480323, other_time: 14.3223571777
epoch time: 39.3871765137s, input_fired_time: 0.1896509379, output_fired_time: 0.0000000000, propagate_forward_time: 38.8068313599, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3906936646
validation[          , genome  7048] predictions:    9797/  10000 (97.97%), best:    9805/10000 (98.05%), error:       720.31451, best error:       632.24744 on epoch:    26, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 506.4427185059s, input_fired_time: 1.5597271919, output_fired_time: 4.3104367256, propagate_forward_time: 240.4499969482, propagate_backward_time: 249.7306976318, weight_update_time: 3.1172807217, other_time: 10.3918609619
epoch time: 44.4577789307s, input_fired_time: 3.2609267235, output_fired_time: 0.0000000000, propagate_forward_time: 40.8061790466, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3906745911
validation[          , genome  7048] predictions:    9777/  10000 (97.77%), best:    9805/10000 (98.05%), error:       756.83234, best error:       632.24744 on epoch:    26, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 526.2323608398s, input_fired_time: 1.5640478134, output_fired_time: 4.4111704826, propagate_forward_time: 240.9489898682, propagate_backward_time: 274.1796875000, weight_update_time: 0.1632415950, other_time: 5.1284790039
epoch time: 43.5778541565s, input_fired_time: 0.1906764209, output_fired_time: 0.0000000000, propagate_forward_time: 40.0434608459, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3437156677
validation[          , genome  7048] predictions:    9722/  10000 (97.22%), best:    9805/10000 (98.05%), error:       920.01703, best error:       632.24744 on epoch:    26, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 525.7728271484s, input_fired_time: 10.5261096954, output_fired_time: 7.3928647041, propagate_forward_time: 219.1904449463, propagate_backward_time: 283.4388427734, weight_update_time: 0.1610478908, other_time: 5.2245483398
epoch time: 44.0157546997s, input_fired_time: 0.1865069717, output_fired_time: 0.0000000000, propagate_forward_time: 43.4433326721, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3859138489
validation[          , genome  7048] predictions:    9772/  10000 (97.72%), best:    9805/10000 (98.05%), error:       733.55792, best error:       632.24744 on epoch:    26, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 509.9672546387s, input_fired_time: 4.5176773071, output_fired_time: 13.3727073669, propagate_forward_time: 186.0368347168, propagate_backward_time: 299.6184387207, weight_update_time: 0.1612694412, other_time: 6.4216003418
epoch time: 43.5462150574s, input_fired_time: 0.1912151724, output_fired_time: 0.0000000000, propagate_forward_time: 39.9025421143, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4524574280
validation[          , genome  7048] predictions:    9814/  10000 (98.14%), best:    9805/10000 (98.05%), error:       645.83740, best error:       632.24744 on epoch:    26, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 529.7116699219s, input_fired_time: 1.5610545874, output_fired_time: 7.3700771332, propagate_forward_time: 211.7737274170, propagate_backward_time: 303.8617248535, weight_update_time: 0.1649047285, other_time: 5.1451110840
epoch time: 43.9536476135s, input_fired_time: 0.1878954619, output_fired_time: 0.0000000000, propagate_forward_time: 43.3891105652, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3766403198
validation[          , genome  7048] predictions:    9764/  10000 (97.64%), best:    9805/10000 (98.05%), error:       800.76355, best error:       632.24744 on epoch:    26, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 501.2498168945s, input_fired_time: 10.5482597351, output_fired_time: 7.4969997406, propagate_forward_time: 167.4181060791, propagate_backward_time: 310.6062316895, weight_update_time: 0.1696403474, other_time: 5.1802062988
epoch time: 44.1181602478s, input_fired_time: 0.1894798875, output_fired_time: 0.0000000000, propagate_forward_time: 40.5925636292, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3361167908
validation[          , genome  7048] predictions:    9797/  10000 (97.97%), best:    9805/10000 (98.05%), error:       665.57642, best error:       632.24744 on epoch:    26, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 453.6752929688s, input_fired_time: 7.5726237297, output_fired_time: 1.3498247862, propagate_forward_time: 175.8220977783, propagate_backward_time: 260.7439880371, weight_update_time: 0.1637637466, other_time: 8.1867370605
epoch time: 35.5090408325s, input_fired_time: 0.1909226775, output_fired_time: 0.0000000000, propagate_forward_time: 34.9315338135, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3865852356
validation[          , genome  7048] predictions:    9753/  10000 (97.53%), best:    9805/10000 (98.05%), error:       856.39874, best error:       632.24744 on epoch:    26, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 481.1921081543s, input_fired_time: 1.5633354187, output_fired_time: 1.3574248552, propagate_forward_time: 173.7369995117, propagate_backward_time: 296.3367614746, weight_update_time: 0.1651543677, other_time: 8.1976013184
epoch time: 47.3799819946s, input_fired_time: 0.1912956387, output_fired_time: 0.0000000000, propagate_forward_time: 46.8027076721, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3859786987
validation[          , genome  7048] predictions:    9811/  10000 (98.11%), best:    9805/10000 (98.05%), error:       647.77557, best error:       632.24744 on epoch:    26, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 531.1185913086s, input_fired_time: 4.6218705177, output_fired_time: 1.3562537432, propagate_forward_time: 180.9170684814, propagate_backward_time: 333.0592651367, weight_update_time: 0.1646875143, other_time: 11.1641235352
epoch time: 36.8660888672s, input_fired_time: 3.1420545578, output_fired_time: 0.0000000000, propagate_forward_time: 30.2713012695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4527320862
validation[          , genome  7048] predictions:    9793/  10000 (97.93%), best:    9805/10000 (98.05%), error:       656.95349, best error:       632.24744 on epoch:    26, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 513.9969482422s, input_fired_time: 10.5605745316, output_fired_time: 1.3941211700, propagate_forward_time: 161.7012176514, propagate_backward_time: 332.0234985352, weight_update_time: 0.1664117277, other_time: 8.3175048828
epoch time: 44.4131622314s, input_fired_time: 0.1924754828, output_fired_time: 0.0000000000, propagate_forward_time: 43.8294677734, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3912200928
validation[          , genome  7048] predictions:    9815/  10000 (98.15%), best:    9815/10000 (98.15%), error:       622.89404, best error:       622.89404 on epoch:    41, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 492.0500183105s, input_fired_time: 4.6347956657, output_fired_time: 4.4393043518, propagate_forward_time: 155.2478485107, propagate_backward_time: 319.5179443359, weight_update_time: 0.1649103165, other_time: 8.2101440430
epoch time: 43.9903450012s, input_fired_time: 0.1899208575, output_fired_time: 0.0000000000, propagate_forward_time: 40.4516487122, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3487739563
validation[          , genome  7048] predictions:    9804/  10000 (98.04%), best:    9815/10000 (98.15%), error:       695.02081, best error:       622.89404 on epoch:    41, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 499.5914306641s, input_fired_time: 7.6970148087, output_fired_time: 10.3284139633, propagate_forward_time: 178.9820861816, propagate_backward_time: 294.5060729980, weight_update_time: 0.1643962413, other_time: 8.0778503418
epoch time: 39.9286651611s, input_fired_time: 0.1901845783, output_fired_time: 0.0000000000, propagate_forward_time: 39.3436508179, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3948287964
validation[          , genome  7048] predictions:    9795/  10000 (97.95%), best:    9815/10000 (98.15%), error:       683.18866, best error:       622.89404 on epoch:    41, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 535.9523315430s, input_fired_time: 4.6935839653, output_fired_time: 4.5358142853, propagate_forward_time: 198.8889770508, propagate_backward_time: 313.6486206055, weight_update_time: 0.1741039306, other_time: 14.1853027344
epoch time: 40.8724288940s, input_fired_time: 0.1893605739, output_fired_time: 0.0000000000, propagate_forward_time: 40.2958145142, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3872528076
validation[          , genome  7048] predictions:    9810/  10000 (98.10%), best:    9810/10000 (98.10%), error:       608.15521, best error:       608.15521 on epoch:    44, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 533.4327392578s, input_fired_time: 7.6144900322, output_fired_time: 4.3648071289, propagate_forward_time: 213.5032501221, propagate_backward_time: 296.5294799805, weight_update_time: 0.1708979905, other_time: 11.4207153320
epoch time: 44.3327407837s, input_fired_time: 0.1948924661, output_fired_time: 0.0000000000, propagate_forward_time: 43.7444343567, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3934135437
validation[          , genome  7048] predictions:    9810/  10000 (98.10%), best:    9810/10000 (98.10%), error:       618.33862, best error:       608.15521 on epoch:    44, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 531.3598632813s, input_fired_time: 4.5543289185, output_fired_time: 4.4738969803, propagate_forward_time: 214.3509063721, propagate_backward_time: 293.9499206543, weight_update_time: 3.1226863861, other_time: 14.0308532715
epoch time: 39.5121116638s, input_fired_time: 0.1955245286, output_fired_time: 0.0000000000, propagate_forward_time: 38.9259300232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3906555176
validation[          , genome  7048] predictions:    9835/  10000 (98.35%), best:    9835/10000 (98.35%), error:       573.89594, best error:       573.89594 on epoch:    46, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 498.1382141113s, input_fired_time: 6.8476886749, output_fired_time: 4.4698510170, propagate_forward_time: 178.5911102295, propagate_backward_time: 302.9323730469, weight_update_time: 0.1705649197, other_time: 5.2971801758
epoch time: 36.0936279297s, input_fired_time: 0.1897821873, output_fired_time: 0.0000000000, propagate_forward_time: 35.5222473145, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3815994263
validation[          , genome  7048] predictions:    9794/  10000 (97.94%), best:    9835/10000 (98.35%), error:       672.95184, best error:       573.89594 on epoch:    46, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 505.6153259277s, input_fired_time: 4.6588740349, output_fired_time: 4.3384447098, propagate_forward_time: 198.7482147217, propagate_backward_time: 289.6611022949, weight_update_time: 0.1657612771, other_time: 8.2087097168
epoch time: 40.2160186768s, input_fired_time: 3.1511514187, output_fired_time: 0.0000000000, propagate_forward_time: 34.9279098511, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1369590759
validation[          , genome  7048] predictions:    9781/  10000 (97.81%), best:    9835/10000 (98.35%), error:       755.13623, best error:       573.89594 on epoch:    46, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 498.1229248047s, input_fired_time: 1.5947624445, output_fired_time: 1.3970925808, propagate_forward_time: 194.1935272217, propagate_backward_time: 280.7536315918, weight_update_time: 0.1699533910, other_time: 20.1839294434
epoch time: 32.7290458679s, input_fired_time: 0.1891570389, output_fired_time: 0.0000000000, propagate_forward_time: 32.1509857178, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3889045715
validation[          , genome  7048] predictions:    9783/  10000 (97.83%), best:    9835/10000 (98.35%), error:       681.56732, best error:       573.89594 on epoch:    46, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 494.0652770996s, input_fired_time: 7.6206707954, output_fired_time: 4.3755259514, propagate_forward_time: 185.1881408691, propagate_backward_time: 285.5622253418, weight_update_time: 0.1707171053, other_time: 11.3187255859
epoch time: 47.3551063538s, input_fired_time: 0.1925152391, output_fired_time: 0.0000000000, propagate_forward_time: 46.7746582031, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3879318237
validation[          , genome  7048] predictions:    9817/  10000 (98.17%), best:    9835/10000 (98.35%), error:       632.05188, best error:       573.89594 on epoch:    46, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 517.9157104492s, input_fired_time: 4.6556820869, output_fired_time: 10.4644136429, propagate_forward_time: 220.8344879150, propagate_backward_time: 276.7717285156, weight_update_time: 0.1712657809, other_time: 5.1893920898
epoch time: 39.8695716858s, input_fired_time: 0.1942227632, output_fired_time: 0.0000000000, propagate_forward_time: 36.3284568787, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3468933105
validation[          , genome  7048] predictions:    9803/  10000 (98.03%), best:    9835/10000 (98.35%), error:       641.35846, best error:       573.89594 on epoch:    46, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 521.3392944336s, input_fired_time: 10.4424972534, output_fired_time: 7.4175405502, propagate_forward_time: 192.9554443359, propagate_backward_time: 294.9995422363, weight_update_time: 0.1652114093, other_time: 15.5242614746
epoch time: 44.1687316895s, input_fired_time: 3.2584910393, output_fired_time: 0.0000000000, propagate_forward_time: 40.5144805908, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3957595825
validation[          , genome  7048] predictions:    9798/  10000 (97.98%), best:    9835/10000 (98.35%), error:       661.70831, best error:       573.89594 on epoch:    46, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 545.7778320313s, input_fired_time: 7.7150402069, output_fired_time: 4.3631410599, propagate_forward_time: 180.4999542236, propagate_backward_time: 336.1107788086, weight_update_time: 0.1676719338, other_time: 17.0889282227
epoch time: 43.5331192017s, input_fired_time: 0.1912681907, output_fired_time: 0.0000000000, propagate_forward_time: 42.9534263611, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3884239197
validation[          , genome  7048] predictions:    9821/  10000 (98.21%), best:    9835/10000 (98.35%), error:       618.69672, best error:       573.89594 on epoch:    46, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 537.2685546875s, input_fired_time: 7.5027141571, output_fired_time: 9.5245609283, propagate_forward_time: 231.8672943115, propagate_backward_time: 283.1009826660, weight_update_time: 0.1673113257, other_time: 5.2730407715
epoch time: 44.0279731750s, input_fired_time: 0.1943091303, output_fired_time: 0.0000000000, propagate_forward_time: 43.4367637634, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3969001770
validation[          , genome  7048] predictions:    9763/  10000 (97.63%), best:    9835/10000 (98.35%), error:       776.69342, best error:       573.89594 on epoch:    46, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 558.5311279297s, input_fired_time: 7.6057534218, output_fired_time: 10.2529163361, propagate_forward_time: 200.8057556152, propagate_backward_time: 325.4995422363, weight_update_time: 3.2318384647, other_time: 14.3671264648
epoch time: 44.1141242981s, input_fired_time: 3.2579135895, output_fired_time: 0.0000000000, propagate_forward_time: 40.4599914551, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3962173462
validation[          , genome  7048] predictions:    9818/  10000 (98.18%), best:    9835/10000 (98.35%), error:       630.73169, best error:       573.89594 on epoch:    46, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 553.7074584961s, input_fired_time: 7.4957613945, output_fired_time: 10.2610654831, propagate_forward_time: 212.4255828857, propagate_backward_time: 312.1063232422, weight_update_time: 3.2326009274, other_time: 11.4187011719
epoch time: 44.1310043335s, input_fired_time: 0.1911814213, output_fired_time: 0.0000000000, propagate_forward_time: 40.8217926025, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1180305481
validation[          , genome  7048] predictions:    9828/  10000 (98.28%), best:    9835/10000 (98.35%), error:       611.51025, best error:       573.89594 on epoch:    46, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 525.2530517578s, input_fired_time: 1.5903155804, output_fired_time: 4.3534440994, propagate_forward_time: 177.6249847412, propagate_backward_time: 336.4066772461, weight_update_time: 0.1666096300, other_time: 5.2775878906
epoch time: 44.2875671387s, input_fired_time: 0.1948518902, output_fired_time: 0.0000000000, propagate_forward_time: 43.6986541748, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3940620422
validation[          , genome  7048] predictions:    9802/  10000 (98.02%), best:    9835/10000 (98.35%), error:       659.16974, best error:       573.89594 on epoch:    46, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 481.7598266602s, input_fired_time: 7.5944356918, output_fired_time: 7.4231147766, propagate_forward_time: 204.3151702881, propagate_backward_time: 260.2190551758, weight_update_time: 0.1671425998, other_time: 2.2080688477
epoch time: 38.3404922485s, input_fired_time: 3.1532852650, output_fired_time: 0.0000000000, propagate_forward_time: 34.8023452759, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3848609924
validation[          , genome  7048] predictions:    9824/  10000 (98.24%), best:    9824/10000 (98.24%), error:       567.10986, best error:       567.10986 on epoch:    58, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 466.5012512207s, input_fired_time: 4.6589045525, output_fired_time: 7.4092831612, propagate_forward_time: 186.2570800781, propagate_backward_time: 262.9968566895, weight_update_time: 0.1691622734, other_time: 5.1791381836
epoch time: 36.0652961731s, input_fired_time: 0.1976490170, output_fired_time: 0.0000000000, propagate_forward_time: 35.4644126892, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4032325745
validation[          , genome  7048] predictions:    9804/  10000 (98.04%), best:    9824/10000 (98.24%), error:       625.60376, best error:       567.10986 on epoch:    58, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 506.4054870605s, input_fired_time: 4.5578899384, output_fired_time: 2.9377937317, propagate_forward_time: 171.7463531494, propagate_backward_time: 309.9482421875, weight_update_time: 0.1683933586, other_time: 17.2152099609
epoch time: 36.7700309753s, input_fired_time: 0.1911588013, output_fired_time: 0.0000000000, propagate_forward_time: 36.2008705139, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3780021667
validation[          , genome  7048] predictions:    9826/  10000 (98.26%), best:    9826/10000 (98.26%), error:       545.59216, best error:       545.59216 on epoch:    60, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 538.2130126953s, input_fired_time: 4.5455546379, output_fired_time: 16.6290588379, propagate_forward_time: 220.7601928711, propagate_backward_time: 294.0828247070, weight_update_time: 0.1660649478, other_time: 2.1954345703
epoch time: 44.3903541565s, input_fired_time: 0.1981967986, output_fired_time: 0.0000000000, propagate_forward_time: 43.7955436707, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3966140747
validation[          , genome  7048] predictions:    9766/  10000 (97.66%), best:    9826/10000 (98.26%), error:       808.79224, best error:       545.59216 on epoch:    60, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 546.3131103516s, input_fired_time: 10.5922317505, output_fired_time: 4.5034084320, propagate_forward_time: 180.2245178223, propagate_backward_time: 342.5925903320, weight_update_time: 0.1745751798, other_time: 8.4003601074
epoch time: 47.8160820007s, input_fired_time: 0.2012736797, output_fired_time: 0.0000000000, propagate_forward_time: 47.2106285095, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4041786194
validation[          , genome  7048] predictions:    9795/  10000 (97.95%), best:    9826/10000 (98.26%), error:       675.35010, best error:       545.59216 on epoch:    60, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 501.6696166992s, input_fired_time: 7.7397513390, output_fired_time: 4.5070629120, propagate_forward_time: 201.2064361572, propagate_backward_time: 275.1399230957, weight_update_time: 3.2379236221, other_time: 13.0764465332
epoch time: 48.0858230591s, input_fired_time: 0.2008575648, output_fired_time: 0.0000000000, propagate_forward_time: 47.4808616638, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4041023254
validation[          , genome  7048] predictions:    9801/  10000 (98.01%), best:    9826/10000 (98.26%), error:       650.71143, best error:       545.59216 on epoch:    60, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 571.6410522461s, input_fired_time: 7.5336470604, output_fired_time: 10.2553586960, propagate_forward_time: 219.4744415283, propagate_backward_time: 320.2217102051, weight_update_time: 3.2239587307, other_time: 14.1559143066
epoch time: 48.1560897827s, input_fired_time: 0.2165260613, output_fired_time: 0.0000000000, propagate_forward_time: 47.5133399963, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4262237549
validation[          , genome  7048] predictions:    9805/  10000 (98.05%), best:    9826/10000 (98.26%), error:       646.02722, best error:       545.59216 on epoch:    60, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 515.4957275391s, input_fired_time: 1.5775010586, output_fired_time: 4.3768539429, propagate_forward_time: 209.3928070068, propagate_backward_time: 291.9603881836, weight_update_time: 0.1662559509, other_time: 8.1881713867
epoch time: 44.1328544617s, input_fired_time: 6.2008876801, output_fired_time: 0.0000000000, propagate_forward_time: 37.5418739319, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3900947571
validation[          , genome  7048] predictions:    9807/  10000 (98.07%), best:    9826/10000 (98.26%), error:       662.95599, best error:       545.59216 on epoch:    60, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_7048.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_7048_1_r1628027998_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.973406
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000423319
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.69328e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 66
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 60
read number_validation_images: 10000
read best_validation_predictions: 9826
read best_validation_error: 545.592
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 7048
read normal distribution: '0 0.31391304731369 -0.625677645206451'
generator_str: '434457907'
read generator: 434457907
read generated_by_map:
1 crossover 1
reading 31 nodes.
reading 92 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 498.850189208984s, input_fired_time: 10.5472631454468, output_fired_time: 4.30094051361084, propagate_forward_time: 196.158737182617, propagate_backward_time: 282.710754394531, weight_update_time: 0.163522973656654, other_time: 5.13250732421875
epoch time: 39.3065414428711s, input_fired_time: 4.56703853607178, output_fired_time: 0, propagate_forward_time: 34.352466583252, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.387035369873047
validation[          , genome  7048] predictions:    9817/  10000 (98.17%), best:    9826/10000 (98.26%), error:       588.65289, best error:       545.59216 on epoch:    60, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 538.8468627930s, input_fired_time: 1.6088390350, output_fired_time: 10.4423952103, propagate_forward_time: 176.2365264893, propagate_backward_time: 342.3346252441, weight_update_time: 0.1678754240, other_time: 8.2245178223
epoch time: 47.7023735046s, input_fired_time: 0.2017825395, output_fired_time: 0.0000000000, propagate_forward_time: 44.0734748840, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4271163940
validation[          , genome  7048] predictions:    9803/  10000 (98.03%), best:    9826/10000 (98.26%), error:       675.11169, best error:       545.59216 on epoch:    60, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 538.5751342773s, input_fired_time: 10.7009220123, output_fired_time: 7.4508662224, propagate_forward_time: 188.6522521973, propagate_backward_time: 323.6213684082, weight_update_time: 0.1702904403, other_time: 8.1497192383
epoch time: 44.7136001587s, input_fired_time: 0.1983229667, output_fired_time: 0.0000000000, propagate_forward_time: 44.1243896484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3908882141
validation[          , genome  7048] predictions:    9794/  10000 (97.94%), best:    9826/10000 (98.26%), error:       688.07361, best error:       545.59216 on epoch:    60, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 550.5336303711s, input_fired_time: 1.5980992317, output_fired_time: 1.4283567667, propagate_forward_time: 227.6142120361, propagate_backward_time: 305.6578063965, weight_update_time: 0.1677501947, other_time: 14.2351989746
epoch time: 44.2445907593s, input_fired_time: 0.1935809106, output_fired_time: 0.0000000000, propagate_forward_time: 43.6635856628, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3874244690
validation[          , genome  7048] predictions:    9824/  10000 (98.24%), best:    9826/10000 (98.26%), error:       581.85663, best error:       545.59216 on epoch:    60, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 538.8446655273s, input_fired_time: 4.6630024910, output_fired_time: 4.4600186348, propagate_forward_time: 201.4092559814, propagate_backward_time: 319.9894409180, weight_update_time: 0.1655250788, other_time: 8.3229370117
epoch time: 36.0985870361s, input_fired_time: 0.1977388263, output_fired_time: 0.0000000000, propagate_forward_time: 35.5095138550, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3913345337
validation[          , genome  7048] predictions:    9803/  10000 (98.03%), best:    9826/10000 (98.26%), error:       645.48755, best error:       545.59216 on epoch:    60, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 461.5102844238s, input_fired_time: 4.5271868706, output_fired_time: 7.3649005890, propagate_forward_time: 176.1573028564, propagate_backward_time: 265.2725219727, weight_update_time: 0.1628047079, other_time: 8.1883544922
epoch time: 40.7975311279s, input_fired_time: 0.1878385991, output_fired_time: 0.0000000000, propagate_forward_time: 40.2245025635, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3851890564
validation[          , genome  7048] predictions:    9816/  10000 (98.16%), best:    9826/10000 (98.26%), error:       618.67322, best error:       545.59216 on epoch:    60, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 469.7965698242s, input_fired_time: 4.5210556984, output_fired_time: 1.3751711845, propagate_forward_time: 207.7972412109, propagate_backward_time: 253.8041687012, weight_update_time: 0.1642930955, other_time: 2.2989196777
epoch time: 48.3611183167s, input_fired_time: 0.2122420371, output_fired_time: 0.0000000000, propagate_forward_time: 47.7341804504, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4146957397
validation[          , genome  7048] predictions:    9801/  10000 (98.01%), best:    9826/10000 (98.26%), error:       642.15283, best error:       545.59216 on epoch:    60, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 530.4526977539s, input_fired_time: 1.6307903528, output_fired_time: 1.4514018297, propagate_forward_time: 200.8104248047, propagate_backward_time: 321.2064819336, weight_update_time: 0.1732085347, other_time: 5.3535766602
epoch time: 40.2182807922s, input_fired_time: 0.1825291067, output_fired_time: 0.0000000000, propagate_forward_time: 39.6594963074, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3762550354
validation[          , genome  7048] predictions:    9810/  10000 (98.10%), best:    9826/10000 (98.26%), error:       608.76154, best error:       545.59216 on epoch:    60, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 523.2752075195s, input_fired_time: 4.5318441391, output_fired_time: 10.4094076157, propagate_forward_time: 221.1972808838, propagate_backward_time: 274.5671691895, weight_update_time: 3.1778273582, other_time: 12.5694885254
epoch time: 44.6344604492s, input_fired_time: 0.1998171955, output_fired_time: 0.0000000000, propagate_forward_time: 44.0261535645, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4084892273
validation[          , genome  7048] predictions:    9802/  10000 (98.02%), best:    9826/10000 (98.26%), error:       630.26221, best error:       545.59216 on epoch:    60, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_7048.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_7048_1_r1628027998_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_7048.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_7048_1_r1628027998_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.979542
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000266796
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.06719e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 75
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 60
read number_validation_images: 10000
read best_validation_predictions: 9826
read best_validation_error: 545.592
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 7048
read normal distribution: '0 0.31391304731369 -0.625677645206451'
generator_str: '2030389472'
read generator: 2030389472
read generated_by_map:
1 crossover 1
reading 31 nodes.
reading 92 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 471.87939453125s, input_fired_time: 7.44119596481323, output_fired_time: 4.3000659942627, propagate_forward_time: 184.786270141602, propagate_backward_time: 270.305541992188, weight_update_time: 0.155055865645409, other_time: 5.04632568359375
epoch time: 38.6126747131348s, input_fired_time: 0.185938239097595, output_fired_time: 0, propagate_forward_time: 38.0449066162109, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.381828308105469
validation[          , genome  7048] predictions:    9840/  10000 (98.40%), best:    9826/10000 (98.26%), error:       558.55212, best error:       545.59216 on epoch:    60, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 479.5506896973s, input_fired_time: 7.5382342339, output_fired_time: 4.2849440575, propagate_forward_time: 174.8434295654, propagate_backward_time: 284.7272644043, weight_update_time: 0.1544696093, other_time: 8.1568298340
epoch time: 40.3487205505s, input_fired_time: 3.2025897503, output_fired_time: 0.0000000000, propagate_forward_time: 36.7659835815, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3801460266
validation[          , genome  7048] predictions:    9816/  10000 (98.16%), best:    9826/10000 (98.26%), error:       627.81024, best error:       545.59216 on epoch:    60, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 488.8531494141s, input_fired_time: 7.5552229881, output_fired_time: 4.2877717018, propagate_forward_time: 185.3733215332, propagate_backward_time: 283.5203552246, weight_update_time: 3.0740437508, other_time: 8.1164550781
epoch time: 28.5950145721s, input_fired_time: 0.1887913495, output_fired_time: 0.0000000000, propagate_forward_time: 28.0250854492, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3811378479
validation[          , genome  7048] predictions:    9813/  10000 (98.13%), best:    9826/10000 (98.26%), error:       629.08960, best error:       545.59216 on epoch:    60, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 489.1935119629s, input_fired_time: 4.4321665764, output_fired_time: 10.0151681900, propagate_forward_time: 187.4564971924, propagate_backward_time: 273.0769958496, weight_update_time: 3.1682264805, other_time: 14.2126770020
epoch time: 39.9589424133s, input_fired_time: 0.1808750033, output_fired_time: 0.0000000000, propagate_forward_time: 39.4081726074, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3698959351
validation[          , genome  7048] predictions:    9781/  10000 (97.81%), best:    9826/10000 (98.26%), error:       718.57343, best error:       545.59216 on epoch:    60, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 473.0934753418s, input_fired_time: 1.5135194063, output_fired_time: 4.2968530655, propagate_forward_time: 166.3108978271, propagate_backward_time: 292.8323364258, weight_update_time: 0.1520838439, other_time: 8.1398925781
epoch time: 39.6898117065s, input_fired_time: 0.1805226803, output_fired_time: 0.0000000000, propagate_forward_time: 39.1387634277, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3705253601
validation[          , genome  7048] predictions:    9813/  10000 (98.13%), best:    9826/10000 (98.26%), error:       636.35858, best error:       545.59216 on epoch:    60, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 463.5752563477s, input_fired_time: 7.4673600197, output_fired_time: 7.3196458817, propagate_forward_time: 195.1694030762, propagate_backward_time: 240.2009124756, weight_update_time: 0.1588135064, other_time: 13.4179534912
epoch time: 40.1901092529s, input_fired_time: 0.1854007244, output_fired_time: 0.0000000000, propagate_forward_time: 39.6222572327, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3824501038
validation[          , genome  7048] predictions:    9804/  10000 (98.04%), best:    9826/10000 (98.26%), error:       639.69354, best error:       545.59216 on epoch:    60, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 472.0295715332s, input_fired_time: 7.1555008888, output_fired_time: 4.2835230827, propagate_forward_time: 195.7233734131, propagate_backward_time: 253.7640686035, weight_update_time: 0.1556840688, other_time: 11.1031188965
epoch time: 43.2686004639s, input_fired_time: 0.1862473786, output_fired_time: 0.0000000000, propagate_forward_time: 42.7083663940, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3739852905
validation[          , genome  7048] predictions:    9803/  10000 (98.03%), best:    9826/10000 (98.26%), error:       720.50714, best error:       545.59216 on epoch:    60, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 474.5546264648s, input_fired_time: 4.5378351212, output_fired_time: 9.9394044876, propagate_forward_time: 184.8378906250, propagate_backward_time: 266.9662475586, weight_update_time: 0.1571961939, other_time: 8.2732543945
epoch time: 35.6671409607s, input_fired_time: 0.1919276416, output_fired_time: 0.0000000000, propagate_forward_time: 35.0839385986, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3912734985
validation[          , genome  7048] predictions:    9837/  10000 (98.37%), best:    9826/10000 (98.26%), error:       562.39709, best error:       545.59216 on epoch:    60, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 491.9774780273s, input_fired_time: 7.5543909073, output_fired_time: 4.2089486122, propagate_forward_time: 180.1916656494, propagate_backward_time: 291.9313964844, weight_update_time: 0.1573719382, other_time: 8.0910644531
epoch time: 31.8880920410s, input_fired_time: 0.1847258210, output_fired_time: 0.0000000000, propagate_forward_time: 31.3282737732, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3750915527
validation[          , genome  7048] predictions:    9808/  10000 (98.08%), best:    9826/10000 (98.26%), error:       612.48602, best error:       545.59216 on epoch:    60, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 497.8815002441s, input_fired_time: 1.5446507931, output_fired_time: 4.3259539604, propagate_forward_time: 170.8328552246, propagate_backward_time: 310.0557861328, weight_update_time: 0.1590731740, other_time: 11.1222534180
epoch time: 31.7848167419s, input_fired_time: 3.1003222466, output_fired_time: 0.0000000000, propagate_forward_time: 28.2986907959, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3858032227
validation[          , genome  7048] predictions:    9818/  10000 (98.18%), best:    9826/10000 (98.26%), error:       577.38867, best error:       545.59216 on epoch:    60, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 479.9832763672s, input_fired_time: 4.5547637939, output_fired_time: 4.3242106438, propagate_forward_time: 203.5066528320, propagate_backward_time: 265.4287719727, weight_update_time: 0.1581861824, other_time: 2.1688842773
epoch time: 39.8217086792s, input_fired_time: 0.1793170571, output_fired_time: 0.0000000000, propagate_forward_time: 30.3308792114, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 9.3115119934
validation[          , genome  7048] predictions:    9822/  10000 (98.22%), best:    9826/10000 (98.26%), error:       582.18799, best error:       545.59216 on epoch:    60, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 485.1737365723s, input_fired_time: 4.5340085030, output_fired_time: 7.2081604004, propagate_forward_time: 183.2895965576, propagate_backward_time: 284.9892578125, weight_update_time: 0.1571321338, other_time: 5.1527099609
epoch time: 28.5616264343s, input_fired_time: 0.1850430965, output_fired_time: 0.0000000000, propagate_forward_time: 28.0075054169, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3690776825
validation[          , genome  7048] predictions:    9832/  10000 (98.32%), best:    9826/10000 (98.26%), error:       573.13696, best error:       545.59216 on epoch:    60, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 456.1629638672s, input_fired_time: 4.5327153206, output_fired_time: 4.2732892036, propagate_forward_time: 150.0748748779, propagate_backward_time: 283.2892456055, weight_update_time: 0.1547017545, other_time: 13.9928588867
epoch time: 35.8315048218s, input_fired_time: 0.1856590062, output_fired_time: 0.0000000000, propagate_forward_time: 35.2688827515, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3769645691
validation[          , genome  7048] predictions:    9784/  10000 (97.84%), best:    9826/10000 (98.26%), error:       681.01294, best error:       545.59216 on epoch:    60, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 477.0368041992s, input_fired_time: 4.4516258240, output_fired_time: 1.2991333008, propagate_forward_time: 179.2835693359, propagate_backward_time: 277.8577270508, weight_update_time: 3.1886086464, other_time: 14.1447448730
epoch time: 44.0808334351s, input_fired_time: 0.2068902254, output_fired_time: 0.0000000000, propagate_forward_time: 43.4687232971, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4052200317
validation[          , genome  7048] predictions:    9826/  10000 (98.26%), best:    9826/10000 (98.26%), error:       562.22729, best error:       545.59216 on epoch:    60, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 533.6804199219s, input_fired_time: 10.7830581665, output_fired_time: 7.3489522934, propagate_forward_time: 192.2679138184, propagate_backward_time: 321.0653686523, weight_update_time: 0.1633846015, other_time: 2.2151184082
epoch time: 43.6318588257s, input_fired_time: 0.1929480284, output_fired_time: 0.0000000000, propagate_forward_time: 43.0507087708, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3882026672
validation[          , genome  7048] predictions:    9820/  10000 (98.20%), best:    9826/10000 (98.26%), error:       571.93567, best error:       545.59216 on epoch:    60, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 513.7973632813s, input_fired_time: 7.6119251251, output_fired_time: 7.3920526505, propagate_forward_time: 194.4706878662, propagate_backward_time: 293.0998535156, weight_update_time: 0.1682338119, other_time: 11.2228393555
epoch time: 36.0341720581s, input_fired_time: 0.1961261928, output_fired_time: 0.0000000000, propagate_forward_time: 35.4408226013, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3972244263
validation[          , genome  7048] predictions:    9843/  10000 (98.43%), best:    9826/10000 (98.26%), error:       558.87354, best error:       545.59216 on epoch:    60, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 483.0926208496s, input_fired_time: 7.6485466957, output_fired_time: 7.2578039169, propagate_forward_time: 160.2907867432, propagate_backward_time: 302.6973876953, weight_update_time: 0.1626263708, other_time: 5.1980590820
epoch time: 43.7129440308s, input_fired_time: 0.1879600883, output_fired_time: 0.0000000000, propagate_forward_time: 43.1436386108, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3813438416
validation[          , genome  7048] predictions:    9820/  10000 (98.20%), best:    9826/10000 (98.26%), error:       626.41620, best error:       545.59216 on epoch:    60, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 502.2355346680s, input_fired_time: 4.5234422684, output_fired_time: 4.4060835838, propagate_forward_time: 181.5948028564, propagate_backward_time: 303.5328369141, weight_update_time: 3.2243607044, other_time: 8.1783447266
epoch time: 39.5295143127s, input_fired_time: 0.1950858235, output_fired_time: 0.0000000000, propagate_forward_time: 38.9467010498, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3877258301
validation[          , genome  7048] predictions:    9834/  10000 (98.34%), best:    9826/10000 (98.26%), error:       575.38617, best error:       545.59216 on epoch:    60, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 526.3000488281s, input_fired_time: 1.5939465761, output_fired_time: 4.3449039459, propagate_forward_time: 212.6594238281, propagate_backward_time: 296.3223571777, weight_update_time: 0.1645938307, other_time: 11.3794250488
epoch time: 36.3127059937s, input_fired_time: 0.1997229457, output_fired_time: 0.0000000000, propagate_forward_time: 35.7141914368, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3987922668
validation[          , genome  7048] predictions:    9816/  10000 (98.16%), best:    9826/10000 (98.26%), error:       604.16876, best error:       545.59216 on epoch:    60, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 510.6023559570s, input_fired_time: 4.5251917839, output_fired_time: 6.8336949348, propagate_forward_time: 204.3818817139, propagate_backward_time: 284.8175659180, weight_update_time: 0.1616140604, other_time: 10.0440063477
epoch time: 44.5491256714s, input_fired_time: 0.2019895911, output_fired_time: 0.0000000000, propagate_forward_time: 43.9483222961, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3988151550
validation[          , genome  7048] predictions:    9827/  10000 (98.27%), best:    9826/10000 (98.26%), error:       615.76990, best error:       545.59216 on epoch:    60, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 513.8801879883s, input_fired_time: 1.5777974129, output_fired_time: 4.3042411804, propagate_forward_time: 180.1052093506, propagate_backward_time: 319.6815795898, weight_update_time: 0.1630014181, other_time: 8.2113647461
epoch time: 43.8372688293s, input_fired_time: 6.3250985146, output_fired_time: 0.0000000000, propagate_forward_time: 37.1285018921, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3836669922
validation[          , genome  7048] predictions:    9828/  10000 (98.28%), best:    9826/10000 (98.26%), error:       581.76111, best error:       545.59216 on epoch:    60, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 514.5565185547s, input_fired_time: 4.5213232040, output_fired_time: 4.3090453148, propagate_forward_time: 172.6540374756, propagate_backward_time: 324.8604125977, weight_update_time: 3.2272946835, other_time: 8.2116699219
epoch time: 40.8349151611s, input_fired_time: 3.1470942497, output_fired_time: 0.0000000000, propagate_forward_time: 37.3031082153, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3847122192
validation[          , genome  7048] predictions:    9824/  10000 (98.24%), best:    9826/10000 (98.26%), error:       566.90637, best error:       545.59216 on epoch:    60, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 505.9297180176s, input_fired_time: 1.5919940472, output_fired_time: 1.3770375252, propagate_forward_time: 209.1005706787, propagate_backward_time: 276.6989440918, weight_update_time: 6.2824792862, other_time: 17.1611633301
epoch time: 40.5793533325s, input_fired_time: 0.1987283379, output_fired_time: 0.0000000000, propagate_forward_time: 39.9833717346, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3972549438
validation[          , genome  7048] predictions:    9836/  10000 (98.36%), best:    9826/10000 (98.26%), error:       565.58423, best error:       545.59216 on epoch:    60, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 530.7400512695s, input_fired_time: 7.6082015038, output_fired_time: 4.4279251099, propagate_forward_time: 189.6618194580, propagate_backward_time: 308.9030761719, weight_update_time: 0.1645612717, other_time: 20.1390380859
epoch time: 40.6661643982s, input_fired_time: 0.1889548004, output_fired_time: 0.0000000000, propagate_forward_time: 40.0930366516, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3841743469
validation[          , genome  7048] predictions:    9836/  10000 (98.36%), best:    9826/10000 (98.26%), error:       566.69543, best error:       545.59216 on epoch:    60, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 477.1628417969s, input_fired_time: 7.7135767937, output_fired_time: 1.3589156866, propagate_forward_time: 173.7710113525, propagate_backward_time: 289.0592956543, weight_update_time: 3.2262058258, other_time: 5.2600402832
epoch time: 37.0228004456s, input_fired_time: 0.1958730519, output_fired_time: 0.0000000000, propagate_forward_time: 36.4360923767, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3908348083
validation[          , genome  7048] predictions:    9805/  10000 (98.05%), best:    9826/10000 (98.26%), error:       649.88348, best error:       545.59216 on epoch:    60, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 518.6491699219s, input_fired_time: 1.5429822206, output_fired_time: 7.2624201775, propagate_forward_time: 171.4124755859, propagate_backward_time: 333.1872253418, weight_update_time: 0.1635341793, other_time: 5.2440795898
epoch time: 44.1572761536s, input_fired_time: 0.1932298988, output_fired_time: 0.0000000000, propagate_forward_time: 43.5796394348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3844070435
validation[          , genome  7048] predictions:    9832/  10000 (98.32%), best:    9826/10000 (98.26%), error:       568.94104, best error:       545.59216 on epoch:    60, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 192.2730865479s, input_fired_time: 3.9085767269, output_fired_time: 0.0000000000, propagate_forward_time: 186.3777923584, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9867248535
best training[          , genome  7048] predictions:   49850/  50000 (99.70%), best:    9826/10000 (98.26%), error:       743.43481, best error:       545.59216 on epoch:    60, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 39.5786323547s, input_fired_time: 0.1881148219, output_fired_time: 0.0000000000, propagate_forward_time: 36.0470314026, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3434867859
   testing[          , genome  7048] predictions:    9844/  10000 (98.44%), best:    9826/10000 (98.26%), error:       494.83102, best error:       545.59216 on epoch:    60, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
01:57:45 (5904): called boinc_finish(0)

</stderr_txt>
]]>