Nameexact_genome_1512568488_42_6703_1
Workunit2604740
Created7 Dec 2017, 21:20:02 UTC
Sent7 Dec 2017, 21:28:33 UTC
Report deadline15 Dec 2017, 9:42:01 UTC
Received8 Dec 2017, 9:39:23 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID62687
Run time1 hours 50 min 32 sec
CPU time1 hours 45 min 36 sec
Validate stateValid
Credit1,877.34
Device peak FLOPS4.32 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size159.13 MB
Peak swap size162.04 MB
Peak disk usage2.71 MB

Stderr output

<core_client_version>7.6.33</core_client_version>
<![CDATA[
<stderr_txt>
_time: 0.0000000000, other_time: 0.5479087830
validation[          , genome  6703] predictions:    9724/  10000 (97.24%), best:    9724/10000 (97.24%), error:       919.56714, best error:       919.56714 on epoch:    20, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 59.1853332520s, input_fired_time: 1.3042718172, output_fired_time: 0.9738323689, propagate_forward_time: 20.9198360443, propagate_backward_time: 32.7901535034, weight_update_time: 0.1719740480, other_time: 3.1972427368
epoch time: 4.8925414085s, input_fired_time: 0.1734573841, output_fired_time: 0.0000000000, propagate_forward_time: 4.1822624207, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5368213654
validation[          , genome  6703] predictions:    9713/  10000 (97.13%), best:    9724/10000 (97.24%), error:       947.47247, best error:       919.56714 on epoch:    20, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 58.8835754395s, input_fired_time: 1.2670975924, output_fired_time: 0.9702341557, propagate_forward_time: 20.8658885956, propagate_backward_time: 32.6417160034, weight_update_time: 0.1822281033, other_time: 3.1386413574
epoch time: 4.8522515297s, input_fired_time: 0.1750137061, output_fired_time: 0.0000000000, propagate_forward_time: 4.1395030022, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5377349854
validation[          , genome  6703] predictions:    9657/  10000 (96.57%), best:    9724/10000 (97.24%), error:      1188.88550, best error:       919.56714 on epoch:    20, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 58.4915161133s, input_fired_time: 1.2510944605, output_fired_time: 0.9640420079, propagate_forward_time: 20.6639328003, propagate_backward_time: 32.5046157837, weight_update_time: 0.1737932116, other_time: 3.1078300476
epoch time: 4.7470536232s, input_fired_time: 0.1605098844, output_fired_time: 0.0000000000, propagate_forward_time: 4.0674195290, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5191240311
validation[          , genome  6703] predictions:    9665/  10000 (96.65%), best:    9724/10000 (97.24%), error:      1093.38135, best error:       919.56714 on epoch:    20, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 59.6804771423s, input_fired_time: 1.3294098377, output_fired_time: 0.9551706314, propagate_forward_time: 20.9549827576, propagate_backward_time: 33.2356567383, weight_update_time: 0.1721765399, other_time: 3.2052574158
epoch time: 4.8929057121s, input_fired_time: 0.1746700108, output_fired_time: 0.0000000000, propagate_forward_time: 4.1631288528, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5551066399
validation[          , genome  6703] predictions:    9724/  10000 (97.24%), best:    9724/10000 (97.24%), error:       970.88757, best error:       919.56714 on epoch:    20, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 59.2591018677s, input_fired_time: 1.3010711670, output_fired_time: 0.9732069373, propagate_forward_time: 20.9308986664, propagate_backward_time: 32.8662719727, weight_update_time: 0.1733049452, other_time: 3.1876525879
epoch time: 4.8815703392s, input_fired_time: 0.1755424440, output_fired_time: 0.0000000000, propagate_forward_time: 4.1816368103, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5243911743
validation[          , genome  6703] predictions:    9715/  10000 (97.15%), best:    9724/10000 (97.24%), error:       925.22791, best error:       919.56714 on epoch:    20, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 59.2368316650s, input_fired_time: 1.2887476683, output_fired_time: 0.9647453427, propagate_forward_time: 20.8999900818, propagate_backward_time: 32.8889961243, weight_update_time: 0.1782257706, other_time: 3.1943550110
epoch time: 4.9466657639s, input_fired_time: 0.1745698899, output_fired_time: 0.0000000000, propagate_forward_time: 4.2111444473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5609512329
validation[          , genome  6703] predictions:    9703/  10000 (97.03%), best:    9724/10000 (97.24%), error:       985.84363, best error:       919.56714 on epoch:    20, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 59.1077651978s, input_fired_time: 1.2902361155, output_fired_time: 0.9774464965, propagate_forward_time: 20.8922233582, propagate_backward_time: 32.7489624023, weight_update_time: 0.1738983393, other_time: 3.1988945007
epoch time: 4.9079289436s, input_fired_time: 0.1771541685, output_fired_time: 0.0000000000, propagate_forward_time: 4.1769928932, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5537819862
validation[          , genome  6703] predictions:    9728/  10000 (97.28%), best:    9724/10000 (97.24%), error:       935.10663, best error:       919.56714 on epoch:    20, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 59.1654281616s, input_fired_time: 1.2825905085, output_fired_time: 0.9571098089, propagate_forward_time: 20.8751621246, propagate_backward_time: 32.8407135010, weight_update_time: 0.1803345978, other_time: 3.2098541260
epoch time: 4.9236531258s, input_fired_time: 0.1761750579, output_fired_time: 0.0000000000, propagate_forward_time: 4.1828818321, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5645961761
validation[          , genome  6703] predictions:    9720/  10000 (97.20%), best:    9724/10000 (97.24%), error:       956.54694, best error:       919.56714 on epoch:    20, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 56.6849174500s, input_fired_time: 1.1071666479, output_fired_time: 0.8304960728, propagate_forward_time: 20.2844734192, propagate_backward_time: 31.8332462311, weight_update_time: 0.1446958333, other_time: 2.6295337677
epoch time: 4.8776288033s, input_fired_time: 0.1750056148, output_fired_time: 0.0000000000, propagate_forward_time: 4.1579227448, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5447006226
validation[          , genome  6703] predictions:    9702/  10000 (97.02%), best:    9724/10000 (97.24%), error:       960.47681, best error:       919.56714 on epoch:    20, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 58.9631614685s, input_fired_time: 1.2885490656, output_fired_time: 0.9495238066, propagate_forward_time: 20.8978023529, propagate_backward_time: 32.6237182617, weight_update_time: 0.1747656912, other_time: 3.2035675049
epoch time: 4.8516383171s, input_fired_time: 0.1710269898, output_fired_time: 0.0000000000, propagate_forward_time: 4.1339597702, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5466513634
validation[          , genome  6703] predictions:    9726/  10000 (97.26%), best:    9724/10000 (97.24%), error:       939.82898, best error:       919.56714 on epoch:    20, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 59.3054122925s, input_fired_time: 1.3025544882, output_fired_time: 0.9688869119, propagate_forward_time: 20.9246654510, propagate_backward_time: 32.9247894287, weight_update_time: 0.1752428561, other_time: 3.1845169067
epoch time: 4.8943099976s, input_fired_time: 0.1733065397, output_fired_time: 0.0000000000, propagate_forward_time: 4.1678457260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5531578064
validation[          , genome  6703] predictions:    9723/  10000 (97.23%), best:    9723/10000 (97.23%), error:       893.80469, best error:       893.80469 on epoch:    31, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 59.1075706482s, input_fired_time: 1.3007029295, output_fired_time: 0.9666553140, propagate_forward_time: 20.9134979248, propagate_backward_time: 32.7866668701, weight_update_time: 0.1741007715, other_time: 3.1400489807
epoch time: 5.1758255959s, input_fired_time: 0.1878797263, output_fired_time: 0.0000000000, propagate_forward_time: 4.4148435593, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5731024742
validation[          , genome  6703] predictions:    9692/  10000 (96.92%), best:    9723/10000 (97.23%), error:      1005.30774, best error:       893.80469 on epoch:    31, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 59.9727020264s, input_fired_time: 1.3170393705, output_fired_time: 1.0009486675, propagate_forward_time: 21.1038398743, propagate_backward_time: 33.3873863220, weight_update_time: 0.1800480783, other_time: 3.1634864807
epoch time: 5.0829105377s, input_fired_time: 0.1863653660, output_fired_time: 0.0000000000, propagate_forward_time: 4.3545303345, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5420150757
validation[          , genome  6703] predictions:    9695/  10000 (96.95%), best:    9723/10000 (97.23%), error:       997.17078, best error:       893.80469 on epoch:    31, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 59.4525375366s, input_fired_time: 1.2965180874, output_fired_time: 0.9567289948, propagate_forward_time: 20.9735546112, propagate_backward_time: 33.0697746277, weight_update_time: 0.1757917553, other_time: 3.1559638977
epoch time: 4.8623480797s, input_fired_time: 0.1761423796, output_fired_time: 0.0000000000, propagate_forward_time: 4.1582016945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5280041695
validation[          , genome  6703] predictions:    9737/  10000 (97.37%), best:    9737/10000 (97.37%), error:       891.66766, best error:       891.66766 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 58.9187545776s, input_fired_time: 1.2794958353, output_fired_time: 0.9830085635, propagate_forward_time: 20.8523254395, propagate_backward_time: 32.7217445374, weight_update_time: 0.1746764928, other_time: 3.0821800232
epoch time: 4.8718824387s, input_fired_time: 0.1741879135, output_fired_time: 0.0000000000, propagate_forward_time: 4.1608867645, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5368075371
validation[          , genome  6703] predictions:    9704/  10000 (97.04%), best:    9737/10000 (97.37%), error:       962.28119, best error:       891.66766 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 58.7071800232s, input_fired_time: 1.2712080479, output_fired_time: 0.9681061506, propagate_forward_time: 20.7784614563, propagate_backward_time: 32.5530967712, weight_update_time: 0.1742139310, other_time: 3.1363067627
epoch time: 4.8727259636s, input_fired_time: 0.1729612797, output_fired_time: 0.0000000000, propagate_forward_time: 4.1615424156, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5382223129
validation[          , genome  6703] predictions:    9718/  10000 (97.18%), best:    9737/10000 (97.37%), error:       934.27423, best error:       891.66766 on epoch:    34, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 58.9933929443s, input_fired_time: 1.2866073847, output_fired_time: 0.9407956600, propagate_forward_time: 20.8380813599, propagate_backward_time: 32.7900314331, weight_update_time: 0.1743499190, other_time: 3.1378784180
epoch time: 4.9261631966s, input_fired_time: 0.1767653376, output_fired_time: 0.0000000000, propagate_forward_time: 4.2075328827, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5418648720
validation[          , genome  6703] predictions:    9714/  10000 (97.14%), best:    9737/10000 (97.37%), error:       941.49219, best error:       891.66766 on epoch:    34, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 59.0827598572s, input_fired_time: 1.2993927002, output_fired_time: 0.9277730584, propagate_forward_time: 20.9314289093, propagate_backward_time: 32.7126731873, weight_update_time: 0.1808673441, other_time: 3.2114906311
epoch time: 4.8911504745s, input_fired_time: 0.1759160161, output_fired_time: 0.0000000000, propagate_forward_time: 4.1689867973, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5462474823
validation[          , genome  6703] predictions:    9651/  10000 (96.51%), best:    9737/10000 (97.37%), error:      1213.72522, best error:       891.66766 on epoch:    34, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 59.2159957886s, input_fired_time: 1.2935931683, output_fired_time: 0.9782550335, propagate_forward_time: 20.9188442230, propagate_backward_time: 32.8133506775, weight_update_time: 0.1824840903, other_time: 3.2119483948
epoch time: 5.0162892342s, input_fired_time: 0.1825441718, output_fired_time: 0.0000000000, propagate_forward_time: 4.2829418182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5508031845
validation[          , genome  6703] predictions:    9729/  10000 (97.29%), best:    9737/10000 (97.37%), error:       909.45703, best error:       891.66766 on epoch:    34, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 57.0822982788s, input_fired_time: 1.1115955114, output_fired_time: 0.8648752570, propagate_forward_time: 20.4286804199, propagate_backward_time: 32.0910186768, weight_update_time: 0.1463977545, other_time: 2.5861282349
epoch time: 4.1908440590s, input_fired_time: 0.0929505080, output_fired_time: 0.0000000000, propagate_forward_time: 3.8273749352, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2705187798
validation[          , genome  6703] predictions:    9682/  10000 (96.82%), best:    9737/10000 (97.37%), error:      1033.10022, best error:       891.66766 on epoch:    34, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 52.1728057861s, input_fired_time: 0.7859746218, output_fired_time: 0.6067194343, propagate_forward_time: 19.1282329559, propagate_backward_time: 30.1012439728, weight_update_time: 0.0870469138, other_time: 1.5506324768
epoch time: 4.2020220757s, input_fired_time: 0.0968233049, output_fired_time: 0.0000000000, propagate_forward_time: 3.8333122730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2718865871
validation[          , genome  6703] predictions:    9675/  10000 (96.75%), best:    9737/10000 (97.37%), error:      1072.74023, best error:       891.66766 on epoch:    34, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 55.7081375122s, input_fired_time: 1.0097696781, output_fired_time: 0.7733695507, propagate_forward_time: 20.0776538849, propagate_backward_time: 31.5980510712, weight_update_time: 0.1319766641, other_time: 2.2492923737
epoch time: 4.7665977478s, input_fired_time: 0.1501158327, output_fired_time: 0.0000000000, propagate_forward_time: 4.1124100685, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5040717125
validation[          , genome  6703] predictions:    9726/  10000 (97.26%), best:    9737/10000 (97.37%), error:       908.23712, best error:       891.66766 on epoch:    34, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 55.5726051331s, input_fired_time: 0.9839522839, output_fired_time: 0.7595726252, propagate_forward_time: 20.0587272644, propagate_backward_time: 31.6572380066, weight_update_time: 0.1220327318, other_time: 2.1131172180
epoch time: 4.8685412407s, input_fired_time: 0.1600700617, output_fired_time: 0.0000000000, propagate_forward_time: 4.2409620285, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4675092697
validation[          , genome  6703] predictions:    9686/  10000 (96.86%), best:    9737/10000 (97.37%), error:      1003.34039, best error:       891.66766 on epoch:    34, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 52.6797294617s, input_fired_time: 0.8235214949, output_fired_time: 0.6300502419, propagate_forward_time: 19.2806339264, propagate_backward_time: 30.2984085083, weight_update_time: 0.0910841972, other_time: 1.6471157074
epoch time: 4.2075505257s, input_fired_time: 0.0965377837, output_fired_time: 0.0000000000, propagate_forward_time: 3.8382763863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2727365494
validation[          , genome  6703] predictions:    9715/  10000 (97.15%), best:    9737/10000 (97.37%), error:       924.32141, best error:       891.66766 on epoch:    34, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 57.7670783997s, input_fired_time: 0.9981769919, output_fired_time: 0.7319875360, propagate_forward_time: 20.8333625793, propagate_backward_time: 33.1729736328, weight_update_time: 0.1214388460, other_time: 2.0305786133
epoch time: 4.8694372177s, input_fired_time: 0.1644473672, output_fired_time: 0.0000000000, propagate_forward_time: 4.1990394592, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5059504509
validation[          , genome  6703] predictions:    9708/  10000 (97.08%), best:    9737/10000 (97.37%), error:       956.51324, best error:       891.66766 on epoch:    34, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 58.1555786133s, input_fired_time: 1.2042895555, output_fired_time: 0.9030107260, propagate_forward_time: 20.6911830902, propagate_backward_time: 32.4787826538, weight_update_time: 0.1595275402, other_time: 2.8783111572
epoch time: 5.1359543800s, input_fired_time: 0.2089348286, output_fired_time: 0.0000000000, propagate_forward_time: 4.2790598869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6479597092
validation[          , genome  6703] predictions:    9705/  10000 (97.05%), best:    9737/10000 (97.37%), error:      1012.24915, best error:       891.66766 on epoch:    34, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 58.2309036255s, input_fired_time: 1.2150198221, output_fired_time: 0.9251602292, propagate_forward_time: 20.7421398163, propagate_backward_time: 32.4904670715, weight_update_time: 0.1589658558, other_time: 2.8581199646
epoch time: 5.0330390930s, input_fired_time: 0.1745667160, output_fired_time: 0.0000000000, propagate_forward_time: 4.3288331032, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5296392441
validation[          , genome  6703] predictions:    9743/  10000 (97.43%), best:    9743/10000 (97.43%), error:       810.52625, best error:       810.52625 on epoch:    47, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 59.3943786621s, input_fired_time: 1.3369076252, output_fired_time: 0.9815297723, propagate_forward_time: 21.0661640167, propagate_backward_time: 32.8670768738, weight_update_time: 0.1747510731, other_time: 3.1427040100
epoch time: 4.4367952347s, input_fired_time: 0.1240414530, output_fired_time: 0.0000000000, propagate_forward_time: 3.9389407635, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3738129139
validation[          , genome  6703] predictions:    9627/  10000 (96.27%), best:    9743/10000 (97.43%), error:      1246.03967, best error:       810.52625 on epoch:    47, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 58.5154914856s, input_fired_time: 1.2337446213, output_fired_time: 0.9186130166, propagate_forward_time: 20.7798862457, propagate_backward_time: 32.6172676086, weight_update_time: 0.1647831947, other_time: 2.9659767151
epoch time: 4.7796683311s, input_fired_time: 0.1600252539, output_fired_time: 0.0000000000, propagate_forward_time: 4.1218838692, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4977593422
validation[          , genome  6703] predictions:    9749/  10000 (97.49%), best:    9743/10000 (97.43%), error:       843.95447, best error:       810.52625 on epoch:    47, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 61.3209686279s, input_fired_time: 1.4963637590, output_fired_time: 1.1352068186, propagate_forward_time: 21.3652458191, propagate_backward_time: 33.6321640015, weight_update_time: 0.2025718391, other_time: 3.6919860840
epoch time: 4.7727146149s, input_fired_time: 0.1614774764, output_fired_time: 0.0000000000, propagate_forward_time: 4.0967302322, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5145068169
validation[          , genome  6703] predictions:    9743/  10000 (97.43%), best:    9743/10000 (97.43%), error:       887.31354, best error:       810.52625 on epoch:    47, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 58.5915451050s, input_fired_time: 1.2394822836, output_fired_time: 0.9585264325, propagate_forward_time: 20.7960891724, propagate_backward_time: 32.6591987610, weight_update_time: 0.1598030627, other_time: 2.9382476807
epoch time: 4.7080221176s, input_fired_time: 0.1514616013, output_fired_time: 0.0000000000, propagate_forward_time: 4.0678019524, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4887585640
validation[          , genome  6703] predictions:    9740/  10000 (97.40%), best:    9743/10000 (97.43%), error:       922.84827, best error:       810.52625 on epoch:    47, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 59.2905883789s, input_fired_time: 1.3322556019, output_fired_time: 0.9879096746, propagate_forward_time: 20.9817752838, propagate_backward_time: 32.8391761780, weight_update_time: 0.1773733646, other_time: 3.1494712830
epoch time: 4.7614941597s, input_fired_time: 0.1572537571, output_fired_time: 0.0000000000, propagate_forward_time: 4.1115126610, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4927277565
validation[          , genome  6703] predictions:    9712/  10000 (97.12%), best:    9743/10000 (97.43%), error:       980.14990, best error:       810.52625 on epoch:    47, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 58.9155693054s, input_fired_time: 1.2434924841, output_fired_time: 0.9460045099, propagate_forward_time: 20.8633117676, propagate_backward_time: 32.9126014709, weight_update_time: 0.1646376252, other_time: 2.9501609802
epoch time: 4.8166699409s, input_fired_time: 0.1667212397, output_fired_time: 0.0000000000, propagate_forward_time: 4.1411213875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5088272095
validation[          , genome  6703] predictions:    9706/  10000 (97.06%), best:    9743/10000 (97.43%), error:      1015.62207, best error:       810.52625 on epoch:    47, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 59.7151947021s, input_fired_time: 1.3696155548, output_fired_time: 1.0259599686, propagate_forward_time: 21.0452957153, propagate_backward_time: 32.9573135376, weight_update_time: 0.1812925637, other_time: 3.3170127869
epoch time: 6.0971179008s, input_fired_time: 0.3162613809, output_fired_time: 0.0000000000, propagate_forward_time: 4.6743364334, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1065201759
validation[          , genome  6703] predictions:    9689/  10000 (96.89%), best:    9743/10000 (97.43%), error:      1071.07983, best error:       810.52625 on epoch:    47, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 58.4194412231s, input_fired_time: 1.2320263386, output_fired_time: 0.9336204529, propagate_forward_time: 20.6827011108, propagate_backward_time: 32.5589561462, weight_update_time: 0.1662559062, other_time: 3.0121383667
epoch time: 4.7598156929s, input_fired_time: 0.1681973338, output_fired_time: 0.0000000000, propagate_forward_time: 4.1080822945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4835362434
validation[          , genome  6703] predictions:    9725/  10000 (97.25%), best:    9743/10000 (97.43%), error:       944.37769, best error:       810.52625 on epoch:    47, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 58.3860092163s, input_fired_time: 1.2350825071, output_fired_time: 0.9209646583, propagate_forward_time: 20.7386302948, propagate_backward_time: 32.5020751953, weight_update_time: 0.1685692966, other_time: 2.9892578125
epoch time: 4.8145899773s, input_fired_time: 0.1635299176, output_fired_time: 0.0000000000, propagate_forward_time: 4.1387414932, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5123186111
validation[          , genome  6703] predictions:    9723/  10000 (97.23%), best:    9743/10000 (97.43%), error:       909.31866, best error:       810.52625 on epoch:    47, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_42_6703.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_42_6703_1_r1808785111_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.963671
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000671669
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 2.68668e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 57
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 47
read number_validation_images: 10000
read best_validation_predictions: 9743
read best_validation_error: 810.526
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6703
read normal distribution: '0 -2.10257506370544 -0.758223056793213'
generator_str: '1748353876'
read generator: 1748353876
read generated_by_map:
1 add_edge 1
reading 34 nodes.
reading 96 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 61.6709251403809s, input_fired_time: 1.33960771560669, output_fired_time: 1.03698468208313, propagate_forward_time: 21.679708480835, propagate_backward_time: 34.7521553039551, weight_update_time: 0.157107561826706, other_time: 2.86246871948242
epoch time: 5.14458084106445s, input_fired_time: 0.20729672908783, output_fired_time: 0, propagate_forward_time: 4.33764791488647, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.599636077880859
validation[          , genome  6703] predictions:    9728/  10000 (97.28%), best:    9743/10000 (97.43%), error:       929.52441, best error:       810.52625 on epoch:    47, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 59.1245574951s, input_fired_time: 1.2609428167, output_fired_time: 0.9262613654, propagate_forward_time: 20.9850311279, propagate_backward_time: 33.1310348511, weight_update_time: 0.1568322927, other_time: 2.8212852478
epoch time: 5.3084034920s, input_fired_time: 0.2376427203, output_fired_time: 0.0000000000, propagate_forward_time: 4.3510212898, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7197394371
validation[          , genome  6703] predictions:    9717/  10000 (97.17%), best:    9743/10000 (97.43%), error:       964.97778, best error:       810.52625 on epoch:    47, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 61.8069381714s, input_fired_time: 1.4862492085, output_fired_time: 1.1288657188, propagate_forward_time: 21.5906734467, propagate_backward_time: 34.1217422485, weight_update_time: 0.1943096071, other_time: 3.4794082642
epoch time: 4.8381438255s, input_fired_time: 0.1656357944, output_fired_time: 0.0000000000, propagate_forward_time: 4.1763367653, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4961714745
validation[          , genome  6703] predictions:    9690/  10000 (96.90%), best:    9743/10000 (97.43%), error:      1079.20898, best error:       810.52625 on epoch:    47, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 59.4626579285s, input_fired_time: 1.2586071491, output_fired_time: 0.9593461156, propagate_forward_time: 21.0802516937, propagate_backward_time: 33.2565193176, weight_update_time: 0.1617308259, other_time: 2.9079322815
epoch time: 4.8231601715s, input_fired_time: 0.1688396633, output_fired_time: 0.0000000000, propagate_forward_time: 4.1732559204, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4810647964
validation[          , genome  6703] predictions:    9730/  10000 (97.30%), best:    9743/10000 (97.43%), error:       931.97589, best error:       810.52625 on epoch:    47, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 60.6296539307s, input_fired_time: 1.3624153137, output_fired_time: 1.0574381351, propagate_forward_time: 21.3712310791, propagate_backward_time: 33.8041648865, weight_update_time: 0.1645807326, other_time: 3.0344047546
epoch time: 4.8539671898s, input_fired_time: 0.1704970598, output_fired_time: 0.0000000000, propagate_forward_time: 4.1894407272, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4940295219
validation[          , genome  6703] predictions:    9744/  10000 (97.44%), best:    9743/10000 (97.43%), error:       882.04944, best error:       810.52625 on epoch:    47, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 62.3935813904s, input_fired_time: 1.3759603500, output_fired_time: 1.0662484169, propagate_forward_time: 21.8844661713, propagate_backward_time: 35.0882644653, weight_update_time: 0.1637614816, other_time: 2.9786376953
epoch time: 5.0256252289s, input_fired_time: 0.1756758541, output_fired_time: 0.0000000000, propagate_forward_time: 4.3450737000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5048756599
validation[          , genome  6703] predictions:    9726/  10000 (97.26%), best:    9743/10000 (97.43%), error:       929.52887, best error:       810.52625 on epoch:    47, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 65.8535537720s, input_fired_time: 1.6149461269, output_fired_time: 1.3514952660, propagate_forward_time: 22.7232055664, propagate_backward_time: 36.5278625488, weight_update_time: 0.2011676580, other_time: 3.6360473633
epoch time: 5.1205782890s, input_fired_time: 0.1871422231, output_fired_time: 0.0000000000, propagate_forward_time: 4.4110255241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5224103928
validation[          , genome  6703] predictions:    9741/  10000 (97.41%), best:    9743/10000 (97.43%), error:       895.24908, best error:       810.52625 on epoch:    47, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 59.7987556458s, input_fired_time: 1.2661541700, output_fired_time: 0.9757320881, propagate_forward_time: 21.1373977661, propagate_backward_time: 33.5043220520, weight_update_time: 0.1639748961, other_time: 2.9151496887
epoch time: 4.8278408051s, input_fired_time: 0.1617552191, output_fired_time: 0.0000000000, propagate_forward_time: 4.1713414192, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4947443008
validation[          , genome  6703] predictions:    9726/  10000 (97.26%), best:    9743/10000 (97.43%), error:       915.94928, best error:       810.52625 on epoch:    47, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 60.1454315186s, input_fired_time: 1.3688583374, output_fired_time: 1.0033192635, propagate_forward_time: 21.1879196167, propagate_backward_time: 33.5223922729, weight_update_time: 0.1719989479, other_time: 3.0629425049
epoch time: 4.7964072227s, input_fired_time: 0.1634054780, output_fired_time: 0.0000000000, propagate_forward_time: 4.1569113731, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4760904312
validation[          , genome  6703] predictions:    9718/  10000 (97.18%), best:    9743/10000 (97.43%), error:       922.68658, best error:       810.52625 on epoch:    47, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 62.9309730530s, input_fired_time: 1.3815873861, output_fired_time: 1.1045145988, propagate_forward_time: 21.9306964874, propagate_backward_time: 35.5632553101, weight_update_time: 0.1626297086, other_time: 2.9509201050
epoch time: 4.8349623680s, input_fired_time: 0.1620743126, output_fired_time: 0.0000000000, propagate_forward_time: 4.1808924675, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4919958115
validation[          , genome  6703] predictions:    9732/  10000 (97.32%), best:    9743/10000 (97.43%), error:       934.20593, best error:       810.52625 on epoch:    47, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 62.0986289978s, input_fired_time: 1.5154697895, output_fired_time: 1.1755840778, propagate_forward_time: 21.6091709137, propagate_backward_time: 34.2547874451, weight_update_time: 0.1970709711, other_time: 3.5436210632
epoch time: 5.1905708313s, input_fired_time: 0.2045333236, output_fired_time: 0.0000000000, propagate_forward_time: 4.3651595116, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6208782196
validation[          , genome  6703] predictions:    9674/  10000 (96.74%), best:    9743/10000 (97.43%), error:      1116.45251, best error:       810.52625 on epoch:    47, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 64.9877243042s, input_fired_time: 1.4437402487, output_fired_time: 1.1823430061, propagate_forward_time: 22.5191307068, propagate_backward_time: 36.8265571594, weight_update_time: 0.1675764620, other_time: 3.0159530640
epoch time: 5.2526612282s, input_fired_time: 0.1914881617, output_fired_time: 0.0000000000, propagate_forward_time: 4.5227560997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5384168625
validation[          , genome  6703] predictions:    9733/  10000 (97.33%), best:    9743/10000 (97.43%), error:       918.98706, best error:       810.52625 on epoch:    47, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 65.7088012695s, input_fired_time: 1.5319833755, output_fired_time: 1.2563695908, propagate_forward_time: 22.6411514282, propagate_backward_time: 37.1530075073, weight_update_time: 0.1698242277, other_time: 3.1262893677
epoch time: 5.5024089813s, input_fired_time: 0.2193609327, output_fired_time: 0.0000000000, propagate_forward_time: 4.6754488945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6075992584
validation[          , genome  6703] predictions:    9740/  10000 (97.40%), best:    9743/10000 (97.43%), error:       894.80731, best error:       810.52625 on epoch:    47, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 65.1693572998s, input_fired_time: 1.4212758541, output_fired_time: 1.2110769749, propagate_forward_time: 22.5230121613, propagate_backward_time: 36.9923400879, weight_update_time: 0.1592077911, other_time: 3.0216522217
epoch time: 5.3021554947s, input_fired_time: 0.2010564804, output_fired_time: 0.0000000000, propagate_forward_time: 4.5610175133, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5400815010
validation[          , genome  6703] predictions:    9749/  10000 (97.49%), best:    9743/10000 (97.43%), error:       872.27911, best error:       810.52625 on epoch:    47, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 64.6436157227s, input_fired_time: 1.4680776596, output_fired_time: 1.1914589405, propagate_forward_time: 22.3906612396, propagate_backward_time: 36.2551269531, weight_update_time: 0.1801448613, other_time: 3.3382873535
epoch time: 6.2943801880s, input_fired_time: 0.2842961848, output_fired_time: 0.0000000000, propagate_forward_time: 5.0709547997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9391293526
validation[          , genome  6703] predictions:    9714/  10000 (97.14%), best:    9743/10000 (97.43%), error:       932.22205, best error:       810.52625 on epoch:    47, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 65.8865814209s, input_fired_time: 1.4945195913, output_fired_time: 1.2617516518, propagate_forward_time: 22.7898445129, propagate_backward_time: 37.1313896179, weight_update_time: 0.1706241816, other_time: 3.2090721130
epoch time: 5.5546293259s, input_fired_time: 0.2138100713, output_fired_time: 0.0000000000, propagate_forward_time: 4.7283148766, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6125044823
validation[          , genome  6703] predictions:    9705/  10000 (97.05%), best:    9743/10000 (97.43%), error:       989.31024, best error:       810.52625 on epoch:    47, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 65.4614639282s, input_fired_time: 1.4462747574, output_fired_time: 1.2211989164, propagate_forward_time: 22.6770610809, propagate_backward_time: 37.1067733765, weight_update_time: 0.1603810340, other_time: 3.0101547241
epoch time: 6.3238048553s, input_fired_time: 0.3107719123, output_fired_time: 0.0000000000, propagate_forward_time: 5.1252012253, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8878316879
validation[          , genome  6703] predictions:    9732/  10000 (97.32%), best:    9743/10000 (97.43%), error:       886.15631, best error:       810.52625 on epoch:    47, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 62.5019302368s, input_fired_time: 1.4073636532, output_fired_time: 1.0854179859, propagate_forward_time: 21.8857059479, propagate_backward_time: 35.0472030640, weight_update_time: 0.1618692279, other_time: 3.0762405396
epoch time: 5.7766051292s, input_fired_time: 0.3098655641, output_fired_time: 0.0000000000, propagate_forward_time: 4.5538673401, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9128723145
validation[          , genome  6703] predictions:    9749/  10000 (97.49%), best:    9743/10000 (97.43%), error:       861.89886, best error:       810.52625 on epoch:    47, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 61.5133857727s, input_fired_time: 1.3337863684, output_fired_time: 1.0414619446, propagate_forward_time: 21.6426601410, propagate_backward_time: 34.5929832458, weight_update_time: 0.1577954292, other_time: 2.9024925232
epoch time: 5.0276899338s, input_fired_time: 0.1674173325, output_fired_time: 0.0000000000, propagate_forward_time: 4.4077019691, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4525704384
validation[          , genome  6703] predictions:    9729/  10000 (97.29%), best:    9743/10000 (97.43%), error:       993.14807, best error:       810.52625 on epoch:    47, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 62.7732963562s, input_fired_time: 1.4636995792, output_fired_time: 1.1625964642, propagate_forward_time: 21.9056148529, propagate_backward_time: 34.9524002075, weight_update_time: 0.1843256801, other_time: 3.2889862061
epoch time: 4.8302426338s, input_fired_time: 0.1526298076, output_fired_time: 0.0000000000, propagate_forward_time: 4.2648134232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4127993584
validation[          , genome  6703] predictions:    9712/  10000 (97.12%), best:    9743/10000 (97.43%), error:       934.69116, best error:       810.52625 on epoch:    47, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 61.0205993652s, input_fired_time: 1.2668681145, output_fired_time: 1.0194780827, propagate_forward_time: 21.4731712341, propagate_backward_time: 34.4409980774, weight_update_time: 0.1492934078, other_time: 2.8200836182
epoch time: 4.8032817841s, input_fired_time: 0.1500670463, output_fired_time: 0.0000000000, propagate_forward_time: 4.2609887123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3922262192
validation[          , genome  6703] predictions:    9723/  10000 (97.23%), best:    9743/10000 (97.43%), error:       919.10919, best error:       810.52625 on epoch:    47, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 62.4221725464s, input_fired_time: 1.4576221704, output_fired_time: 1.1435661316, propagate_forward_time: 21.7525501251, propagate_backward_time: 34.7865486145, weight_update_time: 0.1737367511, other_time: 3.2818870544
epoch time: 4.6886830330s, input_fired_time: 0.1371540278, output_fired_time: 0.0000000000, propagate_forward_time: 4.1746258736, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3769030571
validation[          , genome  6703] predictions:    9719/  10000 (97.19%), best:    9743/10000 (97.43%), error:      1015.36688, best error:       810.52625 on epoch:    47, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 61.4203338623s, input_fired_time: 1.2946408987, output_fired_time: 1.0177806616, propagate_forward_time: 21.7413063049, propagate_backward_time: 34.6339073181, weight_update_time: 0.1452565640, other_time: 2.7327003479
epoch time: 6.7263493538s, input_fired_time: 0.3768745065, output_fired_time: 0.0000000000, propagate_forward_time: 5.2439465523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1055283546
validation[          , genome  6703] predictions:    9691/  10000 (96.91%), best:    9743/10000 (97.43%), error:      1088.57263, best error:       810.52625 on epoch:    47, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 65.7087707520s, input_fired_time: 1.5638490915, output_fired_time: 1.2270696163, propagate_forward_time: 22.6881484985, propagate_backward_time: 36.7786979675, weight_update_time: 0.1817010343, other_time: 3.4510040283
epoch time: 4.7441687584s, input_fired_time: 0.1485019177, output_fired_time: 0.0000000000, propagate_forward_time: 4.2044796944, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3911871910
validation[          , genome  6703] predictions:    9703/  10000 (97.03%), best:    9743/10000 (97.43%), error:      1000.40344, best error:       810.52625 on epoch:    47, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 64.0523071289s, input_fired_time: 1.5027675629, output_fired_time: 1.1900212765, propagate_forward_time: 22.2878360748, propagate_backward_time: 35.7102890015, weight_update_time: 0.1788510680, other_time: 3.3613891602
epoch time: 4.7912445068s, input_fired_time: 0.1550962180, output_fired_time: 0.0000000000, propagate_forward_time: 4.2063961029, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4297523499
validation[          , genome  6703] predictions:    9735/  10000 (97.35%), best:    9743/10000 (97.43%), error:       870.44720, best error:       810.52625 on epoch:    47, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 59.9238815308s, input_fired_time: 1.2477040291, output_fired_time: 0.9520270824, propagate_forward_time: 21.1991004944, propagate_backward_time: 33.8049049377, weight_update_time: 0.1481283605, other_time: 2.7201461792
epoch time: 4.9084815979s, input_fired_time: 0.1682660431, output_fired_time: 0.0000000000, propagate_forward_time: 4.2632093430, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4770064354
validation[          , genome  6703] predictions:    9725/  10000 (97.25%), best:    9743/10000 (97.43%), error:       967.22424, best error:       810.52625 on epoch:    47, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 61.4657135010s, input_fired_time: 1.4167064428, output_fired_time: 1.0841950178, propagate_forward_time: 21.5303421021, propagate_backward_time: 34.1380500793, weight_update_time: 0.1782393903, other_time: 3.2964210510
epoch time: 4.5898594856s, input_fired_time: 0.1292988807, output_fired_time: 0.0000000000, propagate_forward_time: 4.0997214317, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3608393669
validation[          , genome  6703] predictions:    9724/  10000 (97.24%), best:    9743/10000 (97.43%), error:       935.67609, best error:       810.52625 on epoch:    47, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 59.7233924866s, input_fired_time: 1.2239756584, output_fired_time: 0.9244371653, propagate_forward_time: 21.1550331116, propagate_backward_time: 33.7191772461, weight_update_time: 0.1453407854, other_time: 2.7007675171
epoch time: 4.8175678253s, input_fired_time: 0.1518358290, output_fired_time: 0.0000000000, propagate_forward_time: 4.2084207535, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4573111534
validation[          , genome  6703] predictions:    9719/  10000 (97.19%), best:    9743/10000 (97.43%), error:       932.87537, best error:       810.52625 on epoch:    47, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 61.6805343628s, input_fired_time: 1.4433414936, output_fired_time: 1.0883179903, propagate_forward_time: 21.5992660522, propagate_backward_time: 34.2436790466, weight_update_time: 0.1758108288, other_time: 3.3059310913
epoch time: 5.6206173897s, input_fired_time: 0.2294277400, output_fired_time: 0.0000000000, propagate_forward_time: 4.5584568977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8327326775
validation[          , genome  6703] predictions:    9720/  10000 (97.20%), best:    9743/10000 (97.43%), error:       967.05139, best error:       810.52625 on epoch:    47, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 60.6851043701s, input_fired_time: 1.4012298584, output_fired_time: 1.0408669710, propagate_forward_time: 21.3774814606, propagate_backward_time: 33.6696166992, weight_update_time: 0.1694205701, other_time: 3.1959075928
epoch time: 5.6681566238s, input_fired_time: 0.2754174173, output_fired_time: 0.0000000000, propagate_forward_time: 4.5004706383, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8922686577
validation[          , genome  6703] predictions:    9646/  10000 (96.46%), best:    9743/10000 (97.43%), error:      1116.03381, best error:       810.52625 on epoch:    47, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 59.8347091675s, input_fired_time: 1.2297239304, output_fired_time: 0.9383714199, propagate_forward_time: 21.2263698578, propagate_backward_time: 33.7985649109, weight_update_time: 0.1438773870, other_time: 2.6416816711
epoch time: 4.9064459801s, input_fired_time: 0.1664248407, output_fired_time: 0.0000000000, propagate_forward_time: 4.2901101112, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4499111176
validation[          , genome  6703] predictions:    9671/  10000 (96.71%), best:    9743/10000 (97.43%), error:      1068.62585, best error:       810.52625 on epoch:    47, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 62.9782752991s, input_fired_time: 1.4781314135, output_fired_time: 1.1516623497, propagate_forward_time: 21.9363403320, propagate_backward_time: 35.0916137695, weight_update_time: 0.1774047017, other_time: 3.3205299377
epoch time: 4.7723093033s, input_fired_time: 0.1366448998, output_fired_time: 0.0000000000, propagate_forward_time: 4.2528057098, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3828587532
validation[          , genome  6703] predictions:    9741/  10000 (97.41%), best:    9743/10000 (97.43%), error:       895.37390, best error:       810.52625 on epoch:    47, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 61.9771080017s, input_fired_time: 1.3219392300, output_fired_time: 1.0329763889, propagate_forward_time: 21.8024101257, propagate_backward_time: 34.9911575317, weight_update_time: 0.1500488222, other_time: 2.8286247253
epoch time: 4.8732991219s, input_fired_time: 0.1552165896, output_fired_time: 0.0000000000, propagate_forward_time: 4.2846012115, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4334812164
validation[          , genome  6703] predictions:    9629/  10000 (96.29%), best:    9743/10000 (97.43%), error:      1145.07971, best error:       810.52625 on epoch:    47, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 64.2947463989s, input_fired_time: 1.5036628246, output_fired_time: 1.1969470978, propagate_forward_time: 22.3172550201, propagate_backward_time: 35.9446640015, weight_update_time: 0.1715893596, other_time: 3.3322219849
epoch time: 4.6756920815s, input_fired_time: 0.1374469995, output_fired_time: 0.0000000000, propagate_forward_time: 4.1581225395, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3801226616
validation[          , genome  6703] predictions:    9725/  10000 (97.25%), best:    9743/10000 (97.43%), error:       966.33563, best error:       810.52625 on epoch:    47, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 60.4034271240s, input_fired_time: 1.3694339991, output_fired_time: 1.0260250568, propagate_forward_time: 21.2250976563, propagate_backward_time: 33.7786483765, weight_update_time: 0.1648121327, other_time: 3.0042228699
epoch time: 5.4357652664s, input_fired_time: 0.2538587451, output_fired_time: 0.0000000000, propagate_forward_time: 4.4347186089, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7471880913
validation[          , genome  6703] predictions:    9743/  10000 (97.43%), best:    9743/10000 (97.43%), error:       921.65344, best error:       810.52625 on epoch:    47, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 60.3161125183s, input_fired_time: 1.3430343866, output_fired_time: 1.0107395649, propagate_forward_time: 21.2604312897, propagate_backward_time: 33.6476249695, weight_update_time: 0.1657612771, other_time: 3.0542869568
epoch time: 4.6301989555s, input_fired_time: 0.1328737587, output_fired_time: 0.0000000000, propagate_forward_time: 4.1133174896, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3840079308
validation[          , genome  6703] predictions:    9671/  10000 (96.71%), best:    9743/10000 (97.43%), error:      1060.62366, best error:       810.52625 on epoch:    47, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 62.1901321411s, input_fired_time: 1.4383349419, output_fired_time: 1.1111537218, propagate_forward_time: 21.6838645935, propagate_backward_time: 34.6347045898, weight_update_time: 0.1726748943, other_time: 3.3220748901
epoch time: 4.8854002953s, input_fired_time: 0.1753262579, output_fired_time: 0.0000000000, propagate_forward_time: 4.3098602295, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4002137184
validation[          , genome  6703] predictions:    9712/  10000 (97.12%), best:    9743/10000 (97.43%), error:       956.45703, best error:       810.52625 on epoch:    47, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 59.7907066345s, input_fired_time: 1.2375943661, output_fired_time: 0.9524410963, propagate_forward_time: 21.1822414398, propagate_backward_time: 33.7453804016, weight_update_time: 0.1467925310, other_time: 2.6730461121
epoch time: 4.9826717377s, input_fired_time: 0.1801480502, output_fired_time: 0.0000000000, propagate_forward_time: 4.2843666077, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5181570053
validation[          , genome  6703] predictions:    9731/  10000 (97.31%), best:    9743/10000 (97.43%), error:       915.99774, best error:       810.52625 on epoch:    47, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 61.6369209290s, input_fired_time: 1.4056811333, output_fired_time: 1.0744838715, propagate_forward_time: 21.5593013763, propagate_backward_time: 34.3276252747, weight_update_time: 0.1748353690, other_time: 3.2698249817
epoch time: 4.7159810066s, input_fired_time: 0.1452271789, output_fired_time: 0.0000000000, propagate_forward_time: 4.1664705276, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4042835236
validation[          , genome  6703] predictions:    9748/  10000 (97.48%), best:    9743/10000 (97.43%), error:       849.58899, best error:       810.52625 on epoch:    47, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 59.8270988464s, input_fired_time: 1.2423305511, output_fired_time: 0.9382682443, propagate_forward_time: 21.2265834808, propagate_backward_time: 33.7475433350, weight_update_time: 0.1463092715, other_time: 2.6723785400
epoch time: 4.7277359962s, input_fired_time: 0.1453311294, output_fired_time: 0.0000000000, propagate_forward_time: 4.2103371620, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3720679283
validation[          , genome  6703] predictions:    9710/  10000 (97.10%), best:    9743/10000 (97.43%), error:       941.59021, best error:       810.52625 on epoch:    47, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 62.9302597046s, input_fired_time: 1.4839122295, output_fired_time: 1.1527348757, propagate_forward_time: 21.9324073792, propagate_backward_time: 35.0019683838, weight_update_time: 0.1744361371, other_time: 3.3592338562
epoch time: 5.8484945297s, input_fired_time: 0.2788278461, output_fired_time: 0.0000000000, propagate_forward_time: 4.5952806473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9743862152
validation[          , genome  6703] predictions:    9711/  10000 (97.11%), best:    9743/10000 (97.43%), error:      1039.83276, best error:       810.52625 on epoch:    47, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 61.7301635742s, input_fired_time: 1.4190568924, output_fired_time: 1.0912442207, propagate_forward_time: 21.7106914520, propagate_backward_time: 34.2457542419, weight_update_time: 0.1778829396, other_time: 3.2634162903
epoch time: 5.2116389275s, input_fired_time: 0.2045585364, output_fired_time: 0.0000000000, propagate_forward_time: 4.3952522278, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6118283272
validation[          , genome  6703] predictions:    9758/  10000 (97.58%), best:    9743/10000 (97.43%), error:       850.82111, best error:       810.52625 on epoch:    47, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 59.4859619141s, input_fired_time: 1.2324409485, output_fired_time: 0.9295410514, propagate_forward_time: 21.1734619141, propagate_backward_time: 33.5468482971, weight_update_time: 0.1448334157, other_time: 2.6036682129
epoch time: 4.6762180328s, input_fired_time: 0.1386736929, output_fired_time: 0.0000000000, propagate_forward_time: 4.1391539574, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3983902931
validation[          , genome  6703] predictions:    9718/  10000 (97.18%), best:    9743/10000 (97.43%), error:       931.47565, best error:       810.52625 on epoch:    47, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 61.9872817993s, input_fired_time: 1.3930127621, output_fired_time: 1.0863546133, propagate_forward_time: 21.7663917542, propagate_backward_time: 34.5346832275, weight_update_time: 0.1699474454, other_time: 3.2068405151
epoch time: 4.6953797340s, input_fired_time: 0.1442020535, output_fired_time: 0.0000000000, propagate_forward_time: 4.1374406815, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4137368202
validation[          , genome  6703] predictions:    9705/  10000 (97.05%), best:    9743/10000 (97.43%), error:       985.71545, best error:       810.52625 on epoch:    47, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 24.3739566803s, input_fired_time: 0.8044242859, output_fired_time: 0.0000000000, propagate_forward_time: 21.0067081451, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5628242493
best training[          , genome  6703] predictions:   49542/  50000 (99.08%), best:    9743/10000 (97.43%), error:      1775.35852, best error:       810.52625 on epoch:    47, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 5.6140894890s, input_fired_time: 0.2809353769, output_fired_time: 0.0000000000, propagate_forward_time: 4.4909038544, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8422503471
   testing[          , genome  6703] predictions:    9782/  10000 (97.82%), best:    9743/10000 (97.43%), error:       683.72717, best error:       810.52625 on epoch:    47, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
03:29:54 (12264): called boinc_finish(0)

</stderr_txt>
]]>