Nameexact_genome_1509103842_22_16653_1
Workunit2273633
Created27 Oct 2017, 13:46:14 UTC
Sent27 Oct 2017, 14:26:29 UTC
Report deadline13 Dec 2017, 2:38:07 UTC
Received1 Nov 2017, 1:39:19 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID52322
Run time1 days 2 hours 10 min 44 sec
CPU time1 days 2 hours 2 min 54 sec
Validate stateInitial
Credit0.00
Device peak FLOPS2.94 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size169.82 MB
Peak swap size168.38 MB
Peak disk usage6.37 MB

Stderr output

<core_client_version>7.6.33</core_client_version>
<![CDATA[
<stderr_txt>
484526157, other_time: 9.2447204590
epoch time: 67.8258743286s, input_fired_time: 0.9956838489, output_fired_time: 0.0000000000, propagate_forward_time: 65.2608642578, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5693283081
validation[          , genome 16653] predictions:    9858/  10000 (98.58%), best:    9861/10000 (98.61%), error:       474.36810, best error:       439.35510 on epoch:    26, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 864.1490478516s, input_fired_time: 8.0458908081, output_fired_time: 8.2554864883, propagate_forward_time: 326.7193603516, propagate_backward_time: 511.9376220703, weight_update_time: 1.0422163010, other_time: 9.1906738281
epoch time: 68.1401290894s, input_fired_time: 0.9998480082, output_fired_time: 0.0000000000, propagate_forward_time: 65.5863647461, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5539169312
validation[          , genome 16653] predictions:    9858/  10000 (98.58%), best:    9861/10000 (98.61%), error:       448.40741, best error:       439.35510 on epoch:    26, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 863.9997558594s, input_fired_time: 8.0300474167, output_fired_time: 8.2198972702, propagate_forward_time: 327.3904418945, propagate_backward_time: 511.2115783691, weight_update_time: 1.0461499691, other_time: 9.1477966309
epoch time: 67.4137115479s, input_fired_time: 0.9923654199, output_fired_time: 0.0000000000, propagate_forward_time: 64.8433837891, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5779647827
validation[          , genome 16653] predictions:    9850/  10000 (98.50%), best:    9861/10000 (98.61%), error:       505.07538, best error:       439.35510 on epoch:    26, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 860.7736816406s, input_fired_time: 8.0394697189, output_fired_time: 8.2290019989, propagate_forward_time: 325.5996398926, propagate_backward_time: 509.7431640625, weight_update_time: 1.0458952188, other_time: 9.1623535156
epoch time: 67.3612365723s, input_fired_time: 1.0008196831, output_fired_time: 0.0000000000, propagate_forward_time: 64.7835617065, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5768585205
validation[          , genome 16653] predictions:    9836/  10000 (98.36%), best:    9861/10000 (98.61%), error:       513.02679, best error:       439.35510 on epoch:    26, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 862.1000976563s, input_fired_time: 8.0464878082, output_fired_time: 8.2246608734, propagate_forward_time: 326.3825988770, propagate_backward_time: 510.2561950684, weight_update_time: 1.0419986248, other_time: 9.1900939941
epoch time: 67.2854537964s, input_fired_time: 0.9890289307, output_fired_time: 0.0000000000, propagate_forward_time: 64.7286605835, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5677642822
validation[          , genome 16653] predictions:    9871/  10000 (98.71%), best:    9861/10000 (98.61%), error:       444.10638, best error:       439.35510 on epoch:    26, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 863.5037841797s, input_fired_time: 8.0643043518, output_fired_time: 8.2293653488, propagate_forward_time: 326.8194274902, propagate_backward_time: 511.2092285156, weight_update_time: 1.0391339064, other_time: 9.1813964844
epoch time: 67.8903427124s, input_fired_time: 1.0016708374, output_fired_time: 0.0000000000, propagate_forward_time: 65.3114547729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5772171021
validation[          , genome 16653] predictions:    9847/  10000 (98.47%), best:    9861/10000 (98.61%), error:       454.76569, best error:       439.35510 on epoch:    26, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 863.8958129883s, input_fired_time: 8.0642518997, output_fired_time: 8.2254323959, propagate_forward_time: 327.1854858398, propagate_backward_time: 511.2867736816, weight_update_time: 1.0427824259, other_time: 9.1338806152
epoch time: 68.2288131714s, input_fired_time: 1.0066722631, output_fired_time: 0.0000000000, propagate_forward_time: 65.6813583374, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5407791138
validation[          , genome 16653] predictions:    9830/  10000 (98.30%), best:    9861/10000 (98.61%), error:       531.28833, best error:       439.35510 on epoch:    26, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 862.1257324219s, input_fired_time: 8.0567083359, output_fired_time: 8.2444992065, propagate_forward_time: 326.0279541016, propagate_backward_time: 510.6528930664, weight_update_time: 1.0389499664, other_time: 9.1436767578
epoch time: 68.2450790405s, input_fired_time: 0.9973925352, output_fired_time: 0.0000000000, propagate_forward_time: 65.6954421997, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5522460938
validation[          , genome 16653] predictions:    9876/  10000 (98.76%), best:    9876/10000 (98.76%), error:       420.27386, best error:       420.27386 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 861.2541503906s, input_fired_time: 8.0585622787, output_fired_time: 8.1947984695, propagate_forward_time: 325.2533264160, propagate_backward_time: 510.5509033203, weight_update_time: 1.0428125858, other_time: 9.1965332031
epoch time: 67.7784194946s, input_fired_time: 1.0002813339, output_fired_time: 0.0000000000, propagate_forward_time: 65.2275619507, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5505752563
validation[          , genome 16653] predictions:    9864/  10000 (98.64%), best:    9876/10000 (98.76%), error:       445.81503, best error:       420.27386 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 862.1894531250s, input_fired_time: 8.0453548431, output_fired_time: 8.2152004242, propagate_forward_time: 326.5443420410, propagate_backward_time: 510.2197875977, weight_update_time: 1.0361608267, other_time: 9.1647338867
epoch time: 68.1046218872s, input_fired_time: 1.0074199438, output_fired_time: 0.0000000000, propagate_forward_time: 65.5312881470, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5659103394
validation[          , genome 16653] predictions:    9860/  10000 (98.60%), best:    9876/10000 (98.76%), error:       459.04205, best error:       420.27386 on epoch:    34, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 860.6694335938s, input_fired_time: 8.0515460968, output_fired_time: 8.2187013626, propagate_forward_time: 326.0112915039, propagate_backward_time: 509.2344970703, weight_update_time: 1.0373988152, other_time: 9.1533813477
epoch time: 67.2828903198s, input_fired_time: 0.9970548153, output_fired_time: 0.0000000000, propagate_forward_time: 64.6989059448, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5869293213
validation[          , genome 16653] predictions:    9866/  10000 (98.66%), best:    9866/10000 (98.66%), error:       398.78091, best error:       398.78091 on epoch:    37, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 862.5229492188s, input_fired_time: 8.0579366684, output_fired_time: 8.2224788666, propagate_forward_time: 326.1143188477, propagate_backward_time: 510.9437561035, weight_update_time: 1.0465279818, other_time: 9.1844787598
epoch time: 68.2509078979s, input_fired_time: 1.0003589392, output_fired_time: 0.0000000000, propagate_forward_time: 65.6989364624, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5516128540
validation[          , genome 16653] predictions:    9878/  10000 (98.78%), best:    9866/10000 (98.66%), error:       400.11450, best error:       398.78091 on epoch:    37, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 863.7171020508s, input_fired_time: 8.0482254028, output_fired_time: 8.2355737686, propagate_forward_time: 326.9528503418, propagate_backward_time: 511.3370666504, weight_update_time: 1.0454183817, other_time: 9.1434020996
epoch time: 67.2363204956s, input_fired_time: 0.9969247580, output_fired_time: 0.0000000000, propagate_forward_time: 64.6525573730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5868377686
validation[          , genome 16653] predictions:    9875/  10000 (98.75%), best:    9875/10000 (98.75%), error:       383.67593, best error:       383.67593 on epoch:    39, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 862.1746826172s, input_fired_time: 8.0697946548, output_fired_time: 8.2050523758, propagate_forward_time: 325.7766723633, propagate_backward_time: 510.9672851563, weight_update_time: 1.0410944223, other_time: 9.1558227539
epoch time: 67.6364288330s, input_fired_time: 0.9998463988, output_fired_time: 0.0000000000, propagate_forward_time: 65.0745315552, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5620498657
validation[          , genome 16653] predictions:    9844/  10000 (98.44%), best:    9875/10000 (98.75%), error:       500.91257, best error:       383.67593 on epoch:    39, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 862.0967407227s, input_fired_time: 8.0634012222, output_fired_time: 8.2291221619, propagate_forward_time: 326.0173950195, propagate_backward_time: 510.6121826172, weight_update_time: 1.0501503944, other_time: 9.1746215820
epoch time: 67.6077346802s, input_fired_time: 1.0035722256, output_fired_time: 0.0000000000, propagate_forward_time: 65.0473785400, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5567855835
validation[          , genome 16653] predictions:    9846/  10000 (98.46%), best:    9875/10000 (98.75%), error:       480.08234, best error:       383.67593 on epoch:    39, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 862.2715454102s, input_fired_time: 8.0494413376, output_fired_time: 8.2441139221, propagate_forward_time: 326.5899963379, propagate_backward_time: 510.2402954102, weight_update_time: 1.0455749035, other_time: 9.1476440430
epoch time: 67.8218841553s, input_fired_time: 1.0072479248, output_fired_time: 0.0000000000, propagate_forward_time: 65.2444763184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5701599121
validation[          , genome 16653] predictions:    9885/  10000 (98.85%), best:    9875/10000 (98.75%), error:       402.91223, best error:       383.67593 on epoch:    39, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 861.6818237305s, input_fired_time: 8.0382986069, output_fired_time: 8.2525920868, propagate_forward_time: 326.4110107422, propagate_backward_time: 509.8180847168, weight_update_time: 1.0490024090, other_time: 9.1618957520
epoch time: 67.9798507690s, input_fired_time: 1.0004712343, output_fired_time: 0.0000000000, propagate_forward_time: 65.4231185913, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5562591553
validation[          , genome 16653] predictions:    9860/  10000 (98.60%), best:    9875/10000 (98.75%), error:       456.05399, best error:       383.67593 on epoch:    39, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 862.6481933594s, input_fired_time: 8.0510091782, output_fired_time: 8.2226572037, propagate_forward_time: 326.3385314941, propagate_backward_time: 510.8532714844, weight_update_time: 1.0443463326, other_time: 9.1827392578
epoch time: 67.5119857788s, input_fired_time: 0.9985357523, output_fired_time: 0.0000000000, propagate_forward_time: 64.9284057617, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5850448608
validation[          , genome 16653] predictions:    9880/  10000 (98.80%), best:    9875/10000 (98.75%), error:       397.51147, best error:       383.67593 on epoch:    39, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 862.2394409180s, input_fired_time: 8.0539340973, output_fired_time: 8.2227039337, propagate_forward_time: 326.5088806152, propagate_backward_time: 510.2982788086, weight_update_time: 1.0533298254, other_time: 9.1555786133
epoch time: 67.9515304565s, input_fired_time: 0.9959217310, output_fired_time: 0.0000000000, propagate_forward_time: 65.4181671143, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5374450684
validation[          , genome 16653] predictions:    9878/  10000 (98.78%), best:    9878/10000 (98.78%), error:       379.02917, best error:       379.02917 on epoch:    45, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 865.0803833008s, input_fired_time: 8.0786962509, output_fired_time: 8.2601795197, propagate_forward_time: 327.5388793945, propagate_backward_time: 512.0343017578, weight_update_time: 1.0487147570, other_time: 9.1683349609
epoch time: 67.5346221924s, input_fired_time: 0.9977787733, output_fired_time: 0.0000000000, propagate_forward_time: 64.9722900391, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5645523071
validation[          , genome 16653] predictions:    9852/  10000 (98.52%), best:    9878/10000 (98.78%), error:       457.23557, best error:       379.02917 on epoch:    45, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 863.1022949219s, input_fired_time: 8.0652465820, output_fired_time: 8.2332563400, propagate_forward_time: 326.4269714355, propagate_backward_time: 511.2484741211, weight_update_time: 1.0419064760, other_time: 9.1283569336
epoch time: 67.7354660034s, input_fired_time: 1.0082442760, output_fired_time: 0.0000000000, propagate_forward_time: 65.1519699097, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5752487183
validation[          , genome 16653] predictions:    9882/  10000 (98.82%), best:    9878/10000 (98.78%), error:       414.25015, best error:       379.02917 on epoch:    45, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 863.8759155273s, input_fired_time: 8.0515813828, output_fired_time: 8.2459096909, propagate_forward_time: 326.5607910156, propagate_backward_time: 511.8877868652, weight_update_time: 1.0466457605, other_time: 9.1298522949
epoch time: 67.3309020996s, input_fired_time: 0.9943088889, output_fired_time: 0.0000000000, propagate_forward_time: 64.7754440308, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5611495972
validation[          , genome 16653] predictions:    9864/  10000 (98.64%), best:    9878/10000 (98.78%), error:       432.61496, best error:       379.02917 on epoch:    45, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 863.0360107422s, input_fired_time: 8.0653581619, output_fired_time: 8.2474288940, propagate_forward_time: 326.5553588867, propagate_backward_time: 510.9793701172, weight_update_time: 1.0442290306, other_time: 9.1884765625
epoch time: 68.0541229248s, input_fired_time: 1.0044698715, output_fired_time: 0.0000000000, propagate_forward_time: 65.4840927124, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5655593872
validation[          , genome 16653] predictions:    9890/  10000 (98.90%), best:    9878/10000 (98.78%), error:       396.00534, best error:       379.02917 on epoch:    45, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 862.0354003906s, input_fired_time: 8.0636434555, output_fired_time: 8.2236118317, propagate_forward_time: 325.7791748047, propagate_backward_time: 510.8126220703, weight_update_time: 1.0450358391, other_time: 9.1563110352
epoch time: 68.0794525146s, input_fired_time: 1.0042027235, output_fired_time: 0.0000000000, propagate_forward_time: 65.5315246582, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5437240601
validation[          , genome 16653] predictions:    9882/  10000 (98.82%), best:    9878/10000 (98.78%), error:       394.27438, best error:       379.02917 on epoch:    45, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 862.4166870117s, input_fired_time: 8.0524396896, output_fired_time: 8.2264423370, propagate_forward_time: 325.7075805664, propagate_backward_time: 511.2790222168, weight_update_time: 1.0381708145, other_time: 9.1512145996
epoch time: 68.2981643677s, input_fired_time: 1.0096828938, output_fired_time: 0.0000000000, propagate_forward_time: 65.7285766602, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5599060059
validation[          , genome 16653] predictions:    9886/  10000 (98.86%), best:    9886/10000 (98.86%), error:       365.55524, best error:       365.55524 on epoch:    51, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 864.1190185547s, input_fired_time: 8.0505933762, output_fired_time: 8.2608966827, propagate_forward_time: 326.7828369141, propagate_backward_time: 511.8670654297, weight_update_time: 1.0407352448, other_time: 9.1575927734
epoch time: 67.7710723877s, input_fired_time: 1.0041476488, output_fired_time: 0.0000000000, propagate_forward_time: 65.1911621094, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5757598877
validation[          , genome 16653] predictions:    9893/  10000 (98.93%), best:    9886/10000 (98.86%), error:       374.93054, best error:       365.55524 on epoch:    51, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 864.0831298828s, input_fired_time: 8.0712366104, output_fired_time: 8.2193717957, propagate_forward_time: 326.8547668457, propagate_backward_time: 511.7646179199, weight_update_time: 1.0458556414, other_time: 9.1731262207
epoch time: 67.9801712036s, input_fired_time: 1.0016298294, output_fired_time: 0.0000000000, propagate_forward_time: 65.4235153198, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5550231934
validation[          , genome 16653] predictions:    9878/  10000 (98.78%), best:    9886/10000 (98.86%), error:       445.68512, best error:       365.55524 on epoch:    51, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 863.5708007813s, input_fired_time: 8.0815896988, output_fired_time: 8.2341718674, propagate_forward_time: 326.9057617188, propagate_backward_time: 511.1524963379, weight_update_time: 1.0513300896, other_time: 9.1967468262
epoch time: 67.7371520996s, input_fired_time: 1.0106803179, output_fired_time: 0.0000000000, propagate_forward_time: 65.1619720459, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5644989014
validation[          , genome 16653] predictions:    9875/  10000 (98.75%), best:    9886/10000 (98.86%), error:       401.28574, best error:       365.55524 on epoch:    51, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 863.4860229492s, input_fired_time: 8.0952434540, output_fired_time: 8.2311019897, propagate_forward_time: 326.7133178711, propagate_backward_time: 511.2419433594, weight_update_time: 1.0451407433, other_time: 9.2044677734
epoch time: 67.6522445679s, input_fired_time: 1.0046904087, output_fired_time: 0.0000000000, propagate_forward_time: 65.0806884766, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5668640137
validation[          , genome 16653] predictions:    9891/  10000 (98.91%), best:    9891/10000 (98.91%), error:       361.76129, best error:       361.76129 on epoch:    55, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 863.0900878906s, input_fired_time: 8.0830526352, output_fired_time: 8.2373008728, propagate_forward_time: 326.5588989258, propagate_backward_time: 511.0484313965, weight_update_time: 1.0447018147, other_time: 9.1623840332
epoch time: 67.7216491699s, input_fired_time: 1.0142978430, output_fired_time: 0.0000000000, propagate_forward_time: 65.1338500977, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5735015869
validation[          , genome 16653] predictions:    9894/  10000 (98.94%), best:    9891/10000 (98.91%), error:       377.59299, best error:       361.76129 on epoch:    55, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 860.4225463867s, input_fired_time: 8.0847959518, output_fired_time: 8.2607669830, propagate_forward_time: 325.5792541504, propagate_backward_time: 509.3535156250, weight_update_time: 1.0410969257, other_time: 9.1442871094
epoch time: 68.0559997559s, input_fired_time: 1.0045783520, output_fired_time: 0.0000000000, propagate_forward_time: 65.4853591919, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5660629272
validation[          , genome 16653] predictions:    9888/  10000 (98.88%), best:    9891/10000 (98.91%), error:       369.75131, best error:       361.76129 on epoch:    55, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 861.3048095703s, input_fired_time: 8.0872354507, output_fired_time: 8.1893157959, propagate_forward_time: 325.8933410645, propagate_backward_time: 510.0014648438, weight_update_time: 1.0448879004, other_time: 9.1334228516
epoch time: 67.3942489624s, input_fired_time: 1.0118935108, output_fired_time: 0.0000000000, propagate_forward_time: 64.8182067871, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5641479492
validation[          , genome 16653] predictions:    9881/  10000 (98.81%), best:    9891/10000 (98.91%), error:       392.10019, best error:       361.76129 on epoch:    55, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 863.8811035156s, input_fired_time: 8.0823183060, output_fired_time: 8.1963443756, propagate_forward_time: 326.4093933105, propagate_backward_time: 511.9773864746, weight_update_time: 1.0484523773, other_time: 9.2156066895
epoch time: 68.0340347290s, input_fired_time: 1.0046056509, output_fired_time: 0.0000000000, propagate_forward_time: 65.4648590088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5645675659
validation[          , genome 16653] predictions:    9879/  10000 (98.79%), best:    9891/10000 (98.91%), error:       404.66559, best error:       361.76129 on epoch:    55, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 866.7838134766s, input_fired_time: 8.1157684326, output_fired_time: 8.2777910233, propagate_forward_time: 327.6460571289, propagate_backward_time: 513.5352172852, weight_update_time: 1.0480246544, other_time: 9.2089843750
epoch time: 68.3260574341s, input_fired_time: 1.0159273148, output_fired_time: 0.0000000000, propagate_forward_time: 65.7446365356, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5654907227
validation[          , genome 16653] predictions:    9881/  10000 (98.81%), best:    9891/10000 (98.91%), error:       386.17432, best error:       361.76129 on epoch:    55, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 867.7775878906s, input_fired_time: 8.1350202560, output_fired_time: 8.2522640228, propagate_forward_time: 328.5564575195, propagate_backward_time: 513.6450805664, weight_update_time: 1.0508456230, other_time: 9.1887817383
epoch time: 68.9307861328s, input_fired_time: 1.0095832348, output_fired_time: 0.0000000000, propagate_forward_time: 66.3720932007, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5491104126
validation[          , genome 16653] predictions:    9883/  10000 (98.83%), best:    9891/10000 (98.91%), error:       379.45251, best error:       361.76129 on epoch:    55, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 866.6086425781s, input_fired_time: 8.1275377274, output_fired_time: 8.2445116043, propagate_forward_time: 328.2782287598, propagate_backward_time: 512.7214355469, weight_update_time: 1.0494499207, other_time: 9.2369384766
epoch time: 68.2166900635s, input_fired_time: 1.0114064217, output_fired_time: 0.0000000000, propagate_forward_time: 65.6319503784, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5733337402
validation[          , genome 16653] predictions:    9875/  10000 (98.75%), best:    9891/10000 (98.91%), error:       392.25604, best error:       361.76129 on epoch:    55, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 867.1928100586s, input_fired_time: 8.1183776855, output_fired_time: 8.2933692932, propagate_forward_time: 327.8983459473, propagate_backward_time: 513.6872558594, weight_update_time: 1.0493954420, other_time: 9.1954345703
epoch time: 68.3132781982s, input_fired_time: 1.0141923428, output_fired_time: 0.0000000000, propagate_forward_time: 65.7376403809, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5614471436
validation[          , genome 16653] predictions:    9883/  10000 (98.83%), best:    9891/10000 (98.91%), error:       388.56586, best error:       361.76129 on epoch:    55, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 868.7831420898s, input_fired_time: 8.1146783829, output_fired_time: 8.2713670731, propagate_forward_time: 329.1446838379, propagate_backward_time: 514.0728149414, weight_update_time: 1.0501979589, other_time: 9.1796264648
epoch time: 68.0527877808s, input_fired_time: 1.0156489611, output_fired_time: 0.0000000000, propagate_forward_time: 65.4417266846, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5954132080
validation[          , genome 16653] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       358.03336, best error:       358.03336 on epoch:    64, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 870.6244506836s, input_fired_time: 8.1403694153, output_fired_time: 8.3025140762, propagate_forward_time: 329.6409912109, propagate_backward_time: 515.3623046875, weight_update_time: 1.0494093895, other_time: 9.1782836914
epoch time: 68.6333084106s, input_fired_time: 1.0123291016, output_fired_time: 0.0000000000, propagate_forward_time: 66.0438995361, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5770797729
validation[          , genome 16653] predictions:    9885/  10000 (98.85%), best:    9892/10000 (98.92%), error:       378.29434, best error:       358.03336 on epoch:    64, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 870.0653686523s, input_fired_time: 8.1447343826, output_fired_time: 8.3113012314, propagate_forward_time: 329.3094177246, propagate_backward_time: 515.0968627930, weight_update_time: 1.0557274818, other_time: 9.2030639648
epoch time: 68.6984252930s, input_fired_time: 1.0175108910, output_fired_time: 0.0000000000, propagate_forward_time: 66.1174316406, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5634841919
validation[          , genome 16653] predictions:    9890/  10000 (98.90%), best:    9892/10000 (98.92%), error:       360.21063, best error:       358.03336 on epoch:    64, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 868.8597412109s, input_fired_time: 8.1578168869, output_fired_time: 8.2990255356, propagate_forward_time: 328.6994018555, propagate_backward_time: 514.4739990234, weight_update_time: 1.0580209494, other_time: 9.2294921875
epoch time: 67.9481582642s, input_fired_time: 1.0141267776, output_fired_time: 0.0000000000, propagate_forward_time: 65.3651428223, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5688858032
validation[          , genome 16653] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       363.60629, best error:       358.03336 on epoch:    64, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 869.0953369141s, input_fired_time: 8.1587238312, output_fired_time: 8.2770843506, propagate_forward_time: 329.3218078613, propagate_backward_time: 514.1725463867, weight_update_time: 1.0515393019, other_time: 9.1651000977
epoch time: 68.5816345215s, input_fired_time: 1.0137870312, output_fired_time: 0.0000000000, propagate_forward_time: 65.9911270142, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5767211914
validation[          , genome 16653] predictions:    9874/  10000 (98.74%), best:    9892/10000 (98.92%), error:       417.11874, best error:       358.03336 on epoch:    64, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 867.6150512695s, input_fired_time: 8.1400337219, output_fired_time: 8.2455129623, propagate_forward_time: 328.7672119141, propagate_backward_time: 513.2705688477, weight_update_time: 1.0469237566, other_time: 9.1917724609
epoch time: 68.7967910767s, input_fired_time: 1.0165729523, output_fired_time: 0.0000000000, propagate_forward_time: 66.2123565674, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5678634644
validation[          , genome 16653] predictions:    9890/  10000 (98.90%), best:    9892/10000 (98.92%), error:       372.09863, best error:       358.03336 on epoch:    64, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 870.2175292969s, input_fired_time: 8.1671123505, output_fired_time: 8.2699642181, propagate_forward_time: 329.0242309570, propagate_backward_time: 515.5355834961, weight_update_time: 1.0458016396, other_time: 9.2206420898
epoch time: 68.1701660156s, input_fired_time: 1.0048451424, output_fired_time: 0.0000000000, propagate_forward_time: 65.5975265503, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5677947998
validation[          , genome 16653] predictions:    9879/  10000 (98.79%), best:    9892/10000 (98.92%), error:       407.13400, best error:       358.03336 on epoch:    64, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 869.9564208984s, input_fired_time: 8.1608018875, output_fired_time: 8.2995977402, propagate_forward_time: 329.2410278320, propagate_backward_time: 515.0197753906, weight_update_time: 1.0519598722, other_time: 9.2351684570
epoch time: 68.6912994385s, input_fired_time: 1.0190763474, output_fired_time: 0.0000000000, propagate_forward_time: 66.0902862549, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5819396973
validation[          , genome 16653] predictions:    9886/  10000 (98.86%), best:    9892/10000 (98.92%), error:       374.34201, best error:       358.03336 on epoch:    64, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 870.5440673828s, input_fired_time: 8.1792469025, output_fired_time: 8.2651319504, propagate_forward_time: 330.0750122070, propagate_backward_time: 514.7843017578, weight_update_time: 1.0557477474, other_time: 9.2403564453
epoch time: 68.6005935669s, input_fired_time: 1.0152297020, output_fired_time: 0.0000000000, propagate_forward_time: 66.0095138550, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5758514404
validation[          , genome 16653] predictions:    9880/  10000 (98.80%), best:    9892/10000 (98.92%), error:       400.01291, best error:       358.03336 on epoch:    64, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 868.5934448242s, input_fired_time: 8.1536312103, output_fired_time: 8.2909450531, propagate_forward_time: 328.2983703613, propagate_backward_time: 514.6325683594, weight_update_time: 1.0576961040, other_time: 9.2178955078
epoch time: 68.0288391113s, input_fired_time: 1.0137728453, output_fired_time: 0.0000000000, propagate_forward_time: 65.4496612549, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5654067993
validation[          , genome 16653] predictions:    9878/  10000 (98.78%), best:    9892/10000 (98.92%), error:       425.88660, best error:       358.03336 on epoch:    64, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 868.8238525391s, input_fired_time: 8.1460676193, output_fired_time: 8.2461490631, propagate_forward_time: 328.4395446777, propagate_backward_time: 514.7987670898, weight_update_time: 1.0493556261, other_time: 9.1932983398
epoch time: 68.5827178955s, input_fired_time: 1.0161986351, output_fired_time: 0.0000000000, propagate_forward_time: 65.9840621948, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5824584961
validation[          , genome 16653] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       376.67593, best error:       358.03336 on epoch:    64, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 871.4808349609s, input_fired_time: 8.1606626511, output_fired_time: 8.2702188492, propagate_forward_time: 330.2940368652, propagate_backward_time: 515.6007080078, weight_update_time: 1.0529247522, other_time: 9.1552734375
epoch time: 68.2092971802s, input_fired_time: 1.0146381855, output_fired_time: 0.0000000000, propagate_forward_time: 65.6407241821, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5539321899
validation[          , genome 16653] predictions:    9896/  10000 (98.96%), best:    9896/10000 (98.96%), error:       355.35886, best error:       355.35886 on epoch:    75, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 867.5149536133s, input_fired_time: 8.1446199417, output_fired_time: 8.2743911743, propagate_forward_time: 327.8974609375, propagate_backward_time: 514.0289306641, weight_update_time: 1.0534278154, other_time: 9.1695556641
epoch time: 68.6712722778s, input_fired_time: 1.0141583681, output_fired_time: 0.0000000000, propagate_forward_time: 66.0951156616, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5619964600
validation[          , genome 16653] predictions:    9871/  10000 (98.71%), best:    9896/10000 (98.96%), error:       443.52289, best error:       355.35886 on epoch:    75, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 867.5293579102s, input_fired_time: 8.1433057785, output_fired_time: 8.2799539566, propagate_forward_time: 328.0238037109, propagate_backward_time: 513.8834838867, weight_update_time: 1.0484635830, other_time: 9.1987915039
epoch time: 68.3818969727s, input_fired_time: 1.0198742151, output_fired_time: 0.0000000000, propagate_forward_time: 65.7858581543, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5761642456
validation[          , genome 16653] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       352.99744, best error:       352.99744 on epoch:    77, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 870.3648681641s, input_fired_time: 8.1550226212, output_fired_time: 8.2967815399, propagate_forward_time: 329.6032409668, propagate_backward_time: 515.1105346680, weight_update_time: 1.0523718596, other_time: 9.1992797852
epoch time: 67.9361114502s, input_fired_time: 1.0228861570, output_fired_time: 0.0000000000, propagate_forward_time: 65.3336410522, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5795822144
validation[          , genome 16653] predictions:    9893/  10000 (98.93%), best:    9892/10000 (98.92%), error:       363.27261, best error:       352.99744 on epoch:    77, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 868.6439208984s, input_fired_time: 8.1580572128, output_fired_time: 8.2816295624, propagate_forward_time: 328.7742004395, propagate_backward_time: 514.2498168945, weight_update_time: 1.0441696644, other_time: 9.1802368164
epoch time: 68.6717605591s, input_fired_time: 1.0241943598, output_fired_time: 0.0000000000, propagate_forward_time: 66.0545578003, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5930099487
validation[          , genome 16653] predictions:    9844/  10000 (98.44%), best:    9892/10000 (98.92%), error:       508.96289, best error:       352.99744 on epoch:    77, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 870.4271850586s, input_fired_time: 8.1662578583, output_fired_time: 8.2691717148, propagate_forward_time: 329.8947448730, propagate_backward_time: 514.8942871094, weight_update_time: 1.0543618202, other_time: 9.2027587891
epoch time: 68.1575164795s, input_fired_time: 1.0177922249, output_fired_time: 0.0000000000, propagate_forward_time: 65.5738983154, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5658264160
validation[          , genome 16653] predictions:    9888/  10000 (98.88%), best:    9892/10000 (98.92%), error:       372.80469, best error:       352.99744 on epoch:    77, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 871.1587524414s, input_fired_time: 8.1930475235, output_fired_time: 8.3002166748, propagate_forward_time: 329.9838562012, propagate_backward_time: 515.4735107422, weight_update_time: 1.0498478413, other_time: 9.2081298828
epoch time: 68.3168716431s, input_fired_time: 1.0175390244, output_fired_time: 0.0000000000, propagate_forward_time: 65.7209396362, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5783920288
validation[          , genome 16653] predictions:    9891/  10000 (98.91%), best:    9892/10000 (98.92%), error:       354.53040, best error:       352.99744 on epoch:    77, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 870.3435668945s, input_fired_time: 8.1800765991, output_fired_time: 8.2588319778, propagate_forward_time: 329.8429870605, propagate_backward_time: 514.8184204102, weight_update_time: 1.0427045822, other_time: 9.2432250977
epoch time: 68.2312316895s, input_fired_time: 1.0186549425, output_fired_time: 0.0000000000, propagate_forward_time: 65.6248550415, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5877227783
validation[          , genome 16653] predictions:    9897/  10000 (98.97%), best:    9897/10000 (98.97%), error:       352.96716, best error:       352.96716 on epoch:    82, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 871.2349243164s, input_fired_time: 8.1687221527, output_fired_time: 8.2741117477, propagate_forward_time: 329.4630126953, propagate_backward_time: 516.0927124023, weight_update_time: 1.0469850302, other_time: 9.2363891602
epoch time: 68.2407989502s, input_fired_time: 1.0206900835, output_fired_time: 0.0000000000, propagate_forward_time: 65.6603775024, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5597305298
validation[          , genome 16653] predictions:    9887/  10000 (98.87%), best:    9897/10000 (98.97%), error:       355.12799, best error:       352.96716 on epoch:    82, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 871.0424804688s, input_fired_time: 8.1823968887, output_fired_time: 8.3015804291, propagate_forward_time: 329.4547119141, propagate_backward_time: 515.9080200195, weight_update_time: 1.0444750786, other_time: 9.1958007813
epoch time: 68.2648315430s, input_fired_time: 1.0167133808, output_fired_time: 0.0000000000, propagate_forward_time: 65.6556930542, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5924224854
validation[          , genome 16653] predictions:    9890/  10000 (98.90%), best:    9897/10000 (98.97%), error:       366.29355, best error:       352.96716 on epoch:    82, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 871.1849365234s, input_fired_time: 8.1696777344, output_fired_time: 8.2878055573, propagate_forward_time: 329.8998718262, propagate_backward_time: 515.6255493164, weight_update_time: 1.0504455566, other_time: 9.2020874023
epoch time: 68.9328460693s, input_fired_time: 1.0246968269, output_fired_time: 0.0000000000, propagate_forward_time: 66.3567962646, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5513534546
validation[          , genome 16653] predictions:    9887/  10000 (98.87%), best:    9897/10000 (98.97%), error:       382.70941, best error:       352.96716 on epoch:    82, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 871.8728637695s, input_fired_time: 8.1650381088, output_fired_time: 8.3056821823, propagate_forward_time: 330.2543640137, propagate_backward_time: 515.9981689453, weight_update_time: 1.0470076799, other_time: 9.1496582031
epoch time: 68.2894439697s, input_fired_time: 1.0084353685, output_fired_time: 0.0000000000, propagate_forward_time: 65.7297821045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5512237549
validation[          , genome 16653] predictions:    9899/  10000 (98.99%), best:    9899/10000 (98.99%), error:       347.24524, best error:       347.24524 on epoch:    86, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 870.9583129883s, input_fired_time: 8.1710510254, output_fired_time: 8.2853641510, propagate_forward_time: 329.8517150879, propagate_backward_time: 515.5369873047, weight_update_time: 1.0474998951, other_time: 9.1131591797
epoch time: 68.7136764526s, input_fired_time: 1.0231481791, output_fired_time: 0.0000000000, propagate_forward_time: 66.1212463379, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5692825317
validation[          , genome 16653] predictions:    9896/  10000 (98.96%), best:    9899/10000 (98.99%), error:       360.46249, best error:       347.24524 on epoch:    86, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1509103842_22_16653.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1509103842_22_16653_1_r1179361437_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.984631
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000136958
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 5.47831e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 88
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 9899
read best_validation_error: 347.245
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 16653
read normal distribution: '1 0.24388861656189 1.80698049068451'
generator_str: '812569395'
read generator: 812569395
read generated_by_map:
1 split_edge 1
reading 94 nodes.
reading 339 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 728.962951660156s, input_fired_time: 7.16675567626953, output_fired_time: 6.94256210327148, propagate_forward_time: 273.492828369141, propagate_backward_time: 433.209075927734, weight_update_time: 0.900842726230621, other_time: 8.1517333984375
epoch time: 61.2921981811523s, input_fired_time: 0.934844553470612, output_fired_time: 0, propagate_forward_time: 58.9159393310547, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.44141387939453
validation[          , genome 16653] predictions:    9893/  10000 (98.93%), best:    9899/10000 (98.99%), error:       360.25504, best error:       347.24524 on epoch:    86, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 785.0792846680s, input_fired_time: 7.5099172592, output_fired_time: 7.4617156982, propagate_forward_time: 292.8919067383, propagate_backward_time: 468.7263488770, weight_update_time: 0.9488440156, other_time: 8.4894104004
epoch time: 61.7162857056s, input_fired_time: 0.9473549128, output_fired_time: 0.0000000000, propagate_forward_time: 59.3279685974, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4409637451
validation[          , genome 16653] predictions:    9893/  10000 (98.93%), best:    9899/10000 (98.99%), error:       375.81107, best error:       347.24524 on epoch:    86, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 764.9196166992s, input_fired_time: 7.3924961090, output_fired_time: 7.2827286720, propagate_forward_time: 286.9017028809, propagate_backward_time: 454.9652709961, weight_update_time: 0.9264920950, other_time: 8.3774108887
epoch time: 65.2778244019s, input_fired_time: 0.9919109941, output_fired_time: 0.0000000000, propagate_forward_time: 62.7642364502, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5216751099
validation[          , genome 16653] predictions:    9902/  10000 (99.02%), best:    9899/10000 (98.99%), error:       347.67145, best error:       347.24524 on epoch:    86, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 842.4924316406s, input_fired_time: 7.9253034592, output_fired_time: 7.9958987236, propagate_forward_time: 317.3989868164, propagate_backward_time: 500.3784179688, weight_update_time: 1.0032002926, other_time: 8.7938232422
epoch time: 66.4737854004s, input_fired_time: 0.9938789010, output_fired_time: 0.0000000000, propagate_forward_time: 63.9892997742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4906044006
validation[          , genome 16653] predictions:    9897/  10000 (98.97%), best:    9899/10000 (98.99%), error:       358.63138, best error:       347.24524 on epoch:    86, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 860.6555786133s, input_fired_time: 8.0608606339, output_fired_time: 8.1646986008, propagate_forward_time: 326.2210693359, propagate_backward_time: 509.3192749023, weight_update_time: 1.0202974081, other_time: 8.8897094727
epoch time: 68.3516311646s, input_fired_time: 1.0098708868, output_fired_time: 0.0000000000, propagate_forward_time: 65.8355941772, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5061645508
validation[          , genome 16653] predictions:    9898/  10000 (98.98%), best:    9899/10000 (98.99%), error:       358.35159, best error:       347.24524 on epoch:    86, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1509103842_22_16653.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1509103842_22_16653_1_r1179361437_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.985846
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000105975
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 4.23901e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 93
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 86
read number_validation_images: 10000
read best_validation_predictions: 9899
read best_validation_error: 347.245
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 16653
read normal distribution: '1 0.24388861656189 1.80698049068451'
generator_str: '252968482'
read generator: 252968482
read generated_by_map:
1 split_edge 1
reading 94 nodes.
reading 339 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 788.507446289063s, input_fired_time: 7.65219306945801, output_fired_time: 7.5476279258728, propagate_forward_time: 296.258636474609, propagate_backward_time: 468.2373046875, weight_update_time: 0.983163475990295, other_time: 8.81167602539063
epoch time: 55.5346755981445s, input_fired_time: 0.883630454540253, output_fired_time: 0, propagate_forward_time: 53.1829147338867, propagate_backward_time: 0, weight_update_time: 0, other_time: 1.46813201904297
validation[          , genome 16653] predictions:    9897/  10000 (98.97%), best:    9899/10000 (98.99%), error:       360.00964, best error:       347.24524 on epoch:    86, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 763.0189819336s, input_fired_time: 7.5025620461, output_fired_time: 7.4139752388, propagate_forward_time: 284.1206970215, propagate_backward_time: 455.2189025879, weight_update_time: 0.9570648074, other_time: 8.7628173828
epoch time: 64.1215896606s, input_fired_time: 0.9830502868, output_fired_time: 0.0000000000, propagate_forward_time: 61.6037635803, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5347747803
validation[          , genome 16653] predictions:    9894/  10000 (98.94%), best:    9899/10000 (98.99%), error:       359.84793, best error:       347.24524 on epoch:    86, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 814.0340576172s, input_fired_time: 7.8283123970, output_fired_time: 7.8095850945, propagate_forward_time: 305.7517089844, propagate_backward_time: 483.6560058594, weight_update_time: 1.0035730600, other_time: 8.9884643555
epoch time: 57.3148078918s, input_fired_time: 0.9025611281, output_fired_time: 0.0000000000, propagate_forward_time: 54.9301910400, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.4820556641
validation[          , genome 16653] predictions:    9898/  10000 (98.98%), best:    9898/10000 (98.98%), error:       344.52029, best error:       344.52029 on epoch:    95, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 808.7008666992s, input_fired_time: 7.7692303658, output_fired_time: 7.7573251724, propagate_forward_time: 302.9028625488, propagate_backward_time: 481.3217773438, weight_update_time: 1.0071232319, other_time: 8.9496765137
epoch time: 62.3901824951s, input_fired_time: 0.9648081660, output_fired_time: 0.0000000000, propagate_forward_time: 59.9085044861, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5168685913
validation[          , genome 16653] predictions:    9895/  10000 (98.95%), best:    9898/10000 (98.98%), error:       349.22116, best error:       344.52029 on epoch:    95, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 813.4891357422s, input_fired_time: 7.8343605995, output_fired_time: 7.8232870102, propagate_forward_time: 305.1095886230, propagate_backward_time: 483.6958618164, weight_update_time: 1.0145069361, other_time: 9.0260314941
epoch time: 63.7886810303s, input_fired_time: 0.9832488298, output_fired_time: 0.0000000000, propagate_forward_time: 61.2525444031, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5528869629
validation[          , genome 16653] predictions:    9894/  10000 (98.94%), best:    9898/10000 (98.98%), error:       347.91107, best error:       344.52029 on epoch:    95, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 825.7246093750s, input_fired_time: 7.9216270447, output_fired_time: 7.9545445442, propagate_forward_time: 310.2230834961, propagate_backward_time: 490.5798339844, weight_update_time: 1.0175070763, other_time: 9.0455322266
epoch time: 63.1940689087s, input_fired_time: 0.9740709662, output_fired_time: 0.0000000000, propagate_forward_time: 60.6713867188, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5486106873
validation[          , genome 16653] predictions:    9898/  10000 (98.98%), best:    9898/10000 (98.98%), error:       353.29416, best error:       344.52029 on epoch:    95, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 837.0055541992s, input_fired_time: 7.9887280464, output_fired_time: 8.0145387650, propagate_forward_time: 315.1009521484, propagate_backward_time: 496.8498840332, weight_update_time: 1.0191246271, other_time: 9.0514831543
epoch time: 64.9884643555s, input_fired_time: 0.9849610329, output_fired_time: 0.0000000000, propagate_forward_time: 62.4612731934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5422286987
validation[          , genome 16653] predictions:    9898/  10000 (98.98%), best:    9898/10000 (98.98%), error:       351.69391, best error:       344.52029 on epoch:    95, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 844.9286499023s, input_fired_time: 8.0509719849, output_fired_time: 8.0916442871, propagate_forward_time: 318.6875305176, propagate_backward_time: 501.0119018555, weight_update_time: 1.0332814455, other_time: 9.0865783691
epoch time: 66.6814651489s, input_fired_time: 1.0075051785, output_fired_time: 0.0000000000, propagate_forward_time: 64.1136245728, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5603332520
validation[          , genome 16653] predictions:    9895/  10000 (98.95%), best:    9898/10000 (98.98%), error:       353.04788, best error:       344.52029 on epoch:    95, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 334.8261413574s, input_fired_time: 5.0438170433, output_fired_time: 0.0000000000, propagate_forward_time: 321.9595336914, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 7.8227844238
best training[          , genome 16653] predictions:   50000/  50000 (100.00%), best:    9898/10000 (98.98%), error:        84.73866, best error:       344.52029 on epoch:    95, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 66.0720520020s, input_fired_time: 0.9957336783, output_fired_time: 0.0000000000, propagate_forward_time: 63.5328445435, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5434722900
   testing[          , genome 16653] predictions:    9903/  10000 (99.03%), best:    9898/10000 (98.98%), error:       305.97159, best error:       344.52029 on epoch:    95, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
09:38:13 (1980): called boinc_finish(0)

</stderr_txt>
]]>