Nameexact_genome_1512568488_45_6684_0
Workunit2604691
Created7 Dec 2017, 21:19:57 UTC
Sent7 Dec 2017, 21:21:44 UTC
Report deadline19 Dec 2017, 4:14:32 UTC
Received8 Dec 2017, 12:39:34 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID53180
Run time2 hours 43 min 48 sec
CPU time2 hours 28 min 25 sec
Validate stateValid
Credit2,821.67
Device peak FLOPS3.91 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size156.63 MB
Peak swap size159.68 MB
Peak disk usage2.30 MB

Stderr output

<core_client_version>7.8.2</core_client_version>
<![CDATA[
<stderr_txt>
(97.83%), error:       812.64258, best error:       758.02814 on epoch:    16, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 82.5251922607s, input_fired_time: 1.0398677588, output_fired_time: 0.8070669174, propagate_forward_time: 30.0356292725, propagate_backward_time: 48.8959693909, weight_update_time: 0.0981045365, other_time: 1.7466545105
epoch time: 6.5785183907s, input_fired_time: 0.1221425161, output_fired_time: 0.0000000000, propagate_forward_time: 6.1466703415, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3097057343
validation[          , genome  6684] predictions:    9762/  10000 (97.62%), best:    9783/10000 (97.83%), error:       811.29962, best error:       758.02814 on epoch:    16, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 82.2022476196s, input_fired_time: 1.0296187401, output_fired_time: 0.8041081429, propagate_forward_time: 29.9624519348, propagate_backward_time: 48.6401100159, weight_update_time: 0.1045297980, other_time: 1.7659606934
epoch time: 6.4637408257s, input_fired_time: 0.1190634668, output_fired_time: 0.0000000000, propagate_forward_time: 6.0307626724, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3139147758
validation[          , genome  6684] predictions:    9790/  10000 (97.90%), best:    9790/10000 (97.90%), error:       676.48474, best error:       676.48474 on epoch:    28, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 81.5965957642s, input_fired_time: 1.0248216391, output_fired_time: 0.7966947556, propagate_forward_time: 29.8348102570, propagate_backward_time: 48.2138557434, weight_update_time: 0.0980498344, other_time: 1.7264213562
epoch time: 6.4014973640s, input_fired_time: 0.1186007038, output_fired_time: 0.0000000000, propagate_forward_time: 5.9743351936, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3085613251
validation[          , genome  6684] predictions:    9745/  10000 (97.45%), best:    9790/10000 (97.90%), error:       881.78937, best error:       676.48474 on epoch:    28, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 82.3841400146s, input_fired_time: 1.0363289118, output_fired_time: 0.8029726148, propagate_forward_time: 30.0071239471, propagate_backward_time: 48.7892303467, weight_update_time: 0.1043753624, other_time: 1.7484817505
epoch time: 6.4815511703s, input_fired_time: 0.1187095046, output_fired_time: 0.0000000000, propagate_forward_time: 6.0507416725, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3120999336
validation[          , genome  6684] predictions:    9766/  10000 (97.66%), best:    9790/10000 (97.90%), error:       774.07800, best error:       676.48474 on epoch:    28, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 84.3249664307s, input_fired_time: 1.0506314039, output_fired_time: 0.8423712850, propagate_forward_time: 30.7424716949, propagate_backward_time: 49.8819465637, weight_update_time: 0.1086101979, other_time: 1.8075523376
epoch time: 6.5486893654s, input_fired_time: 0.1199149713, output_fired_time: 0.0000000000, propagate_forward_time: 6.1227235794, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3060507774
validation[          , genome  6684] predictions:    9769/  10000 (97.69%), best:    9790/10000 (97.90%), error:       782.34198, best error:       676.48474 on epoch:    28, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 95.5582656860s, input_fired_time: 1.0977896452, output_fired_time: 0.8215591908, propagate_forward_time: 41.2921524048, propagate_backward_time: 50.5625801086, weight_update_time: 0.1136835814, other_time: 1.7841911316
epoch time: 6.2495470047s, input_fired_time: 0.1178349257, output_fired_time: 0.0000000000, propagate_forward_time: 5.8338303566, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2978816032
validation[          , genome  6684] predictions:    9784/  10000 (97.84%), best:    9790/10000 (97.90%), error:       722.01447, best error:       676.48474 on epoch:    28, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 80.2656860352s, input_fired_time: 1.0022073984, output_fired_time: 0.7787651420, propagate_forward_time: 29.2861213684, propagate_backward_time: 47.5102233887, weight_update_time: 0.0969941989, other_time: 1.6883735657
epoch time: 6.2466578484s, input_fired_time: 0.1181184798, output_fired_time: 0.0000000000, propagate_forward_time: 5.8287181854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2998213768
validation[          , genome  6684] predictions:    9757/  10000 (97.57%), best:    9790/10000 (97.90%), error:       787.55597, best error:       676.48474 on epoch:    28, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 80.8823394775s, input_fired_time: 1.0050864220, output_fired_time: 0.7961965799, propagate_forward_time: 29.5122985840, propagate_backward_time: 47.8618240356, weight_update_time: 0.0973887444, other_time: 1.7069320679
epoch time: 6.2645783424s, input_fired_time: 0.1186605766, output_fired_time: 0.0000000000, propagate_forward_time: 5.8478369713, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2980809212
validation[          , genome  6684] predictions:    9750/  10000 (97.50%), best:    9790/10000 (97.90%), error:       812.08722, best error:       676.48474 on epoch:    28, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 81.0047302246s, input_fired_time: 1.0311611891, output_fired_time: 0.7901296020, propagate_forward_time: 29.5716133118, propagate_backward_time: 47.8956031799, weight_update_time: 0.0975352749, other_time: 1.7162246704
epoch time: 6.4024152756s, input_fired_time: 0.1265975088, output_fired_time: 0.0000000000, propagate_forward_time: 5.9740247726, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3017930984
validation[          , genome  6684] predictions:    9732/  10000 (97.32%), best:    9790/10000 (97.90%), error:       846.94843, best error:       676.48474 on epoch:    28, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 82.0661544800s, input_fired_time: 1.0221869946, output_fired_time: 0.8137474656, propagate_forward_time: 29.8912162781, propagate_backward_time: 48.6016921997, weight_update_time: 0.0984140113, other_time: 1.7373085022
epoch time: 6.3068666458s, input_fired_time: 0.1199418455, output_fired_time: 0.0000000000, propagate_forward_time: 5.8795852661, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3073396683
validation[          , genome  6684] predictions:    9744/  10000 (97.44%), best:    9790/10000 (97.90%), error:       864.33221, best error:       676.48474 on epoch:    28, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 81.3123092651s, input_fired_time: 1.0210912228, output_fired_time: 0.7909579277, propagate_forward_time: 29.6268463135, propagate_backward_time: 48.1203689575, weight_update_time: 0.0983807519, other_time: 1.7530517578
epoch time: 6.4099607468s, input_fired_time: 0.1214490458, output_fired_time: 0.0000000000, propagate_forward_time: 5.9725971222, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3159146309
validation[          , genome  6684] predictions:    9790/  10000 (97.90%), best:    9790/10000 (97.90%), error:       723.14642, best error:       676.48474 on epoch:    28, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 81.3329544067s, input_fired_time: 1.0166388750, output_fired_time: 0.7941051722, propagate_forward_time: 29.6624717712, propagate_backward_time: 48.1244544983, weight_update_time: 0.1007014364, other_time: 1.7352828979
epoch time: 6.5472040176s, input_fired_time: 0.1437123865, output_fired_time: 0.0000000000, propagate_forward_time: 6.1026062965, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3008852005
validation[          , genome  6684] predictions:    9794/  10000 (97.94%), best:    9790/10000 (97.90%), error:       744.39807, best error:       676.48474 on epoch:    28, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 80.7689208984s, input_fired_time: 1.0073727369, output_fired_time: 0.7918169498, propagate_forward_time: 29.4178466797, propagate_backward_time: 47.8307914734, weight_update_time: 0.0975103155, other_time: 1.7210960388
epoch time: 6.2715864182s, input_fired_time: 0.1176739931, output_fired_time: 0.0000000000, propagate_forward_time: 5.8494272232, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3044853210
validation[          , genome  6684] predictions:    9755/  10000 (97.55%), best:    9790/10000 (97.90%), error:       809.30786, best error:       676.48474 on epoch:    28, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 81.3909912109s, input_fired_time: 1.0105313063, output_fired_time: 0.7965645790, propagate_forward_time: 29.6913089752, propagate_backward_time: 48.1535720825, weight_update_time: 0.0986744612, other_time: 1.7390213013
epoch time: 6.2747249603s, input_fired_time: 0.1183427870, output_fired_time: 0.0000000000, propagate_forward_time: 5.8564348221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2999472618
validation[          , genome  6684] predictions:    9769/  10000 (97.69%), best:    9790/10000 (97.90%), error:       805.27625, best error:       676.48474 on epoch:    28, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 80.5659408569s, input_fired_time: 1.0138050318, output_fired_time: 0.7872797251, propagate_forward_time: 29.3766536713, propagate_backward_time: 47.6832313538, weight_update_time: 0.0971487239, other_time: 1.7049751282
epoch time: 6.2824845314s, input_fired_time: 0.1180398166, output_fired_time: 0.0000000000, propagate_forward_time: 5.8576617241, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3067831993
validation[          , genome  6684] predictions:    9791/  10000 (97.91%), best:    9790/10000 (97.90%), error:       733.82904, best error:       676.48474 on epoch:    28, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 81.0713958740s, input_fired_time: 1.0150556564, output_fired_time: 0.7890257239, propagate_forward_time: 29.5865173340, propagate_backward_time: 47.9526290894, weight_update_time: 0.0987499654, other_time: 1.7281723022
epoch time: 6.3419022560s, input_fired_time: 0.1207786500, output_fired_time: 0.0000000000, propagate_forward_time: 5.9168567657, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3042669296
validation[          , genome  6684] predictions:    9775/  10000 (97.75%), best:    9790/10000 (97.90%), error:       725.91473, best error:       676.48474 on epoch:    28, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 80.8706741333s, input_fired_time: 1.0185440779, output_fired_time: 0.7929905653, propagate_forward_time: 29.4889354706, propagate_backward_time: 47.8282928467, weight_update_time: 0.0971113145, other_time: 1.7419052124
epoch time: 6.3447546959s, input_fired_time: 0.1196810007, output_fired_time: 0.0000000000, propagate_forward_time: 5.9203028679, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3047709465
validation[          , genome  6684] predictions:    9763/  10000 (97.63%), best:    9790/10000 (97.90%), error:       782.76288, best error:       676.48474 on epoch:    28, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 80.6672058105s, input_fired_time: 1.0032638311, output_fired_time: 0.7928192616, propagate_forward_time: 29.3923225403, propagate_backward_time: 47.7715110779, weight_update_time: 0.0975666642, other_time: 1.7072906494
epoch time: 6.2758569717s, input_fired_time: 0.1187005490, output_fired_time: 0.0000000000, propagate_forward_time: 5.8524084091, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3047480583
validation[          , genome  6684] predictions:    9783/  10000 (97.83%), best:    9790/10000 (97.90%), error:       731.15454, best error:       676.48474 on epoch:    28, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 81.7304992676s, input_fired_time: 1.0212733746, output_fired_time: 0.8151284456, propagate_forward_time: 29.7992515564, propagate_backward_time: 48.3530654907, weight_update_time: 0.1053941846, other_time: 1.7417793274
epoch time: 6.2513103485s, input_fired_time: 0.1180695221, output_fired_time: 0.0000000000, propagate_forward_time: 5.8332014084, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3000392914
validation[          , genome  6684] predictions:    9778/  10000 (97.78%), best:    9790/10000 (97.90%), error:       775.16302, best error:       676.48474 on epoch:    28, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 81.9816360474s, input_fired_time: 1.0416910648, output_fired_time: 0.8044957519, propagate_forward_time: 29.8368091583, propagate_backward_time: 48.5400962830, weight_update_time: 0.0995439589, other_time: 1.7585411072
epoch time: 6.3246688843s, input_fired_time: 0.1211108193, output_fired_time: 0.0000000000, propagate_forward_time: 5.8947501183, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3088078499
validation[          , genome  6684] predictions:    9769/  10000 (97.69%), best:    9790/10000 (97.90%), error:       729.33270, best error:       676.48474 on epoch:    28, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 83.0969924927s, input_fired_time: 1.0417693853, output_fired_time: 0.8038875461, propagate_forward_time: 30.2866344452, propagate_backward_time: 49.1773834229, weight_update_time: 0.1004175097, other_time: 1.7873153687
epoch time: 6.3785958290s, input_fired_time: 0.1205220073, output_fired_time: 0.0000000000, propagate_forward_time: 5.9536318779, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3044419289
validation[          , genome  6684] predictions:    9786/  10000 (97.86%), best:    9790/10000 (97.90%), error:       707.99945, best error:       676.48474 on epoch:    28, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 85.8624572754s, input_fired_time: 1.0701854229, output_fired_time: 0.8249938488, propagate_forward_time: 31.1925487518, propagate_backward_time: 50.8904342651, weight_update_time: 0.1018015742, other_time: 1.8842926025
epoch time: 6.3759984970s, input_fired_time: 0.1204064786, output_fired_time: 0.0000000000, propagate_forward_time: 5.9492220879, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3063697815
validation[          , genome  6684] predictions:    9772/  10000 (97.72%), best:    9790/10000 (97.90%), error:       762.91919, best error:       676.48474 on epoch:    28, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 98.7951889038s, input_fired_time: 1.0971647501, output_fired_time: 0.8785321712, propagate_forward_time: 42.2319946289, propagate_backward_time: 52.6935119629, weight_update_time: 0.1101269275, other_time: 1.8939819336
epoch time: 7.2782039642s, input_fired_time: 0.1343289614, output_fired_time: 0.0000000000, propagate_forward_time: 6.7849245071, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3589506149
validation[          , genome  6684] predictions:    9771/  10000 (97.71%), best:    9790/10000 (97.90%), error:       796.46442, best error:       676.48474 on epoch:    28, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 82.5637359619s, input_fired_time: 1.0560414791, output_fired_time: 0.8166037798, propagate_forward_time: 30.1098098755, propagate_backward_time: 48.8209419250, weight_update_time: 0.0997061282, other_time: 1.7603416443
epoch time: 8.2242422104s, input_fired_time: 0.1426736414, output_fired_time: 0.0000000000, propagate_forward_time: 7.7163634300, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3652052879
validation[          , genome  6684] predictions:    9784/  10000 (97.84%), best:    9790/10000 (97.90%), error:       705.70868, best error:       676.48474 on epoch:    28, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 149.9061584473s, input_fired_time: 1.3431742191, output_fired_time: 1.0107823610, propagate_forward_time: 56.3586845398, propagate_backward_time: 88.7096786499, weight_update_time: 0.4315351844, other_time: 2.4838333130
epoch time: 20.1948509216s, input_fired_time: 0.2054514438, output_fired_time: 0.0000000000, propagate_forward_time: 19.4217815399, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5676174164
validation[          , genome  6684] predictions:    9761/  10000 (97.61%), best:    9790/10000 (97.90%), error:       791.47235, best error:       676.48474 on epoch:    28, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 197.0462188721s, input_fired_time: 1.3895040751, output_fired_time: 21.4863967896, propagate_forward_time: 58.7145462036, propagate_backward_time: 112.9682922363, weight_update_time: 0.1329910904, other_time: 2.4874801636
epoch time: 17.8974018097s, input_fired_time: 0.1482342929, output_fired_time: 0.0000000000, propagate_forward_time: 17.3342075348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4149608612
validation[          , genome  6684] predictions:    9774/  10000 (97.74%), best:    9790/10000 (97.90%), error:       748.31396, best error:       676.48474 on epoch:    28, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 188.3106689453s, input_fired_time: 11.5536279678, output_fired_time: 11.2958631516, propagate_forward_time: 48.5711250305, propagate_backward_time: 114.3694305420, weight_update_time: 0.1285510063, other_time: 2.5206146240
epoch time: 7.3835225105s, input_fired_time: 0.1566476822, output_fired_time: 0.0000000000, propagate_forward_time: 6.8137397766, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4131350517
validation[          , genome  6684] predictions:    9790/  10000 (97.90%), best:    9790/10000 (97.90%), error:       689.79791, best error:       676.48474 on epoch:    28, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 240.4835815430s, input_fired_time: 1.4004880190, output_fired_time: 1.0869387388, propagate_forward_time: 120.4988555908, propagate_backward_time: 114.9734497070, weight_update_time: 0.1406505853, other_time: 2.5238494873
epoch time: 8.1763877869s, input_fired_time: 0.1707609594, output_fired_time: 0.0000000000, propagate_forward_time: 7.5694851875, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4361414909
validation[          , genome  6684] predictions:    9801/  10000 (98.01%), best:    9790/10000 (97.90%), error:       690.57312, best error:       676.48474 on epoch:    28, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 192.7989807129s, input_fired_time: 1.5010876656, output_fired_time: 1.1782711744, propagate_forward_time: 101.0417404175, propagate_backward_time: 86.4436111450, weight_update_time: 0.1861551553, other_time: 2.6342773438
epoch time: 38.2981224060s, input_fired_time: 0.1559302360, output_fired_time: 0.0000000000, propagate_forward_time: 37.7528839111, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3893089294
validation[          , genome  6684] predictions:    9805/  10000 (98.05%), best:    9805/10000 (98.05%), error:       639.15912, best error:       639.15912 on epoch:    55, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 257.9668579102s, input_fired_time: 1.5400795937, output_fired_time: 1.2404564619, propagate_forward_time: 133.5746307373, propagate_backward_time: 118.8536529541, weight_update_time: 0.1683721542, other_time: 2.7580413818
epoch time: 49.8457336426s, input_fired_time: 0.1906582117, output_fired_time: 0.0000000000, propagate_forward_time: 49.1378021240, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5172729492
validation[          , genome  6684] predictions:    9781/  10000 (97.81%), best:    9805/10000 (98.05%), error:       732.42474, best error:       639.15912 on epoch:    55, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 301.6772460938s, input_fired_time: 1.5943791866, output_fired_time: 1.2552778721, propagate_forward_time: 83.1843643188, propagate_backward_time: 202.6696472168, weight_update_time: 0.1702328324, other_time: 12.9735717773
epoch time: 29.7314414978s, input_fired_time: 0.1852738708, output_fired_time: 0.0000000000, propagate_forward_time: 29.0257568359, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5204105377
validation[          , genome  6684] predictions:    9803/  10000 (98.03%), best:    9805/10000 (98.05%), error:       702.74872, best error:       639.15912 on epoch:    55, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 194.7863769531s, input_fired_time: 11.6907358170, output_fired_time: 11.5002193451, propagate_forward_time: 50.8360671997, propagate_backward_time: 107.8000717163, weight_update_time: 0.1633316725, other_time: 12.9592895508
epoch time: 39.2388420105s, input_fired_time: 0.1746057719, output_fired_time: 0.0000000000, propagate_forward_time: 7.8537344933, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 31.2105007172
validation[          , genome  6684] predictions:    9796/  10000 (97.96%), best:    9805/10000 (98.05%), error:       674.24329, best error:       639.15912 on epoch:    55, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 212.6415252686s, input_fired_time: 1.4997589588, output_fired_time: 1.1908223629, propagate_forward_time: 80.8166122437, propagate_backward_time: 126.4861602783, weight_update_time: 0.1396824867, other_time: 2.6481781006
epoch time: 8.5048837662s, input_fired_time: 0.1727123111, output_fired_time: 0.0000000000, propagate_forward_time: 7.8648657799, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4673056602
validation[          , genome  6684] predictions:    9797/  10000 (97.97%), best:    9805/10000 (98.05%), error:       673.26416, best error:       639.15912 on epoch:    55, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 251.3695983887s, input_fired_time: 1.5702507496, output_fired_time: 1.3505159616, propagate_forward_time: 103.9402618408, propagate_backward_time: 141.6877136230, weight_update_time: 0.1658159047, other_time: 2.8208618164
epoch time: 10.6349563599s, input_fired_time: 0.2135851383, output_fired_time: 0.0000000000, propagate_forward_time: 9.8446493149, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5767221451
validation[          , genome  6684] predictions:    9809/  10000 (98.09%), best:    9809/10000 (98.09%), error:       634.85077, best error:       634.85077 on epoch:    60, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 298.3954772949s, input_fired_time: 1.6361143589, output_fired_time: 1.3753306866, propagate_forward_time: 75.9139709473, propagate_backward_time: 216.4523315430, weight_update_time: 0.1487911344, other_time: 3.0177307129
epoch time: 10.0055475235s, input_fired_time: 0.2122136503, output_fired_time: 0.0000000000, propagate_forward_time: 9.2639913559, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5293426514
validation[          , genome  6684] predictions:    9747/  10000 (97.47%), best:    9809/10000 (98.09%), error:       829.65051, best error:       634.85077 on epoch:    60, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 266.5538330078s, input_fired_time: 1.6551755667, output_fired_time: 1.3807387352, propagate_forward_time: 116.2943725586, propagate_backward_time: 134.0130157471, weight_update_time: 0.1670471281, other_time: 13.2105255127
epoch time: 7.4204115868s, input_fired_time: 0.1561741084, output_fired_time: 0.0000000000, propagate_forward_time: 6.8717918396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3924455643
validation[          , genome  6684] predictions:    9773/  10000 (97.73%), best:    9809/10000 (98.09%), error:       756.02582, best error:       634.85077 on epoch:    60, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 137.3810882568s, input_fired_time: 1.3010233641, output_fired_time: 0.9482126236, propagate_forward_time: 65.5489501953, propagate_backward_time: 67.3997268677, weight_update_time: 0.1203733012, other_time: 2.1831741333
epoch time: 7.0240864754s, input_fired_time: 0.1373583972, output_fired_time: 0.0000000000, propagate_forward_time: 6.5319204330, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3548078537
validation[          , genome  6684] predictions:    9798/  10000 (97.98%), best:    9809/10000 (98.09%), error:       692.28754, best error:       634.85077 on epoch:    60, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 87.2934799194s, input_fired_time: 1.0638643503, output_fired_time: 0.8821726441, propagate_forward_time: 31.6333827972, propagate_backward_time: 51.8466606140, weight_update_time: 0.1118968874, other_time: 1.8673973083
epoch time: 6.5365180969s, input_fired_time: 0.1251408756, output_fired_time: 0.0000000000, propagate_forward_time: 6.0953812599, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3159961700
validation[          , genome  6684] predictions:    9801/  10000 (98.01%), best:    9809/10000 (98.09%), error:       696.13892, best error:       634.85077 on epoch:    60, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 86.2328643799s, input_fired_time: 1.0803645849, output_fired_time: 0.8366527557, propagate_forward_time: 31.4038028717, propagate_backward_time: 51.0560455322, weight_update_time: 0.1009554714, other_time: 1.8559951782
epoch time: 6.5333132744s, input_fired_time: 0.1231754497, output_fired_time: 0.0000000000, propagate_forward_time: 6.0787205696, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3314170837
validation[          , genome  6684] predictions:    9791/  10000 (97.91%), best:    9809/10000 (98.09%), error:       713.01770, best error:       634.85077 on epoch:    60, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 83.1411285400s, input_fired_time: 1.0454676151, output_fired_time: 0.8135025501, propagate_forward_time: 30.2489509583, propagate_backward_time: 49.2794380188, weight_update_time: 0.0997887626, other_time: 1.7537689209
epoch time: 7.1010198593s, input_fired_time: 0.1336330473, output_fired_time: 0.0000000000, propagate_forward_time: 6.6428585052, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3245282173
validation[          , genome  6684] predictions:    9783/  10000 (97.83%), best:    9809/10000 (98.09%), error:       714.12299, best error:       634.85077 on epoch:    60, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 89.1600494385s, input_fired_time: 1.2016606331, output_fired_time: 0.9134212732, propagate_forward_time: 32.4923973083, propagate_backward_time: 52.5983238220, weight_update_time: 0.1072703525, other_time: 1.9542465210
epoch time: 7.7529530525s, input_fired_time: 0.1370672733, output_fired_time: 0.0000000000, propagate_forward_time: 7.2493410110, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3665447235
validation[          , genome  6684] predictions:    9579/  10000 (95.79%), best:    9809/10000 (98.09%), error:      1285.93298, best error:       634.85077 on epoch:    60, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 83.2119293213s, input_fired_time: 1.0541602373, output_fired_time: 0.8052324653, propagate_forward_time: 30.3514881134, propagate_backward_time: 49.1487884521, weight_update_time: 0.1013145372, other_time: 1.8522644043
epoch time: 6.4378709793s, input_fired_time: 0.1224253029, output_fired_time: 0.0000000000, propagate_forward_time: 6.0141935349, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3012523651
validation[          , genome  6684] predictions:    9793/  10000 (97.93%), best:    9809/10000 (98.09%), error:       693.79150, best error:       634.85077 on epoch:    60, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 81.7779846191s, input_fired_time: 1.0359424353, output_fired_time: 0.8013479114, propagate_forward_time: 29.7624950409, propagate_backward_time: 48.4202270508, weight_update_time: 0.1030950844, other_time: 1.7579727173
epoch time: 6.3621878624s, input_fired_time: 0.1229651794, output_fired_time: 0.0000000000, propagate_forward_time: 5.9338774681, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3053450584
validation[          , genome  6684] predictions:    9806/  10000 (98.06%), best:    9809/10000 (98.09%), error:       666.13287, best error:       634.85077 on epoch:    60, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 81.8160705566s, input_fired_time: 1.0364823341, output_fired_time: 0.7977638245, propagate_forward_time: 29.8552246094, propagate_backward_time: 48.3839263916, weight_update_time: 0.0994174927, other_time: 1.7426757813
epoch time: 6.4640202522s, input_fired_time: 0.1277921051, output_fired_time: 0.0000000000, propagate_forward_time: 6.0282082558, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3080201149
validation[          , genome  6684] predictions:    9804/  10000 (98.04%), best:    9809/10000 (98.09%), error:       675.50458, best error:       634.85077 on epoch:    60, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 81.8367233276s, input_fired_time: 1.0500981808, output_fired_time: 0.7980565429, propagate_forward_time: 29.8036727905, propagate_backward_time: 48.4495277405, weight_update_time: 0.0994373634, other_time: 1.7353706360
epoch time: 6.4996562004s, input_fired_time: 0.1318048835, output_fired_time: 0.0000000000, propagate_forward_time: 6.0544691086, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3133821487
validation[          , genome  6684] predictions:    9789/  10000 (97.89%), best:    9809/10000 (98.09%), error:       738.22931, best error:       634.85077 on epoch:    60, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 84.5766143799s, input_fired_time: 1.0761491060, output_fired_time: 0.8262802958, propagate_forward_time: 30.8901596069, propagate_backward_time: 49.9325103760, weight_update_time: 0.0998351425, other_time: 1.8515167236
epoch time: 6.3762073517s, input_fired_time: 0.1253335029, output_fired_time: 0.0000000000, propagate_forward_time: 5.9416923523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3091816902
validation[          , genome  6684] predictions:    9807/  10000 (98.07%), best:    9809/10000 (98.09%), error:       655.12115, best error:       634.85077 on epoch:    60, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 83.3716583252s, input_fired_time: 1.0993274450, output_fired_time: 0.8113539815, propagate_forward_time: 30.4209575653, propagate_backward_time: 49.2329902649, weight_update_time: 0.1005410701, other_time: 1.8070259094
epoch time: 6.6686043739s, input_fired_time: 0.1240362525, output_fired_time: 0.0000000000, propagate_forward_time: 6.2320294380, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3125386238
validation[          , genome  6684] predictions:    9787/  10000 (97.87%), best:    9809/10000 (98.09%), error:       721.48859, best error:       634.85077 on epoch:    60, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_45_6684.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_45_6684_0_r2003580056_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.978991
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000280838
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.12335e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 74
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 60
read number_validation_images: 10000
read best_validation_predictions: 9809
read best_validation_error: 634.851
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6684
read normal distribution: '1 0 0'
generator_str: '1966661789'
read generator: 1966661789
read generated_by_map:
1 disable_edge 1
reading 29 nodes.
reading 70 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_45_6684.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_45_6684_0_r2003580056_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.978991
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000280838
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.12335e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 74
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 60
read number_validation_images: 10000
read best_validation_predictions: 9809
read best_validation_error: 634.851
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6684
read normal distribution: '1 0 0'
generator_str: '1966661789'
read generator: 1966661789
read generated_by_map:
1 disable_edge 1
reading 29 nodes.
reading 70 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 87.1216583251953s, input_fired_time: 1.09416317939758, output_fired_time: 0.84476625919342, propagate_forward_time: 31.7936859130859, propagate_backward_time: 51.614387512207, weight_update_time: 0.101650558412075, other_time: 1.774658203125
epoch time: 6.48858022689819s, input_fired_time: 0.128944382071495, output_fired_time: 0, propagate_forward_time: 6.05513525009155, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.304500579833984
validation[          , genome  6684] predictions:    9800/  10000 (98.00%), best:    9809/10000 (98.09%), error:       695.76947, best error:       634.85077 on epoch:    60, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 82.5641021729s, input_fired_time: 1.0621533394, output_fired_time: 0.8119850755, propagate_forward_time: 30.0297317505, propagate_backward_time: 48.9180335999, weight_update_time: 0.0981010273, other_time: 1.7421913147
epoch time: 6.6991715431s, input_fired_time: 0.1266736537, output_fired_time: 0.0000000000, propagate_forward_time: 6.2549524307, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3175454140
validation[          , genome  6684] predictions:    9789/  10000 (97.89%), best:    9809/10000 (98.09%), error:       730.16083, best error:       634.85077 on epoch:    60, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 118.0028228760s, input_fired_time: 1.4162590504, output_fired_time: 0.9298972487, propagate_forward_time: 55.5929222107, propagate_backward_time: 57.9623451233, weight_update_time: 0.1200414598, other_time: 2.1014022827
epoch time: 6.9787988663s, input_fired_time: 0.1371562034, output_fired_time: 0.0000000000, propagate_forward_time: 6.4797015190, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3619413376
validation[          , genome  6684] predictions:    9792/  10000 (97.92%), best:    9809/10000 (98.09%), error:       718.85040, best error:       634.85077 on epoch:    60, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 88.7623519897s, input_fired_time: 1.1590932608, output_fired_time: 0.8567255735, propagate_forward_time: 32.2636451721, propagate_backward_time: 52.5668144226, weight_update_time: 0.1114016101, other_time: 1.9160690308
epoch time: 8.3277816772s, input_fired_time: 0.1544150859, output_fired_time: 0.0000000000, propagate_forward_time: 7.7374243736, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4359421730
validation[          , genome  6684] predictions:    9799/  10000 (97.99%), best:    9809/10000 (98.09%), error:       679.82379, best error:       634.85077 on epoch:    60, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 104.6566085815s, input_fired_time: 1.2194406986, output_fired_time: 0.8859561086, propagate_forward_time: 44.6642074585, propagate_backward_time: 55.8250770569, weight_update_time: 0.1150012240, other_time: 2.0619239807
epoch time: 6.2961301804s, input_fired_time: 0.1213303059, output_fired_time: 0.0000000000, propagate_forward_time: 5.8750395775, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2997603416
validation[          , genome  6684] predictions:    9791/  10000 (97.91%), best:    9809/10000 (98.09%), error:       714.63336, best error:       634.85077 on epoch:    60, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 83.9386596680s, input_fired_time: 1.0782265663, output_fired_time: 0.8318321705, propagate_forward_time: 30.3516654968, propagate_backward_time: 49.7923965454, weight_update_time: 0.0986674577, other_time: 1.8845405579
epoch time: 6.6474933624s, input_fired_time: 0.1335235536, output_fired_time: 0.0000000000, propagate_forward_time: 6.1832385063, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3307313919
validation[          , genome  6684] predictions:    9799/  10000 (97.99%), best:    9809/10000 (98.09%), error:       689.05817, best error:       634.85077 on epoch:    60, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 84.1911163330s, input_fired_time: 1.0908333063, output_fired_time: 0.8460691571, propagate_forward_time: 30.5655727386, propagate_backward_time: 49.8265571594, weight_update_time: 0.1011586487, other_time: 1.8620796204
epoch time: 6.5640630722s, input_fired_time: 0.1326073706, output_fired_time: 0.0000000000, propagate_forward_time: 6.1010384560, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3304171562
validation[          , genome  6684] predictions:    9813/  10000 (98.13%), best:    9813/10000 (98.13%), error:       629.89948, best error:       629.89948 on epoch:    80, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 83.2029418945s, input_fired_time: 1.1137682199, output_fired_time: 0.8211256862, propagate_forward_time: 30.2285423279, propagate_backward_time: 49.2335700989, weight_update_time: 0.1066761985, other_time: 1.8059310913
epoch time: 6.5784459114s, input_fired_time: 0.1264263242, output_fired_time: 0.0000000000, propagate_forward_time: 6.1158494949, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3361701965
validation[          , genome  6684] predictions:    9785/  10000 (97.85%), best:    9813/10000 (98.13%), error:       715.61871, best error:       629.89948 on epoch:    80, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 85.1508483887s, input_fired_time: 1.1049954891, output_fired_time: 0.8514124751, propagate_forward_time: 31.0694122314, propagate_backward_time: 50.2922782898, weight_update_time: 0.1012190878, other_time: 1.8327522278
epoch time: 6.4356555939s, input_fired_time: 0.1241040602, output_fired_time: 0.0000000000, propagate_forward_time: 5.9391183853, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3724331856
validation[          , genome  6684] predictions:    9791/  10000 (97.91%), best:    9813/10000 (98.13%), error:       694.06140, best error:       629.89948 on epoch:    80, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 91.2034225464s, input_fired_time: 1.2454843521, output_fired_time: 0.9693752527, propagate_forward_time: 32.9752883911, propagate_backward_time: 53.8799133301, weight_update_time: 0.1212168708, other_time: 2.1333618164
epoch time: 7.1582050323s, input_fired_time: 0.1495095193, output_fired_time: 0.0000000000, propagate_forward_time: 6.6325874329, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3761081696
validation[          , genome  6684] predictions:    9807/  10000 (98.07%), best:    9813/10000 (98.13%), error:       671.40424, best error:       629.89948 on epoch:    80, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 97.4814834595s, input_fired_time: 1.2630908489, output_fired_time: 1.0473945141, propagate_forward_time: 35.4013862610, propagate_backward_time: 57.5163803101, weight_update_time: 0.1232198030, other_time: 2.2532310486
epoch time: 7.8043422699s, input_fired_time: 0.1662950367, output_fired_time: 0.0000000000, propagate_forward_time: 7.2132716179, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4247756004
validation[          , genome  6684] predictions:    9774/  10000 (97.74%), best:    9813/10000 (98.13%), error:       736.04327, best error:       629.89948 on epoch:    80, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 93.0777740479s, input_fired_time: 1.2165052891, output_fired_time: 0.9915898442, propagate_forward_time: 33.6993141174, propagate_backward_time: 55.0564308167, weight_update_time: 0.1110654846, other_time: 2.1139297485
epoch time: 6.6782073975s, input_fired_time: 0.1393290162, output_fired_time: 0.0000000000, propagate_forward_time: 6.2070703506, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3318080902
validation[          , genome  6684] predictions:    9805/  10000 (98.05%), best:    9813/10000 (98.13%), error:       648.12677, best error:       629.89948 on epoch:    80, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 99.6136169434s, input_fired_time: 1.2031763792, output_fired_time: 0.8974813223, propagate_forward_time: 32.6481285095, propagate_backward_time: 62.9000587463, weight_update_time: 0.1108856872, other_time: 1.9647636414
epoch time: 7.0416560173s, input_fired_time: 0.1440333128, output_fired_time: 0.0000000000, propagate_forward_time: 6.5428814888, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3547410965
validation[          , genome  6684] predictions:    9794/  10000 (97.94%), best:    9813/10000 (98.13%), error:       703.03815, best error:       629.89948 on epoch:    80, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 89.2927093506s, input_fired_time: 1.1653957367, output_fired_time: 0.9119186997, propagate_forward_time: 32.3378334045, propagate_backward_time: 52.8448982239, weight_update_time: 0.1068508178, other_time: 2.0326614380
epoch time: 6.7966251373s, input_fired_time: 0.1412663311, output_fired_time: 0.0000000000, propagate_forward_time: 6.3158836365, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3394751549
validation[          , genome  6684] predictions:    9787/  10000 (97.87%), best:    9813/10000 (98.13%), error:       748.96924, best error:       629.89948 on epoch:    80, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 88.8070907593s, input_fired_time: 1.1527932882, output_fired_time: 0.9151554704, propagate_forward_time: 32.1636505127, propagate_backward_time: 52.5753517151, weight_update_time: 0.1107423306, other_time: 2.0001411438
epoch time: 7.3741440773s, input_fired_time: 0.1484256536, output_fired_time: 0.0000000000, propagate_forward_time: 6.8272280693, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3984904289
validation[          , genome  6684] predictions:    9805/  10000 (98.05%), best:    9813/10000 (98.13%), error:       641.02118, best error:       629.89948 on epoch:    80, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 97.0484466553s, input_fired_time: 1.3165329695, output_fired_time: 1.0277464390, propagate_forward_time: 35.0830039978, propagate_backward_time: 57.3736190796, weight_update_time: 0.1289676875, other_time: 2.2475395203
epoch time: 7.4854969978s, input_fired_time: 0.1524655819, output_fired_time: 0.0000000000, propagate_forward_time: 6.9290499687, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4039816856
validation[          , genome  6684] predictions:    9791/  10000 (97.91%), best:    9813/10000 (98.13%), error:       696.37177, best error:       629.89948 on epoch:    80, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 100.9133682251s, input_fired_time: 1.3654098511, output_fired_time: 1.1050001383, propagate_forward_time: 36.2724723816, propagate_backward_time: 59.5789337158, weight_update_time: 0.1316284239, other_time: 2.5915489197
epoch time: 8.0514707565s, input_fired_time: 0.1829731166, output_fired_time: 0.0000000000, propagate_forward_time: 7.4270710945, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4414267540
validation[          , genome  6684] predictions:    9797/  10000 (97.97%), best:    9813/10000 (98.13%), error:       682.27008, best error:       629.89948 on epoch:    80, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 121.6810684204s, input_fired_time: 1.3780943155, output_fired_time: 1.0808442831, propagate_forward_time: 56.6013107300, propagate_backward_time: 60.2327651978, weight_update_time: 0.1256089807, other_time: 2.3880538940
epoch time: 6.8989596367s, input_fired_time: 0.1367955953, output_fired_time: 0.0000000000, propagate_forward_time: 6.4318909645, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3302731514
validation[          , genome  6684] predictions:    9765/  10000 (97.65%), best:    9813/10000 (98.13%), error:       816.56470, best error:       629.89948 on epoch:    80, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 95.6067276001s, input_fired_time: 1.3348094225, output_fired_time: 0.9989743829, propagate_forward_time: 34.5819435120, propagate_backward_time: 56.5267410278, weight_update_time: 0.1239935830, other_time: 2.1642570496
epoch time: 7.9081268311s, input_fired_time: 0.1663926691, output_fired_time: 0.0000000000, propagate_forward_time: 7.3100748062, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4316592216
validation[          , genome  6684] predictions:    9805/  10000 (98.05%), best:    9813/10000 (98.13%), error:       676.53802, best error:       629.89948 on epoch:    80, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 149.7513427734s, input_fired_time: 1.3856141567, output_fired_time: 1.0492308140, propagate_forward_time: 56.1647071838, propagate_backward_time: 88.7346191406, weight_update_time: 0.1362325102, other_time: 2.4171752930
epoch time: 7.5273766518s, input_fired_time: 0.1600560844, output_fired_time: 0.0000000000, propagate_forward_time: 6.9606041908, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4067163467
validation[          , genome  6684] predictions:    9783/  10000 (97.83%), best:    9813/10000 (98.13%), error:       736.05139, best error:       629.89948 on epoch:    80, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 102.4159927368s, input_fired_time: 1.4040373564, output_fired_time: 1.1036008596, propagate_forward_time: 37.0397033691, propagate_backward_time: 60.3364143372, weight_update_time: 0.1380183846, other_time: 2.5322380066
epoch time: 8.0138654709s, input_fired_time: 0.1715554744, output_fired_time: 0.0000000000, propagate_forward_time: 7.4323329926, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4099769592
validation[          , genome  6684] predictions:    9796/  10000 (97.96%), best:    9813/10000 (98.13%), error:       654.93054, best error:       629.89948 on epoch:    80, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 152.5067749023s, input_fired_time: 1.4038537741, output_fired_time: 1.0804932117, propagate_forward_time: 86.6803588867, propagate_backward_time: 60.8924560547, weight_update_time: 0.1319327652, other_time: 2.4496154785
epoch time: 6.6947350502s, input_fired_time: 0.1324646622, output_fired_time: 0.0000000000, propagate_forward_time: 6.2339429855, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3283271790
validation[          , genome  6684] predictions:    9823/  10000 (98.23%), best:    9813/10000 (98.13%), error:       657.41791, best error:       629.89948 on epoch:    80, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 90.6848297119s, input_fired_time: 1.2008198500, output_fired_time: 0.9302291274, propagate_forward_time: 32.8649330139, propagate_backward_time: 53.7260589600, weight_update_time: 0.1114754379, other_time: 1.9627876282
epoch time: 6.7666435242s, input_fired_time: 0.1339328736, output_fired_time: 0.0000000000, propagate_forward_time: 6.2929353714, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3397750854
validation[          , genome  6684] predictions:    9814/  10000 (98.14%), best:    9813/10000 (98.13%), error:       653.68433, best error:       629.89948 on epoch:    80, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 86.4991531372s, input_fired_time: 1.1103681326, output_fired_time: 0.8466607928, propagate_forward_time: 31.4416923523, propagate_backward_time: 51.2211227417, weight_update_time: 0.1032304093, other_time: 1.8793067932
epoch time: 6.4363765717s, input_fired_time: 0.1301274300, output_fired_time: 0.0000000000, propagate_forward_time: 5.9912614822, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3149876595
validation[          , genome  6684] predictions:    9798/  10000 (97.98%), best:    9813/10000 (98.13%), error:       667.14294, best error:       629.89948 on epoch:    80, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 251.7113189697s, input_fired_time: 1.5341789722, output_fired_time: 31.2922039032, propagate_forward_time: 100.3337631226, propagate_backward_time: 115.8528213501, weight_update_time: 0.1225013733, other_time: 2.6983489990
epoch time: 7.3245453835s, input_fired_time: 0.1514947414, output_fired_time: 0.0000000000, propagate_forward_time: 6.7953000069, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3777508736
validation[          , genome  6684] predictions:    9758/  10000 (97.58%), best:    9813/10000 (98.13%), error:       818.10767, best error:       629.89948 on epoch:    80, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 99.1205520630s, input_fired_time: 1.2987935543, output_fired_time: 1.0243517160, propagate_forward_time: 35.7847099304, propagate_backward_time: 58.6886062622, weight_update_time: 0.1223522946, other_time: 2.3240928650
epoch time: 8.1187114716s, input_fired_time: 0.1606729776, output_fired_time: 0.0000000000, propagate_forward_time: 7.5287594795, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4292788506
validation[          , genome  6684] predictions:    9815/  10000 (98.15%), best:    9813/10000 (98.13%), error:       642.16406, best error:       629.89948 on epoch:    80, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 100.9659271240s, input_fired_time: 1.3037757874, output_fired_time: 1.1081023216, propagate_forward_time: 36.4762496948, propagate_backward_time: 59.7259941101, weight_update_time: 0.1428511143, other_time: 2.3518104553
epoch time: 7.3339028358s, input_fired_time: 0.1559814662, output_fired_time: 0.0000000000, propagate_forward_time: 6.7918887138, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3860325813
validation[          , genome  6684] predictions:    9801/  10000 (98.01%), best:    9813/10000 (98.13%), error:       663.91132, best error:       629.89948 on epoch:    80, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 38.5265769958s, input_fired_time: 0.7919835448, output_fired_time: 0.0000000000, propagate_forward_time: 35.6620216370, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0725708008
best training[          , genome  6684] predictions:   49830/  50000 (99.66%), best:    9813/10000 (98.13%), error:       859.38696, best error:       629.89948 on epoch:    80, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 7.7341732979s, input_fired_time: 0.1476298124, output_fired_time: 0.0000000000, propagate_forward_time: 7.1334624290, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4530811310
   testing[          , genome  6684] predictions:    9805/  10000 (98.05%), best:    9813/10000 (98.13%), error:       567.37024, best error:       629.89948 on epoch:    80, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
13:20:51 (2080): called boinc_finish(0)

</stderr_txt>
]]>