Nameexact_genome_1512568488_42_6807_0
Workunit2605244
Created7 Dec 2017, 21:58:26 UTC
Sent7 Dec 2017, 22:04:23 UTC
Report deadline17 Dec 2017, 19:46:05 UTC
Received8 Dec 2017, 12:26:12 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID55829
Run time4 hours 52 min 40 sec
CPU time4 hours 48 min 38 sec
Validate stateValid
Credit2,475.99
Device peak FLOPS2.39 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.33
Peak working set size153.81 MB
Peak swap size155.27 MB
Peak disk usage1.38 MB

Stderr output

<core_client_version>7.6.31</core_client_version>
<![CDATA[
<stderr_txt>
1, output_fired_time: 0.0000000000, propagate_forward_time: 12.1206359863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9006538391
validation[          , genome  6807] predictions:    9739/  10000 (97.39%), best:    9775/10000 (97.75%), error:       855.56439, best error:       759.63489 on epoch:     8, epoch:   14/100, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 159.7384033203s, input_fired_time: 2.4980556965, output_fired_time: 1.9422169924, propagate_forward_time: 60.7859420776, propagate_backward_time: 89.7259979248, weight_update_time: 0.2354952842, other_time: 4.7861862183
epoch time: 13.1614847183s, input_fired_time: 0.2791513205, output_fired_time: 0.0000000000, propagate_forward_time: 12.0432367325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8390970230
validation[          , genome  6807] predictions:    9700/  10000 (97.00%), best:    9775/10000 (97.75%), error:       979.53937, best error:       759.63489 on epoch:     8, epoch:   15/100, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 159.0882873535s, input_fired_time: 2.4818646908, output_fired_time: 1.9035549164, propagate_forward_time: 60.7776031494, propagate_backward_time: 89.1698532104, weight_update_time: 0.2355398983, other_time: 4.7554244995
epoch time: 13.4378614426s, input_fired_time: 0.3160438836, output_fired_time: 0.0000000000, propagate_forward_time: 12.2592782974, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8625392914
validation[          , genome  6807] predictions:    9733/  10000 (97.33%), best:    9775/10000 (97.75%), error:       869.32703, best error:       759.63489 on epoch:     8, epoch:   16/100, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 159.2126007080s, input_fired_time: 2.5035531521, output_fired_time: 1.9092770815, propagate_forward_time: 60.8641090393, propagate_backward_time: 89.1961135864, weight_update_time: 0.2367500663, other_time: 4.7395553589
epoch time: 13.3846883774s, input_fired_time: 0.2759968638, output_fired_time: 0.0000000000, propagate_forward_time: 12.2561492920, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8525419235
validation[          , genome  6807] predictions:    9764/  10000 (97.64%), best:    9764/10000 (97.64%), error:       734.64240, best error:       734.64240 on epoch:    17, epoch:   17/100, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 159.9895935059s, input_fired_time: 2.4974000454, output_fired_time: 1.9200799465, propagate_forward_time: 61.1305389404, propagate_backward_time: 89.6473464966, weight_update_time: 0.2560984790, other_time: 4.7942276001
epoch time: 13.2412071228s, input_fired_time: 0.3011394739, output_fired_time: 0.0000000000, propagate_forward_time: 12.0918807983, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8481864929
validation[          , genome  6807] predictions:    9742/  10000 (97.42%), best:    9764/10000 (97.64%), error:       823.67078, best error:       734.64240 on epoch:    17, epoch:   18/100, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 159.6840515137s, input_fired_time: 2.4572196007, output_fired_time: 1.8959175348, propagate_forward_time: 61.0892295837, propagate_backward_time: 89.3788604736, weight_update_time: 0.2368557006, other_time: 4.8628234863
epoch time: 13.3019390106s, input_fired_time: 0.2786665261, output_fired_time: 0.0000000000, propagate_forward_time: 12.1532793045, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8699932098
validation[          , genome  6807] predictions:    9770/  10000 (97.70%), best:    9764/10000 (97.64%), error:       739.62756, best error:       734.64240 on epoch:    17, epoch:   19/100, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 160.1377105713s, input_fired_time: 2.4695258141, output_fired_time: 1.9490956068, propagate_forward_time: 61.2555084229, propagate_backward_time: 89.6247634888, weight_update_time: 0.2344055921, other_time: 4.8388137817
epoch time: 13.2630786896s, input_fired_time: 0.2842919230, output_fired_time: 0.0000000000, propagate_forward_time: 12.1210422516, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8577442169
validation[          , genome  6807] predictions:    9731/  10000 (97.31%), best:    9764/10000 (97.64%), error:       886.61298, best error:       734.64240 on epoch:    17, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 159.1543884277s, input_fired_time: 2.4704589844, output_fired_time: 1.8891783953, propagate_forward_time: 60.9935531616, propagate_backward_time: 89.0109405518, weight_update_time: 0.2388538867, other_time: 4.7902603149
epoch time: 13.4675664902s, input_fired_time: 0.3480750322, output_fired_time: 0.0000000000, propagate_forward_time: 12.2410211563, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8784704208
validation[          , genome  6807] predictions:    9723/  10000 (97.23%), best:    9764/10000 (97.64%), error:       862.40540, best error:       734.64240 on epoch:    17, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 159.8474578857s, input_fired_time: 2.5113189220, output_fired_time: 1.9689410925, propagate_forward_time: 61.0726737976, propagate_backward_time: 89.4794845581, weight_update_time: 0.2344238758, other_time: 4.8150253296
epoch time: 13.1654195786s, input_fired_time: 0.2826939523, output_fired_time: 0.0000000000, propagate_forward_time: 12.0419588089, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8407669067
validation[          , genome  6807] predictions:    9768/  10000 (97.68%), best:    9768/10000 (97.68%), error:       715.60901, best error:       715.60901 on epoch:    22, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 159.6105346680s, input_fired_time: 2.4630942345, output_fired_time: 1.8925430775, propagate_forward_time: 60.9519386292, propagate_backward_time: 89.5066146851, weight_update_time: 0.2612046599, other_time: 4.7963485718
epoch time: 13.3606691360s, input_fired_time: 0.3037471175, output_fired_time: 0.0000000000, propagate_forward_time: 12.1945486069, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8623733521
validation[          , genome  6807] predictions:    9735/  10000 (97.35%), best:    9768/10000 (97.68%), error:       840.10358, best error:       715.60901 on epoch:    22, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 159.5191650391s, input_fired_time: 2.4686934948, output_fired_time: 1.9452110529, propagate_forward_time: 60.8528099060, propagate_backward_time: 89.4722213745, weight_update_time: 0.2413461208, other_time: 4.7802352905
epoch time: 13.4668331146s, input_fired_time: 0.2780051231, output_fired_time: 0.0000000000, propagate_forward_time: 12.3377828598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8510446548
validation[          , genome  6807] predictions:    9772/  10000 (97.72%), best:    9772/10000 (97.72%), error:       715.31415, best error:       715.31415 on epoch:    24, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 160.4132080078s, input_fired_time: 2.5311932564, output_fired_time: 1.9792050123, propagate_forward_time: 61.1528129578, propagate_backward_time: 89.9677581787, weight_update_time: 0.2373564690, other_time: 4.7822418213
epoch time: 13.3049535751s, input_fired_time: 0.2875314057, output_fired_time: 0.0000000000, propagate_forward_time: 12.1709079742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8465137482
validation[          , genome  6807] predictions:    9711/  10000 (97.11%), best:    9772/10000 (97.72%), error:       871.08533, best error:       715.31415 on epoch:    24, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 159.4143066406s, input_fired_time: 2.5329902172, output_fired_time: 1.9117990732, propagate_forward_time: 61.0901908875, propagate_backward_time: 89.1106338501, weight_update_time: 0.2373454124, other_time: 4.7686843872
epoch time: 13.4296236038s, input_fired_time: 0.2771677673, output_fired_time: 0.0000000000, propagate_forward_time: 12.2974977493, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8549585342
validation[          , genome  6807] predictions:    9748/  10000 (97.48%), best:    9772/10000 (97.72%), error:       822.88074, best error:       715.31415 on epoch:    24, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 163.4949188232s, input_fired_time: 2.5063989162, output_fired_time: 1.9310024977, propagate_forward_time: 62.3065147400, propagate_backward_time: 91.6108169556, weight_update_time: 0.2428126782, other_time: 5.1401901245
epoch time: 13.5661859512s, input_fired_time: 0.2845595479, output_fired_time: 0.0000000000, propagate_forward_time: 12.1809158325, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1007108688
validation[          , genome  6807] predictions:    9745/  10000 (97.45%), best:    9772/10000 (97.72%), error:       805.12170, best error:       715.31415 on epoch:    24, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 159.6590270996s, input_fired_time: 2.4971451759, output_fired_time: 1.9356720448, propagate_forward_time: 60.9422683716, propagate_backward_time: 89.4919357300, weight_update_time: 0.2378811687, other_time: 4.7920074463
epoch time: 13.3076457977s, input_fired_time: 0.2809348106, output_fired_time: 0.0000000000, propagate_forward_time: 12.1605682373, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8661422729
validation[          , genome  6807] predictions:    9761/  10000 (97.61%), best:    9772/10000 (97.72%), error:       789.76672, best error:       715.31415 on epoch:    24, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 159.6232910156s, input_fired_time: 2.4713730812, output_fired_time: 1.9600200653, propagate_forward_time: 60.7474517822, propagate_backward_time: 89.6429901123, weight_update_time: 0.2386035919, other_time: 4.8014526367
epoch time: 13.2088613510s, input_fired_time: 0.2857742310, output_fired_time: 0.0000000000, propagate_forward_time: 12.0632839203, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8598031998
validation[          , genome  6807] predictions:    9705/  10000 (97.05%), best:    9772/10000 (97.72%), error:      1012.43500, best error:       715.31415 on epoch:    24, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 158.9737243652s, input_fired_time: 2.5531404018, output_fired_time: 1.8910272121, propagate_forward_time: 60.7253265381, propagate_backward_time: 89.0439987183, weight_update_time: 0.2444104403, other_time: 4.7602310181
epoch time: 13.4815082550s, input_fired_time: 0.2790737152, output_fired_time: 0.0000000000, propagate_forward_time: 12.3410902023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8613443375
validation[          , genome  6807] predictions:    9764/  10000 (97.64%), best:    9772/10000 (97.72%), error:       741.73364, best error:       715.31415 on epoch:    24, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 159.9362030029s, input_fired_time: 2.5099761486, output_fired_time: 1.9168045521, propagate_forward_time: 61.1696166992, propagate_backward_time: 89.5680389404, weight_update_time: 0.2355703562, other_time: 4.7717590332
epoch time: 13.1966304779s, input_fired_time: 0.2789888382, output_fired_time: 0.0000000000, propagate_forward_time: 12.0758295059, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8418121338
validation[          , genome  6807] predictions:    9775/  10000 (97.75%), best:    9772/10000 (97.72%), error:       717.90167, best error:       715.31415 on epoch:    24, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 159.8152465820s, input_fired_time: 2.4776425362, output_fired_time: 1.9025031328, propagate_forward_time: 61.2842102051, propagate_backward_time: 89.2842941284, weight_update_time: 0.2365635633, other_time: 4.8666000366
epoch time: 13.3177986145s, input_fired_time: 0.2807242274, output_fired_time: 0.0000000000, propagate_forward_time: 12.1673755646, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8696985245
validation[          , genome  6807] predictions:    9731/  10000 (97.31%), best:    9772/10000 (97.72%), error:       850.00037, best error:       715.31415 on epoch:    24, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 159.7304992676s, input_fired_time: 2.4606034756, output_fired_time: 1.9018598795, propagate_forward_time: 61.0682220459, propagate_backward_time: 89.5269851685, weight_update_time: 0.2351533622, other_time: 4.7728347778
epoch time: 13.2755775452s, input_fired_time: 0.2860159278, output_fired_time: 0.0000000000, propagate_forward_time: 12.1048669815, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8846950531
validation[          , genome  6807] predictions:    9763/  10000 (97.63%), best:    9772/10000 (97.72%), error:       781.02002, best error:       715.31415 on epoch:    24, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 159.9634704590s, input_fired_time: 2.5240294933, output_fired_time: 1.9499855042, propagate_forward_time: 61.0165290833, propagate_backward_time: 89.5708999634, weight_update_time: 0.2372132093, other_time: 4.9020309448
epoch time: 13.2700796127s, input_fired_time: 0.3250667751, output_fired_time: 0.0000000000, propagate_forward_time: 12.0854663849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8595466614
validation[          , genome  6807] predictions:    9772/  10000 (97.72%), best:    9772/10000 (97.72%), error:       699.34766, best error:       699.34766 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 159.0629119873s, input_fired_time: 2.4538810253, output_fired_time: 1.9528362751, propagate_forward_time: 60.8725852966, propagate_backward_time: 88.9948043823, weight_update_time: 0.2399387658, other_time: 4.7887954712
epoch time: 13.2834939957s, input_fired_time: 0.2836200595, output_fired_time: 0.0000000000, propagate_forward_time: 12.1505374908, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8493366241
validation[          , genome  6807] predictions:    9728/  10000 (97.28%), best:    9772/10000 (97.72%), error:       897.50873, best error:       699.34766 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 159.7513427734s, input_fired_time: 2.4669802189, output_fired_time: 1.9315646887, propagate_forward_time: 60.9801979065, propagate_backward_time: 89.6275634766, weight_update_time: 0.2356252968, other_time: 4.7450408936
epoch time: 13.2699499130s, input_fired_time: 0.2835826576, output_fired_time: 0.0000000000, propagate_forward_time: 12.1070518494, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8793153763
validation[          , genome  6807] predictions:    9793/  10000 (97.93%), best:    9793/10000 (97.93%), error:       694.21301, best error:       694.21301 on epoch:    36, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 159.5242767334s, input_fired_time: 2.4617376328, output_fired_time: 1.8964596987, propagate_forward_time: 60.7606239319, propagate_backward_time: 89.6617355347, weight_update_time: 0.2365522981, other_time: 4.7437362671
epoch time: 13.2648077011s, input_fired_time: 0.2823797166, output_fired_time: 0.0000000000, propagate_forward_time: 12.1132287979, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8691987991
validation[          , genome  6807] predictions:    9746/  10000 (97.46%), best:    9793/10000 (97.93%), error:       871.59167, best error:       694.21301 on epoch:    36, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 159.6417388916s, input_fired_time: 2.4606635571, output_fired_time: 1.9125797749, propagate_forward_time: 60.9136505127, propagate_backward_time: 89.4529647827, weight_update_time: 0.2444030792, other_time: 4.9018783569
epoch time: 13.3739719391s, input_fired_time: 0.2779158354, output_fired_time: 0.0000000000, propagate_forward_time: 12.2449531555, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8511028290
validation[          , genome  6807] predictions:    9731/  10000 (97.31%), best:    9793/10000 (97.93%), error:       918.99408, best error:       694.21301 on epoch:    36, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 159.6746215820s, input_fired_time: 2.4695763588, output_fired_time: 1.9874514341, propagate_forward_time: 60.6435165405, propagate_backward_time: 89.8188323975, weight_update_time: 0.2376870364, other_time: 4.7552413940
epoch time: 13.2852706909s, input_fired_time: 0.3217462897, output_fired_time: 0.0000000000, propagate_forward_time: 12.1049013138, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8586235046
validation[          , genome  6807] predictions:    9682/  10000 (96.82%), best:    9793/10000 (97.93%), error:      1022.25519, best error:       694.21301 on epoch:    36, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 159.5922698975s, input_fired_time: 2.4475867748, output_fired_time: 1.8930974007, propagate_forward_time: 61.1878089905, propagate_backward_time: 89.2954254150, weight_update_time: 0.2412991822, other_time: 4.7683563232
epoch time: 13.2952718735s, input_fired_time: 0.2757317722, output_fired_time: 0.0000000000, propagate_forward_time: 12.1755809784, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8439588547
validation[          , genome  6807] predictions:    9748/  10000 (97.48%), best:    9793/10000 (97.93%), error:       776.14935, best error:       694.21301 on epoch:    36, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 159.4063262939s, input_fired_time: 2.4557967186, output_fired_time: 1.8934241533, propagate_forward_time: 60.8352546692, propagate_backward_time: 89.4422988892, weight_update_time: 0.2382081747, other_time: 4.7795639038
epoch time: 13.2562074661s, input_fired_time: 0.2733123899, output_fired_time: 0.0000000000, propagate_forward_time: 12.1173219681, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8655729294
validation[          , genome  6807] predictions:    9743/  10000 (97.43%), best:    9793/10000 (97.93%), error:       813.06781, best error:       694.21301 on epoch:    36, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 159.1430358887s, input_fired_time: 2.4699494839, output_fired_time: 1.9358421564, propagate_forward_time: 60.5825958252, propagate_backward_time: 89.4234695435, weight_update_time: 0.2390994430, other_time: 4.7311782837
epoch time: 13.2643241882s, input_fired_time: 0.2897547781, output_fired_time: 0.0000000000, propagate_forward_time: 12.1320514679, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8425178528
validation[          , genome  6807] predictions:    9698/  10000 (96.98%), best:    9793/10000 (97.93%), error:      1014.35065, best error:       694.21301 on epoch:    36, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 159.3383941650s, input_fired_time: 2.4793391228, output_fired_time: 1.8895188570, propagate_forward_time: 61.0743713379, propagate_backward_time: 89.1258773804, weight_update_time: 0.2353379130, other_time: 4.7692794800
epoch time: 13.2506790161s, input_fired_time: 0.2757977545, output_fired_time: 0.0000000000, propagate_forward_time: 12.1240539551, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8508272171
validation[          , genome  6807] predictions:    9768/  10000 (97.68%), best:    9793/10000 (97.93%), error:       773.61340, best error:       694.21301 on epoch:    36, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 159.4192199707s, input_fired_time: 2.5119256973, output_fired_time: 1.8917238712, propagate_forward_time: 60.6829986572, propagate_backward_time: 89.6041717529, weight_update_time: 0.2366806716, other_time: 4.7283935547
epoch time: 13.3600072861s, input_fired_time: 0.2768754661, output_fired_time: 0.0000000000, propagate_forward_time: 12.2391843796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8439474106
validation[          , genome  6807] predictions:    9792/  10000 (97.92%), best:    9792/10000 (97.92%), error:       693.75189, best error:       693.75189 on epoch:    44, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 159.7223968506s, input_fired_time: 2.4963519573, output_fired_time: 1.9715056419, propagate_forward_time: 60.8902626038, propagate_backward_time: 89.6296310425, weight_update_time: 0.2374047935, other_time: 4.7346420288
epoch time: 13.3219680786s, input_fired_time: 0.2814400792, output_fired_time: 0.0000000000, propagate_forward_time: 12.1825027466, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8580255508
validation[          , genome  6807] predictions:    9295/  10000 (92.95%), best:    9792/10000 (97.92%), error:      2840.20923, best error:       693.75189 on epoch:    44, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

resetting weights to those on epoch: 44
epoch time: 159.3963775635s, input_fired_time: 2.4513850212, output_fired_time: 1.8840500116, propagate_forward_time: 60.8116149902, propagate_backward_time: 89.4707946777, weight_update_time: 0.2391690165, other_time: 4.7785339355
epoch time: 13.3072633743s, input_fired_time: 0.2756051123, output_fired_time: 0.0000000000, propagate_forward_time: 12.1728458405, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8588123322
validation[          , genome  6807] predictions:    9767/  10000 (97.67%), best:    9792/10000 (97.92%), error:       760.60815, best error:       693.75189 on epoch:    44, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 159.7188110352s, input_fired_time: 2.4725210667, output_fired_time: 1.8879010677, propagate_forward_time: 61.0588226318, propagate_backward_time: 89.5752487183, weight_update_time: 0.2410923094, other_time: 4.7243270874
epoch time: 13.2450113297s, input_fired_time: 0.2769956887, output_fired_time: 0.0000000000, propagate_forward_time: 12.1135568619, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8544588089
validation[          , genome  6807] predictions:    9780/  10000 (97.80%), best:    9792/10000 (97.92%), error:       776.03552, best error:       693.75189 on epoch:    44, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 159.5904998779s, input_fired_time: 2.4837124348, output_fired_time: 1.9161812067, propagate_forward_time: 61.2154312134, propagate_backward_time: 89.2122802734, weight_update_time: 0.2412271798, other_time: 4.7628860474
epoch time: 13.2724103928s, input_fired_time: 0.2746546566, output_fired_time: 0.0000000000, propagate_forward_time: 12.1401176453, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8576383591
validation[          , genome  6807] predictions:    9743/  10000 (97.43%), best:    9792/10000 (97.92%), error:       839.06464, best error:       693.75189 on epoch:    44, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 159.3446044922s, input_fired_time: 2.5029611588, output_fired_time: 1.8816920519, propagate_forward_time: 60.8792228699, propagate_backward_time: 89.3463973999, weight_update_time: 0.2372823209, other_time: 4.7343215942
epoch time: 13.3518972397s, input_fired_time: 0.2762032449, output_fired_time: 0.0000000000, propagate_forward_time: 12.2179279327, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8577661514
validation[          , genome  6807] predictions:    9796/  10000 (97.96%), best:    9796/10000 (97.96%), error:       660.24396, best error:       660.24396 on epoch:    49, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 159.0449218750s, input_fired_time: 2.4993848801, output_fired_time: 1.9235285521, propagate_forward_time: 60.7966461182, propagate_backward_time: 89.0874252319, weight_update_time: 0.2351630032, other_time: 4.7379379272
epoch time: 13.2988500595s, input_fired_time: 0.2776041627, output_fired_time: 0.0000000000, propagate_forward_time: 12.1548690796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8663768768
validation[          , genome  6807] predictions:    9759/  10000 (97.59%), best:    9796/10000 (97.96%), error:       806.84222, best error:       660.24396 on epoch:    49, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 159.4575653076s, input_fired_time: 2.4842936993, output_fired_time: 1.9023523331, propagate_forward_time: 61.0753860474, propagate_backward_time: 89.2245941162, weight_update_time: 0.2378519773, other_time: 4.7709274292
epoch time: 13.3730792999s, input_fired_time: 0.3056847155, output_fired_time: 0.0000000000, propagate_forward_time: 12.2122116089, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8551826477
validation[          , genome  6807] predictions:    9759/  10000 (97.59%), best:    9796/10000 (97.96%), error:       803.01788, best error:       660.24396 on epoch:    49, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 159.2042846680s, input_fired_time: 2.4766216278, output_fired_time: 1.9507106543, propagate_forward_time: 60.8572425842, propagate_backward_time: 89.1436462402, weight_update_time: 0.2445106804, other_time: 4.7760620117
epoch time: 13.4060344696s, input_fired_time: 0.3015793264, output_fired_time: 0.0000000000, propagate_forward_time: 12.2493886948, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8550662994
validation[          , genome  6807] predictions:    9779/  10000 (97.79%), best:    9796/10000 (97.96%), error:       692.88501, best error:       660.24396 on epoch:    49, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 159.7298126221s, input_fired_time: 2.4659218788, output_fired_time: 1.9579044580, propagate_forward_time: 61.0044250488, propagate_backward_time: 89.5331192017, weight_update_time: 0.2359758765, other_time: 4.7684402466
epoch time: 13.2905321121s, input_fired_time: 0.2764052153, output_fired_time: 0.0000000000, propagate_forward_time: 12.1542968750, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8598299026
validation[          , genome  6807] predictions:    9733/  10000 (97.33%), best:    9796/10000 (97.96%), error:       863.85669, best error:       660.24396 on epoch:    49, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 159.3315734863s, input_fired_time: 2.5085916519, output_fired_time: 1.8994507790, propagate_forward_time: 60.9238433838, propagate_backward_time: 89.2444000244, weight_update_time: 0.2402880788, other_time: 4.7552947998
epoch time: 13.2055339813s, input_fired_time: 0.2787154615, output_fired_time: 0.0000000000, propagate_forward_time: 12.0801792145, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8466396332
validation[          , genome  6807] predictions:    9668/  10000 (96.68%), best:    9796/10000 (97.96%), error:      1173.05457, best error:       660.24396 on epoch:    49, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 159.7435150146s, input_fired_time: 2.4786882401, output_fired_time: 1.8963456154, propagate_forward_time: 61.0892333984, propagate_backward_time: 89.4889526367, weight_update_time: 0.2358063906, other_time: 4.7902984619
epoch time: 13.3980035782s, input_fired_time: 0.2800205946, output_fired_time: 0.0000000000, propagate_forward_time: 12.2530965805, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8648862839
validation[          , genome  6807] predictions:    9730/  10000 (97.30%), best:    9796/10000 (97.96%), error:       882.62354, best error:       660.24396 on epoch:    49, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 159.1682739258s, input_fired_time: 2.4596657753, output_fired_time: 1.9510201216, propagate_forward_time: 60.7330322266, propagate_backward_time: 89.2528152466, weight_update_time: 0.2352946699, other_time: 4.7717361450
epoch time: 13.2173471451s, input_fired_time: 0.2834648192, output_fired_time: 0.0000000000, propagate_forward_time: 12.0609159470, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8729667664
validation[          , genome  6807] predictions:    9761/  10000 (97.61%), best:    9796/10000 (97.96%), error:       776.38263, best error:       660.24396 on epoch:    49, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 159.5540618896s, input_fired_time: 2.4769489765, output_fired_time: 1.9893202782, propagate_forward_time: 60.7070884705, propagate_backward_time: 89.6117858887, weight_update_time: 0.2357646376, other_time: 4.7689208984
epoch time: 13.4449310303s, input_fired_time: 0.2771562934, output_fired_time: 0.0000000000, propagate_forward_time: 12.3164911270, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8512840271
validation[          , genome  6807] predictions:    9747/  10000 (97.47%), best:    9796/10000 (97.96%), error:       809.06445, best error:       660.24396 on epoch:    49, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 159.2136840820s, input_fired_time: 2.4992246628, output_fired_time: 1.8840930462, propagate_forward_time: 60.7107009888, propagate_backward_time: 89.3591384888, weight_update_time: 0.2372155786, other_time: 4.7605285645
epoch time: 13.2800540924s, input_fired_time: 0.2890895307, output_fired_time: 0.0000000000, propagate_forward_time: 12.1027965546, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8881683350
validation[          , genome  6807] predictions:    9768/  10000 (97.68%), best:    9796/10000 (97.96%), error:       788.49957, best error:       660.24396 on epoch:    49, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 160.1775360107s, input_fired_time: 2.4636650085, output_fired_time: 1.9067300558, propagate_forward_time: 61.2846946716, propagate_backward_time: 89.7091522217, weight_update_time: 0.2371310741, other_time: 4.8132934570
epoch time: 13.2081975937s, input_fired_time: 0.2803330123, output_fired_time: 0.0000000000, propagate_forward_time: 12.0689945221, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8588705063
validation[          , genome  6807] predictions:    9777/  10000 (97.77%), best:    9796/10000 (97.96%), error:       747.44049, best error:       660.24396 on epoch:    49, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 162.7102050781s, input_fired_time: 2.5444788933, output_fired_time: 1.9913394451, propagate_forward_time: 62.0867004395, propagate_backward_time: 91.2003860474, weight_update_time: 0.2353291661, other_time: 4.8873062134
epoch time: 13.2152166367s, input_fired_time: 0.2835942209, output_fired_time: 0.0000000000, propagate_forward_time: 12.0617818832, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8698406219
validation[          , genome  6807] predictions:    9749/  10000 (97.49%), best:    9796/10000 (97.96%), error:       837.06372, best error:       660.24396 on epoch:    49, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 159.3804626465s, input_fired_time: 2.4675602913, output_fired_time: 1.9008597136, propagate_forward_time: 60.7619476318, propagate_backward_time: 89.4533081055, weight_update_time: 0.2412105948, other_time: 4.7967834473
epoch time: 13.2270193100s, input_fired_time: 0.2820508182, output_fired_time: 0.0000000000, propagate_forward_time: 12.0776052475, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8673629761
validation[          , genome  6807] predictions:    9780/  10000 (97.80%), best:    9796/10000 (97.96%), error:       689.66748, best error:       660.24396 on epoch:    49, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 159.6998291016s, input_fired_time: 2.5407292843, output_fired_time: 1.9298316240, propagate_forward_time: 60.6681671143, propagate_backward_time: 89.7031250000, weight_update_time: 0.2369402796, other_time: 4.8579864502
epoch time: 13.3095178604s, input_fired_time: 0.2817334235, output_fired_time: 0.0000000000, propagate_forward_time: 12.1701335907, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8576507568
validation[          , genome  6807] predictions:    9743/  10000 (97.43%), best:    9796/10000 (97.96%), error:       909.45764, best error:       660.24396 on epoch:    49, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 159.7205352783s, input_fired_time: 2.5157957077, output_fired_time: 1.9291014671, propagate_forward_time: 60.8881416321, propagate_backward_time: 89.6183471680, weight_update_time: 0.2367697060, other_time: 4.7691497803
epoch time: 13.3114585876s, input_fired_time: 0.2780342996, output_fired_time: 0.0000000000, propagate_forward_time: 12.1770963669, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8563280106
validation[          , genome  6807] predictions:    9786/  10000 (97.86%), best:    9796/10000 (97.96%), error:       686.24774, best error:       660.24396 on epoch:    49, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 159.9218292236s, input_fired_time: 2.4619166851, output_fired_time: 1.8978914022, propagate_forward_time: 60.7692565918, propagate_backward_time: 89.9661102295, weight_update_time: 0.2372446954, other_time: 4.8266601562
epoch time: 13.2417697906s, input_fired_time: 0.2827079892, output_fired_time: 0.0000000000, propagate_forward_time: 12.0868501663, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8722114563
validation[          , genome  6807] predictions:    9801/  10000 (98.01%), best:    9796/10000 (97.96%), error:       681.27814, best error:       660.24396 on epoch:    49, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 159.5246429443s, input_fired_time: 2.4528687000, output_fired_time: 1.9206346273, propagate_forward_time: 60.6602210999, propagate_backward_time: 89.6937637329, weight_update_time: 0.2431249917, other_time: 4.7971572876
epoch time: 13.3765659332s, input_fired_time: 0.2860420346, output_fired_time: 0.0000000000, propagate_forward_time: 12.2321300507, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8583936691
validation[          , genome  6807] predictions:    9751/  10000 (97.51%), best:    9796/10000 (97.96%), error:       816.61926, best error:       660.24396 on epoch:    49, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 159.6998748779s, input_fired_time: 2.5359013081, output_fired_time: 1.9033150673, propagate_forward_time: 60.8995780945, propagate_backward_time: 89.5577239990, weight_update_time: 0.2411836088, other_time: 4.8033447266
epoch time: 13.2571668625s, input_fired_time: 0.2866134942, output_fired_time: 0.0000000000, propagate_forward_time: 12.1135587692, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8569946289
validation[          , genome  6807] predictions:    9755/  10000 (97.55%), best:    9796/10000 (97.96%), error:       847.41150, best error:       660.24396 on epoch:    49, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 159.6282348633s, input_fired_time: 2.4711134434, output_fired_time: 1.9199849367, propagate_forward_time: 60.9460105896, propagate_backward_time: 89.4708938599, weight_update_time: 0.2404204607, other_time: 4.8202285767
epoch time: 13.3262033463s, input_fired_time: 0.2873391509, output_fired_time: 0.0000000000, propagate_forward_time: 12.1684360504, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8704280853
validation[          , genome  6807] predictions:    9688/  10000 (96.88%), best:    9796/10000 (97.96%), error:      1086.11475, best error:       660.24396 on epoch:    49, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 159.8546142578s, input_fired_time: 2.4701969624, output_fired_time: 1.9460232258, propagate_forward_time: 60.9813957214, propagate_backward_time: 89.6873855591, weight_update_time: 0.2375244647, other_time: 4.7695999146
epoch time: 13.3484773636s, input_fired_time: 0.2858747542, output_fired_time: 0.0000000000, propagate_forward_time: 12.1949529648, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8676500320
validation[          , genome  6807] predictions:    9747/  10000 (97.47%), best:    9796/10000 (97.96%), error:       824.89252, best error:       660.24396 on epoch:    49, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 159.4976654053s, input_fired_time: 2.5038220882, output_fired_time: 1.9546582699, propagate_forward_time: 60.7978973389, propagate_backward_time: 89.4613342285, weight_update_time: 0.2372270375, other_time: 4.7799682617
epoch time: 13.2428979874s, input_fired_time: 0.2840844691, output_fired_time: 0.0000000000, propagate_forward_time: 12.0705747604, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8882389069
validation[          , genome  6807] predictions:    9758/  10000 (97.58%), best:    9796/10000 (97.96%), error:       795.87390, best error:       660.24396 on epoch:    49, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 159.6757354736s, input_fired_time: 2.4547705650, output_fired_time: 1.9186525345, propagate_forward_time: 60.8062286377, propagate_backward_time: 89.6887054443, weight_update_time: 0.2413553447, other_time: 4.8073730469
epoch time: 13.3751955032s, input_fired_time: 0.2863626778, output_fired_time: 0.0000000000, propagate_forward_time: 12.2318162918, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8570165634
validation[          , genome  6807] predictions:    9795/  10000 (97.95%), best:    9796/10000 (97.96%), error:       668.49524, best error:       660.24396 on epoch:    49, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 160.0573730469s, input_fired_time: 2.5411083698, output_fired_time: 1.9448535442, propagate_forward_time: 61.0732383728, propagate_backward_time: 89.6941452026, weight_update_time: 0.2386706620, other_time: 4.8040237427
epoch time: 13.4106359482s, input_fired_time: 0.3066257238, output_fired_time: 0.0000000000, propagate_forward_time: 12.2480735779, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8559370041
validation[          , genome  6807] predictions:    9799/  10000 (97.99%), best:    9799/10000 (97.99%), error:       657.19019, best error:       657.19019 on epoch:    71, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 159.6589965820s, input_fired_time: 2.4749462605, output_fired_time: 1.8991169930, propagate_forward_time: 60.9236526489, propagate_backward_time: 89.5248260498, weight_update_time: 0.2406282276, other_time: 4.8364486694
epoch time: 13.2824821472s, input_fired_time: 0.2783287466, output_fired_time: 0.0000000000, propagate_forward_time: 12.1423788071, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8617744446
validation[          , genome  6807] predictions:    9806/  10000 (98.06%), best:    9799/10000 (97.99%), error:       674.50256, best error:       657.19019 on epoch:    71, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 160.1665344238s, input_fired_time: 2.5231676102, output_fired_time: 1.9025635719, propagate_forward_time: 61.0328826904, propagate_backward_time: 89.8705978394, weight_update_time: 0.2405577153, other_time: 4.8373336792
epoch time: 13.2086973190s, input_fired_time: 0.2835633159, output_fired_time: 0.0000000000, propagate_forward_time: 12.0592737198, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8658599854
validation[          , genome  6807] predictions:    9772/  10000 (97.72%), best:    9799/10000 (97.99%), error:       779.79108, best error:       657.19019 on epoch:    71, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 159.8869934082s, input_fired_time: 2.4454250336, output_fired_time: 1.9159157276, propagate_forward_time: 61.0525283813, propagate_backward_time: 89.6749038696, weight_update_time: 0.2449271381, other_time: 4.7982330322
epoch time: 13.3032417297s, input_fired_time: 0.2805230916, output_fired_time: 0.0000000000, propagate_forward_time: 12.1640796661, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8586387634
validation[          , genome  6807] predictions:    9505/  10000 (95.05%), best:    9799/10000 (97.99%), error:      1793.50159, best error:       657.19019 on epoch:    71, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

resetting weights to those on epoch: 71
epoch time: 159.3953552246s, input_fired_time: 2.5174951553, output_fired_time: 1.9118624926, propagate_forward_time: 60.6370544434, propagate_backward_time: 89.5139236450, weight_update_time: 0.2394976020, other_time: 4.8150100708
epoch time: 13.2358331680s, input_fired_time: 0.2863227725, output_fired_time: 0.0000000000, propagate_forward_time: 12.0767135620, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8727970123
validation[          , genome  6807] predictions:    9788/  10000 (97.88%), best:    9799/10000 (97.99%), error:       716.72260, best error:       657.19019 on epoch:    71, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 159.2758483887s, input_fired_time: 2.4841208458, output_fired_time: 1.9278506041, propagate_forward_time: 60.8740272522, propagate_backward_time: 89.1650009155, weight_update_time: 0.2475294322, other_time: 4.8248519897
epoch time: 13.3249416351s, input_fired_time: 0.2848326266, output_fired_time: 0.0000000000, propagate_forward_time: 12.1405801773, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8995285034
validation[          , genome  6807] predictions:    9689/  10000 (96.89%), best:    9799/10000 (97.99%), error:      1053.80774, best error:       657.19019 on epoch:    71, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 159.8050842285s, input_fired_time: 2.4903142452, output_fired_time: 1.9060522318, propagate_forward_time: 60.6427688599, propagate_backward_time: 89.9603576660, weight_update_time: 0.2405186743, other_time: 4.8055953979
epoch time: 13.3961715698s, input_fired_time: 0.2812111974, output_fired_time: 0.0000000000, propagate_forward_time: 12.2501974106, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8647632599
validation[          , genome  6807] predictions:    9764/  10000 (97.64%), best:    9799/10000 (97.99%), error:       749.67303, best error:       657.19019 on epoch:    71, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 159.5367431641s, input_fired_time: 2.4978215694, output_fired_time: 1.9230272770, propagate_forward_time: 60.8025436401, propagate_backward_time: 89.5134658813, weight_update_time: 0.2364873290, other_time: 4.7998809814
epoch time: 13.3954858780s, input_fired_time: 0.2902942300, output_fired_time: 0.0000000000, propagate_forward_time: 12.2224721909, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8827190399
validation[          , genome  6807] predictions:    9471/  10000 (94.71%), best:    9799/10000 (97.99%), error:      1881.48547, best error:       657.19019 on epoch:    71, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

resetting weights to those on epoch: 71
epoch time: 160.1183624268s, input_fired_time: 2.4809782505, output_fired_time: 1.9756474495, propagate_forward_time: 61.2128868103, propagate_backward_time: 89.5870895386, weight_update_time: 0.2478314340, other_time: 4.8617630005
epoch time: 13.2837371826s, input_fired_time: 0.2976117134, output_fired_time: 0.0000000000, propagate_forward_time: 12.1174306870, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8686952591
validation[          , genome  6807] predictions:    9790/  10000 (97.90%), best:    9799/10000 (97.99%), error:       732.78217, best error:       657.19019 on epoch:    71, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 159.7330780029s, input_fired_time: 2.4985551834, output_fired_time: 1.9202622175, propagate_forward_time: 60.7634544373, propagate_backward_time: 89.7354736328, weight_update_time: 0.2356174290, other_time: 4.8153381348
epoch time: 13.2545404434s, input_fired_time: 0.2838755548, output_fired_time: 0.0000000000, propagate_forward_time: 12.0948524475, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8758125305
validation[          , genome  6807] predictions:    9806/  10000 (98.06%), best:    9799/10000 (97.99%), error:       697.13861, best error:       657.19019 on epoch:    71, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 158.9219970703s, input_fired_time: 2.4775667191, output_fired_time: 1.9072757959, propagate_forward_time: 60.6052665710, propagate_backward_time: 89.1080322266, weight_update_time: 0.2359012216, other_time: 4.8238525391
epoch time: 13.3148460388s, input_fired_time: 0.2860895395, output_fired_time: 0.0000000000, propagate_forward_time: 12.1480617523, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8806943893
validation[          , genome  6807] predictions:    9775/  10000 (97.75%), best:    9799/10000 (97.99%), error:       739.54346, best error:       657.19019 on epoch:    71, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 159.8148193359s, input_fired_time: 2.4941422939, output_fired_time: 1.9050732851, propagate_forward_time: 60.9243965149, propagate_backward_time: 89.5931625366, weight_update_time: 0.2376791388, other_time: 4.8980484009
epoch time: 13.2529411316s, input_fired_time: 0.2822797298, output_fired_time: 0.0000000000, propagate_forward_time: 12.1031799316, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8674812317
validation[          , genome  6807] predictions:    9799/  10000 (97.99%), best:    9799/10000 (97.99%), error:       690.96277, best error:       657.19019 on epoch:    71, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 159.3061065674s, input_fired_time: 2.4726941586, output_fired_time: 1.9436787367, propagate_forward_time: 60.6741600037, propagate_backward_time: 89.3767318726, weight_update_time: 0.2370451987, other_time: 4.8388442993
epoch time: 13.2153091431s, input_fired_time: 0.2775343657, output_fired_time: 0.0000000000, propagate_forward_time: 12.0829162598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8548583984
validation[          , genome  6807] predictions:    9805/  10000 (98.05%), best:    9799/10000 (97.99%), error:       702.74878, best error:       657.19019 on epoch:    71, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 159.3112182617s, input_fired_time: 2.4913587570, output_fired_time: 1.9100208282, propagate_forward_time: 60.4130973816, propagate_backward_time: 89.7057037354, weight_update_time: 0.2407183200, other_time: 4.7910308838
epoch time: 13.3496856689s, input_fired_time: 0.2824118733, output_fired_time: 0.0000000000, propagate_forward_time: 12.1820564270, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8852176666
validation[          , genome  6807] predictions:    9755/  10000 (97.55%), best:    9799/10000 (97.99%), error:       730.38599, best error:       657.19019 on epoch:    71, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 159.7736968994s, input_fired_time: 2.4944891930, output_fired_time: 1.9225698709, propagate_forward_time: 60.6407699585, propagate_backward_time: 89.7926101685, weight_update_time: 0.2360093147, other_time: 4.9232482910
epoch time: 13.2301425934s, input_fired_time: 0.2836115956, output_fired_time: 0.0000000000, propagate_forward_time: 12.0891437531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8573875427
validation[          , genome  6807] predictions:    9753/  10000 (97.53%), best:    9799/10000 (97.99%), error:       857.35284, best error:       657.19019 on epoch:    71, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 159.8573608398s, input_fired_time: 2.5996861458, output_fired_time: 1.9355783463, propagate_forward_time: 60.6954765320, propagate_backward_time: 89.8197402954, weight_update_time: 0.2390761077, other_time: 4.8068771362
epoch time: 13.2432403564s, input_fired_time: 0.2877346575, output_fired_time: 0.0000000000, propagate_forward_time: 12.0858173370, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8696880341
validation[          , genome  6807] predictions:    9697/  10000 (96.97%), best:    9799/10000 (97.99%), error:       952.83887, best error:       657.19019 on epoch:    71, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 159.7771453857s, input_fired_time: 2.4787733555, output_fired_time: 1.9497001171, propagate_forward_time: 61.0585861206, propagate_backward_time: 89.4403381348, weight_update_time: 0.2382532954, other_time: 4.8497390747
epoch time: 13.2913665771s, input_fired_time: 0.2851338685, output_fired_time: 0.0000000000, propagate_forward_time: 12.1221780777, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8840541840
validation[          , genome  6807] predictions:    9759/  10000 (97.59%), best:    9799/10000 (97.99%), error:       799.62701, best error:       657.19019 on epoch:    71, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 159.6009979248s, input_fired_time: 2.5022900105, output_fired_time: 1.9254217148, propagate_forward_time: 60.8151550293, propagate_backward_time: 89.4930496216, weight_update_time: 0.2370190024, other_time: 4.8650894165
epoch time: 13.2471532822s, input_fired_time: 0.2885115445, output_fired_time: 0.0000000000, propagate_forward_time: 12.0864677429, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8721742630
validation[          , genome  6807] predictions:    9783/  10000 (97.83%), best:    9799/10000 (97.99%), error:       705.19781, best error:       657.19019 on epoch:    71, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 159.5565643311s, input_fired_time: 2.4779238701, output_fired_time: 2.0079302788, propagate_forward_time: 60.8152351379, propagate_backward_time: 89.4117813110, weight_update_time: 0.2381525487, other_time: 4.8436965942
epoch time: 13.3776473999s, input_fired_time: 0.2820648551, output_fired_time: 0.0000000000, propagate_forward_time: 12.1619205475, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9336624146
validation[          , genome  6807] predictions:    9780/  10000 (97.80%), best:    9799/10000 (97.99%), error:       732.59216, best error:       657.19019 on epoch:    71, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 159.5414733887s, input_fired_time: 2.4873521328, output_fired_time: 1.9053359032, propagate_forward_time: 61.0707778931, propagate_backward_time: 89.1891174316, weight_update_time: 0.2376293093, other_time: 4.8888931274
epoch time: 13.2643556595s, input_fired_time: 0.2802849114, output_fired_time: 0.0000000000, propagate_forward_time: 12.1081876755, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8758831024
validation[          , genome  6807] predictions:    9731/  10000 (97.31%), best:    9799/10000 (97.99%), error:       875.95081, best error:       657.19019 on epoch:    71, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 159.8905029297s, input_fired_time: 2.4922478199, output_fired_time: 1.9225867987, propagate_forward_time: 60.9340667725, propagate_backward_time: 89.6849975586, weight_update_time: 0.2370746583, other_time: 4.8565979004
epoch time: 13.3360786438s, input_fired_time: 0.2975006998, output_fired_time: 0.0000000000, propagate_forward_time: 12.1635255814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8750524521
validation[          , genome  6807] predictions:    9783/  10000 (97.83%), best:    9799/10000 (97.99%), error:       740.15564, best error:       657.19019 on epoch:    71, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 159.7035827637s, input_fired_time: 2.5583760738, output_fired_time: 1.9240739346, propagate_forward_time: 60.8909835815, propagate_backward_time: 89.3837356567, weight_update_time: 0.2399634421, other_time: 4.9464111328
epoch time: 13.4623212814s, input_fired_time: 0.2872203588, output_fired_time: 0.0000000000, propagate_forward_time: 12.2656688690, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9094324112
validation[          , genome  6807] predictions:    9753/  10000 (97.53%), best:    9799/10000 (97.99%), error:       834.17407, best error:       657.19019 on epoch:    71, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 161.0321197510s, input_fired_time: 2.6402468681, output_fired_time: 2.0043191910, propagate_forward_time: 61.1641998291, propagate_backward_time: 90.2970046997, weight_update_time: 0.2405256629, other_time: 4.9263534546
epoch time: 13.3712844849s, input_fired_time: 0.2820520997, output_fired_time: 0.0000000000, propagate_forward_time: 12.1762094498, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9130229950
validation[          , genome  6807] predictions:    9789/  10000 (97.89%), best:    9799/10000 (97.99%), error:       714.46173, best error:       657.19019 on epoch:    71, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 159.3390045166s, input_fired_time: 2.5565071106, output_fired_time: 1.9290703535, propagate_forward_time: 60.8412742615, propagate_backward_time: 89.0279235840, weight_update_time: 0.2381007671, other_time: 4.9842224121
epoch time: 13.1773519516s, input_fired_time: 0.2850322127, output_fired_time: 0.0000000000, propagate_forward_time: 12.0388240814, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8534955978
validation[          , genome  6807] predictions:    9786/  10000 (97.86%), best:    9799/10000 (97.99%), error:       698.07788, best error:       657.19019 on epoch:    71, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 159.6846466064s, input_fired_time: 2.5159239769, output_fired_time: 1.9574987888, propagate_forward_time: 60.7950439453, propagate_backward_time: 89.5867004395, weight_update_time: 0.2371607721, other_time: 4.8294677734
epoch time: 13.1712818146s, input_fired_time: 0.2944174111, output_fired_time: 0.0000000000, propagate_forward_time: 11.9990777969, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8777866364
validation[          , genome  6807] predictions:    9759/  10000 (97.59%), best:    9799/10000 (97.99%), error:       788.91589, best error:       657.19019 on epoch:    71, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 159.5688629150s, input_fired_time: 2.4841339588, output_fired_time: 2.0307371616, propagate_forward_time: 60.8204078674, propagate_backward_time: 89.2796325684, weight_update_time: 0.2383852452, other_time: 4.9539642334
epoch time: 13.4020090103s, input_fired_time: 0.2836440504, output_fired_time: 0.0000000000, propagate_forward_time: 12.2278242111, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8905410767
validation[          , genome  6807] predictions:    9802/  10000 (98.02%), best:    9799/10000 (97.99%), error:       668.26611, best error:       657.19019 on epoch:    71, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 159.7276306152s, input_fired_time: 2.5021104813, output_fired_time: 1.9186831713, propagate_forward_time: 60.8739662170, propagate_backward_time: 89.5969238281, weight_update_time: 0.2401639819, other_time: 4.8359527588
epoch time: 13.3945779800s, input_fired_time: 0.2869247794, output_fired_time: 0.0000000000, propagate_forward_time: 12.2451696396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8624839783
validation[          , genome  6807] predictions:    9792/  10000 (97.92%), best:    9799/10000 (97.99%), error:       710.28748, best error:       657.19019 on epoch:    71, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 159.6091613770s, input_fired_time: 2.5072154999, output_fired_time: 1.9448370934, propagate_forward_time: 60.7506141663, propagate_backward_time: 89.5576171875, weight_update_time: 0.2366386205, other_time: 4.8488769531
epoch time: 13.4465465546s, input_fired_time: 0.2835135162, output_fired_time: 0.0000000000, propagate_forward_time: 12.1879138947, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9751195908
validation[          , genome  6807] predictions:    9771/  10000 (97.71%), best:    9799/10000 (97.99%), error:       770.84937, best error:       657.19019 on epoch:    71, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 159.8631744385s, input_fired_time: 2.5124106407, output_fired_time: 1.9530000687, propagate_forward_time: 61.0066184998, propagate_backward_time: 89.5072555542, weight_update_time: 0.2404809743, other_time: 4.8838882446
epoch time: 13.4145364761s, input_fired_time: 0.3230171204, output_fired_time: 0.0000000000, propagate_forward_time: 12.1996545792, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8918647766
validation[          , genome  6807] predictions:    9780/  10000 (97.80%), best:    9799/10000 (97.99%), error:       727.16138, best error:       657.19019 on epoch:    71, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 159.4940643311s, input_fired_time: 2.4913616180, output_fired_time: 1.9096596241, propagate_forward_time: 60.9612045288, propagate_backward_time: 89.2883300781, weight_update_time: 0.2377868146, other_time: 4.8435134888
epoch time: 13.2882642746s, input_fired_time: 0.2883683145, output_fired_time: 0.0000000000, propagate_forward_time: 12.1400184631, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8598775864
validation[          , genome  6807] predictions:    9783/  10000 (97.83%), best:    9799/10000 (97.99%), error:       777.66327, best error:       657.19019 on epoch:    71, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 66.7514038086s, input_fired_time: 1.4274765253, output_fired_time: 0.0000000000, propagate_forward_time: 60.9625358582, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.3613929749
best training[          , genome  6807] predictions:   49736/  50000 (99.47%), best:    9799/10000 (97.99%), error:      1169.63354, best error:       657.19019 on epoch:    71, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 14.1272077560s, input_fired_time: 0.2833731174, output_fired_time: 0.0000000000, propagate_forward_time: 12.4754419327, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3683929443
   testing[          , genome  6807] predictions:    9830/  10000 (98.30%), best:    9799/10000 (97.99%), error:       576.63287, best error:       657.19019 on epoch:    71, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
11:48:45 (11303): called boinc_finish(0)

</stderr_txt>
]]>