Nameexact_genome_1512568488_43_6690_0
Workunit2604756
Created7 Dec 2017, 21:20:07 UTC
Sent7 Dec 2017, 21:28:33 UTC
Report deadline18 Dec 2017, 22:12:19 UTC
Received8 Dec 2017, 5:48:45 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID62687
Run time2 hours 34 min 31 sec
CPU time2 hours 27 min 21 sec
Validate stateValid
Credit2,757.60
Device peak FLOPS4.32 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size156.87 MB
Peak swap size159.82 MB
Peak disk usage2.26 MB

Stderr output

<core_client_version>7.6.33</core_client_version>
<![CDATA[
<stderr_txt>
_time: 0.0000000000, other_time: 0.5712490082
validation[          , genome  6690] predictions:    9730/  10000 (97.30%), best:    9745/10000 (97.45%), error:       922.07764, best error:       877.56354 on epoch:    19, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 82.3506469727s, input_fired_time: 1.4878033400, output_fired_time: 1.1869019270, propagate_forward_time: 29.0972175598, propagate_backward_time: 47.0884094238, weight_update_time: 0.1844991595, other_time: 3.4903144836
epoch time: 6.5911197662s, input_fired_time: 0.2049307823, output_fired_time: 0.0000000000, propagate_forward_time: 5.7953810692, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5908079147
validation[          , genome  6690] predictions:    9713/  10000 (97.13%), best:    9745/10000 (97.45%), error:       946.74280, best error:       877.56354 on epoch:    19, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 82.0312652588s, input_fired_time: 1.4857228994, output_fired_time: 1.1644624472, propagate_forward_time: 29.1173362732, propagate_backward_time: 46.8758926392, weight_update_time: 0.1788802147, other_time: 3.3878517151
epoch time: 6.5066194534s, input_fired_time: 0.1839704663, output_fired_time: 0.0000000000, propagate_forward_time: 5.7651562691, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5574927330
validation[          , genome  6690] predictions:    9732/  10000 (97.32%), best:    9732/10000 (97.32%), error:       874.85577, best error:       874.85577 on epoch:    22, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 81.4559097290s, input_fired_time: 1.3543995619, output_fired_time: 1.0416367054, propagate_forward_time: 28.9732780457, propagate_backward_time: 47.3267555237, weight_update_time: 0.1472319663, other_time: 2.7598419189
epoch time: 6.3769330978s, input_fired_time: 0.1650119275, output_fired_time: 0.0000000000, propagate_forward_time: 5.6894364357, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5224847794
validation[          , genome  6690] predictions:    9703/  10000 (97.03%), best:    9732/10000 (97.32%), error:       980.35730, best error:       874.85577 on epoch:    22, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 81.9614639282s, input_fired_time: 1.2454426289, output_fired_time: 0.9801959991, propagate_forward_time: 29.3540267944, propagate_backward_time: 47.6902656555, weight_update_time: 0.1413709074, other_time: 2.6915321350
epoch time: 6.6226162910s, input_fired_time: 0.1956144720, output_fired_time: 0.0000000000, propagate_forward_time: 5.8520050049, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5749969482
validation[          , genome  6690] predictions:    9671/  10000 (96.71%), best:    9732/10000 (97.32%), error:      1065.77429, best error:       874.85577 on epoch:    22, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 82.0236129761s, input_fired_time: 1.4505863190, output_fired_time: 1.1745263338, propagate_forward_time: 28.9974536896, propagate_backward_time: 47.1490478516, weight_update_time: 0.1684324741, other_time: 3.2519989014
epoch time: 6.5250611305s, input_fired_time: 0.1893660575, output_fired_time: 0.0000000000, propagate_forward_time: 5.7585473061, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5771479607
validation[          , genome  6690] predictions:    9747/  10000 (97.47%), best:    9747/10000 (97.47%), error:       869.59149, best error:       869.59149 on epoch:    25, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 83.3023147583s, input_fired_time: 1.4787546396, output_fired_time: 1.2133316994, propagate_forward_time: 29.4054679871, propagate_backward_time: 47.9172363281, weight_update_time: 0.1749679446, other_time: 3.2875251770
epoch time: 7.2906465530s, input_fired_time: 0.2325976640, output_fired_time: 0.0000000000, propagate_forward_time: 6.4056777954, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6523709297
validation[          , genome  6690] predictions:    9734/  10000 (97.34%), best:    9747/10000 (97.47%), error:       901.95667, best error:       869.59149 on epoch:    25, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 86.1669311523s, input_fired_time: 1.5746848583, output_fired_time: 1.3076554537, propagate_forward_time: 30.2324295044, propagate_backward_time: 49.6994819641, weight_update_time: 0.1789241582, other_time: 3.3526802063
epoch time: 6.6035060883s, input_fired_time: 0.1936251670, output_fired_time: 0.0000000000, propagate_forward_time: 5.8008852005, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6089959145
validation[          , genome  6690] predictions:    9756/  10000 (97.56%), best:    9756/10000 (97.56%), error:       828.72327, best error:       828.72327 on epoch:    27, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 81.3787612915s, input_fired_time: 1.4290096760, output_fired_time: 1.1274604797, propagate_forward_time: 28.8803272247, propagate_backward_time: 46.6080131531, weight_update_time: 0.1832899153, other_time: 3.3339576721
epoch time: 6.5557689667s, input_fired_time: 0.1922298968, output_fired_time: 0.0000000000, propagate_forward_time: 5.7741189003, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5894203186
validation[          , genome  6690] predictions:    9734/  10000 (97.34%), best:    9756/10000 (97.56%), error:       873.22168, best error:       828.72327 on epoch:    27, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 80.8073043823s, input_fired_time: 1.4101401567, output_fired_time: 1.1063107252, propagate_forward_time: 28.7561874390, propagate_backward_time: 46.4177818298, weight_update_time: 0.1691493392, other_time: 3.1168861389
epoch time: 6.3651294708s, input_fired_time: 0.1763849258, output_fired_time: 0.0000000000, propagate_forward_time: 5.6427459717, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5459985733
validation[          , genome  6690] predictions:    9692/  10000 (96.92%), best:    9756/10000 (97.56%), error:      1016.65167, best error:       828.72327 on epoch:    27, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 80.6045761108s, input_fired_time: 1.3982529640, output_fired_time: 1.1198540926, propagate_forward_time: 28.6549949646, propagate_backward_time: 46.2108840942, weight_update_time: 0.1712173522, other_time: 3.2205848694
epoch time: 6.5021762848s, input_fired_time: 0.1887051612, output_fired_time: 0.0000000000, propagate_forward_time: 5.7447619438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5687093735
validation[          , genome  6690] predictions:    9744/  10000 (97.44%), best:    9756/10000 (97.56%), error:       836.29315, best error:       828.72327 on epoch:    27, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 81.1180877686s, input_fired_time: 1.4461297989, output_fired_time: 1.1319137812, propagate_forward_time: 28.8691883087, propagate_backward_time: 46.4643592834, weight_update_time: 0.1750513017, other_time: 3.2065010071
epoch time: 6.5159478188s, input_fired_time: 0.1852614880, output_fired_time: 0.0000000000, propagate_forward_time: 5.7633457184, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5673408508
validation[          , genome  6690] predictions:    9763/  10000 (97.63%), best:    9763/10000 (97.63%), error:       814.80426, best error:       814.80426 on epoch:    31, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 83.2744674683s, input_fired_time: 1.5065714121, output_fired_time: 1.2044868469, propagate_forward_time: 29.3880596161, propagate_backward_time: 47.9811668396, weight_update_time: 0.1647941619, other_time: 3.1941871643
epoch time: 6.5288033485s, input_fired_time: 0.1945446134, output_fired_time: 0.0000000000, propagate_forward_time: 5.7575683594, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5766901970
validation[          , genome  6690] predictions:    9769/  10000 (97.69%), best:    9769/10000 (97.69%), error:       806.32117, best error:       806.32117 on epoch:    32, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 79.2345581055s, input_fired_time: 1.3104809523, output_fired_time: 1.0436842442, propagate_forward_time: 28.3092422485, propagate_backward_time: 45.6316795349, weight_update_time: 0.1573180407, other_time: 2.9394721985
epoch time: 5.9892048836s, input_fired_time: 0.1353083998, output_fired_time: 0.0000000000, propagate_forward_time: 5.4890298843, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3648667336
validation[          , genome  6690] predictions:    9741/  10000 (97.41%), best:    9769/10000 (97.69%), error:       868.30853, best error:       806.32117 on epoch:    32, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 79.8443374634s, input_fired_time: 1.3581559658, output_fired_time: 1.0506968498, propagate_forward_time: 28.4821357727, propagate_backward_time: 45.9611320496, weight_update_time: 0.1601169854, other_time: 2.9922180176
epoch time: 6.4648523331s, input_fired_time: 0.1850873232, output_fired_time: 0.0000000000, propagate_forward_time: 5.6938419342, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5859231949
validation[          , genome  6690] predictions:    9740/  10000 (97.40%), best:    9769/10000 (97.69%), error:       865.14905, best error:       806.32117 on epoch:    32, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 82.0497131348s, input_fired_time: 1.4497902393, output_fired_time: 1.1597075462, propagate_forward_time: 29.0373802185, propagate_backward_time: 47.1302375793, weight_update_time: 0.1676999927, other_time: 3.2725982666
epoch time: 7.1183929443s, input_fired_time: 0.2270736843, output_fired_time: 0.0000000000, propagate_forward_time: 6.2909092903, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6004099846
validation[          , genome  6690] predictions:    9773/  10000 (97.73%), best:    9773/10000 (97.73%), error:       789.24591, best error:       789.24591 on epoch:    35, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 82.1142730713s, input_fired_time: 1.4615541697, output_fired_time: 1.1702494621, propagate_forward_time: 29.1253547668, propagate_backward_time: 47.0317077637, weight_update_time: 0.1808698028, other_time: 3.3254051208
epoch time: 6.5727171898s, input_fired_time: 0.1915921271, output_fired_time: 0.0000000000, propagate_forward_time: 5.8036737442, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5774512291
validation[          , genome  6690] predictions:    9737/  10000 (97.37%), best:    9773/10000 (97.73%), error:       905.95850, best error:       789.24591 on epoch:    35, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 82.0842285156s, input_fired_time: 1.4484559298, output_fired_time: 1.1806510687, propagate_forward_time: 29.0448589325, propagate_backward_time: 46.9212913513, weight_update_time: 0.1859976053, other_time: 3.4889717102
epoch time: 6.6273140907s, input_fired_time: 0.1980538517, output_fired_time: 0.0000000000, propagate_forward_time: 5.8340215683, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5952386856
validation[          , genome  6690] predictions:    9755/  10000 (97.55%), best:    9773/10000 (97.73%), error:       809.23492, best error:       789.24591 on epoch:    35, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 81.5141448975s, input_fired_time: 1.4418417215, output_fired_time: 1.1375612020, propagate_forward_time: 28.8951892853, propagate_backward_time: 46.7749977112, weight_update_time: 0.1731325388, other_time: 3.2645606995
epoch time: 6.6225414276s, input_fired_time: 0.1989010870, output_fired_time: 0.0000000000, propagate_forward_time: 5.8134717941, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6101684570
validation[          , genome  6690] predictions:    9752/  10000 (97.52%), best:    9773/10000 (97.73%), error:       829.56262, best error:       789.24591 on epoch:    35, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 81.8809280396s, input_fired_time: 1.4646061659, output_fired_time: 1.1448051929, propagate_forward_time: 29.1185855865, propagate_backward_time: 46.8118209839, weight_update_time: 0.1732828617, other_time: 3.3411102295
epoch time: 6.6279644966s, input_fired_time: 0.1984615326, output_fired_time: 0.0000000000, propagate_forward_time: 5.8338956833, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5956072807
validation[          , genome  6690] predictions:    9746/  10000 (97.46%), best:    9773/10000 (97.73%), error:       845.85681, best error:       789.24591 on epoch:    35, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 81.9435882568s, input_fired_time: 1.4584047794, output_fired_time: 1.1586551666, propagate_forward_time: 28.9756298065, propagate_backward_time: 46.8769607544, weight_update_time: 0.1810670495, other_time: 3.4739379883
epoch time: 6.7178196907s, input_fired_time: 0.2177470773, output_fired_time: 0.0000000000, propagate_forward_time: 5.9236092567, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5764632225
validation[          , genome  6690] predictions:    9741/  10000 (97.41%), best:    9773/10000 (97.73%), error:       877.18024, best error:       789.24591 on epoch:    35, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_6690.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512568488_43_6690_0_r2125434009_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.930178
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00152608
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 6.10432e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 41
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 35
read number_validation_images: 10000
read best_validation_predictions: 9773
read best_validation_error: 789.246
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 6690
read normal distribution: '0 0.0998763516545296 -1.21585154533386'
generator_str: '144693292'
read generator: 144693292
read generated_by_map:
1 merge_node 1
reading 28 nodes.
reading 63 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 78.6688385009766s, input_fired_time: 1.16183423995972, output_fired_time: 0.866829693317413, propagate_forward_time: 28.4738616943359, propagate_backward_time: 45.8580627441406, weight_update_time: 0.12617053091526, other_time: 2.30825042724609
epoch time: 6.06482982635498s, input_fired_time: 0.135683819651604, output_fired_time: 0, propagate_forward_time: 5.54893827438354, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.380207538604736
validation[          , genome  6690] predictions:    9722/  10000 (97.22%), best:    9773/10000 (97.73%), error:       928.68848, best error:       789.24591 on epoch:    35, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 80.0881652832s, input_fired_time: 1.2589150667, output_fired_time: 0.9640820622, propagate_forward_time: 28.8954792023, propagate_backward_time: 46.3351745605, weight_update_time: 0.1434108764, other_time: 2.6345138550
epoch time: 6.3656125069s, input_fired_time: 0.1612784117, output_fired_time: 0.0000000000, propagate_forward_time: 5.7375802994, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4667539597
validation[          , genome  6690] predictions:    9733/  10000 (97.33%), best:    9773/10000 (97.73%), error:       886.30627, best error:       789.24591 on epoch:    35, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 78.4521865845s, input_fired_time: 1.1832545996, output_fired_time: 0.9020352960, propagate_forward_time: 28.3360366821, propagate_backward_time: 45.5674781799, weight_update_time: 0.1340067685, other_time: 2.4633750916
epoch time: 5.7443714142s, input_fired_time: 0.1080493107, output_fired_time: 0.0000000000, propagate_forward_time: 5.3481101990, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2882118225
validation[          , genome  6690] predictions:    9751/  10000 (97.51%), best:    9773/10000 (97.73%), error:       816.18298, best error:       789.24591 on epoch:    35, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 73.4075546265s, input_fired_time: 0.9228221178, output_fired_time: 0.7051367760, propagate_forward_time: 26.8767013550, propagate_backward_time: 43.2155532837, weight_update_time: 0.0911362916, other_time: 1.6873397827
epoch time: 5.7721319199s, input_fired_time: 0.1109358817, output_fired_time: 0.0000000000, propagate_forward_time: 5.3677854538, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2934107780
validation[          , genome  6690] predictions:    9745/  10000 (97.45%), best:    9773/10000 (97.73%), error:       833.37616, best error:       789.24591 on epoch:    35, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 80.9184799194s, input_fired_time: 1.2557990551, output_fired_time: 0.9776486754, propagate_forward_time: 29.0554771423, propagate_backward_time: 47.0731506348, weight_update_time: 0.1385362893, other_time: 2.5564079285
epoch time: 6.4707455635s, input_fired_time: 0.1621523499, output_fired_time: 0.0000000000, propagate_forward_time: 5.8700895309, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4385037422
validation[          , genome  6690] predictions:    9764/  10000 (97.64%), best:    9773/10000 (97.73%), error:       795.64465, best error:       789.24591 on epoch:    35, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 82.8353805542s, input_fired_time: 1.3047933578, output_fired_time: 1.0331588984, propagate_forward_time: 29.5587787628, propagate_backward_time: 48.3161430359, weight_update_time: 0.1496894956, other_time: 2.6225090027
epoch time: 5.7634820938s, input_fired_time: 0.1100625247, output_fired_time: 0.0000000000, propagate_forward_time: 5.3588280678, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2945914268
validation[          , genome  6690] predictions:    9760/  10000 (97.60%), best:    9773/10000 (97.73%), error:       813.03961, best error:       789.24591 on epoch:    35, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 80.2432174683s, input_fired_time: 1.2294145823, output_fired_time: 0.9616891742, propagate_forward_time: 28.8272819519, propagate_backward_time: 46.6819381714, weight_update_time: 0.1368190497, other_time: 2.5428886414
epoch time: 6.4860911369s, input_fired_time: 0.1671529710, output_fired_time: 0.0000000000, propagate_forward_time: 5.8431396484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4757986069
validation[          , genome  6690] predictions:    9646/  10000 (96.46%), best:    9773/10000 (97.73%), error:      1141.87915, best error:       789.24591 on epoch:    35, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 82.9962844849s, input_fired_time: 1.3181396723, output_fired_time: 1.0620808601, propagate_forward_time: 29.6303119659, propagate_backward_time: 48.3845710754, weight_update_time: 0.1416806281, other_time: 2.6011848450
epoch time: 5.7878503799s, input_fired_time: 0.1104808748, output_fired_time: 0.0000000000, propagate_forward_time: 5.3771348000, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.3002347946
validation[          , genome  6690] predictions:    9742/  10000 (97.42%), best:    9773/10000 (97.73%), error:       855.08862, best error:       789.24591 on epoch:    35, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 73.7654876709s, input_fired_time: 0.9369265437, output_fired_time: 0.7216421962, propagate_forward_time: 26.9746532440, propagate_backward_time: 43.4022941589, weight_update_time: 0.0927620158, other_time: 1.7299690247
epoch time: 5.8171210289s, input_fired_time: 0.1140585616, output_fired_time: 0.0000000000, propagate_forward_time: 5.4083609581, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.2947015762
validation[          , genome  6690] predictions:    9756/  10000 (97.56%), best:    9756/10000 (97.56%), error:       788.59265, best error:       788.59265 on epoch:    49, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 82.7932128906s, input_fired_time: 1.2482136488, output_fired_time: 0.9992368221, propagate_forward_time: 29.6265163422, propagate_backward_time: 48.5543098450, weight_update_time: 0.1255321950, other_time: 2.3649330139
epoch time: 7.0333127975s, input_fired_time: 0.2064007521, output_fired_time: 0.0000000000, propagate_forward_time: 6.3396167755, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4872951508
validation[          , genome  6690] predictions:    9744/  10000 (97.44%), best:    9756/10000 (97.56%), error:       850.53857, best error:       788.59265 on epoch:    49, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 83.0886840820s, input_fired_time: 1.3050625324, output_fired_time: 1.0429825783, propagate_forward_time: 29.7372722626, propagate_backward_time: 48.3511276245, weight_update_time: 0.1429302692, other_time: 2.6522369385
epoch time: 6.8211760521s, input_fired_time: 0.1853271574, output_fired_time: 0.0000000000, propagate_forward_time: 6.1520361900, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4838128090
validation[          , genome  6690] predictions:    9769/  10000 (97.69%), best:    9756/10000 (97.56%), error:       797.14661, best error:       788.59265 on epoch:    49, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 80.2017135620s, input_fired_time: 1.1973880529, output_fired_time: 0.9214702845, propagate_forward_time: 28.8321495056, propagate_backward_time: 46.8343772888, weight_update_time: 0.1290508211, other_time: 2.4163284302
epoch time: 6.7931571007s, input_fired_time: 0.1845551878, output_fired_time: 0.0000000000, propagate_forward_time: 6.1432318687, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.4653701782
validation[          , genome  6690] predictions:    9776/  10000 (97.76%), best:    9776/10000 (97.76%), error:       732.78796, best error:       732.78796 on epoch:    52, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 82.9286804199s, input_fired_time: 1.3266415596, output_fired_time: 1.0345499516, propagate_forward_time: 29.6021556854, propagate_backward_time: 48.2557106018, weight_update_time: 0.1443343610, other_time: 2.7096138000
epoch time: 6.6429843903s, input_fired_time: 0.1765989959, output_fired_time: 0.0000000000, propagate_forward_time: 5.9605555534, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5058298111
validation[          , genome  6690] predictions:    9766/  10000 (97.66%), best:    9776/10000 (97.76%), error:       775.14795, best error:       732.78796 on epoch:    52, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 84.2499313354s, input_fired_time: 1.4998879433, output_fired_time: 1.1616624594, propagate_forward_time: 29.9829483032, propagate_backward_time: 48.1735267639, weight_update_time: 0.1780897379, other_time: 3.4319114685
epoch time: 6.9252233505s, input_fired_time: 0.2211860269, output_fired_time: 0.0000000000, propagate_forward_time: 6.0413188934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6627182961
validation[          , genome  6690] predictions:    9754/  10000 (97.54%), best:    9776/10000 (97.76%), error:       857.18304, best error:       732.78796 on epoch:    52, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 88.8701171875s, input_fired_time: 1.7427765131, output_fired_time: 1.3866432905, propagate_forward_time: 31.1379508972, propagate_backward_time: 51.0428237915, weight_update_time: 0.1946956068, other_time: 3.5599250793
epoch time: 7.6299486160s, input_fired_time: 0.2590062022, output_fired_time: 0.0000000000, propagate_forward_time: 6.6459703445, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7249722481
validation[          , genome  6690] predictions:    9754/  10000 (97.54%), best:    9776/10000 (97.76%), error:       848.93890, best error:       732.78796 on epoch:    52, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 94.0547790527s, input_fired_time: 1.9174019098, output_fired_time: 1.5858128071, propagate_forward_time: 32.7255172729, propagate_backward_time: 54.2689018250, weight_update_time: 0.1794024855, other_time: 3.5571403503
epoch time: 7.6172242165s, input_fired_time: 0.2559593618, output_fired_time: 0.0000000000, propagate_forward_time: 6.6449522972, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7163124084
validation[          , genome  6690] predictions:    9702/  10000 (97.02%), best:    9776/10000 (97.76%), error:       994.16882, best error:       732.78796 on epoch:    52, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 94.4986495972s, input_fired_time: 1.9144818783, output_fired_time: 1.5932003260, propagate_forward_time: 32.8233680725, propagate_backward_time: 54.6212387085, weight_update_time: 0.1777295023, other_time: 3.5463600159
epoch time: 7.2418370247s, input_fired_time: 0.2450571358, output_fired_time: 0.0000000000, propagate_forward_time: 6.3428034782, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6539764404
validation[          , genome  6690] predictions:    9748/  10000 (97.48%), best:    9776/10000 (97.76%), error:       827.64258, best error:       732.78796 on epoch:    52, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 90.2552261353s, input_fired_time: 1.7382926941, output_fired_time: 1.5710374117, propagate_forward_time: 31.4281845093, propagate_backward_time: 51.9865112305, weight_update_time: 0.1795915067, other_time: 3.5312042236
epoch time: 6.8482499123s, input_fired_time: 0.2139255553, output_fired_time: 0.0000000000, propagate_forward_time: 5.9940171242, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6403074265
validation[          , genome  6690] predictions:    9769/  10000 (97.69%), best:    9776/10000 (97.76%), error:       791.08594, best error:       732.78796 on epoch:    52, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 94.0601425171s, input_fired_time: 1.9152350426, output_fired_time: 1.5472129583, propagate_forward_time: 32.7767906189, propagate_backward_time: 54.3186683655, weight_update_time: 0.1787207425, other_time: 3.5022354126
epoch time: 7.6298894882s, input_fired_time: 0.2660865188, output_fired_time: 0.0000000000, propagate_forward_time: 6.6551132202, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7086896896
validation[          , genome  6690] predictions:    9769/  10000 (97.69%), best:    9776/10000 (97.76%), error:       746.03876, best error:       732.78796 on epoch:    52, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 95.9645385742s, input_fired_time: 1.9860979319, output_fired_time: 1.5964709520, propagate_forward_time: 33.3838386536, propagate_backward_time: 55.6026573181, weight_update_time: 0.1701222658, other_time: 3.3954696655
epoch time: 7.5567092896s, input_fired_time: 0.2559858859, output_fired_time: 0.0000000000, propagate_forward_time: 6.6036000252, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6971235275
validation[          , genome  6690] predictions:    9739/  10000 (97.39%), best:    9776/10000 (97.76%), error:       884.49915, best error:       732.78796 on epoch:    52, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 96.2333602905s, input_fired_time: 1.9891804457, output_fired_time: 1.6284259558, propagate_forward_time: 33.3750762939, propagate_backward_time: 55.7431869507, weight_update_time: 0.1792795211, other_time: 3.4974899292
epoch time: 7.6243515015s, input_fired_time: 0.2619372010, output_fired_time: 0.0000000000, propagate_forward_time: 6.6330509186, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7293634415
validation[          , genome  6690] predictions:    9747/  10000 (97.47%), best:    9776/10000 (97.76%), error:       850.69214, best error:       732.78796 on epoch:    52, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 95.6636657715s, input_fired_time: 1.9983131886, output_fired_time: 1.5862927437, propagate_forward_time: 33.2569351196, propagate_backward_time: 55.3874549866, weight_update_time: 0.1743794680, other_time: 3.4346656799
epoch time: 7.5377140045s, input_fired_time: 0.2610377371, output_fired_time: 0.0000000000, propagate_forward_time: 6.5903148651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6863613129
validation[          , genome  6690] predictions:    9776/  10000 (97.76%), best:    9776/10000 (97.76%), error:       756.15662, best error:       732.78796 on epoch:    52, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 93.5804061890s, input_fired_time: 1.9175360203, output_fired_time: 1.5630838871, propagate_forward_time: 32.6397781372, propagate_backward_time: 54.0036087036, weight_update_time: 0.1744367480, other_time: 3.4563980103
epoch time: 6.7013416290s, input_fired_time: 0.2070273757, output_fired_time: 0.0000000000, propagate_forward_time: 5.8844709396, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6098432541
validation[          , genome  6690] predictions:    9745/  10000 (97.45%), best:    9776/10000 (97.76%), error:       874.60089, best error:       732.78796 on epoch:    52, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 93.8114929199s, input_fired_time: 1.9489122629, output_fired_time: 1.5357891321, propagate_forward_time: 32.7858963013, propagate_backward_time: 54.2163009644, weight_update_time: 0.1679084003, other_time: 3.3245925903
epoch time: 7.5478911400s, input_fired_time: 0.2591495216, output_fired_time: 0.0000000000, propagate_forward_time: 6.5956888199, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6930527687
validation[          , genome  6690] predictions:    9753/  10000 (97.53%), best:    9776/10000 (97.76%), error:       875.72369, best error:       732.78796 on epoch:    52, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 95.3452072144s, input_fired_time: 1.9999961853, output_fired_time: 1.6241165400, propagate_forward_time: 33.2516632080, propagate_backward_time: 55.1531105042, weight_update_time: 0.1667610854, other_time: 3.3163261414
epoch time: 7.4985804558s, input_fired_time: 0.2561790943, output_fired_time: 0.0000000000, propagate_forward_time: 6.5549554825, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6874456406
validation[          , genome  6690] predictions:    9736/  10000 (97.36%), best:    9776/10000 (97.76%), error:       869.62128, best error:       732.78796 on epoch:    52, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 94.2097015381s, input_fired_time: 1.9362632036, output_fired_time: 1.5992896557, propagate_forward_time: 32.8080673218, propagate_backward_time: 54.4445190430, weight_update_time: 0.1727745831, other_time: 3.4215621948
epoch time: 7.5279121399s, input_fired_time: 0.2527219951, output_fired_time: 0.0000000000, propagate_forward_time: 6.6006608009, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6745295525
validation[          , genome  6690] predictions:    9744/  10000 (97.44%), best:    9776/10000 (97.76%), error:       858.65387, best error:       732.78796 on epoch:    52, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 89.7682571411s, input_fired_time: 1.7741571665, output_fired_time: 1.4657545090, propagate_forward_time: 31.3723564148, propagate_backward_time: 51.6227493286, weight_update_time: 0.1876314729, other_time: 3.5332450867
epoch time: 6.7025837898s, input_fired_time: 0.2135635167, output_fired_time: 0.0000000000, propagate_forward_time: 5.8600234985, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6289968491
validation[          , genome  6690] predictions:    9744/  10000 (97.44%), best:    9776/10000 (97.76%), error:       801.62811, best error:       732.78796 on epoch:    52, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 85.6127700806s, input_fired_time: 1.6069424152, output_fired_time: 1.3769994974, propagate_forward_time: 30.0830039978, propagate_backward_time: 49.0158348083, weight_update_time: 0.1923953444, other_time: 3.5299911499
epoch time: 6.7840614319s, input_fired_time: 0.2186287940, output_fired_time: 0.0000000000, propagate_forward_time: 5.9521284103, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6133041382
validation[          , genome  6690] predictions:    9757/  10000 (97.57%), best:    9776/10000 (97.76%), error:       823.95288, best error:       732.78796 on epoch:    52, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 84.9505538940s, input_fired_time: 1.5879129171, output_fired_time: 1.3122671843, propagate_forward_time: 29.8347492218, propagate_backward_time: 48.4498176575, weight_update_time: 0.1997213811, other_time: 3.7658119202
epoch time: 6.7222862244s, input_fired_time: 0.2123428136, output_fired_time: 0.0000000000, propagate_forward_time: 5.8834896088, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6264538765
validation[          , genome  6690] predictions:    9739/  10000 (97.39%), best:    9776/10000 (97.76%), error:       877.85248, best error:       732.78796 on epoch:    52, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 84.1244201660s, input_fired_time: 1.5907931328, output_fired_time: 1.3103014231, propagate_forward_time: 29.6062984467, propagate_backward_time: 47.8539428711, weight_update_time: 0.2003062218, other_time: 3.7630844116
epoch time: 6.9861140251s, input_fired_time: 0.2255223095, output_fired_time: 0.0000000000, propagate_forward_time: 6.1195273399, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6410641670
validation[          , genome  6690] predictions:    9735/  10000 (97.35%), best:    9776/10000 (97.76%), error:       843.68353, best error:       732.78796 on epoch:    52, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 84.7699356079s, input_fired_time: 1.6184306145, output_fired_time: 1.3117376566, propagate_forward_time: 29.7984886169, propagate_backward_time: 48.3821678162, weight_update_time: 0.1929315627, other_time: 3.6591110229
epoch time: 7.4404253960s, input_fired_time: 0.2502947748, output_fired_time: 0.0000000000, propagate_forward_time: 6.4969401360, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6931905746
validation[          , genome  6690] predictions:    9746/  10000 (97.46%), best:    9776/10000 (97.76%), error:       851.82208, best error:       732.78796 on epoch:    52, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 89.9575500488s, input_fired_time: 1.7652851343, output_fired_time: 1.4910826683, propagate_forward_time: 31.3908729553, propagate_backward_time: 51.7165985107, weight_update_time: 0.1831201315, other_time: 3.5937156677
epoch time: 6.9568452835s, input_fired_time: 0.2312892228, output_fired_time: 0.0000000000, propagate_forward_time: 6.0804109573, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6451449394
validation[          , genome  6690] predictions:    9750/  10000 (97.50%), best:    9776/10000 (97.76%), error:       823.56769, best error:       732.78796 on epoch:    52, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 87.2299499512s, input_fired_time: 1.6416738033, output_fired_time: 1.4389991760, propagate_forward_time: 30.6151580811, propagate_backward_time: 49.9916534424, weight_update_time: 0.1865618974, other_time: 3.5424652100
epoch time: 7.5795369148s, input_fired_time: 0.2587094605, output_fired_time: 0.0000000000, propagate_forward_time: 6.6092624664, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7115650177
validation[          , genome  6690] predictions:    9732/  10000 (97.32%), best:    9776/10000 (97.76%), error:       885.77692, best error:       732.78796 on epoch:    52, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

epoch time: 92.7216033936s, input_fired_time: 1.9398813248, output_fired_time: 1.5904257298, propagate_forward_time: 32.4580574036, propagate_backward_time: 53.5927772522, weight_update_time: 0.1613257080, other_time: 3.1404647827
epoch time: 7.4838104248s, input_fired_time: 0.2547869682, output_fired_time: 0.0000000000, propagate_forward_time: 6.5326814651, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6963419914
validation[          , genome  6690] predictions:    9756/  10000 (97.56%), best:    9776/10000 (97.76%), error:       813.48041, best error:       732.78796 on epoch:    52, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 92.6838607788s, input_fired_time: 1.8534045219, output_fired_time: 1.6621556282, propagate_forward_time: 32.0773849487, propagate_backward_time: 53.5713577271, weight_update_time: 0.1805062890, other_time: 3.5195617676
epoch time: 6.7290740013s, input_fired_time: 0.2121701241, output_fired_time: 0.0000000000, propagate_forward_time: 5.8993897438, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6175141335
validation[          , genome  6690] predictions:    9712/  10000 (97.12%), best:    9776/10000 (97.76%), error:       976.94073, best error:       732.78796 on epoch:    52, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

epoch time: 86.3991317749s, input_fired_time: 1.6404184103, output_fired_time: 1.4325429201, propagate_forward_time: 30.2329044342, propagate_backward_time: 49.7178077698, weight_update_time: 0.1762440950, other_time: 3.3754615784
epoch time: 6.6254220009s, input_fired_time: 0.2026358098, output_fired_time: 0.0000000000, propagate_forward_time: 5.8129510880, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6098351479
validation[          , genome  6690] predictions:    9764/  10000 (97.64%), best:    9776/10000 (97.76%), error:       801.28699, best error:       732.78796 on epoch:    52, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

epoch time: 83.5888519287s, input_fired_time: 1.5616363287, output_fired_time: 1.2773007154, propagate_forward_time: 29.4549293518, propagate_backward_time: 47.5954780579, weight_update_time: 0.2024808228, other_time: 3.6995086670
epoch time: 7.1113996506s, input_fired_time: 0.2277065963, output_fired_time: 0.0000000000, propagate_forward_time: 6.2215948105, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6620984077
validation[          , genome  6690] predictions:    9749/  10000 (97.49%), best:    9776/10000 (97.76%), error:       845.06787, best error:       732.78796 on epoch:    52, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 86.6722640991s, input_fired_time: 1.6800230742, output_fired_time: 1.4052609205, propagate_forward_time: 30.3475131989, propagate_backward_time: 49.5765037537, weight_update_time: 0.1962846518, other_time: 3.6629676819
epoch time: 7.1724228859s, input_fired_time: 0.2358403802, output_fired_time: 0.0000000000, propagate_forward_time: 6.2652897835, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6712927818
validation[          , genome  6690] predictions:    9764/  10000 (97.64%), best:    9776/10000 (97.76%), error:       833.33276, best error:       732.78796 on epoch:    52, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

epoch time: 86.4229660034s, input_fired_time: 1.6617043018, output_fired_time: 1.3536462784, propagate_forward_time: 30.3470249176, propagate_backward_time: 49.4698677063, weight_update_time: 0.1869639456, other_time: 3.5907249451
epoch time: 6.7401103973s, input_fired_time: 0.2070121765, output_fired_time: 0.0000000000, propagate_forward_time: 5.9186301231, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6144680977
validation[          , genome  6690] predictions:    9761/  10000 (97.61%), best:    9776/10000 (97.76%), error:       763.69537, best error:       732.78796 on epoch:    52, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 86.8847579956s, input_fired_time: 1.6770726442, output_fired_time: 1.3966367245, propagate_forward_time: 30.5178108215, propagate_backward_time: 49.5803565979, weight_update_time: 0.1919447631, other_time: 3.7128829956
epoch time: 7.0472187996s, input_fired_time: 0.2340371162, output_fired_time: 0.0000000000, propagate_forward_time: 6.1552662849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6579155922
validation[          , genome  6690] predictions:    9764/  10000 (97.64%), best:    9776/10000 (97.76%), error:       785.73218, best error:       732.78796 on epoch:    52, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 83.8237609863s, input_fired_time: 1.6020570993, output_fired_time: 1.2786657810, propagate_forward_time: 29.5936527252, propagate_backward_time: 47.7035522461, weight_update_time: 0.1880681664, other_time: 3.6458358765
epoch time: 6.8981614113s, input_fired_time: 0.2280645669, output_fired_time: 0.0000000000, propagate_forward_time: 6.0318322182, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6382646561
validation[          , genome  6690] predictions:    9730/  10000 (97.30%), best:    9776/10000 (97.76%), error:       918.35596, best error:       732.78796 on epoch:    52, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 85.8002548218s, input_fired_time: 1.6632031202, output_fired_time: 1.3525053263, propagate_forward_time: 30.1741466522, propagate_backward_time: 48.7903366089, weight_update_time: 0.1979386806, other_time: 3.8200607300
epoch time: 6.8711175919s, input_fired_time: 0.2256793380, output_fired_time: 0.0000000000, propagate_forward_time: 5.9946937561, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6507444382
validation[          , genome  6690] predictions:    9753/  10000 (97.53%), best:    9776/10000 (97.76%), error:       872.35449, best error:       732.78796 on epoch:    52, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 88.0415344238s, input_fired_time: 1.7268031836, output_fired_time: 1.4528027773, propagate_forward_time: 30.8217716217, propagate_backward_time: 50.2627830505, weight_update_time: 0.1935797483, other_time: 3.7773704529
epoch time: 6.8745846748s, input_fired_time: 0.2228844911, output_fired_time: 0.0000000000, propagate_forward_time: 6.0273132324, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6243867874
validation[          , genome  6690] predictions:    9782/  10000 (97.82%), best:    9776/10000 (97.76%), error:       762.17450, best error:       732.78796 on epoch:    52, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 84.5935134888s, input_fired_time: 1.6277047396, output_fired_time: 1.3076034784, propagate_forward_time: 29.8356227875, propagate_backward_time: 47.9949378967, weight_update_time: 0.1985998601, other_time: 3.8276405334
epoch time: 6.9935498238s, input_fired_time: 0.2259007543, output_fired_time: 0.0000000000, propagate_forward_time: 6.1116538048, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6559953690
validation[          , genome  6690] predictions:    9754/  10000 (97.54%), best:    9776/10000 (97.76%), error:       824.84888, best error:       732.78796 on epoch:    52, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

epoch time: 84.2922592163s, input_fired_time: 1.4514095783, output_fired_time: 1.1911025047, propagate_forward_time: 29.7092342377, propagate_backward_time: 48.8728790283, weight_update_time: 0.1661812663, other_time: 3.0676345825
epoch time: 6.7742519379s, input_fired_time: 0.1965328902, output_fired_time: 0.0000000000, propagate_forward_time: 6.0405206680, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5371985435
validation[          , genome  6690] predictions:    9728/  10000 (97.28%), best:    9776/10000 (97.76%), error:       819.04858, best error:       732.78796 on epoch:    52, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

epoch time: 86.6504669189s, input_fired_time: 1.4830493927, output_fired_time: 1.1272872686, propagate_forward_time: 30.6088447571, propagate_backward_time: 50.6164550781, weight_update_time: 0.1527597904, other_time: 2.8148307800
epoch time: 6.7124404907s, input_fired_time: 0.2000738829, output_fired_time: 0.0000000000, propagate_forward_time: 5.9407315254, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5716352463
validation[          , genome  6690] predictions:    9765/  10000 (97.65%), best:    9776/10000 (97.76%), error:       749.46484, best error:       732.78796 on epoch:    52, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 84.8149108887s, input_fired_time: 1.5355681181, output_fired_time: 1.2622711658, propagate_forward_time: 29.9412879944, propagate_backward_time: 48.7115440369, weight_update_time: 0.1806113720, other_time: 3.3642425537
epoch time: 6.6817975044s, input_fired_time: 0.2056598514, output_fired_time: 0.0000000000, propagate_forward_time: 5.8883738518, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5877637863
validation[          , genome  6690] predictions:    9676/  10000 (96.76%), best:    9776/10000 (97.76%), error:      1061.33386, best error:       732.78796 on epoch:    52, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 84.6929931641s, input_fired_time: 1.5823931694, output_fired_time: 1.3000220060, propagate_forward_time: 29.8799324036, propagate_backward_time: 48.3917465210, weight_update_time: 0.1813528240, other_time: 3.5389060974
epoch time: 6.4555029869s, input_fired_time: 0.1821793467, output_fired_time: 0.0000000000, propagate_forward_time: 5.7203898430, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5529336929
validation[          , genome  6690] predictions:    9738/  10000 (97.38%), best:    9776/10000 (97.76%), error:       896.44104, best error:       732.78796 on epoch:    52, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 83.3084869385s, input_fired_time: 1.5040909052, output_fired_time: 1.2305058241, propagate_forward_time: 29.4573364258, propagate_backward_time: 47.6755676270, weight_update_time: 0.1788025945, other_time: 3.4409866333
epoch time: 8.1674556732s, input_fired_time: 0.3444000781, output_fired_time: 0.0000000000, propagate_forward_time: 6.6883816719, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.1346740723
validation[          , genome  6690] predictions:    9725/  10000 (97.25%), best:    9776/10000 (97.76%), error:       925.70691, best error:       732.78796 on epoch:    52, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 87.9441070557s, input_fired_time: 1.7600198984, output_fired_time: 1.4084188938, propagate_forward_time: 30.7459926605, propagate_backward_time: 50.0940513611, weight_update_time: 0.1979501694, other_time: 3.9356269836
epoch time: 7.0472683907s, input_fired_time: 0.2438594848, output_fired_time: 0.0000000000, propagate_forward_time: 6.1017384529, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7016706467
validation[          , genome  6690] predictions:    9736/  10000 (97.36%), best:    9776/10000 (97.76%), error:       878.27930, best error:       732.78796 on epoch:    52, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 84.7611999512s, input_fired_time: 1.6049071550, output_fired_time: 1.2869333029, propagate_forward_time: 29.7834911346, propagate_backward_time: 48.4309387207, weight_update_time: 0.1913948357, other_time: 3.6549301147
epoch time: 6.4862785339s, input_fired_time: 0.1941806525, output_fired_time: 0.0000000000, propagate_forward_time: 5.7321982384, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5598998070
validation[          , genome  6690] predictions:    9779/  10000 (97.79%), best:    9776/10000 (97.76%), error:       781.58759, best error:       732.78796 on epoch:    52, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 82.4046936035s, input_fired_time: 1.4929906130, output_fired_time: 1.1667289734, propagate_forward_time: 29.1251964569, propagate_backward_time: 47.2530479431, weight_update_time: 0.1732589155, other_time: 3.3667335510
epoch time: 6.5045623779s, input_fired_time: 0.1878295988, output_fired_time: 0.0000000000, propagate_forward_time: 5.7538018227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5629310608
validation[          , genome  6690] predictions:    9754/  10000 (97.54%), best:    9776/10000 (97.76%), error:       798.58936, best error:       732.78796 on epoch:    52, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 87.9902801514s, input_fired_time: 1.7508450747, output_fired_time: 1.4647355080, propagate_forward_time: 30.5391845703, propagate_backward_time: 50.2007408142, weight_update_time: 0.2044048607, other_time: 4.0347709656
epoch time: 6.6254906654s, input_fired_time: 0.1917357296, output_fired_time: 0.0000000000, propagate_forward_time: 5.8550930023, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5786619186
validation[          , genome  6690] predictions:    9755/  10000 (97.55%), best:    9776/10000 (97.76%), error:       829.51019, best error:       732.78796 on epoch:    52, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 82.2183990479s, input_fired_time: 1.4574961662, output_fired_time: 1.1669625044, propagate_forward_time: 29.1207408905, propagate_backward_time: 47.1757621765, weight_update_time: 0.1675099880, other_time: 3.2974357605
epoch time: 6.5627608299s, input_fired_time: 0.1910057366, output_fired_time: 0.0000000000, propagate_forward_time: 5.7994399071, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5723152161
validation[          , genome  6690] predictions:    9730/  10000 (97.30%), best:    9776/10000 (97.76%), error:       852.53339, best error:       732.78796 on epoch:    52, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 83.6029891968s, input_fired_time: 1.5773124695, output_fired_time: 1.2305309772, propagate_forward_time: 29.5059623718, propagate_backward_time: 47.7491378784, weight_update_time: 0.1809345186, other_time: 3.5400505066
epoch time: 6.4505844116s, input_fired_time: 0.1845738739, output_fired_time: 0.0000000000, propagate_forward_time: 5.6890025139, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5770082474
validation[          , genome  6690] predictions:    9783/  10000 (97.83%), best:    9776/10000 (97.76%), error:       748.56085, best error:       732.78796 on epoch:    52, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 82.6551589966s, input_fired_time: 1.5259394646, output_fired_time: 1.1664868593, propagate_forward_time: 29.3242702484, propagate_backward_time: 47.1638183594, weight_update_time: 0.1784970760, other_time: 3.4746398926
epoch time: 8.5969810486s, input_fired_time: 0.3984130025, output_fired_time: 0.0000000000, propagate_forward_time: 6.9029593468, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2956089973
validation[          , genome  6690] predictions:    9714/  10000 (97.14%), best:    9776/10000 (97.76%), error:       922.46478, best error:       732.78796 on epoch:    52, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 88.5995864868s, input_fired_time: 1.7504923344, output_fired_time: 1.4865294695, propagate_forward_time: 30.8922462463, propagate_backward_time: 50.6498680115, weight_update_time: 0.1935561001, other_time: 3.8204498291
epoch time: 6.7064404488s, input_fired_time: 0.2047731727, output_fired_time: 0.0000000000, propagate_forward_time: 5.9073185921, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.5943489075
validation[          , genome  6690] predictions:    9763/  10000 (97.63%), best:    9776/10000 (97.76%), error:       812.18781, best error:       732.78796 on epoch:    52, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 85.9171600342s, input_fired_time: 1.6447483301, output_fired_time: 1.3307245970, propagate_forward_time: 30.1930904388, propagate_backward_time: 49.0783500671, weight_update_time: 0.1866216063, other_time: 3.6702461243
epoch time: 6.6494054794s, input_fired_time: 0.2047827393, output_fired_time: 0.0000000000, propagate_forward_time: 5.8387904167, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6058320999
validation[          , genome  6690] predictions:    9743/  10000 (97.43%), best:    9776/10000 (97.76%), error:       840.44556, best error:       732.78796 on epoch:    52, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 85.3491973877s, input_fired_time: 1.6037224531, output_fired_time: 1.2865799665, propagate_forward_time: 30.0900058746, propagate_backward_time: 48.9660797119, weight_update_time: 0.1720173061, other_time: 3.4028091431
epoch time: 6.6982455254s, input_fired_time: 0.2102853060, output_fired_time: 0.0000000000, propagate_forward_time: 5.8863410950, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6016192436
validation[          , genome  6690] predictions:    9739/  10000 (97.39%), best:    9776/10000 (97.76%), error:       826.66742, best error:       732.78796 on epoch:    52, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 85.3129348755s, input_fired_time: 1.7557392120, output_fired_time: 1.3421039581, propagate_forward_time: 29.9699935913, propagate_backward_time: 48.1804771423, weight_update_time: 0.2002874464, other_time: 4.0646247864
epoch time: 6.7485079765s, input_fired_time: 0.2182045877, output_fired_time: 0.0000000000, propagate_forward_time: 5.8933620453, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.6369414330
validation[          , genome  6690] predictions:    9750/  10000 (97.50%), best:    9776/10000 (97.76%), error:       809.44189, best error:       732.78796 on epoch:    52, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 33.6048316956s, input_fired_time: 1.0146479607, output_fired_time: 0.0000000000, propagate_forward_time: 29.4675922394, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1225910187
best training[          , genome  6690] predictions:   49526/  50000 (99.05%), best:    9776/10000 (97.76%), error:      1708.55530, best error:       732.78796 on epoch:    52, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 7.1948332787s, input_fired_time: 0.2466468364, output_fired_time: 0.0000000000, propagate_forward_time: 6.2101192474, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.7380671501
   testing[          , genome  6690] predictions:    9789/  10000 (97.89%), best:    9776/10000 (97.76%), error:       698.11621, best error:       732.78796 on epoch:    52, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
23:40:27 (5628): called boinc_finish(0)

</stderr_txt>
]]>