Nameexact_genome_1511526933_34_18346_2
Workunit2485194
Created26 Nov 2017, 15:17:39 UTC
Sent26 Nov 2017, 15:43:40 UTC
Report deadline22 Dec 2017, 4:28:34 UTC
Received30 Nov 2017, 13:52:48 UTC
Server stateOver
OutcomeComputation error
Client stateCompute error
Exit status194 (0xc2) EXIT_ABORTED_BY_CLIENT
Computer ID49051
Run time8 hours 25 min 13 sec
CPU time8 hours 24 min 18 sec
Validate stateInvalid
Credit0.00
Device peak FLOPS1.56 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size172.01 MB
Peak swap size177.07 MB
Peak disk usage5.73 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 194 (0xc2)</message>
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346_2_r901556117_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18346
read normal distribution: '1 -1.24322175979614 0.0412701778113842'
generator_str: '1062400132'
read generator: 1062400132
read generated_by_map:
1 split_node 1
reading 81 nodes.
reading 283 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 1579977366
post shuffle 1: 63937354
epoch time: 484.304351806641s, input_fired_time: 13.7622165679932, output_fired_time: 12.5410060882568, propagate_forward_time: 179.397186279297, propagate_backward_time: 258.936981201172, weight_update_time: 0.974333822727203, other_time: 19.6669616699219
epoch time: 42.576732635498s, input_fired_time: 1.76796567440033, output_fired_time: 0, propagate_forward_time: 37.3964157104492, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.41234970092773
validation[          , genome 18346] predictions:    9727/  10000 (97.27%), best:    9727/10000 (97.27%), error:       873.54010, best error:       873.54010 on epoch:     0, epoch:    0/100, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 499.3514709473s, input_fired_time: 16.0356693268, output_fired_time: 14.8839197159, propagate_forward_time: 181.7269439697, propagate_backward_time: 262.3446350098, weight_update_time: 1.1600930691, other_time: 24.3603210449
epoch time: 46.0810852051s, input_fired_time: 2.5078604221, output_fired_time: 0.0000000000, propagate_forward_time: 38.3720664978, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2011566162
validation[          , genome 18346] predictions:    9676/  10000 (96.76%), best:    9727/10000 (97.27%), error:       982.01593, best error:       873.54010 on epoch:     0, epoch:    1/100, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 499.6901245117s, input_fired_time: 15.9955692291, output_fired_time: 14.5618705750, propagate_forward_time: 182.3937988281, propagate_backward_time: 263.2796325684, weight_update_time: 1.1480530500, other_time: 23.4592590332
epoch time: 45.5409393311s, input_fired_time: 2.4724986553, output_fired_time: 0.0000000000, propagate_forward_time: 38.1541023254, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.9143371582
validation[          , genome 18346] predictions:    9779/  10000 (97.79%), best:    9779/10000 (97.79%), error:       763.21753, best error:       763.21753 on epoch:     2, epoch:    2/100, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 498.7876281738s, input_fired_time: 16.1015243530, output_fired_time: 14.5006589890, propagate_forward_time: 182.1561737061, propagate_backward_time: 262.5486755371, weight_update_time: 1.1136536598, other_time: 23.4805603027
epoch time: 44.5952987671s, input_fired_time: 2.5784544945, output_fired_time: 0.0000000000, propagate_forward_time: 36.8767051697, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1401405334
validation[          , genome 18346] predictions:    9744/  10000 (97.44%), best:    9779/10000 (97.79%), error:       819.05298, best error:       763.21753 on epoch:     2, epoch:    3/100, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 495.8107604980s, input_fired_time: 15.6196861267, output_fired_time: 14.2029685974, propagate_forward_time: 181.1913604736, propagate_backward_time: 261.8276367188, weight_update_time: 1.1095438004, other_time: 22.9691162109
epoch time: 43.3376350403s, input_fired_time: 2.0342590809, output_fired_time: 0.0000000000, propagate_forward_time: 37.2595787048, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0437965393
validation[          , genome 18346] predictions:    9744/  10000 (97.44%), best:    9779/10000 (97.79%), error:       847.88843, best error:       763.21753 on epoch:     2, epoch:    4/100, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 500.1326293945s, input_fired_time: 16.1006393433, output_fired_time: 14.7737932205, propagate_forward_time: 181.5720214844, propagate_backward_time: 262.5422363281, weight_update_time: 1.1967865229, other_time: 25.1439208984
epoch time: 43.1535606384s, input_fired_time: 1.9063689709, output_fired_time: 0.0000000000, propagate_forward_time: 37.6280784607, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6191139221
validation[          , genome 18346] predictions:    9789/  10000 (97.89%), best:    9789/10000 (97.89%), error:       658.94537, best error:       658.94537 on epoch:     5, epoch:    5/100, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 506.3531799316s, input_fired_time: 16.9366340637, output_fired_time: 15.2679576874, propagate_forward_time: 183.3129119873, propagate_backward_time: 264.8387451172, weight_update_time: 1.2322829962, other_time: 25.9969482422
epoch time: 45.5726776123s, input_fired_time: 2.4781553745, output_fired_time: 0.0000000000, propagate_forward_time: 38.4766883850, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.6178321838
validation[          , genome 18346] predictions:    9804/  10000 (98.04%), best:    9804/10000 (98.04%), error:       609.97736, best error:       609.97736 on epoch:     6, epoch:    6/100, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 498.6325378418s, input_fired_time: 15.8106355667, output_fired_time: 14.3849296570, propagate_forward_time: 182.1073303223, propagate_backward_time: 262.7083740234, weight_update_time: 1.1153007746, other_time: 23.6212768555
epoch time: 44.6184768677s, input_fired_time: 2.2735733986, output_fired_time: 0.0000000000, propagate_forward_time: 38.1202354431, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2246665955
validation[          , genome 18346] predictions:    9811/  10000 (98.11%), best:    9804/10000 (98.04%), error:       620.08563, best error:       609.97736 on epoch:     6, epoch:    7/100, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 503.8121643066s, input_fired_time: 17.3288288116, output_fired_time: 15.6176576614, propagate_forward_time: 181.2286224365, propagate_backward_time: 262.3567504883, weight_update_time: 1.2880485058, other_time: 27.2803344727
epoch time: 44.0795593262s, input_fired_time: 2.1394040585, output_fired_time: 0.0000000000, propagate_forward_time: 37.8657722473, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0743827820
validation[          , genome 18346] predictions:    9718/  10000 (97.18%), best:    9804/10000 (98.04%), error:       875.76697, best error:       609.97736 on epoch:     6, epoch:    8/100, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 497.7142333984s, input_fired_time: 15.9543418884, output_fired_time: 14.6429538727, propagate_forward_time: 180.9985656738, propagate_backward_time: 261.4289855957, weight_update_time: 1.1666977406, other_time: 24.6893920898
epoch time: 43.3036575317s, input_fired_time: 1.9564130306, output_fired_time: 0.0000000000, propagate_forward_time: 37.8535728455, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4936714172
validation[          , genome 18346] predictions:    9802/  10000 (98.02%), best:    9804/10000 (98.04%), error:       613.27612, best error:       609.97736 on epoch:     6, epoch:    9/100, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 497.0158691406s, input_fired_time: 15.8675022125, output_fired_time: 14.4547271729, propagate_forward_time: 181.2130432129, propagate_backward_time: 261.4698486328, weight_update_time: 1.1453253031, other_time: 24.0107727051
epoch time: 45.2033004761s, input_fired_time: 2.3512384892, output_fired_time: 0.0000000000, propagate_forward_time: 38.0423202515, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8097419739
validation[          , genome 18346] predictions:    9813/  10000 (98.13%), best:    9813/10000 (98.13%), error:       600.52380, best error:       600.52380 on epoch:    10, epoch:   10/100, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 502.9003295898s, input_fired_time: 16.5249462128, output_fired_time: 14.8851709366, propagate_forward_time: 183.3829956055, propagate_backward_time: 264.1175231934, weight_update_time: 1.1609458923, other_time: 23.9897155762
epoch time: 45.3208541870s, input_fired_time: 2.3515262604, output_fired_time: 0.0000000000, propagate_forward_time: 38.2102966309, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7590332031
validation[          , genome 18346] predictions:    9821/  10000 (98.21%), best:    9821/10000 (98.21%), error:       588.46539, best error:       588.46539 on epoch:    11, epoch:   11/100, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 501.9822082520s, input_fired_time: 16.5364990234, output_fired_time: 15.2787399292, propagate_forward_time: 181.8295898438, propagate_backward_time: 262.7092895508, weight_update_time: 1.1917384863, other_time: 25.6280822754
epoch time: 46.1100234985s, input_fired_time: 2.6670637131, output_fired_time: 0.0000000000, propagate_forward_time: 38.5441894531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.8987693787
validation[          , genome 18346] predictions:    9828/  10000 (98.28%), best:    9828/10000 (98.28%), error:       568.75537, best error:       568.75537 on epoch:    12, epoch:   12/100, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 504.8979492188s, input_fired_time: 16.6132087708, output_fired_time: 15.2045278549, propagate_forward_time: 183.3186798096, propagate_backward_time: 264.7146911621, weight_update_time: 1.2053992748, other_time: 25.0468444824
epoch time: 44.0156631470s, input_fired_time: 2.1386468410, output_fired_time: 0.0000000000, propagate_forward_time: 37.8199653625, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0570526123
validation[          , genome 18346] predictions:    9835/  10000 (98.35%), best:    9835/10000 (98.35%), error:       536.60022, best error:       536.60022 on epoch:    13, epoch:   13/100, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 500.3086242676s, input_fired_time: 16.2405662537, output_fired_time: 14.7426328659, propagate_forward_time: 182.1454925537, propagate_backward_time: 262.4782104492, weight_update_time: 1.1895236969, other_time: 24.7017211914
epoch time: 43.0896110535s, input_fired_time: 1.9162546396, output_fired_time: 0.0000000000, propagate_forward_time: 37.6299247742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5434303284
validation[          , genome 18346] predictions:    9784/  10000 (97.84%), best:    9835/10000 (98.35%), error:       711.20251, best error:       536.60022 on epoch:    13, epoch:   14/100, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 496.0341186523s, input_fired_time: 15.6606483459, output_fired_time: 14.3590478897, propagate_forward_time: 181.0187377930, propagate_backward_time: 261.3612976074, weight_update_time: 1.1053112745, other_time: 23.6343994141
epoch time: 44.4122772217s, input_fired_time: 2.4543156624, output_fired_time: 0.0000000000, propagate_forward_time: 37.2552680969, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7026939392
validation[          , genome 18346] predictions:    9818/  10000 (98.18%), best:    9835/10000 (98.35%), error:       605.17059, best error:       536.60022 on epoch:    13, epoch:   15/100, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 495.2643127441s, input_fired_time: 15.1241130829, output_fired_time: 13.9683017731, propagate_forward_time: 181.5948486328, propagate_backward_time: 261.8050842285, weight_update_time: 1.0958615541, other_time: 22.7719726563
epoch time: 43.0632781982s, input_fired_time: 2.2997484207, output_fired_time: 0.0000000000, propagate_forward_time: 36.0274848938, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.7360458374
validation[          , genome 18346] predictions:    9845/  10000 (98.45%), best:    9845/10000 (98.45%), error:       502.38638, best error:       502.38638 on epoch:    16, epoch:   16/100, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 497.3505249023s, input_fired_time: 15.7114324570, output_fired_time: 14.2704143524, propagate_forward_time: 181.6920013428, propagate_backward_time: 262.0267639160, weight_update_time: 1.1355768442, other_time: 23.6499328613
epoch time: 45.1056709290s, input_fired_time: 2.6085925102, output_fired_time: 0.0000000000, propagate_forward_time: 37.1921234131, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3049545288
validation[          , genome 18346] predictions:    9830/  10000 (98.30%), best:    9845/10000 (98.45%), error:       555.53638, best error:       502.38638 on epoch:    16, epoch:   17/100, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 494.8009643555s, input_fired_time: 15.2828483582, output_fired_time: 14.1905403137, propagate_forward_time: 181.1195373535, propagate_backward_time: 261.3085327148, weight_update_time: 1.1020013094, other_time: 22.8995056152
epoch time: 43.4189720154s, input_fired_time: 1.9220649004, output_fired_time: 0.0000000000, propagate_forward_time: 37.8085021973, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6884040833
validation[          , genome 18346] predictions:    9823/  10000 (98.23%), best:    9845/10000 (98.45%), error:       566.42694, best error:       502.38638 on epoch:    16, epoch:   18/100, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 496.6020507813s, input_fired_time: 15.9000864029, output_fired_time: 14.5107507706, propagate_forward_time: 181.2475128174, propagate_backward_time: 261.6867675781, weight_update_time: 1.1466567516, other_time: 23.2569580078
epoch time: 46.0357131958s, input_fired_time: 2.5527029037, output_fired_time: 0.0000000000, propagate_forward_time: 38.3566856384, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.1263237000
validation[          , genome 18346] predictions:    9839/  10000 (98.39%), best:    9845/10000 (98.45%), error:       527.80774, best error:       502.38638 on epoch:    16, epoch:   19/100, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 489.2990417480s, input_fired_time: 14.5870008469, output_fired_time: 13.4973506927, propagate_forward_time: 180.2303771973, propagate_backward_time: 259.6104736328, weight_update_time: 1.0363844633, other_time: 21.3738403320
epoch time: 43.3915939331s, input_fired_time: 2.1003751755, output_fired_time: 0.0000000000, propagate_forward_time: 37.0627784729, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2284393311
validation[          , genome 18346] predictions:    9794/  10000 (97.94%), best:    9845/10000 (98.45%), error:       648.32898, best error:       502.38638 on epoch:    16, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 487.7415771484s, input_fired_time: 14.6257772446, output_fired_time: 13.2143173218, propagate_forward_time: 179.8541870117, propagate_backward_time: 259.2527770996, weight_update_time: 1.0177824497, other_time: 20.7945251465
epoch time: 42.6929664612s, input_fired_time: 1.8480478525, output_fired_time: 0.0000000000, propagate_forward_time: 37.4255638123, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4193534851
validation[          , genome 18346] predictions:    9843/  10000 (98.43%), best:    9845/10000 (98.45%), error:       524.62335, best error:       502.38638 on epoch:    16, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 487.7336425781s, input_fired_time: 14.1246385574, output_fired_time: 12.7466688156, propagate_forward_time: 180.6837310791, propagate_backward_time: 259.8261718750, weight_update_time: 1.0005869865, other_time: 20.3524169922
epoch time: 42.9220924377s, input_fired_time: 1.8235143423, output_fired_time: 0.0000000000, propagate_forward_time: 37.6031532288, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4954261780
validation[          , genome 18346] predictions:    9841/  10000 (98.41%), best:    9845/10000 (98.45%), error:       512.83582, best error:       502.38638 on epoch:    16, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 485.5717163086s, input_fired_time: 14.2966995239, output_fired_time: 13.1736793518, propagate_forward_time: 179.1045227051, propagate_backward_time: 258.3519287109, weight_update_time: 0.9878780246, other_time: 20.6448974609
epoch time: 44.2451591492s, input_fired_time: 2.1477994919, output_fired_time: 0.0000000000, propagate_forward_time: 37.9594001770, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1379585266
validation[          , genome 18346] predictions:    9832/  10000 (98.32%), best:    9845/10000 (98.45%), error:       518.68524, best error:       502.38638 on epoch:    16, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 488.6561584473s, input_fired_time: 14.3308649063, output_fired_time: 13.0605907440, propagate_forward_time: 180.2093505859, propagate_backward_time: 259.9076843262, weight_update_time: 1.0291992426, other_time: 21.1476745605
epoch time: 43.8720588684s, input_fired_time: 2.0463957787, output_fired_time: 0.0000000000, propagate_forward_time: 37.9423942566, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8832702637
validation[          , genome 18346] predictions:    9814/  10000 (98.14%), best:    9845/10000 (98.45%), error:       610.56055, best error:       502.38638 on epoch:    16, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 493.2103576660s, input_fired_time: 15.0791711807, output_fired_time: 13.8864974976, propagate_forward_time: 180.4094696045, propagate_backward_time: 260.3523254395, weight_update_time: 1.1269135475, other_time: 23.4828796387
epoch time: 44.3429603577s, input_fired_time: 2.1378469467, output_fired_time: 0.0000000000, propagate_forward_time: 38.0836715698, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.1214408875
validation[          , genome 18346] predictions:    9813/  10000 (98.13%), best:    9845/10000 (98.45%), error:       557.68982, best error:       502.38638 on epoch:    16, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 489.1920776367s, input_fired_time: 14.3091602325, output_fired_time: 12.9819841385, propagate_forward_time: 180.4762725830, propagate_backward_time: 260.1421813965, weight_update_time: 1.0441039801, other_time: 21.2824401855
epoch time: 42.9617919922s, input_fired_time: 1.8958342075, output_fired_time: 0.0000000000, propagate_forward_time: 37.4670715332, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5988845825
validation[          , genome 18346] predictions:    9829/  10000 (98.29%), best:    9845/10000 (98.45%), error:       559.18530, best error:       502.38638 on epoch:    16, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 490.0939025879s, input_fired_time: 14.5284404755, output_fired_time: 13.1550674438, propagate_forward_time: 180.6398010254, propagate_backward_time: 260.1375427246, weight_update_time: 1.0581754446, other_time: 21.6330566406
epoch time: 44.8680267334s, input_fired_time: 2.3299639225, output_fired_time: 0.0000000000, propagate_forward_time: 38.2741546631, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2639083862
validation[          , genome 18346] predictions:    9853/  10000 (98.53%), best:    9853/10000 (98.53%), error:       480.12561, best error:       480.12561 on epoch:    27, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 487.9357604980s, input_fired_time: 14.4175100327, output_fired_time: 13.1738710403, propagate_forward_time: 179.9789276123, propagate_backward_time: 259.1990966797, weight_update_time: 1.0417824984, other_time: 21.1663818359
epoch time: 43.1823577881s, input_fired_time: 1.9693473577, output_fired_time: 0.0000000000, propagate_forward_time: 37.5823593140, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6306495667
validation[          , genome 18346] predictions:    9846/  10000 (98.46%), best:    9853/10000 (98.53%), error:       518.48285, best error:       480.12561 on epoch:    27, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 481.1058654785s, input_fired_time: 13.6503038406, output_fired_time: 12.3422431946, propagate_forward_time: 178.7100982666, propagate_backward_time: 256.8283081055, weight_update_time: 0.9615675211, other_time: 19.5748901367
epoch time: 44.0331344604s, input_fired_time: 2.1393949986, output_fired_time: 0.0000000000, propagate_forward_time: 37.9098167419, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9839210510
validation[          , genome 18346] predictions:    9858/  10000 (98.58%), best:    9858/10000 (98.58%), error:       465.20010, best error:       465.20010 on epoch:    29, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 485.6575927734s, input_fired_time: 13.6129169464, output_fired_time: 12.6382007599, propagate_forward_time: 180.1700439453, propagate_backward_time: 259.4977111816, weight_update_time: 0.9863831401, other_time: 19.7387084961
epoch time: 43.2200088501s, input_fired_time: 1.9553740025, output_fired_time: 0.0000000000, propagate_forward_time: 37.6073455811, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.6572875977
validation[          , genome 18346] predictions:    9885/  10000 (98.85%), best:    9885/10000 (98.85%), error:       391.88693, best error:       391.88693 on epoch:    30, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 484.5713195801s, input_fired_time: 14.0772218704, output_fired_time: 12.8396701813, propagate_forward_time: 178.8516845703, propagate_backward_time: 258.2875061035, weight_update_time: 1.0169607401, other_time: 20.5152587891
epoch time: 44.0917320251s, input_fired_time: 2.1158242226, output_fired_time: 0.0000000000, propagate_forward_time: 37.7617568970, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2141494751
validation[          , genome 18346] predictions:    9850/  10000 (98.50%), best:    9885/10000 (98.85%), error:       483.81613, best error:       391.88693 on epoch:    30, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 479.4966125488s, input_fired_time: 13.2563295364, output_fired_time: 12.0062007904, propagate_forward_time: 178.5697326660, propagate_backward_time: 256.7843627930, weight_update_time: 0.9293023944, other_time: 18.8800048828
epoch time: 41.9837989807s, input_fired_time: 1.7306228876, output_fired_time: 0.0000000000, propagate_forward_time: 37.1417160034, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1114616394
validation[          , genome 18346] predictions:    9859/  10000 (98.59%), best:    9885/10000 (98.85%), error:       457.57886, best error:       391.88693 on epoch:    30, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 484.6832275391s, input_fired_time: 14.4506359100, output_fired_time: 13.1245174408, propagate_forward_time: 178.2523040771, propagate_backward_time: 257.4682006836, weight_update_time: 1.0477255583, other_time: 21.3875732422
epoch time: 43.3574714661s, input_fired_time: 1.9489594698, output_fired_time: 0.0000000000, propagate_forward_time: 37.6433982849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7651138306
validation[          , genome 18346] predictions:    9841/  10000 (98.41%), best:    9885/10000 (98.85%), error:       521.43158, best error:       391.88693 on epoch:    30, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 483.1523437500s, input_fired_time: 13.4793167114, output_fired_time: 12.4233436584, propagate_forward_time: 179.2198638916, propagate_backward_time: 258.5386657715, weight_update_time: 0.9531430006, other_time: 19.4911804199
epoch time: 43.7227554321s, input_fired_time: 2.0024621487, output_fired_time: 0.0000000000, propagate_forward_time: 37.8134307861, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9068641663
validation[          , genome 18346] predictions:    9852/  10000 (98.52%), best:    9885/10000 (98.85%), error:       465.45029, best error:       391.88693 on epoch:    30, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 480.9660339355s, input_fired_time: 13.3534488678, output_fired_time: 12.2666759491, propagate_forward_time: 178.5012969971, propagate_backward_time: 257.7864074707, weight_update_time: 0.9501510262, other_time: 19.0581970215
epoch time: 41.9657363892s, input_fired_time: 1.6553726196, output_fired_time: 0.0000000000, propagate_forward_time: 37.2743072510, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.0360565186
validation[          , genome 18346] predictions:    9866/  10000 (98.66%), best:    9885/10000 (98.85%), error:       443.22784, best error:       391.88693 on epoch:    30, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 479.9297485352s, input_fired_time: 13.4987783432, output_fired_time: 12.1452140808, propagate_forward_time: 178.1499633789, propagate_backward_time: 256.8412170410, weight_update_time: 0.9709863067, other_time: 19.2945861816
epoch time: 42.5443420410s, input_fired_time: 1.7788665295, output_fired_time: 0.0000000000, propagate_forward_time: 37.4762229919, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2892532349
validation[          , genome 18346] predictions:    9824/  10000 (98.24%), best:    9885/10000 (98.85%), error:       532.19049, best error:       391.88693 on epoch:    30, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346_2_r901556117_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.916554
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.00187363
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 7.49451e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 37
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 30
read number_validation_images: 10000
read best_validation_predictions: 9885
read best_validation_error: 391.887
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18346
read normal distribution: '1 -1.24322175979614 0.0412701778113842'
generator_str: '1966613678'
read generator: 1966613678
read generated_by_map:
1 split_node 1
reading 81 nodes.
reading 283 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 471.241668701172s, input_fired_time: 12.0096387863159, output_fired_time: 11.0201978683472, propagate_forward_time: 177.055877685547, propagate_backward_time: 254.885879516602, weight_update_time: 0.816257178783417, other_time: 16.2700653076172
epoch time: 42.7805938720703s, input_fired_time: 1.86752152442932, output_fired_time: 0, propagate_forward_time: 37.3326950073242, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.58037567138672
validation[          , genome 18346] predictions:    9870/  10000 (98.70%), best:    9885/10000 (98.85%), error:       462.92258, best error:       391.88693 on epoch:    30, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 484.9636535645s, input_fired_time: 13.9512605667, output_fired_time: 12.7673206329, propagate_forward_time: 179.3206481934, propagate_backward_time: 258.6769714355, weight_update_time: 0.9891050458, other_time: 20.2474365234
epoch time: 43.5189704895s, input_fired_time: 2.0319201946, output_fired_time: 0.0000000000, propagate_forward_time: 37.6063613892, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8806877136
validation[          , genome 18346] predictions:    9856/  10000 (98.56%), best:    9885/10000 (98.85%), error:       497.43536, best error:       391.88693 on epoch:    30, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 497.4747314453s, input_fired_time: 16.0879707336, output_fired_time: 14.6952772141, propagate_forward_time: 180.9543151855, propagate_backward_time: 261.3642883301, weight_update_time: 1.1399499178, other_time: 24.3728637695
epoch time: 43.3183097839s, input_fired_time: 2.0694017410, output_fired_time: 0.0000000000, propagate_forward_time: 37.3092155457, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9396934509
validation[          , genome 18346] predictions:    9873/  10000 (98.73%), best:    9885/10000 (98.85%), error:       427.53885, best error:       391.88693 on epoch:    30, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 501.4089355469s, input_fired_time: 16.3465366364, output_fired_time: 14.7377119064, propagate_forward_time: 182.4798736572, propagate_backward_time: 262.9019775391, weight_update_time: 1.1596984863, other_time: 24.9428710938
epoch time: 47.3317680359s, input_fired_time: 2.9946718216, output_fired_time: 0.0000000000, propagate_forward_time: 38.7414703369, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.5956268311
validation[          , genome 18346] predictions:    9853/  10000 (98.53%), best:    9885/10000 (98.85%), error:       499.18488, best error:       391.88693 on epoch:    30, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 497.0604858398s, input_fired_time: 15.8377132416, output_fired_time: 14.3572492599, propagate_forward_time: 181.3069915771, propagate_backward_time: 261.4451599121, weight_update_time: 1.1653435230, other_time: 24.1133728027
epoch time: 44.2900199890s, input_fired_time: 2.1819219589, output_fired_time: 0.0000000000, propagate_forward_time: 37.8205642700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2875328064
validation[          , genome 18346] predictions:    9848/  10000 (98.48%), best:    9885/10000 (98.85%), error:       485.36475, best error:       391.88693 on epoch:    30, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 505.6759643555s, input_fired_time: 17.2306461334, output_fired_time: 15.7024869919, propagate_forward_time: 183.1639099121, propagate_backward_time: 263.5614013672, weight_update_time: 1.2179522514, other_time: 26.0175170898
epoch time: 43.8920783997s, input_fired_time: 2.0710690022, output_fired_time: 0.0000000000, propagate_forward_time: 37.7411155701, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0798950195
validation[          , genome 18346] predictions:    9859/  10000 (98.59%), best:    9885/10000 (98.85%), error:       446.08008, best error:       391.88693 on epoch:    30, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 500.4439697266s, input_fired_time: 16.3779830933, output_fired_time: 14.7672653198, propagate_forward_time: 181.8692626953, propagate_backward_time: 262.1375122070, weight_update_time: 1.2091141939, other_time: 25.2919311523
epoch time: 44.8590126038s, input_fired_time: 2.4436693192, output_fired_time: 0.0000000000, propagate_forward_time: 37.8237800598, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5915641785
validation[          , genome 18346] predictions:    9887/  10000 (98.87%), best:    9885/10000 (98.85%), error:       428.39609, best error:       391.88693 on epoch:    30, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 501.0502929688s, input_fired_time: 16.5209865570, output_fired_time: 15.3031578064, propagate_forward_time: 181.9875030518, propagate_backward_time: 262.1705627441, weight_update_time: 1.1818526983, other_time: 25.0680847168
epoch time: 42.7730598450s, input_fired_time: 1.8380495310, output_fired_time: 0.0000000000, propagate_forward_time: 37.4248008728, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5102081299
validation[          , genome 18346] predictions:    9859/  10000 (98.59%), best:    9885/10000 (98.85%), error:       482.70874, best error:       391.88693 on epoch:    30, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1511526933_34_18346_2_r901556117_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.941274
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.001243
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 4.97201e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 45
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 30
read number_validation_images: 10000
read best_validation_predictions: 9885
read best_validation_error: 391.887
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 18346
read normal distribution: '1 -1.24322175979614 0.0412701778113842'
generator_str: '1389903550'
read generator: 1389903550
read generated_by_map:
1 split_node 1
reading 81 nodes.
reading 283 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 458.446319580078s, input_fired_time: 9.95962715148926, output_fired_time: 9.39540004730225, propagate_forward_time: 175.312072753906, propagate_backward_time: 252.511444091797, weight_update_time: 0.630298435688019, other_time: 11.2677917480469
epoch time: 40.7776641845703s, input_fired_time: 1.31696176528931, output_fired_time: 0, propagate_forward_time: 37.1093444824219, propagate_backward_time: 0, weight_update_time: 0, other_time: 2.35135650634766
validation[          , genome 18346] predictions:    9890/  10000 (98.90%), best:    9890/10000 (98.90%), error:       385.65643, best error:       385.65643 on epoch:    45, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 467.3337707520s, input_fired_time: 11.2856693268, output_fired_time: 10.4157638550, propagate_forward_time: 176.7341461182, propagate_backward_time: 254.5069732666, weight_update_time: 0.7421565652, other_time: 14.3912200928
epoch time: 41.0391502380s, input_fired_time: 1.4256293774, output_fired_time: 0.0000000000, propagate_forward_time: 37.1703338623, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4431877136
validation[          , genome 18346] predictions:    9868/  10000 (98.68%), best:    9890/10000 (98.90%), error:       427.66602, best error:       385.65643 on epoch:    45, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 470.9412536621s, input_fired_time: 11.9031391144, output_fired_time: 10.8547563553, propagate_forward_time: 177.2539367676, propagate_backward_time: 255.2903900146, weight_update_time: 0.8131953478, other_time: 15.6390228271
epoch time: 41.4609756470s, input_fired_time: 1.7231181860, output_fired_time: 0.0000000000, propagate_forward_time: 36.5879364014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.1499214172
validation[          , genome 18346] predictions:    9875/  10000 (98.75%), best:    9890/10000 (98.90%), error:       428.49603, best error:       385.65643 on epoch:    45, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724

epoch time: 469.4043884277s, input_fired_time: 12.1530227661, output_fired_time: 11.1813383102, propagate_forward_time: 175.9276428223, propagate_backward_time: 254.0016326904, weight_update_time: 0.8260860443, other_time: 16.1407623291
epoch time: 41.5130500793s, input_fired_time: 1.5713076591, output_fired_time: 0.0000000000, propagate_forward_time: 37.0482597351, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8934822083
validation[          , genome 18346] predictions:    9855/  10000 (98.55%), best:    9890/10000 (98.90%), error:       457.51425, best error:       385.65643 on epoch:    45, epoch:   48/100, mu: 0.9482238293, learning_rate: 0.0010657189, weight_decay: 0.0000426288

epoch time: 472.5892639160s, input_fired_time: 12.0136299133, output_fired_time: 10.9771194458, propagate_forward_time: 178.1685028076, propagate_backward_time: 255.8737487793, weight_update_time: 0.7916194797, other_time: 15.5562438965
epoch time: 41.3562240601s, input_fired_time: 1.5059957504, output_fired_time: 0.0000000000, propagate_forward_time: 37.1187133789, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7315139771
validation[          , genome 18346] predictions:    9869/  10000 (98.69%), best:    9890/10000 (98.90%), error:       419.93146, best error:       385.65643 on epoch:    45, epoch:   49/100, mu: 0.9503126144, learning_rate: 0.0010124330, weight_decay: 0.0000404973

epoch time: 474.7042236328s, input_fired_time: 12.0518503189, output_fired_time: 11.1856460571, propagate_forward_time: 178.4920349121, propagate_backward_time: 256.7995910645, weight_update_time: 0.8269985914, other_time: 16.1751098633
epoch time: 41.7677612305s, input_fired_time: 1.5219745636, output_fired_time: 0.0000000000, propagate_forward_time: 37.2471466064, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9986419678
validation[          , genome 18346] predictions:    9863/  10000 (98.63%), best:    9890/10000 (98.90%), error:       449.33563, best error:       385.65643 on epoch:    45, epoch:   50/100, mu: 0.9522969723, learning_rate: 0.0009618113, weight_decay: 0.0000384725

epoch time: 470.2270507813s, input_fired_time: 11.9507446289, output_fired_time: 11.0513086319, propagate_forward_time: 176.4269104004, propagate_backward_time: 254.4473724365, weight_update_time: 0.8205900788, other_time: 16.3507232666
epoch time: 42.5179710388s, input_fired_time: 1.8408210278, output_fired_time: 0.0000000000, propagate_forward_time: 37.4426078796, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.2345428467
validation[          , genome 18346] predictions:    9862/  10000 (98.62%), best:    9890/10000 (98.90%), error:       456.24246, best error:       385.65643 on epoch:    45, epoch:   51/100, mu: 0.9541821480, learning_rate: 0.0009137207, weight_decay: 0.0000365488

epoch time: 476.7117614746s, input_fired_time: 12.1832380295, output_fired_time: 11.1788969040, propagate_forward_time: 177.6059265137, propagate_backward_time: 259.1636657715, weight_update_time: 0.8173505664, other_time: 16.5800476074
epoch time: 42.6168708801s, input_fired_time: 1.8295110464, output_fired_time: 0.0000000000, propagate_forward_time: 37.4472045898, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3401565552
validation[          , genome 18346] predictions:    9861/  10000 (98.61%), best:    9890/10000 (98.90%), error:       449.42050, best error:       385.65643 on epoch:    45, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 474.6984252930s, input_fired_time: 12.2303056717, output_fired_time: 11.3460283279, propagate_forward_time: 177.9842376709, propagate_backward_time: 256.6396484375, weight_update_time: 0.8478838801, other_time: 16.4981689453
epoch time: 41.6127815247s, input_fired_time: 1.5281965733, output_fired_time: 0.0000000000, propagate_forward_time: 37.1118927002, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9726905823
validation[          , genome 18346] predictions:    9866/  10000 (98.66%), best:    9890/10000 (98.90%), error:       435.00464, best error:       385.65643 on epoch:    45, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 488.5308227539s, input_fired_time: 14.7020702362, output_fired_time: 13.4736700058, propagate_forward_time: 179.4870910645, propagate_backward_time: 259.0733032227, weight_update_time: 1.0547912121, other_time: 21.7947082520
epoch time: 42.9872436523s, input_fired_time: 2.1465222836, output_fired_time: 0.0000000000, propagate_forward_time: 36.6015968323, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2391242981
validation[          , genome 18346] predictions:    9863/  10000 (98.63%), best:    9890/10000 (98.90%), error:       443.27142, best error:       385.65643 on epoch:    45, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 489.8009948730s, input_fired_time: 15.0314893723, output_fired_time: 13.4307594299, propagate_forward_time: 179.5734558105, propagate_backward_time: 259.1835021973, weight_update_time: 1.0755649805, other_time: 22.5817871094
epoch time: 45.0834197998s, input_fired_time: 2.2730047703, output_fired_time: 0.0000000000, propagate_forward_time: 38.2824020386, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.5280113220
validation[          , genome 18346] predictions:    9860/  10000 (98.60%), best:    9890/10000 (98.90%), error:       471.30795, best error:       385.65643 on epoch:    45, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693


</stderr_txt>
]]>