Nameexact_genome_1512185335_40_9595_1
Workunit2566397
Created5 Dec 2017, 23:03:03 UTC
Sent5 Dec 2017, 23:10:22 UTC
Report deadline26 Dec 2017, 20:56:23 UTC
Received7 Dec 2017, 21:18:58 UTC
Server stateOver
OutcomeComputation error
Client stateCompute error
Exit status194 (0xc2) EXIT_ABORTED_BY_CLIENT
Computer ID49051
Run time5 hours 0 min 6 sec
CPU time4 hours 59 min 41 sec
Validate stateInvalid
Credit0.00
Device peak FLOPS1.56 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size159.69 MB
Peak swap size164.44 MB
Peak disk usage3.56 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code 194 (0xc2)</message>
<stderr_txt>
arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1512185335_40_9595.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1512185335_40_9595_1_r247439743_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from input file: '../../projects/csgrid.org_csg/exact_genome_1512185335_40_9595.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.5
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.0125
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 0.0005
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 0
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 0
read number_validation_images: 10000
read best_validation_predictions: 0
read best_validation_error: 1e+07
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 9595
read normal distribution: '0 1.2301869392395 -0.614859640598297'
generator_str: '1989068771'
read generator: 1989068771
read generated_by_map:
1 change_size 1
reading 45 nodes.
reading 134 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 0
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
generator min: 1, generator max: 2147483646
pre shuffle 1: 400901348
post shuffle 1: 758956352
epoch time: 342.536895751953s, input_fired_time: 3.51772570610046, output_fired_time: 3.36131501197815, propagate_forward_time: 131.272399902344, propagate_backward_time: 199.728637695313, weight_update_time: 0.304887056350708, other_time: 4.65679931640625
epoch time: 27.6866970062256s, input_fired_time: 0.436761140823364, output_fired_time: 0, propagate_forward_time: 26.4133796691895, propagate_backward_time: 0, weight_update_time: 0, other_time: 0.836555480957031
validation[          , genome  9595] predictions:    9779/  10000 (97.79%), best:    9779/10000 (97.79%), error:       769.23743, best error:       769.23743 on epoch:     0, epoch:    0/100, mu: 0.5000000000, learning_rate: 0.0125000002, weight_decay: 0.0005000000

epoch time: 342.0099792480s, input_fired_time: 3.5330982208, output_fired_time: 3.3412098885, propagate_forward_time: 130.8984375000, propagate_backward_time: 199.5873718262, weight_update_time: 0.3040802181, other_time: 4.6498413086
epoch time: 27.4540138245s, input_fired_time: 0.4319365919, output_fired_time: 0.0000000000, propagate_forward_time: 26.2029838562, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8190937042
validation[          , genome  9595] predictions:    9769/  10000 (97.69%), best:    9779/10000 (97.79%), error:       792.09125, best error:       769.23743 on epoch:     0, epoch:    1/100, mu: 0.5245000124, learning_rate: 0.0118749999, weight_decay: 0.0004750000

epoch time: 342.0075073242s, input_fired_time: 3.5358357430, output_fired_time: 3.3501696587, propagate_forward_time: 130.8001708984, propagate_backward_time: 199.6724853516, weight_update_time: 0.3050049245, other_time: 4.6488647461
epoch time: 27.2781562805s, input_fired_time: 0.4310948551, output_fired_time: 0.0000000000, propagate_forward_time: 26.0212917328, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8257694244
validation[          , genome  9595] predictions:    9765/  10000 (97.65%), best:    9779/10000 (97.79%), error:       819.94519, best error:       769.23743 on epoch:     0, epoch:    2/100, mu: 0.5477750301, learning_rate: 0.0112812500, weight_decay: 0.0004512500

epoch time: 342.5316772461s, input_fired_time: 3.5424270630, output_fired_time: 3.3520073891, propagate_forward_time: 131.1496276855, propagate_backward_time: 199.8236236572, weight_update_time: 0.3058384955, other_time: 4.6639862061
epoch time: 27.3245182037s, input_fired_time: 0.4361140430, output_fired_time: 0.0000000000, propagate_forward_time: 26.0504131317, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8379917145
validation[          , genome  9595] predictions:    9811/  10000 (98.11%), best:    9811/10000 (98.11%), error:       694.87250, best error:       694.87250 on epoch:     3, epoch:    3/100, mu: 0.5698862672, learning_rate: 0.0107171871, weight_decay: 0.0004286875

epoch time: 342.4452209473s, input_fired_time: 3.5379345417, output_fired_time: 3.3503923416, propagate_forward_time: 130.9062194824, propagate_backward_time: 199.9977264404, weight_update_time: 0.3059224486, other_time: 4.6529388428
epoch time: 27.5020866394s, input_fired_time: 0.4360525906, output_fired_time: 0.0000000000, propagate_forward_time: 26.2383117676, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8277225494
validation[          , genome  9595] predictions:    9724/  10000 (97.24%), best:    9811/10000 (98.11%), error:       845.66608, best error:       694.87250 on epoch:     3, epoch:    4/100, mu: 0.5908919573, learning_rate: 0.0101813274, weight_decay: 0.0004072531

epoch time: 342.3241577148s, input_fired_time: 3.5471582413, output_fired_time: 3.3535075188, propagate_forward_time: 130.9013824463, propagate_backward_time: 199.8496551514, weight_update_time: 0.3049942553, other_time: 4.6724548340
epoch time: 27.4668292999s, input_fired_time: 0.4326247573, output_fired_time: 0.0000000000, propagate_forward_time: 26.2082386017, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8259658813
validation[          , genome  9595] predictions:    9784/  10000 (97.84%), best:    9811/10000 (98.11%), error:       748.29663, best error:       694.87250 on epoch:     3, epoch:    5/100, mu: 0.6108473539, learning_rate: 0.0096722608, weight_decay: 0.0003868904

epoch time: 343.6328125000s, input_fired_time: 3.5415706635, output_fired_time: 3.3569197655, propagate_forward_time: 131.3019104004, propagate_backward_time: 200.7977752686, weight_update_time: 0.3051793873, other_time: 4.6346282959
epoch time: 27.4225749969s, input_fired_time: 0.4326426685, output_fired_time: 0.0000000000, propagate_forward_time: 26.1571292877, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8328037262
validation[          , genome  9595] predictions:    9771/  10000 (97.71%), best:    9811/10000 (98.11%), error:       727.21808, best error:       694.87250 on epoch:     3, epoch:    6/100, mu: 0.6298049688, learning_rate: 0.0091886474, weight_decay: 0.0003675459

epoch time: 342.9492187500s, input_fired_time: 3.5463957787, output_fired_time: 3.3507533073, propagate_forward_time: 130.7325592041, propagate_backward_time: 200.6360015869, weight_update_time: 0.3060698807, other_time: 4.6835327148
epoch time: 27.5846519470s, input_fired_time: 0.4345759749, output_fired_time: 0.0000000000, propagate_forward_time: 26.3227481842, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8273277283
validation[          , genome  9595] predictions:    9803/  10000 (98.03%), best:    9803/10000 (98.03%), error:       654.76782, best error:       654.76782 on epoch:     7, epoch:    7/100, mu: 0.6478147507, learning_rate: 0.0087292148, weight_decay: 0.0003491686

epoch time: 343.7144470215s, input_fired_time: 3.5457651615, output_fired_time: 3.3484945297, propagate_forward_time: 131.1138458252, propagate_backward_time: 201.0652465820, weight_update_time: 0.3050080240, other_time: 4.6410980225
epoch time: 27.2087650299s, input_fired_time: 0.4337034822, output_fired_time: 0.0000000000, propagate_forward_time: 25.9539318085, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8211288452
validation[          , genome  9595] predictions:    9801/  10000 (98.01%), best:    9801/10000 (98.01%), error:       650.98853, best error:       650.98853 on epoch:     8, epoch:    8/100, mu: 0.6649240255, learning_rate: 0.0082927542, weight_decay: 0.0003317102

epoch time: 344.5301818848s, input_fired_time: 3.5551331043, output_fired_time: 3.3541848660, propagate_forward_time: 131.2435607910, propagate_backward_time: 201.7065582275, weight_update_time: 0.3061861396, other_time: 4.6707305908
epoch time: 27.5590209961s, input_fired_time: 0.4369449019, output_fired_time: 0.0000000000, propagate_forward_time: 26.2949218750, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8271541595
validation[          , genome  9595] predictions:    9789/  10000 (97.89%), best:    9801/10000 (98.01%), error:       724.83624, best error:       650.98853 on epoch:     8, epoch:    9/100, mu: 0.6811778545, learning_rate: 0.0078781163, weight_decay: 0.0003151247

epoch time: 342.5893249512s, input_fired_time: 3.5494816303, output_fired_time: 3.3336758614, propagate_forward_time: 130.3214874268, propagate_backward_time: 200.6853332520, weight_update_time: 0.3070330322, other_time: 4.6993560791
epoch time: 27.4904727936s, input_fired_time: 0.4357945621, output_fired_time: 0.0000000000, propagate_forward_time: 26.2274513245, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8272266388
validation[          , genome  9595] predictions:    9804/  10000 (98.04%), best:    9801/10000 (98.01%), error:       661.54071, best error:       650.98853 on epoch:     8, epoch:   10/100, mu: 0.6966189742, learning_rate: 0.0074842102, weight_decay: 0.0002993684

epoch time: 342.4953308105s, input_fired_time: 3.5426390171, output_fired_time: 3.3326046467, propagate_forward_time: 130.1687622070, propagate_backward_time: 200.7916564941, weight_update_time: 0.3054478467, other_time: 4.6596679688
epoch time: 27.4141349792s, input_fired_time: 0.4345191717, output_fired_time: 0.0000000000, propagate_forward_time: 26.1624183655, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8171977997
validation[          , genome  9595] predictions:    9809/  10000 (98.09%), best:    9809/10000 (98.09%), error:       630.25787, best error:       630.25787 on epoch:    11, epoch:   11/100, mu: 0.7112880349, learning_rate: 0.0071099997, weight_decay: 0.0002844000

epoch time: 344.4849853516s, input_fired_time: 3.5556027889, output_fired_time: 3.3352305889, propagate_forward_time: 130.8264312744, propagate_backward_time: 202.0789642334, weight_update_time: 0.3076331913, other_time: 4.6887512207
epoch time: 27.5736255646s, input_fired_time: 0.4353014231, output_fired_time: 0.0000000000, propagate_forward_time: 26.3127956390, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8255290985
validation[          , genome  9595] predictions:    9762/  10000 (97.62%), best:    9809/10000 (98.09%), error:       816.84497, best error:       630.25787 on epoch:    11, epoch:   12/100, mu: 0.7252236605, learning_rate: 0.0067544994, weight_decay: 0.0002701800

epoch time: 343.8784790039s, input_fired_time: 3.5493028164, output_fired_time: 3.3410179615, propagate_forward_time: 130.2718353271, propagate_backward_time: 202.0388793945, weight_update_time: 0.3072311580, other_time: 4.6774444580
epoch time: 27.1520614624s, input_fired_time: 0.4332176745, output_fired_time: 0.0000000000, propagate_forward_time: 25.8823413849, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8365020752
validation[          , genome  9595] predictions:    9791/  10000 (97.91%), best:    9809/10000 (98.09%), error:       655.59192, best error:       630.25787 on epoch:    11, epoch:   13/100, mu: 0.7384625077, learning_rate: 0.0064167744, weight_decay: 0.0002566710

epoch time: 346.0338439941s, input_fired_time: 3.5623619556, output_fired_time: 3.3613018990, propagate_forward_time: 131.2282257080, propagate_backward_time: 203.2137451172, weight_update_time: 0.3070716858, other_time: 4.6682281494
epoch time: 27.3998279572s, input_fired_time: 0.4378384650, output_fired_time: 0.0000000000, propagate_forward_time: 26.1263370514, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8356533051
validation[          , genome  9595] predictions:    9813/  10000 (98.13%), best:    9809/10000 (98.09%), error:       674.29889, best error:       630.25787 on epoch:    11, epoch:   14/100, mu: 0.7510393858, learning_rate: 0.0060959356, weight_decay: 0.0002438374

epoch time: 343.8147888184s, input_fired_time: 3.5494341850, output_fired_time: 3.3371388912, propagate_forward_time: 130.0665740967, propagate_backward_time: 202.1826629639, weight_update_time: 0.3079291880, other_time: 4.6789855957
epoch time: 27.2555522919s, input_fired_time: 0.4317020476, output_fired_time: 0.0000000000, propagate_forward_time: 26.0056838989, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8181667328
validation[          , genome  9595] predictions:    9832/  10000 (98.32%), best:    9832/10000 (98.32%), error:       584.29272, best error:       584.29272 on epoch:    15, epoch:   15/100, mu: 0.7629874349, learning_rate: 0.0057911389, weight_decay: 0.0002316456

epoch time: 345.4218139648s, input_fired_time: 3.5553295612, output_fired_time: 3.3390650749, propagate_forward_time: 130.7944793701, propagate_backward_time: 203.0579376221, weight_update_time: 0.3090152442, other_time: 4.6750183105
epoch time: 27.5201911926s, input_fired_time: 0.4366639853, output_fired_time: 0.0000000000, propagate_forward_time: 26.2557525635, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8277740479
validation[          , genome  9595] predictions:    9771/  10000 (97.71%), best:    9832/10000 (98.32%), error:       756.38708, best error:       584.29272 on epoch:    15, epoch:   16/100, mu: 0.7743380666, learning_rate: 0.0055015818, weight_decay: 0.0002200633

epoch time: 345.5211791992s, input_fired_time: 3.5558621883, output_fired_time: 3.3463873863, propagate_forward_time: 130.5894165039, propagate_backward_time: 203.3416900635, weight_update_time: 0.3112205863, other_time: 4.6878509521
epoch time: 27.2416553497s, input_fired_time: 0.4358272851, output_fired_time: 0.0000000000, propagate_forward_time: 25.9806823730, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8251457214
validation[          , genome  9595] predictions:    9720/  10000 (97.20%), best:    9832/10000 (98.32%), error:       834.34875, best error:       584.29272 on epoch:    15, epoch:   17/100, mu: 0.7851211429, learning_rate: 0.0052265027, weight_decay: 0.0002090601

epoch time: 345.9985351563s, input_fired_time: 3.5609638691, output_fired_time: 3.3495502472, propagate_forward_time: 130.7971954346, propagate_backward_time: 203.6263732910, weight_update_time: 0.3111698627, other_time: 4.6644439697
epoch time: 27.3991508484s, input_fired_time: 0.4367919564, output_fired_time: 0.0000000000, propagate_forward_time: 26.1402988434, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8220596313
validation[          , genome  9595] predictions:    9794/  10000 (97.94%), best:    9832/10000 (98.32%), error:       695.34436, best error:       584.29272 on epoch:    15, epoch:   18/100, mu: 0.7953650951, learning_rate: 0.0049651773, weight_decay: 0.0001986071

epoch time: 345.3841857910s, input_fired_time: 3.5576813221, output_fired_time: 3.3397560120, propagate_forward_time: 130.6700286865, propagate_backward_time: 203.1524047852, weight_update_time: 0.3132361174, other_time: 4.6643218994
epoch time: 27.5192337036s, input_fired_time: 0.4365215898, output_fired_time: 0.0000000000, propagate_forward_time: 26.2671871185, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8155250549
validation[          , genome  9595] predictions:    9825/  10000 (98.25%), best:    9832/10000 (98.32%), error:       603.75043, best error:       584.29272 on epoch:    15, epoch:   19/100, mu: 0.8050968647, learning_rate: 0.0047169183, weight_decay: 0.0001886768

epoch time: 346.1304626465s, input_fired_time: 3.5650980473, output_fired_time: 3.3473565578, propagate_forward_time: 130.8196868896, propagate_backward_time: 203.7124328613, weight_update_time: 0.3151216805, other_time: 4.6858978271
epoch time: 27.5848503113s, input_fired_time: 0.4377361536, output_fired_time: 0.0000000000, propagate_forward_time: 26.3251476288, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8219661713
validation[          , genome  9595] predictions:    9797/  10000 (97.97%), best:    9832/10000 (98.32%), error:       677.42542, best error:       584.29272 on epoch:    15, epoch:   20/100, mu: 0.8143420219, learning_rate: 0.0044810725, weight_decay: 0.0001792429

epoch time: 347.0582580566s, input_fired_time: 3.5689299107, output_fired_time: 3.3645427227, propagate_forward_time: 131.1531372070, propagate_backward_time: 204.3084869385, weight_update_time: 0.3175550103, other_time: 4.6631622314
epoch time: 27.3170108795s, input_fired_time: 0.4343594015, output_fired_time: 0.0000000000, propagate_forward_time: 26.0638389587, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8188133240
validation[          , genome  9595] predictions:    9800/  10000 (98.00%), best:    9832/10000 (98.32%), error:       637.09320, best error:       584.29272 on epoch:    15, epoch:   21/100, mu: 0.8231249452, learning_rate: 0.0042570187, weight_decay: 0.0001702808

epoch time: 346.7019653320s, input_fired_time: 3.5634841919, output_fired_time: 3.3410010338, propagate_forward_time: 130.8641662598, propagate_backward_time: 204.2656555176, weight_update_time: 0.3196141720, other_time: 4.6676635742
epoch time: 27.5353927612s, input_fired_time: 0.4396566451, output_fired_time: 0.0000000000, propagate_forward_time: 26.2767505646, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8189849854
validation[          , genome  9595] predictions:    9812/  10000 (98.12%), best:    9832/10000 (98.32%), error:       646.69824, best error:       584.29272 on epoch:    15, epoch:   22/100, mu: 0.8314687014, learning_rate: 0.0040441677, weight_decay: 0.0001617667

epoch time: 347.5750427246s, input_fired_time: 3.5731701851, output_fired_time: 3.3485205173, propagate_forward_time: 131.1137237549, propagate_backward_time: 204.8570861816, weight_update_time: 0.3239841461, other_time: 4.6825408936
epoch time: 27.8018054962s, input_fired_time: 0.4415858388, output_fired_time: 0.0000000000, propagate_forward_time: 26.5315494537, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8286705017
validation[          , genome  9595] predictions:    9789/  10000 (97.89%), best:    9832/10000 (98.32%), error:       693.22272, best error:       584.29272 on epoch:    15, epoch:   23/100, mu: 0.8393952847, learning_rate: 0.0038419592, weight_decay: 0.0001536784

epoch time: 346.7506713867s, input_fired_time: 3.5658085346, output_fired_time: 3.3361663818, propagate_forward_time: 130.8764190674, propagate_backward_time: 204.2911376953, weight_update_time: 0.3290251493, other_time: 4.6811370850
epoch time: 26.9939575195s, input_fired_time: 0.4351637363, output_fired_time: 0.0000000000, propagate_forward_time: 25.7318534851, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8269405365
validation[          , genome  9595] predictions:    9829/  10000 (98.29%), best:    9832/10000 (98.32%), error:       592.10583, best error:       584.29272 on epoch:    15, epoch:   24/100, mu: 0.8469254971, learning_rate: 0.0036498611, weight_decay: 0.0001459945

epoch time: 345.6825866699s, input_fired_time: 3.5711364746, output_fired_time: 3.3291323185, propagate_forward_time: 130.3325195313, propagate_backward_time: 203.7481384277, weight_update_time: 0.3324716091, other_time: 4.7016601563
epoch time: 27.4855861664s, input_fired_time: 0.4387928247, output_fired_time: 0.0000000000, propagate_forward_time: 26.2146892548, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8321037292
validation[          , genome  9595] predictions:    9784/  10000 (97.84%), best:    9832/10000 (98.32%), error:       665.45380, best error:       584.29272 on epoch:    15, epoch:   25/100, mu: 0.8540792465, learning_rate: 0.0034673680, weight_decay: 0.0001386948

epoch time: 348.0393371582s, input_fired_time: 3.5773637295, output_fired_time: 3.3539042473, propagate_forward_time: 131.2774963379, propagate_backward_time: 205.1271972656, weight_update_time: 0.3405345082, other_time: 4.7033691406
epoch time: 27.4372920990s, input_fired_time: 0.4426445961, output_fired_time: 0.0000000000, propagate_forward_time: 26.1510334015, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8436145782
validation[          , genome  9595] predictions:    9821/  10000 (98.21%), best:    9832/10000 (98.32%), error:       594.55353, best error:       584.29272 on epoch:    15, epoch:   26/100, mu: 0.8608753085, learning_rate: 0.0032939995, weight_decay: 0.0001317600

epoch time: 346.9793701172s, input_fired_time: 3.5732142925, output_fired_time: 3.3449535370, propagate_forward_time: 130.4027099609, propagate_backward_time: 204.9339447021, weight_update_time: 0.3469457924, other_time: 4.7245635986
epoch time: 27.5361366272s, input_fired_time: 0.4390872717, output_fired_time: 0.0000000000, propagate_forward_time: 26.2827930450, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8142566681
validation[          , genome  9595] predictions:    9817/  10000 (98.17%), best:    9832/10000 (98.32%), error:       597.10443, best error:       584.29272 on epoch:    15, epoch:   27/100, mu: 0.8673315644, learning_rate: 0.0031292995, weight_decay: 0.0001251720

epoch time: 348.6251525879s, input_fired_time: 3.5704424381, output_fired_time: 3.3568866253, propagate_forward_time: 131.0386962891, propagate_backward_time: 205.9650421143, weight_update_time: 0.3524419665, other_time: 4.6941070557
epoch time: 27.7630767822s, input_fired_time: 0.4466939867, output_fired_time: 0.0000000000, propagate_forward_time: 26.4690189362, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8473644257
validation[          , genome  9595] predictions:    9813/  10000 (98.13%), best:    9832/10000 (98.32%), error:       616.48145, best error:       584.29272 on epoch:    15, epoch:   28/100, mu: 0.8734650016, learning_rate: 0.0029728345, weight_decay: 0.0001189134

epoch time: 348.4109497070s, input_fired_time: 3.5807039738, output_fired_time: 3.3531389236, propagate_forward_time: 131.0884552002, propagate_backward_time: 205.6535491943, weight_update_time: 0.3632719517, other_time: 4.7350769043
epoch time: 27.5413913727s, input_fired_time: 0.4436209500, output_fired_time: 0.0000000000, propagate_forward_time: 26.2675075531, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8302631378
validation[          , genome  9595] predictions:    9783/  10000 (97.83%), best:    9832/10000 (98.32%), error:       707.22498, best error:       584.29272 on epoch:    15, epoch:   29/100, mu: 0.8792917728, learning_rate: 0.0028241926, weight_decay: 0.0001129677

epoch time: 347.6257019043s, input_fired_time: 3.5822408199, output_fired_time: 3.3427538872, propagate_forward_time: 130.7057495117, propagate_backward_time: 205.2482147217, weight_update_time: 0.3723615110, other_time: 4.7467498779
epoch time: 27.4816417694s, input_fired_time: 0.4429953396, output_fired_time: 0.0000000000, propagate_forward_time: 26.2065219879, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8321247101
validation[          , genome  9595] predictions:    9818/  10000 (98.18%), best:    9832/10000 (98.32%), error:       591.02191, best error:       584.29272 on epoch:    15, epoch:   30/100, mu: 0.8848271966, learning_rate: 0.0026829829, weight_decay: 0.0001073193

epoch time: 348.0374450684s, input_fired_time: 3.5846824646, output_fired_time: 3.3416543007, propagate_forward_time: 131.1467895508, propagate_backward_time: 205.2075653076, weight_update_time: 0.3825535476, other_time: 4.7567596436
epoch time: 27.4069938660s, input_fired_time: 0.4403426051, output_fired_time: 0.0000000000, propagate_forward_time: 26.1345214844, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8321304321
validation[          , genome  9595] predictions:    9831/  10000 (98.31%), best:    9831/10000 (98.31%), error:       562.24426, best error:       562.24426 on epoch:    31, epoch:   31/100, mu: 0.8900858164, learning_rate: 0.0025488338, weight_decay: 0.0001019534

epoch time: 347.7509460449s, input_fired_time: 3.6229298115, output_fired_time: 3.3498699665, propagate_forward_time: 130.8569030762, propagate_backward_time: 204.9480438232, weight_update_time: 0.4048442543, other_time: 4.9731903076
epoch time: 27.7871742249s, input_fired_time: 0.4619423151, output_fired_time: 0.0000000000, propagate_forward_time: 26.4133853912, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9118461609
validation[          , genome  9595] predictions:    9826/  10000 (98.26%), best:    9826/10000 (98.26%), error:       544.91748, best error:       544.91748 on epoch:    32, epoch:   32/100, mu: 0.8950815201, learning_rate: 0.0024213921, weight_decay: 0.0000968557

epoch time: 350.0468750000s, input_fired_time: 3.7043890953, output_fired_time: 3.3924360275, propagate_forward_time: 131.2309265137, propagate_backward_time: 206.3010559082, weight_update_time: 0.4450584948, other_time: 5.4180908203
epoch time: 27.6536273956s, input_fired_time: 0.4692624509, output_fired_time: 0.0000000000, propagate_forward_time: 26.2415771484, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9427871704
validation[          , genome  9595] predictions:    9793/  10000 (97.93%), best:    9826/10000 (98.26%), error:       671.00873, best error:       544.91748 on epoch:    32, epoch:   33/100, mu: 0.8998274207, learning_rate: 0.0023003225, weight_decay: 0.0000920129

epoch time: 349.5113830566s, input_fired_time: 3.6959421635, output_fired_time: 3.3812308311, propagate_forward_time: 131.1501312256, propagate_backward_time: 205.8780517578, weight_update_time: 0.4579346180, other_time: 5.4060211182
epoch time: 27.5832633972s, input_fired_time: 0.4581423402, output_fired_time: 0.0000000000, propagate_forward_time: 26.2181816101, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9069385529
validation[          , genome  9595] predictions:    9827/  10000 (98.27%), best:    9827/10000 (98.27%), error:       539.10931, best error:       539.10931 on epoch:    34, epoch:   34/100, mu: 0.9043360353, learning_rate: 0.0021853063, weight_decay: 0.0000874123

epoch time: 349.8087768555s, input_fired_time: 3.6884315014, output_fired_time: 3.3970136642, propagate_forward_time: 131.3280487061, propagate_backward_time: 206.0205841064, weight_update_time: 0.4662822187, other_time: 5.3746948242
epoch time: 27.5080795288s, input_fired_time: 0.4539167583, output_fired_time: 0.0000000000, propagate_forward_time: 26.1565551758, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8976078033
validation[          , genome  9595] predictions:    9844/  10000 (98.44%), best:    9827/10000 (98.27%), error:       555.81995, best error:       539.10931 on epoch:    34, epoch:   35/100, mu: 0.9086192250, learning_rate: 0.0020760410, weight_decay: 0.0000830417

epoch time: 350.0696411133s, input_fired_time: 3.7122037411, output_fired_time: 3.3980462551, propagate_forward_time: 131.1932067871, propagate_backward_time: 206.2619476318, weight_update_time: 0.4849790931, other_time: 5.5042572021
epoch time: 27.5978126526s, input_fired_time: 0.4579518735, output_fired_time: 0.0000000000, propagate_forward_time: 26.2433338165, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.8965263367
validation[          , genome  9595] predictions:    9830/  10000 (98.30%), best:    9827/10000 (98.27%), error:       582.93628, best error:       539.10931 on epoch:    34, epoch:   36/100, mu: 0.9126882553, learning_rate: 0.0019722388, weight_decay: 0.0000788896

epoch time: 351.0777893066s, input_fired_time: 3.7009906769, output_fired_time: 3.3991031647, propagate_forward_time: 131.7067871094, propagate_backward_time: 206.8435821533, weight_update_time: 0.4899068773, other_time: 5.4273223877
epoch time: 27.8220157623s, input_fired_time: 0.4670977294, output_fired_time: 0.0000000000, propagate_forward_time: 26.4028453827, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9520721436
validation[          , genome  9595] predictions:    9842/  10000 (98.42%), best:    9842/10000 (98.42%), error:       516.26855, best error:       516.26855 on epoch:    37, epoch:   37/100, mu: 0.9165538549, learning_rate: 0.0018736268, weight_decay: 0.0000749451

epoch time: 349.8740844727s, input_fired_time: 3.7019081116, output_fired_time: 3.3970437050, propagate_forward_time: 131.1878662109, propagate_backward_time: 206.1706848145, weight_update_time: 0.5000150204, other_time: 5.4165954590
epoch time: 27.5047149658s, input_fired_time: 0.4589850605, output_fired_time: 0.0000000000, propagate_forward_time: 26.1339931488, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9117374420
validation[          , genome  9595] predictions:    9783/  10000 (97.83%), best:    9842/10000 (98.42%), error:       700.66632, best error:       516.26855 on epoch:    37, epoch:   38/100, mu: 0.9202261567, learning_rate: 0.0017799455, weight_decay: 0.0000711978

epoch time: 350.4558410645s, input_fired_time: 3.7065711021, output_fired_time: 3.3895273209, propagate_forward_time: 131.4512634277, propagate_backward_time: 206.4882507324, weight_update_time: 0.5074101686, other_time: 5.4202270508
epoch time: 27.4850006104s, input_fired_time: 0.4616909921, output_fired_time: 0.0000000000, propagate_forward_time: 26.0958709717, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9274387360
validation[          , genome  9595] predictions:    9843/  10000 (98.43%), best:    9842/10000 (98.42%), error:       532.15265, best error:       516.26855 on epoch:    37, epoch:   39/100, mu: 0.9237148762, learning_rate: 0.0016909481, weight_decay: 0.0000676379

epoch time: 349.7478942871s, input_fired_time: 3.7081282139, output_fired_time: 3.3958203793, propagate_forward_time: 131.0466766357, propagate_backward_time: 206.1062622070, weight_update_time: 0.5153714418, other_time: 5.4910125732
epoch time: 27.8874855042s, input_fired_time: 0.4654510319, output_fired_time: 0.0000000000, propagate_forward_time: 26.4922389984, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9297962189
validation[          , genome  9595] predictions:    9839/  10000 (98.39%), best:    9842/10000 (98.42%), error:       559.37891, best error:       516.26855 on epoch:    37, epoch:   40/100, mu: 0.9270291328, learning_rate: 0.0016064007, weight_decay: 0.0000642560

epoch time: 350.4817199707s, input_fired_time: 3.7176840305, output_fired_time: 3.3920822144, propagate_forward_time: 131.4729919434, propagate_backward_time: 206.3804016113, weight_update_time: 0.5209302902, other_time: 5.5185546875
epoch time: 27.8096580505s, input_fired_time: 0.4631717205, output_fired_time: 0.0000000000, propagate_forward_time: 26.4349308014, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9115562439
validation[          , genome  9595] predictions:    9844/  10000 (98.44%), best:    9842/10000 (98.42%), error:       561.82910, best error:       516.26855 on epoch:    37, epoch:   41/100, mu: 0.9301776886, learning_rate: 0.0015260807, weight_decay: 0.0000610432

epoch time: 349.3797302246s, input_fired_time: 3.7181732655, output_fired_time: 3.3939721584, propagate_forward_time: 130.8279113770, propagate_backward_time: 205.9324340820, weight_update_time: 0.5237482786, other_time: 5.5072326660
epoch time: 27.3491115570s, input_fired_time: 0.4634434879, output_fired_time: 0.0000000000, propagate_forward_time: 25.9492855072, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9363822937
validation[          , genome  9595] predictions:    9837/  10000 (98.37%), best:    9842/10000 (98.42%), error:       546.86652, best error:       516.26855 on epoch:    37, epoch:   42/100, mu: 0.9331688285, learning_rate: 0.0014497766, weight_decay: 0.0000579911

epoch time: 348.3464965820s, input_fired_time: 3.7239053249, output_fired_time: 3.3892807961, propagate_forward_time: 130.2044219971, propagate_backward_time: 205.4351501465, weight_update_time: 0.5272315145, other_time: 5.5937347412
epoch time: 27.9326000214s, input_fired_time: 0.4732385576, output_fired_time: 0.0000000000, propagate_forward_time: 26.5076904297, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9516716003
validation[          , genome  9595] predictions:    9823/  10000 (98.23%), best:    9842/10000 (98.42%), error:       605.33594, best error:       516.26855 on epoch:    37, epoch:   43/100, mu: 0.9360103607, learning_rate: 0.0013772878, weight_decay: 0.0000550915

epoch time: 350.5271301270s, input_fired_time: 3.7153213024, output_fired_time: 3.3991243839, propagate_forward_time: 131.3928680420, propagate_backward_time: 206.5773010254, weight_update_time: 0.5229117274, other_time: 5.4424896240
epoch time: 27.3907585144s, input_fired_time: 0.4677678645, output_fired_time: 0.0000000000, propagate_forward_time: 25.9736328125, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9493579865
validation[          , genome  9595] predictions:    9822/  10000 (98.22%), best:    9842/10000 (98.42%), error:       614.08990, best error:       516.26855 on epoch:    37, epoch:   44/100, mu: 0.9387098551, learning_rate: 0.0013084234, weight_decay: 0.0000523369

epoch time: 349.1994323730s, input_fired_time: 3.7272770405, output_fired_time: 3.3957474232, propagate_forward_time: 130.7594757080, propagate_backward_time: 205.7538909912, weight_update_time: 0.5280035138, other_time: 5.5630493164
epoch time: 27.4461746216s, input_fired_time: 0.4679250717, output_fired_time: 0.0000000000, propagate_forward_time: 26.0471267700, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9311218262
validation[          , genome  9595] predictions:    9849/  10000 (98.49%), best:    9849/10000 (98.49%), error:       487.32907, best error:       487.32907 on epoch:    45, epoch:   45/100, mu: 0.9412743449, learning_rate: 0.0012430021, weight_decay: 0.0000497201

epoch time: 348.4278259277s, input_fired_time: 3.7134687901, output_fired_time: 3.3830566406, propagate_forward_time: 130.2463073730, propagate_backward_time: 205.5760192871, weight_update_time: 0.5241943002, other_time: 5.5089721680
epoch time: 27.8417243958s, input_fired_time: 0.4679664671, output_fired_time: 0.0000000000, propagate_forward_time: 26.4448585510, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9288997650
validation[          , genome  9595] predictions:    9849/  10000 (98.49%), best:    9849/10000 (98.49%), error:       537.93250, best error:       487.32907 on epoch:    45, epoch:   46/100, mu: 0.9437106252, learning_rate: 0.0011808521, weight_decay: 0.0000472341

epoch time: 350.8922424316s, input_fired_time: 3.7276680470, output_fired_time: 3.4057717323, propagate_forward_time: 131.5547027588, propagate_backward_time: 206.7021789551, weight_update_time: 0.5235567689, other_time: 5.5019378662
epoch time: 27.3088569641s, input_fired_time: 0.4685057402, output_fired_time: 0.0000000000, propagate_forward_time: 25.8981246948, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 0.9422264099
validation[          , genome  9595] predictions:    9828/  10000 (98.28%), best:    9849/10000 (98.49%), error:       620.45160, best error:       487.32907 on epoch:    45, epoch:   47/100, mu: 0.9460250735, learning_rate: 0.0011218095, weight_decay: 0.0000448724


</stderr_txt>
]]>