Nameexact_genome_1508499903_21_13769_2
Workunit2259216
Created3 Dec 2017, 6:50:25 UTC
Sent3 Dec 2017, 7:04:52 UTC
Report deadline13 Jan 2018, 5:59:07 UTC
Received5 Dec 2017, 2:18:45 UTC
Server stateOver
OutcomeSuccess
Client stateDone
Exit status0 (0x0)
Computer ID64142
Run time1 days 10 hours 35 min 50 sec
CPU time20 hours 53 min 43 sec
Validate stateValid
Credit10,238.59
Device peak FLOPS3.08 GFLOPS
Application versionEXACT Batch Norm With Scaled FMP CNN Trainer v0.34
Peak working set size171.18 MB
Peak swap size174.24 MB
Peak disk usage5.10 MB

Stderr output

<core_client_version>7.8.3</core_client_version>
<![CDATA[
<stderr_txt>
.0528421402, propagate_forward_time: 363.8505859375, propagate_backward_time: 576.7850952148, weight_update_time: 1.0069806576, other_time: 13.3106689453
epoch time: 76.5235900879s, input_fired_time: 1.1417378187, output_fired_time: 0.0000000000, propagate_forward_time: 72.7335739136, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6482772827
validation[          , genome 13769] predictions:    9868/  10000 (98.68%), best:    9876/10000 (98.76%), error:       435.49982, best error:       373.98804 on epoch:    50, epoch:   52/100, mu: 0.9559730291, learning_rate: 0.0008680347, weight_decay: 0.0000347214

epoch time: 964.1719360352s, input_fired_time: 12.1436309814, output_fired_time: 10.5705337524, propagate_forward_time: 357.8248291016, propagate_backward_time: 571.1658325195, weight_update_time: 1.4293889999, other_time: 12.4671020508
epoch time: 75.8515930176s, input_fired_time: 1.6420141459, output_fired_time: 0.0000000000, propagate_forward_time: 72.5576782227, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6519012451
validation[          , genome 13769] predictions:    9873/  10000 (98.73%), best:    9876/10000 (98.76%), error:       395.41034, best error:       373.98804 on epoch:    50, epoch:   53/100, mu: 0.9576743841, learning_rate: 0.0008246329, weight_decay: 0.0000329853

epoch time: 969.8624267578s, input_fired_time: 12.4895143509, output_fired_time: 11.3635158539, propagate_forward_time: 363.9963378906, propagate_backward_time: 569.4747314453, weight_update_time: 0.9283933640, other_time: 12.5383300781
epoch time: 75.9132614136s, input_fired_time: 1.3905410767, output_fired_time: 0.0000000000, propagate_forward_time: 72.3753356934, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1473846436
validation[          , genome 13769] predictions:    9873/  10000 (98.73%), best:    9876/10000 (98.76%), error:       379.30478, best error:       373.98804 on epoch:    50, epoch:   54/100, mu: 0.9592906833, learning_rate: 0.0007834013, weight_decay: 0.0000313361

epoch time: 961.6230468750s, input_fired_time: 12.6176385880, output_fired_time: 12.7374734879, propagate_forward_time: 360.3237915039, propagate_backward_time: 563.8142700195, weight_update_time: 0.6784595847, other_time: 12.1298828125
epoch time: 75.7290649414s, input_fired_time: 2.1433100700, output_fired_time: 0.0000000000, propagate_forward_time: 71.4249801636, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1607742310
validation[          , genome 13769] predictions:    9883/  10000 (98.83%), best:    9876/10000 (98.76%), error:       376.50760, best error:       373.98804 on epoch:    50, epoch:   55/100, mu: 0.9608261585, learning_rate: 0.0007442312, weight_decay: 0.0000297693

epoch time: 960.6406250000s, input_fired_time: 11.5889644623, output_fired_time: 10.7097988129, propagate_forward_time: 359.6601867676, propagate_backward_time: 566.2858276367, weight_update_time: 0.6781455278, other_time: 12.3958129883
epoch time: 75.1928405762s, input_fired_time: 1.3992818594, output_fired_time: 0.0000000000, propagate_forward_time: 72.1424179077, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6511383057
validation[          , genome 13769] predictions:    9886/  10000 (98.86%), best:    9886/10000 (98.86%), error:       361.83279, best error:       361.83279 on epoch:    56, epoch:   56/100, mu: 0.9622848630, learning_rate: 0.0007070196, weight_decay: 0.0000282808

epoch time: 917.3265991211s, input_fired_time: 12.5676822662, output_fired_time: 10.6298170090, propagate_forward_time: 338.8110961914, propagate_backward_time: 543.1607666016, weight_update_time: 0.9080543518, other_time: 12.1572265625
epoch time: 67.9002075195s, input_fired_time: 1.0614280701, output_fired_time: 0.0000000000, propagate_forward_time: 65.0194931030, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.8192825317
validation[          , genome 13769] predictions:    9864/  10000 (98.64%), best:    9886/10000 (98.86%), error:       437.13544, best error:       361.83279 on epoch:    56, epoch:   57/100, mu: 0.9636706114, learning_rate: 0.0006716687, weight_decay: 0.0000268668

epoch time: 874.9291381836s, input_fired_time: 9.8145618439, output_fired_time: 10.7666721344, propagate_forward_time: 322.7155456543, propagate_backward_time: 520.9709472656, weight_update_time: 1.1368129253, other_time: 10.6613769531
epoch time: 70.1983642578s, input_fired_time: 1.0836706161, output_fired_time: 0.0000000000, propagate_forward_time: 67.0167236328, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0979690552
validation[          , genome 13769] predictions:    9856/  10000 (98.56%), best:    9886/10000 (98.86%), error:       431.98364, best error:       361.83279 on epoch:    56, epoch:   58/100, mu: 0.9649870992, learning_rate: 0.0006380852, weight_decay: 0.0000255234

epoch time: 873.4266357422s, input_fired_time: 10.5686330795, output_fired_time: 11.2588748932, propagate_forward_time: 326.7556152344, propagate_backward_time: 512.8661499023, weight_update_time: 0.8931357861, other_time: 11.9774169922
epoch time: 67.3314132690s, input_fired_time: 1.8035587072, output_fired_time: 0.0000000000, propagate_forward_time: 62.9715042114, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.5563507080
validation[          , genome 13769] predictions:    9877/  10000 (98.77%), best:    9886/10000 (98.86%), error:       422.64600, best error:       361.83279 on epoch:    56, epoch:   59/100, mu: 0.9662377238, learning_rate: 0.0006061810, weight_decay: 0.0000242472

epoch time: 871.6846923828s, input_fired_time: 11.7931470871, output_fired_time: 10.7675523758, propagate_forward_time: 322.6249084473, propagate_backward_time: 513.5836181641, weight_update_time: 0.6373894811, other_time: 12.9154052734
epoch time: 68.2611160278s, input_fired_time: 1.5672966242, output_fired_time: 0.0000000000, propagate_forward_time: 63.8763122559, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8175048828
validation[          , genome 13769] predictions:    9873/  10000 (98.73%), best:    9886/10000 (98.86%), error:       412.84570, best error:       361.83279 on epoch:    56, epoch:   60/100, mu: 0.9674258232, learning_rate: 0.0005758719, weight_decay: 0.0000230349

epoch time: 826.5019531250s, input_fired_time: 10.2716941833, output_fired_time: 10.1260766983, propagate_forward_time: 301.3648071289, propagate_backward_time: 494.0888061523, weight_update_time: 0.6152488589, other_time: 10.6505737305
epoch time: 65.5120697021s, input_fired_time: 1.5424143076, output_fired_time: 0.0000000000, propagate_forward_time: 61.6812019348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.2884521484
validation[          , genome 13769] predictions:    9876/  10000 (98.76%), best:    9876/10000 (98.76%), error:       361.66315, best error:       361.66315 on epoch:    61, epoch:   61/100, mu: 0.9685545564, learning_rate: 0.0005470783, weight_decay: 0.0000218831

epoch time: 835.5396118164s, input_fired_time: 11.0733041763, output_fired_time: 9.4676418304, propagate_forward_time: 305.9829711914, propagate_backward_time: 497.5703125000, weight_update_time: 0.6177843213, other_time: 11.4453735352
epoch time: 62.5537338257s, input_fired_time: 1.5020313263, output_fired_time: 0.0000000000, propagate_forward_time: 59.2983093262, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7533950806
validation[          , genome 13769] predictions:    9864/  10000 (98.64%), best:    9876/10000 (98.76%), error:       408.18549, best error:       361.66315 on epoch:    61, epoch:   62/100, mu: 0.9696268439, learning_rate: 0.0005197244, weight_decay: 0.0000207890

epoch time: 766.9713745117s, input_fired_time: 11.6744089127, output_fired_time: 8.9025630951, propagate_forward_time: 281.9905700684, propagate_backward_time: 454.1029968262, weight_update_time: 0.5843073726, other_time: 10.3007812500
epoch time: 56.3502464294s, input_fired_time: 1.4327557087, output_fired_time: 0.0000000000, propagate_forward_time: 53.2613182068, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6561737061
validation[          , genome 13769] predictions:    9885/  10000 (98.85%), best:    9876/10000 (98.76%), error:       365.20111, best error:       361.66315 on epoch:    61, epoch:   63/100, mu: 0.9706454873, learning_rate: 0.0004937382, weight_decay: 0.0000197495

epoch time: 738.6849365234s, input_fired_time: 10.2274780273, output_fired_time: 7.1392822266, propagate_forward_time: 260.9893798828, propagate_backward_time: 450.4449462891, weight_update_time: 0.5700412393, other_time: 9.8838500977
epoch time: 61.7729988098s, input_fired_time: 1.4971009493, output_fired_time: 0.0000000000, propagate_forward_time: 58.2991104126, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.9767875671
validation[          , genome 13769] predictions:    9891/  10000 (98.91%), best:    9891/10000 (98.91%), error:       349.39658, best error:       349.39658 on epoch:    64, epoch:   64/100, mu: 0.9716132283, learning_rate: 0.0004690512, weight_decay: 0.0000187621

epoch time: 749.3938598633s, input_fired_time: 9.5639238358, output_fired_time: 6.9893550873, propagate_forward_time: 274.4902954102, propagate_backward_time: 446.4101257324, weight_update_time: 0.5757394433, other_time: 11.9401550293
epoch time: 51.7132072449s, input_fired_time: 0.8755795360, output_fired_time: 0.0000000000, propagate_forward_time: 49.2350196838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6026077271
validation[          , genome 13769] predictions:    9890/  10000 (98.90%), best:    9891/10000 (98.91%), error:       352.92313, best error:       349.39658 on epoch:    64, epoch:   65/100, mu: 0.9725325704, learning_rate: 0.0004455987, weight_decay: 0.0000178240

epoch time: 700.3082275391s, input_fired_time: 11.2751588821, output_fired_time: 7.8258008957, propagate_forward_time: 247.4541473389, propagate_backward_time: 422.5982666016, weight_update_time: 0.8012878895, other_time: 11.1548461914
epoch time: 44.5890121460s, input_fired_time: 1.0470416546, output_fired_time: 0.0000000000, propagate_forward_time: 41.7901496887, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.7518196106
validation[          , genome 13769] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       343.61972, best error:       343.61972 on epoch:    66, epoch:   66/100, mu: 0.9734059572, learning_rate: 0.0004233187, weight_decay: 0.0000169328

epoch time: 622.4913330078s, input_fired_time: 8.8075609207, output_fired_time: 7.4337635040, propagate_forward_time: 215.4871063232, propagate_backward_time: 381.3239440918, weight_update_time: 0.7623875141, other_time: 9.4389343262
epoch time: 45.3320198059s, input_fired_time: 1.0630623102, output_fired_time: 0.0000000000, propagate_forward_time: 42.7463874817, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5225715637
validation[          , genome 13769] predictions:    9866/  10000 (98.66%), best:    9892/10000 (98.92%), error:       401.46152, best error:       343.61972 on epoch:    66, epoch:   67/100, mu: 0.9742356539, learning_rate: 0.0004021528, weight_decay: 0.0000160861

epoch time: 668.3659667969s, input_fired_time: 8.3227014542, output_fired_time: 8.0649938583, propagate_forward_time: 242.6776428223, propagate_backward_time: 399.3206481934, weight_update_time: 0.5372223258, other_time: 9.9799804688
epoch time: 52.9670066833s, input_fired_time: 1.3917797804, output_fired_time: 0.0000000000, propagate_forward_time: 49.9645729065, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.6106529236
validation[          , genome 13769] predictions:    9875/  10000 (98.75%), best:    9892/10000 (98.92%), error:       375.45798, best error:       343.61972 on epoch:    66, epoch:   68/100, mu: 0.9750238657, learning_rate: 0.0003820452, weight_decay: 0.0000152818

epoch time: 672.3637695313s, input_fired_time: 9.1055488586, output_fired_time: 7.5998601913, propagate_forward_time: 237.8598937988, propagate_backward_time: 406.8070983887, weight_update_time: 0.5376781821, other_time: 10.9913940430
epoch time: 42.7207603455s, input_fired_time: 0.7791094780, output_fired_time: 0.0000000000, propagate_forward_time: 40.7071380615, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.2345123291
validation[          , genome 13769] predictions:    9879/  10000 (98.79%), best:    9892/10000 (98.92%), error:       354.75259, best error:       343.61972 on epoch:    66, epoch:   69/100, mu: 0.9757726789, learning_rate: 0.0003629429, weight_decay: 0.0000145177

epoch time: 647.0737915039s, input_fired_time: 10.4745092392, output_fired_time: 8.6488199234, propagate_forward_time: 233.4060211182, propagate_backward_time: 383.6796264648, weight_update_time: 0.7763031125, other_time: 10.8648681641
epoch time: 46.7025604248s, input_fired_time: 1.0722945929, output_fired_time: 0.0000000000, propagate_forward_time: 43.5957450867, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0345191956
validation[          , genome 13769] predictions:    9888/  10000 (98.88%), best:    9892/10000 (98.92%), error:       357.29111, best error:       343.61972 on epoch:    66, epoch:   70/100, mu: 0.9764840603, learning_rate: 0.0003447958, weight_decay: 0.0000137918

epoch time: 690.9887695313s, input_fired_time: 9.9746007919, output_fired_time: 8.2365732193, propagate_forward_time: 248.4456939697, propagate_backward_time: 413.9696655273, weight_update_time: 0.5486829877, other_time: 10.3622436523
epoch time: 55.2513198853s, input_fired_time: 1.9221215248, output_fired_time: 0.0000000000, propagate_forward_time: 51.1806678772, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.1485290527
validation[          , genome 13769] predictions:    9890/  10000 (98.90%), best:    9892/10000 (98.92%), error:       373.76138, best error:       343.61972 on epoch:    66, epoch:   71/100, mu: 0.9771598577, learning_rate: 0.0003275560, weight_decay: 0.0000131022

epoch time: 694.3492431641s, input_fired_time: 9.9988660812, output_fired_time: 8.3110237122, propagate_forward_time: 253.3273010254, propagate_backward_time: 412.6452331543, weight_update_time: 0.5442774892, other_time: 10.0668334961
epoch time: 52.7468833923s, input_fired_time: 0.8902106881, output_fired_time: 0.0000000000, propagate_forward_time: 50.4908599854, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.3658142090
validation[          , genome 13769] predictions:    9888/  10000 (98.88%), best:    9892/10000 (98.92%), error:       362.63007, best error:       343.61972 on epoch:    66, epoch:   72/100, mu: 0.9778018594, learning_rate: 0.0003111782, weight_decay: 0.0000124471

epoch time: 695.2966308594s, input_fired_time: 8.9996843338, output_fired_time: 8.0723218918, propagate_forward_time: 241.4530181885, propagate_backward_time: 427.1790771484, weight_update_time: 0.5461659431, other_time: 9.5925292969
epoch time: 52.4548606873s, input_fired_time: 1.3929241896, output_fired_time: 0.0000000000, propagate_forward_time: 49.4665603638, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 1.5953750610
validation[          , genome 13769] predictions:    9862/  10000 (98.62%), best:    9892/10000 (98.92%), error:       414.17429, best error:       343.61972 on epoch:    66, epoch:   73/100, mu: 0.9784117937, learning_rate: 0.0002956192, weight_decay: 0.0000118248

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769_2_r1554237706_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.978991
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000280838
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.12335e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 74
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 66
read number_validation_images: 10000
read best_validation_predictions: 9892
read best_validation_error: 343.62
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 13769
read normal distribution: '0 0.00225635175593197 1.14715433120728'
generator_str: '1484577045'
read generator: 1484577045
read generated_by_map:
1 merge_node 1
reading 71 nodes.
reading 226 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 1971.86743164063s, input_fired_time: 21.0596122741699, output_fired_time: 21.9189414978027, propagate_forward_time: 735.730834960938, propagate_backward_time: 1168.20593261719, weight_update_time: 1.49616241455078, other_time: 24.9521484375
epoch time: 184.210342407227s, input_fired_time: 5.67255210876465, output_fired_time: 0, propagate_forward_time: 173.204925537109, propagate_backward_time: 0, weight_update_time: 0, other_time: 5.33287048339844
validation[          , genome 13769] predictions:    9880/  10000 (98.80%), best:    9892/10000 (98.92%), error:       386.56396, best error:       343.61972 on epoch:    66, epoch:   74/100, mu: 0.9789912105, learning_rate: 0.0002808383, weight_decay: 0.0000112335

epoch time: 2589.8059082031s, input_fired_time: 33.1368331909, output_fired_time: 34.1259956360, propagate_forward_time: 959.1007690430, propagate_backward_time: 1531.2767333984, weight_update_time: 3.0681531429, other_time: 32.1656494141
epoch time: 211.2156677246s, input_fired_time: 4.0418090820, output_fired_time: 0.0000000000, propagate_forward_time: 199.1239166260, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 8.0499420166
validation[          , genome 13769] predictions:    9878/  10000 (98.78%), best:    9892/10000 (98.92%), error:       387.93964, best error:       343.61972 on epoch:    66, epoch:   75/100, mu: 0.9795416594, learning_rate: 0.0002667964, weight_decay: 0.0000106719

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769_2_r1554237706_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.980065
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000253457
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 1.01383e-05
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 76
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 66
read number_validation_images: 10000
read best_validation_predictions: 9892
read best_validation_error: 343.62
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 13769
read normal distribution: '0 0.00225635175593197 1.14715433120728'
generator_str: '1479099891'
read generator: 1479099891
read generated_by_map:
1 merge_node 1
reading 71 nodes.
reading 226 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 1410.8427734375s, input_fired_time: 16.5029163360596, output_fired_time: 15.7922286987305, propagate_forward_time: 525.050415039063, propagate_backward_time: 836.887939453125, weight_update_time: 1.83856534957886, other_time: 16.6092529296875
epoch time: 110.466987609863s, input_fired_time: 1.74315059185028, output_fired_time: 0, propagate_forward_time: 105.330451965332, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.39338684082031
validation[          , genome 13769] predictions:    9873/  10000 (98.73%), best:    9892/10000 (98.92%), error:       417.93869, best error:       343.61972 on epoch:    66, epoch:   76/100, mu: 0.9800645709, learning_rate: 0.0002534565, weight_decay: 0.0000101383

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769_2_r1554237706_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.980561
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000240784
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 9.63135e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 77
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 66
read number_validation_images: 10000
read best_validation_predictions: 9892
read best_validation_error: 343.62
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 13769
read normal distribution: '0 0.00225635175593197 1.14715433120728'
generator_str: '1126354397'
read generator: 1126354397
read generated_by_map:
1 merge_node 1
reading 71 nodes.
reading 226 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 1456.06604003906s, input_fired_time: 19.2629318237305, output_fired_time: 15.4309387207031, propagate_forward_time: 543.107421875, propagate_backward_time: 858.674438476563, weight_update_time: 1.5729855298996, other_time: 19.59033203125
epoch time: 103.964828491211s, input_fired_time: 2.67205381393433, output_fired_time: 0, propagate_forward_time: 98.1432723999023, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.14950561523438
validation[          , genome 13769] predictions:    9883/  10000 (98.83%), best:    9892/10000 (98.92%), error:       384.66080, best error:       343.61972 on epoch:    66, epoch:   77/100, mu: 0.9805613160, learning_rate: 0.0002407837, weight_decay: 0.0000096314

epoch time: 1317.8305664063s, input_fired_time: 17.1131191254, output_fired_time: 14.0964822769, propagate_forward_time: 497.7630615234, propagate_backward_time: 773.2873535156, weight_update_time: 1.1792355776, other_time: 15.5705566406
epoch time: 99.4764480591s, input_fired_time: 1.9360939264, output_fired_time: 0.0000000000, propagate_forward_time: 92.2244262695, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.3159255981
validation[          , genome 13769] predictions:    9892/  10000 (98.92%), best:    9892/10000 (98.92%), error:       354.34415, best error:       343.61972 on epoch:    66, epoch:   78/100, mu: 0.9810332656, learning_rate: 0.0002287445, weight_decay: 0.0000091498

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769_2_r1554237706_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.981482
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000217307
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 8.69229e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 79
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 66
read number_validation_images: 10000
read best_validation_predictions: 9892
read best_validation_error: 343.62
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 13769
read normal distribution: '0 0.00225635175593197 1.14715433120728'
generator_str: '739298881'
read generator: 739298881
read generated_by_map:
1 merge_node 1
reading 71 nodes.
reading 226 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 1745.83728027344s, input_fired_time: 18.9613437652588, output_fired_time: 19.5191230773926, propagate_forward_time: 654.900268554688, propagate_backward_time: 1029.5966796875, weight_update_time: 1.53178262710571, other_time: 22.85986328125
epoch time: 152.101806640625s, input_fired_time: 2.67223334312439, output_fired_time: 0, propagate_forward_time: 145.576583862305, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.85299682617188
validation[          , genome 13769] predictions:    9890/  10000 (98.90%), best:    9892/10000 (98.92%), error:       366.72339, best error:       343.61972 on epoch:    66, epoch:   79/100, mu: 0.9814816117, learning_rate: 0.0002173073, weight_decay: 0.0000086923

epoch time: 1950.9895019531s, input_fired_time: 27.1264972687, output_fired_time: 23.0805778503, propagate_forward_time: 723.3449707031, propagate_backward_time: 1151.3179931641, weight_update_time: 1.9316529036, other_time: 26.1195068359
epoch time: 147.0435333252s, input_fired_time: 3.5690510273, output_fired_time: 0.0000000000, propagate_forward_time: 140.0487518311, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.4257354736
validation[          , genome 13769] predictions:    9886/  10000 (98.86%), best:    9892/10000 (98.92%), error:       354.86765, best error:       343.61972 on epoch:    66, epoch:   80/100, mu: 0.9819075465, learning_rate: 0.0002064419, weight_decay: 0.0000082577

epoch time: 1705.3978271484s, input_fired_time: 20.5815048218, output_fired_time: 20.3774185181, propagate_forward_time: 640.5121459961, propagate_backward_time: 1004.9987792969, weight_update_time: 1.6788476706, other_time: 18.9279174805
epoch time: 125.7149963379s, input_fired_time: 2.7229132652, output_fired_time: 0.0000000000, propagate_forward_time: 119.6322174072, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.3598632813
validation[          , genome 13769] predictions:    9884/  10000 (98.84%), best:    9892/10000 (98.92%), error:       371.57764, best error:       343.61972 on epoch:    66, epoch:   81/100, mu: 0.9823121428, learning_rate: 0.0001961198, weight_decay: 0.0000078448

epoch time: 1276.5560302734s, input_fired_time: 15.5523786545, output_fired_time: 16.5955219269, propagate_forward_time: 475.2770080566, propagate_backward_time: 752.6628417969, weight_update_time: 1.3649708033, other_time: 16.4682617188
epoch time: 83.9216003418s, input_fired_time: 1.7317887545, output_fired_time: 0.0000000000, propagate_forward_time: 80.1062240601, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.0835876465
validation[          , genome 13769] predictions:    9883/  10000 (98.83%), best:    9892/10000 (98.92%), error:       361.71448, best error:       343.61972 on epoch:    66, epoch:   82/100, mu: 0.9826965332, learning_rate: 0.0001863138, weight_decay: 0.0000074526

epoch time: 1241.6678466797s, input_fired_time: 14.8370761871, output_fired_time: 13.8902025223, propagate_forward_time: 462.0949707031, propagate_backward_time: 734.6273193359, weight_update_time: 1.7022418976, other_time: 16.2182617188
epoch time: 105.0804138184s, input_fired_time: 2.0868675709, output_fired_time: 0.0000000000, propagate_forward_time: 99.2514266968, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7421188354
validation[          , genome 13769] predictions:    9881/  10000 (98.81%), best:    9892/10000 (98.92%), error:       391.99066, best error:       343.61972 on epoch:    66, epoch:   83/100, mu: 0.9830617309, learning_rate: 0.0001769981, weight_decay: 0.0000070799

epoch time: 1352.7772216797s, input_fired_time: 16.9884376526, output_fired_time: 16.5920944214, propagate_forward_time: 500.7114868164, propagate_backward_time: 801.5189208984, weight_update_time: 1.3972686529, other_time: 16.9663696289
epoch time: 111.9398422241s, input_fired_time: 2.0983474255, output_fired_time: 0.0000000000, propagate_forward_time: 107.4187164307, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.4227752686
validation[          , genome 13769] predictions:    9884/  10000 (98.84%), best:    9892/10000 (98.92%), error:       354.94138, best error:       343.61972 on epoch:    66, epoch:   84/100, mu: 0.9834086299, learning_rate: 0.0001681482, weight_decay: 0.0000067259

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769_2_r1554237706_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.983738
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000159741
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 6.38963e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 85
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 66
read number_validation_images: 10000
read best_validation_predictions: 9892
read best_validation_error: 343.62
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 13769
read normal distribution: '0 0.00225635175593197 1.14715433120728'
generator_str: '559128189'
read generator: 559128189
read generated_by_map:
1 merge_node 1
reading 71 nodes.
reading 226 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 1712.6826171875s, input_fired_time: 21.2805957794189, output_fired_time: 20.61061668396, propagate_forward_time: 641.697082519531, propagate_backward_time: 1008.19384765625, weight_update_time: 1.81984400749207, other_time: 20.900390625
epoch time: 147.697647094727s, input_fired_time: 2.77608466148376, output_fired_time: 0, propagate_forward_time: 140.862747192383, propagate_backward_time: 0, weight_update_time: 0, other_time: 4.05882263183594
validation[          , genome 13769] predictions:    9886/  10000 (98.86%), best:    9892/10000 (98.92%), error:       374.56863, best error:       343.61972 on epoch:    66, epoch:   85/100, mu: 0.9837381840, learning_rate: 0.0001597408, weight_decay: 0.0000063896

arguments:
	'projects/csgrid.org_csg/exact_client_0.34_windows_x86_64.exe'
	'--training_file'
	'training_samples.bin'
	'--validation_file'
	'validation_samples.bin'
	'--testing_file'
	'testing_samples.bin'
	'--genome_file'
	'input_genome.txt'
	'--output_file'
	'output_genome.txt'
	'--checkpoint_file'
	'checkpoint.txt'
converting arguments to vector
boincified training filename: '../../projects/csgrid.org_csg/mnist_training_50k.bin'
boincified validation filename: '../../projects/csgrid.org_csg/mnist_validation_10k.bin'
boincified testing filename: '../../projects/csgrid.org_csg/mnist_testing_new.bin'
boincified genome filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769.txt'
boincified output filename: '../../projects/csgrid.org_csg/exact_genome_1508499903_21_13769_2_r1554237706_0'
boincified checkpoint filename: 'checkpoint.txt'
loading genome
starting from checkpoint file: 'checkpoint.txt'
read CNN_Genome file with version string: 'v0.33'
read exact_id: -1
read genome_id: -1
read initial_mu: 0.5
read mu: 0.984051
read mu_delta: 0.95
read initial_learning_rate: 0.0125
read learning_rate: 0.000151754
read learning_rate_delta: 0.95
read initial_weight_decay: 0.0005
read weight_decay: 6.07015e-06
read weight_decay_delta: 0.95
read batch_size: 50
read epsilon: 1e-07
read alpha: 0.1
read input_dropout_probability: 0
read hidden_dropout_probability: 0
read velocity_reset: 1000
read epoch: 86
read max_epochs: 100
read reset_weights: 0
read padding: 2
read best_epoch: 66
read number_validation_images: 10000
read best_validation_predictions: 9892
read best_validation_error: 343.62
read number_training_images: 50000
read training_predictions: 0
read training_error: 1e+07
read number_test_images: 10000
read test_predictions: 0
read test_error: 1e+07
read generation_id: 13769
read normal distribution: '0 0.00225635175593197 1.14715433120728'
generator_str: '1718904938'
read generator: 1718904938
read generated_by_map:
1 merge_node 1
reading 71 nodes.
reading 226 edges.
number input nodes: 1
number softmax nodes: 10
order_size: 50000
parsed input file
loaded genome
loading images
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 4923 images.
reading image set with 5742 images.
reading image set with 4958 images.
reading image set with 5131 images.
reading image set with 4842 images.
reading image set with 4421 images.
reading image set with 4918 images.
reading image set with 5265 images.
reading image set with 4851 images.
reading image set with 4949 images.
image_size: 1x28x28 = 784
read 50000 images.
    class    0: 4923
    class    1: 5742
    class    2: 4958
    class    3: 5131
    class    4: 4842
    class    5: 4421
    class    6: 4918
    class    7: 5265
    class    8: 4851
    class    9: 4949
calculating averages and standard deviations for images
average pixel value for channel 0: 0.130530551075935
pixel variance for channel 0: 0.0948646068572998
pixel standard deviation for channel 0: 0.308000981807709
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
reading image set with 1000 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 1000
    class    1: 1000
    class    2: 1000
    class    3: 1000
    class    4: 1000
    class    5: 1000
    class    6: 1000
    class    7: 1000
    class    8: 1000
    class    9: 1000
number_classes: 10
channels: 1
width: 28
height: 28
reading image set with 980 images.
reading image set with 1135 images.
reading image set with 1032 images.
reading image set with 1010 images.
reading image set with 982 images.
reading image set with 892 images.
reading image set with 958 images.
reading image set with 1028 images.
reading image set with 974 images.
reading image set with 1009 images.
image_size: 1x28x28 = 784
read 10000 images.
    class    0: 980
    class    1: 1135
    class    2: 1032
    class    3: 1010
    class    4: 982
    class    5: 892
    class    6: 958
    class    7: 1028
    class    8: 974
    class    9: 1009
loaded images
starting backpropagation!
epoch time: 1804.28430175781s, input_fired_time: 22.3324069976807, output_fired_time: 22.8468151092529, propagate_forward_time: 666.770935058594, propagate_backward_time: 1067.81079101563, weight_update_time: 1.00697946548462, other_time: 24.5234375
epoch time: 124.534324645996s, input_fired_time: 2.21914577484131, output_fired_time: 0, propagate_forward_time: 118.614585876465, propagate_backward_time: 0, weight_update_time: 0, other_time: 3.70059204101563
validation[          , genome 13769] predictions:    9893/  10000 (98.93%), best:    9893/10000 (98.93%), error:       342.87149, best error:       342.87149 on epoch:    86, epoch:   86/100, mu: 0.9840512872, learning_rate: 0.0001517538, weight_decay: 0.0000060702

epoch time: 1547.3343505859s, input_fired_time: 19.9801883698, output_fired_time: 18.5297451019, propagate_forward_time: 580.3991088867, propagate_backward_time: 910.4996948242, weight_update_time: 1.7084878683, other_time: 17.9255371094
epoch time: 126.2080078125s, input_fired_time: 2.5283725262, output_fired_time: 0.0000000000, propagate_forward_time: 119.7614135742, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9182205200
validation[          , genome 13769] predictions:    9891/  10000 (98.91%), best:    9893/10000 (98.93%), error:       358.64908, best error:       342.87149 on epoch:    86, epoch:   87/100, mu: 0.9843487144, learning_rate: 0.0001441661, weight_decay: 0.0000057666

epoch time: 1612.9151611328s, input_fired_time: 23.1542415619, output_fired_time: 18.6253356934, propagate_forward_time: 597.9707031250, propagate_backward_time: 954.1220703125, weight_update_time: 0.9910928011, other_time: 19.0427246094
epoch time: 129.4116668701s, input_fired_time: 2.5733404160, output_fired_time: 0.0000000000, propagate_forward_time: 123.2433013916, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5950241089
validation[          , genome 13769] predictions:    9887/  10000 (98.87%), best:    9893/10000 (98.93%), error:       355.76611, best error:       342.87149 on epoch:    86, epoch:   88/100, mu: 0.9846313000, learning_rate: 0.0001369578, weight_decay: 0.0000054783

epoch time: 1608.9396972656s, input_fired_time: 23.6153240204, output_fired_time: 20.0996341705, propagate_forward_time: 599.2068481445, propagate_backward_time: 944.6260375977, weight_update_time: 2.2722110748, other_time: 21.3918457031
epoch time: 132.1771545410s, input_fired_time: 2.5469810963, output_fired_time: 0.0000000000, propagate_forward_time: 125.6137619019, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0164108276
validation[          , genome 13769] predictions:    9885/  10000 (98.85%), best:    9893/10000 (98.93%), error:       363.07294, best error:       342.87149 on epoch:    86, epoch:   89/100, mu: 0.9848997593, learning_rate: 0.0001301099, weight_decay: 0.0000052044

epoch time: 1582.8394775391s, input_fired_time: 21.5686988831, output_fired_time: 18.9365768433, propagate_forward_time: 583.3364868164, propagate_backward_time: 938.0449218750, weight_update_time: 1.5381805897, other_time: 20.9528198242
epoch time: 116.7875137329s, input_fired_time: 2.1401624680, output_fired_time: 0.0000000000, propagate_forward_time: 110.3575820923, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.2897720337
validation[          , genome 13769] predictions:    9893/  10000 (98.93%), best:    9893/10000 (98.93%), error:       365.16821, best error:       342.87149 on epoch:    86, epoch:   90/100, mu: 0.9851547480, learning_rate: 0.0001236044, weight_decay: 0.0000049442

epoch time: 1632.2830810547s, input_fired_time: 18.8528060913, output_fired_time: 19.7596874237, propagate_forward_time: 607.0318603516, propagate_backward_time: 965.6179199219, weight_update_time: 1.4594335556, other_time: 21.0208740234
epoch time: 137.6021728516s, input_fired_time: 2.4571852684, output_fired_time: 0.0000000000, propagate_forward_time: 129.8992614746, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 5.2457275391
validation[          , genome 13769] predictions:    9870/  10000 (98.70%), best:    9893/10000 (98.93%), error:       390.69873, best error:       342.87149 on epoch:    86, epoch:   91/100, mu: 0.9853969812, learning_rate: 0.0001174242, weight_decay: 0.0000046970

epoch time: 1664.7719726563s, input_fired_time: 21.3397178650, output_fired_time: 19.7371883392, propagate_forward_time: 619.6378784180, propagate_backward_time: 980.7363891602, weight_update_time: 1.6201782227, other_time: 23.3208007813
epoch time: 160.7356109619s, input_fired_time: 3.7284641266, output_fired_time: 0.0000000000, propagate_forward_time: 153.2294006348, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.7777404785
validation[          , genome 13769] predictions:    9886/  10000 (98.86%), best:    9886/10000 (98.86%), error:       342.66592, best error:       342.66592 on epoch:    92, epoch:   92/100, mu: 0.9856271148, learning_rate: 0.0001115530, weight_decay: 0.0000044621

epoch time: 2204.8308105469s, input_fired_time: 27.3910408020, output_fired_time: 28.2848720551, propagate_forward_time: 816.9595336914, propagate_backward_time: 1305.0134277344, weight_update_time: 2.7101302147, other_time: 27.1818847656
epoch time: 140.7248229980s, input_fired_time: 2.8029153347, output_fired_time: 0.0000000000, propagate_forward_time: 134.1050109863, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.8168945313
validation[          , genome 13769] predictions:    9883/  10000 (98.83%), best:    9886/10000 (98.86%), error:       362.48392, best error:       342.66592 on epoch:    92, epoch:   93/100, mu: 0.9858457446, learning_rate: 0.0001059753, weight_decay: 0.0000042390

epoch time: 1768.0540771484s, input_fired_time: 24.5390911102, output_fired_time: 19.0875091553, propagate_forward_time: 664.7391357422, propagate_backward_time: 1038.1500244141, weight_update_time: 1.5818114281, other_time: 21.5383300781
epoch time: 133.6254119873s, input_fired_time: 2.5531566143, output_fired_time: 0.0000000000, propagate_forward_time: 127.0334930420, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 4.0387573242
validation[          , genome 13769] predictions:    9890/  10000 (98.90%), best:    9886/10000 (98.86%), error:       363.18549, best error:       342.66592 on epoch:    92, epoch:   94/100, mu: 0.9860534668, learning_rate: 0.0001006766, weight_decay: 0.0000040271

epoch time: 1549.9357910156s, input_fired_time: 17.5174579620, output_fired_time: 19.8565769196, propagate_forward_time: 589.1262207031, propagate_backward_time: 902.2919921875, weight_update_time: 1.4293936491, other_time: 21.1435546875
epoch time: 109.5431518555s, input_fired_time: 1.6732307673, output_fired_time: 0.0000000000, propagate_forward_time: 103.9288940430, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.9410247803
validation[          , genome 13769] predictions:    9892/  10000 (98.92%), best:    9886/10000 (98.86%), error:       357.39633, best error:       342.66592 on epoch:    92, epoch:   95/100, mu: 0.9862508178, learning_rate: 0.0000956427, weight_decay: 0.0000038257

epoch time: 1316.6280517578s, input_fired_time: 15.7406015396, output_fired_time: 15.5186147690, propagate_forward_time: 490.3725585938, propagate_backward_time: 777.8748168945, weight_update_time: 1.3983287811, other_time: 17.1215209961
epoch time: 100.1359786987s, input_fired_time: 2.2825002670, output_fired_time: 0.0000000000, propagate_forward_time: 94.9609680176, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8925094604
validation[          , genome 13769] predictions:    9882/  10000 (98.82%), best:    9886/10000 (98.86%), error:       382.67819, best error:       342.66592 on epoch:    92, epoch:   96/100, mu: 0.9864382744, learning_rate: 0.0000908606, weight_decay: 0.0000036344

epoch time: 1255.2637939453s, input_fired_time: 16.3222045898, output_fired_time: 13.8540363312, propagate_forward_time: 474.0350952148, propagate_backward_time: 735.0905151367, weight_update_time: 1.4295388460, other_time: 15.9620361328
epoch time: 96.2416534424s, input_fired_time: 1.9473464489, output_fired_time: 0.0000000000, propagate_forward_time: 91.3860015869, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.9083023071
validation[          , genome 13769] predictions:    9892/  10000 (98.92%), best:    9886/10000 (98.86%), error:       345.38431, best error:       342.66592 on epoch:    92, epoch:   97/100, mu: 0.9866163731, learning_rate: 0.0000863176, weight_decay: 0.0000034527

epoch time: 1176.8643798828s, input_fired_time: 14.5406723022, output_fired_time: 14.4642562866, propagate_forward_time: 441.3783569336, propagate_backward_time: 689.6791992188, weight_update_time: 0.9592146277, other_time: 16.8019409180
epoch time: 85.3238372803s, input_fired_time: 1.2225232124, output_fired_time: 0.0000000000, propagate_forward_time: 81.4772262573, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.6240844727
validation[          , genome 13769] predictions:    9888/  10000 (98.88%), best:    9886/10000 (98.86%), error:       350.48868, best error:       342.66592 on epoch:    92, epoch:   98/100, mu: 0.9867855310, learning_rate: 0.0000820017, weight_decay: 0.0000032801

epoch time: 1388.5792236328s, input_fired_time: 18.9985332489, output_fired_time: 16.1164512634, propagate_forward_time: 510.4414062500, propagate_backward_time: 824.6610717773, weight_update_time: 1.3198742867, other_time: 18.3617553711
epoch time: 117.5430755615s, input_fired_time: 2.3119859695, output_fired_time: 0.0000000000, propagate_forward_time: 111.6860733032, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 3.5450134277
validation[          , genome 13769] predictions:    9895/  10000 (98.95%), best:    9895/10000 (98.95%), error:       338.82526, best error:       338.82526 on epoch:    99, epoch:   99/100, mu: 0.9869462252, learning_rate: 0.0000779016, weight_decay: 0.0000031161

epoch time: 1425.8837890625s, input_fired_time: 18.4897365570, output_fired_time: 16.2369899750, propagate_forward_time: 525.3551635742, propagate_backward_time: 848.9035034180, weight_update_time: 1.4644708633, other_time: 16.8984375000
epoch time: 113.3852386475s, input_fired_time: 1.9800808430, output_fired_time: 0.0000000000, propagate_forward_time: 108.6894683838, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.7156906128
validation[          , genome 13769] predictions:    9888/  10000 (98.88%), best:    9888/10000 (98.88%), error:       335.45731, best error:       335.45731 on epoch:   100, epoch:  100/100, mu: 0.9870989323, learning_rate: 0.0000740065, weight_decay: 0.0000029603

evaluating best weights on full training data.
evaluting training set with running mean/variance:
epoch time: 542.9287109375s, input_fired_time: 12.1365909576, output_fired_time: 0.0000000000, propagate_forward_time: 515.5260620117, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 15.2660522461
best training[          , genome 13769] predictions:   49999/  50000 (100.00%), best:    9888/10000 (98.88%), error:       142.54543, best error:       335.45731 on epoch:   100, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
evaluating best weights on test data.
evaluting test set with running mean/variance:
epoch time: 101.3003997803s, input_fired_time: 2.3407835960, output_fired_time: 0.0000000000, propagate_forward_time: 96.1510467529, propagate_backward_time: 0.0000000000, weight_update_time: 0.0000000000, other_time: 2.8085708618
   testing[          , genome 13769] predictions:    9892/  10000 (98.92%), best:    9888/10000 (98.88%), error:       331.70551, best error:       335.45731 on epoch:   100, epoch:  101/100, mu: 0.9872440100, learning_rate: 0.0000703062, weight_decay: 0.0000028122
backpropagation finished successfully!
03:18:21 (8592): called boinc_finish(0)

</stderr_txt>
]]>