- This topic has 10 replies, 3 voices, and was last updated 8 years, 3 months ago by
.
-
It’s also worth mentioning that with the TrueRNGPro, you can turn off the whitening algorithm. This gives you access to the raw random numbers before any xoring or merging of samples is done.
Typically the reason you see a even distribution is due to the whitening. The raw values (if they are of interest to you) produce more of a bell curve. Which when plotted — look like a bell. The center of the bell being the most common values, and the outlayers 0 and 255 being the least common number. Some of our customers are more interested in this or implement their own whitening algorithm using this data. This is more in line with your flipping a coin analogy and getting heads more times in a row than you do from a random number generator.
As for the ‘un-generatable’ numbers, I’m still a little confused because if you check the distribution of a sample set, you will see that all bytes have a pretty even distribution, if you count the frequency of each byte (which you can do with “ent -c” in linux) . If you were to plot the count of each byte, they will be within the same order of magnitude of each other, which means all numbers are generated at about the same frequency.
I hope this helps further answer your question.
Thanks for the details! Again, I may just be putting my ignorance on display and I appreciate the info and discussion. I think your comment about whitening vs. raw and the bell curve is likely what I have in mind. If you don’t mind clarification discussion:
I certainly understand for physical processes generating unbiased bits we are absolutely good. And I haven’t gotten to play with TrueRNG yet, so this is purely a theoretical understanding question from looking at PRNG and random.org articles and data in the past, and wondering if it applies to something like TrueRNGPro.
When articles talk about using sound sources or random.org’s atmospheric samples, they talk about adjusting the A->D 0/1 bit conversion levels range based on a running window analyzing bit deviation from 50%, so input data doesn’t drift toward all zeros or ones because of input amplitude changes. But isn’t an artifact of that reducing the chance to generate “honest” runs of such bits? Similar considerations appear with the selection of PRNG algorithm details. Maybe this doesn’t apply to all physical methods of generating bits?
(Trivial example of my fear/understanding: a heuristic A->D window or PRNG algorithm forced to produce exactly 50% 0::1 ratio across 4 bits would be bad! That forces the random values to be: 1100, 1010, 1001, 0101, 0110, 0011. Only 6 values out of 16 allowed by algorithm, and neither the “interesting for testing purposes” extremes of 0000 or 1111 included!)
So I was under the impression that an artifact of heuristic tuning is the longer runs of bits are made even less likely than the purely statistical likelihood of a 50% per coin-flip run. In another example poor quality algorithm, perhaps generating 8 zero bits in a row is a perfect 1 in 2^8 likelihood. But generating 16 zero bits in a row is only half as likely as a perfect 1 in 2^16, and the algorithm will never allow the generation 32 zero bits in a row.
I also thought this was acceptable and on purpose because of the use in cryptography – all zero bytes are often unwanted, much less 4 or 8 of them in a row.
Even assuming I’m not just ignorant of the real details, possible I am down some weird thought path and need my thinking corrected? Perhaps the statistical degradation involved don’t kick in until 2^billion or something, and the analysis window makes sure we have appropriate approximately 50% bit distribution over a megabyte, not something short and bad like a window of 64 bits?
Again, this question in my brain was around randomly testing inputs in a 2^32 or 2^64 input space and wondering if the numbers really were equally possible or if binary numbers with “long” runs of same bits in the larger space were less likely. (ex: is 0x0000001 w/ run of 27 0s really as likely as numbers like 0xa5a5a5a5 with runs of only 2)? Easily solved for testing purposes by randomly/exhaustively testing ranges around interesting numbers like zero, but piqued my curiosity about generating randomness.
Yes, the outputs that you get from a TrueRNG/TrueRNGpro running in normal mode are equally probable.
We use the avalanche effect in a semiconductor junction in two generators which are captured by 10 bit ADCs. The ADC bias is dealt with using an algorithm that maximizes the entropy and “spreads” the bias across the entire sample.
Each 10 bit ADC sample from each generator has about 6.5 – 7 bits of Shannon entropy. These are combined using an entropy mixing algorithm (multiplication in a galois field — similar to a CRC) to concentrate the entropy. This takes in eight 10-bit samples and outputs four 8 bit bytes. So we are taking in 52 to 56 bits of entropy and mixing them to saturate the entropy across all 32 output bits (i.e. maximizing entropy). This is then XOR’d with a whitening sequence.
So, yes, the probability of getting each byte is equal. The reality of a capture will show some variation among the bytes actually captured. Similarly, each 32 bit word will be equally likely. In a pseudo random generator, the algorithm usually guarantees that each symbol will appear an equal number of times in a finite sequence. The issue is that if a generator has a period of 256 bytes (small number for the sake of this argument) and you have 255 bytes output, then you know exactly what the next byte will be. It may be harder to guess if you have a longer period but the problem is the same.
Here is the frequency count output of the ent tool ran on a 1MB capture from a TrueRNGpro (running under Linux on a Raspberry PI 3). The actual counts of each byte vary between runs with no discernible pattern. You can see below that the occurance of each byte is about 4000 counts (1024000 / 256).
root@raspberrypi:/home/pi/workspace/TrueRNG_Windows_Python# ent -c test.bin Value Char Occurrences Fraction 0 4113 0.004017 1 3918 0.003826 2 3993 0.003899 3 4072 0.003977 4 4071 0.003976 5 3983 0.003890 6 3870 0.003779 7 4044 0.003949 8 4129 0.004032 9 4022 0.003928 10 4021 0.003927 11 3928 0.003836 12 4006 0.003912 13 3912 0.003820 14 3954 0.003861 15 4035 0.003940 16 4073 0.003978 17 3925 0.003833 18 3972 0.003879 19 4058 0.003963 20 3942 0.003850 21 3945 0.003853 22 4015 0.003921 23 4038 0.003943 24 3887 0.003796 25 3893 0.003802 26 4090 0.003994 27 4048 0.003953 28 3978 0.003885 29 3931 0.003839 30 4011 0.003917 31 3965 0.003872 32 4002 0.003908 33 ! 4045 0.003950 34 " 4002 0.003908 35 # 4084 0.003988 36 $ 3904 0.003812 37 % 4023 0.003929 38 & 4034 0.003939 39 ' 3877 0.003786 40 ( 3913 0.003821 41 ) 3909 0.003817 42 * 4041 0.003946 43 + 3967 0.003874 44 , 4036 0.003941 45 - 3967 0.003874 46 . 4022 0.003928 47 / 4031 0.003937 48 0 3994 0.003900 49 1 4042 0.003947 50 2 3980 0.003887 51 3 3960 0.003867 52 4 4092 0.003996 53 5 3861 0.003771 54 6 3939 0.003847 55 7 3949 0.003856 56 8 4021 0.003927 57 9 4035 0.003940 58 : 4040 0.003945 59 ; 4123 0.004026 60 < 3997 0.003903 61 = 3977 0.003884 62 > 4053 0.003958 63 ? 3957 0.003864 64 @ 4103 0.004007 65 A 3928 0.003836 66 B 4016 0.003922 67 C 4051 0.003956 68 D 3935 0.003843 69 E 3950 0.003857 70 F 4024 0.003930 71 G 3997 0.003903 72 H 4058 0.003963 73 I 3948 0.003855 74 J 3928 0.003836 75 K 4133 0.004036 76 L 4053 0.003958 77 M 3976 0.003883 78 N 4070 0.003975 79 O 4068 0.003973 80 P 4037 0.003942 81 Q 3994 0.003900 82 R 4010 0.003916 83 S 3981 0.003888 84 T 4114 0.004018 85 U 3960 0.003867 86 V 3940 0.003848 87 W 4020 0.003926 88 X 4056 0.003961 89 Y 3937 0.003845 90 Z 3946 0.003854 91 [ 4105 0.004009 92 \ 4050 0.003955 93 ] 4044 0.003949 94 ^ 4076 0.003980 95 _ 3870 0.003779 96 X 3955 0.003862 97 a 3982 0.003889 98 b 3935 0.003843 99 c 3912 0.003820 100 d 3962 0.003869 101 e 4069 0.003974 102 f 3989 0.003896 103 g 3963 0.003870 104 h 3904 0.003812 105 i 4014 0.003920 106 j 3970 0.003877 107 k 4054 0.003959 108 l 4016 0.003922 109 m 3977 0.003884 110 n 3986 0.003893 111 o 4012 0.003918 112 p 3926 0.003834 113 q 4030 0.003936 114 r 4019 0.003925 115 s 4011 0.003917 116 t 3890 0.003799 117 u 4036 0.003941 118 v 3897 0.003806 119 w 3955 0.003862 120 x 4006 0.003912 121 y 3988 0.003895 122 z 3907 0.003815 123 { 4042 0.003947 124 | 4032 0.003938 125 } 4044 0.003949 126 ~ 3921 0.003829 127 4000 0.003906 128 4032 0.003938 129 4059 0.003964 130 4109 0.004013 131 3932 0.003840 132 3992 0.003898 133 3981 0.003888 134 4022 0.003928 135 4028 0.003934 136 4066 0.003971 137 3983 0.003890 138 3941 0.003849 139 3957 0.003864 140 3886 0.003795 141 4041 0.003946 142 3957 0.003864 143 3986 0.003893 144 3898 0.003807 145 4110 0.004014 146 3935 0.003843 147 4010 0.003916 148 4094 0.003998 149 3938 0.003846 150 3952 0.003859 151 3970 0.003877 152 3974 0.003881 153 4013 0.003919 154 3971 0.003878 155 4021 0.003927 156 4064 0.003969 157 4061 0.003966 158 4138 0.004041 159 4004 0.003910 160 4145 0.004048 161 â–’ 4006 0.003912 162 â–’ 4087 0.003991 163 â–’ 3890 0.003799 164 â–’ 3992 0.003898 165 â–’ 3941 0.003849 166 â–’ 4137 0.004040 167 â–’ 4017 0.003923 168 â–’ 4011 0.003917 169 â–’ 3993 0.003899 170 â–’ 3881 0.003790 171 â–’ 4037 0.003942 172 â–’ 3988 0.003895 173 â–’ 4020 0.003926 174 â–’ 4130 0.004033 175 â–’ 4059 0.003964 176 â–’ 3991 0.003897 177 â–’ 3947 0.003854 178 â–’ 3965 0.003872 179 â–’ 4053 0.003958 180 â–’ 3962 0.003869 181 â–’ 4027 0.003933 182 â–’ 3902 0.003811 183 â–’ 3871 0.003780 184 â–’ 4088 0.003992 185 â–’ 4071 0.003976 186 â–’ 4070 0.003975 187 â–’ 4021 0.003927 188 â–’ 3923 0.003831 189 â–’ 3967 0.003874 190 â–’ 3991 0.003897 191 â–’ 3905 0.003813 192 â–’ 4125 0.004028 193 â–’ 3934 0.003842 194 â–’ 4063 0.003968 195 â–’ 3880 0.003789 196 â–’ 4064 0.003969 197 â–’ 4037 0.003942 198 â–’ 4028 0.003934 199 â–’ 4057 0.003962 200 â–’ 4024 0.003930 201 â–’ 3988 0.003895 202 â–’ 3943 0.003851 203 â–’ 3950 0.003857 204 â–’ 3924 0.003832 205 â–’ 4079 0.003983 206 â–’ 4062 0.003967 207 â–’ 3945 0.003853 208 â–’ 4047 0.003952 209 â–’ 3923 0.003831 210 â–’ 4008 0.003914 211 â–’ 3938 0.003846 212 â–’ 4095 0.003999 213 â–’ 3943 0.003851 214 â–’ 4089 0.003993 215 â–’ 3965 0.003872 216 â–’ 3876 0.003785 217 â–’ 4022 0.003928 218 â–’ 4100 0.004004 219 â–’ 4097 0.004001 220 â–’ 4036 0.003941 221 â–’ 3963 0.003870 222 â–’ 4082 0.003986 223 â–’ 4106 0.004010 224 â–’ 4019 0.003925 225 â–’ 4049 0.003954 226 â–’ 3945 0.003853 227 â–’ 4023 0.003929 228 â–’ 4002 0.003908 229 â–’ 4029 0.003935 230 â–’ 4042 0.003947 231 â–’ 4048 0.003953 232 â–’ 4004 0.003910 233 â–’ 3884 0.003793 234 â–’ 3947 0.003854 235 â–’ 3953 0.003860 236 â–’ 4111 0.004015 237 â–’ 4042 0.003947 238 â–’ 4033 0.003938 239 â–’ 4059 0.003964 240 â–’ 3976 0.003883 241 â–’ 3996 0.003902 242 â–’ 4120 0.004023 243 â–’ 4098 0.004002 244 â–’ 3983 0.003890 245 â–’ 3971 0.003878 246 â–’ 4027 0.003933 247 â–’ 3931 0.003839 248 â–’ 4020 0.003926 249 â–’ 4024 0.003930 250 â–’ 3939 0.003847 251 â–’ 3935 0.003843 252 â–’ 3965 0.003872 253 â–’ 3937 0.003845 254 â–’ 3938 0.003846 255 â–’ 3843 0.003753 Total: 1024000 1.000000 Entropy = 7.999812 bits per byte. Optimum compression would reduce the size of this 1024000 byte file by 0 percent. Chi square distribution for 1024000 samples is 266.78, and randomly would exceed this value 50.00 percent of the times. Arithmetic mean value of data bytes is 127.5356 (127.5 = random). Monte Carlo value for Pi is 3.141973211 (error 0.01 percent). Serial correlation coefficient is 0.000399 (totally uncorrelated = 0.0).Here is another run:
root@raspberrypi:/home/pi/workspace/TrueRNG_Windows_Python# ent -c test.bin Value Char Occurrences Fraction 0 3853 0.003763 1 4002 0.003908 2 4085 0.003989 3 4008 0.003914 4 4010 0.003916 5 3903 0.003812 6 4044 0.003949 7 3982 0.003889 8 4003 0.003909 9 3987 0.003894 10 3970 0.003877 11 3954 0.003861 12 3988 0.003895 13 3982 0.003889 14 4114 0.004018 15 3962 0.003869 16 4079 0.003983 17 4028 0.003934 18 3982 0.003889 19 4047 0.003952 20 3946 0.003854 21 4023 0.003929 22 4010 0.003916 23 3990 0.003896 24 3996 0.003902 25 4033 0.003938 26 4016 0.003922 27 3920 0.003828 28 3956 0.003863 29 3915 0.003823 30 4011 0.003917 31 3926 0.003834 32 3976 0.003883 33 ! 4061 0.003966 34 " 4044 0.003949 35 # 4060 0.003965 36 $ 4075 0.003979 37 % 4059 0.003964 38 & 3949 0.003856 39 ' 3960 0.003867 40 ( 3958 0.003865 41 ) 3900 0.003809 42 * 3947 0.003854 43 + 3993 0.003899 44 , 3975 0.003882 45 - 4037 0.003942 46 . 3936 0.003844 47 / 3980 0.003887 48 0 3855 0.003765 49 1 4032 0.003938 50 2 3999 0.003905 51 3 3919 0.003827 52 4 4094 0.003998 53 5 3900 0.003809 54 6 3943 0.003851 55 7 4033 0.003938 56 8 3990 0.003896 57 9 3960 0.003867 58 : 4021 0.003927 59 ; 3992 0.003898 60 < 3954 0.003861 61 = 3875 0.003784 62 > 4014 0.003920 63 ? 4075 0.003979 64 @ 4008 0.003914 65 A 4091 0.003995 66 B 4049 0.003954 67 C 3990 0.003896 68 D 4014 0.003920 69 E 3976 0.003883 70 F 3865 0.003774 71 G 4077 0.003981 72 H 3958 0.003865 73 I 4034 0.003939 74 J 3980 0.003887 75 K 3991 0.003897 76 L 3933 0.003841 77 M 3927 0.003835 78 N 3929 0.003837 79 O 3967 0.003874 80 P 4101 0.004005 81 Q 3943 0.003851 82 R 4058 0.003963 83 S 4005 0.003911 84 T 3951 0.003858 85 U 4060 0.003965 86 V 3944 0.003852 87 W 3975 0.003882 88 X 3912 0.003820 89 Y 3999 0.003905 90 Z 3910 0.003818 91 [ 4008 0.003914 92 \ 4008 0.003914 93 ] 3952 0.003859 94 ^ 3972 0.003879 95 _ 3933 0.003841 96 X 4026 0.003932 97 a 4015 0.003921 98 b 3846 0.003756 99 c 3953 0.003860 100 d 3971 0.003878 101 e 3992 0.003898 102 f 4095 0.003999 103 g 4013 0.003919 104 h 3906 0.003814 105 i 3889 0.003798 106 j 3885 0.003794 107 k 4029 0.003935 108 l 4021 0.003927 109 m 3961 0.003868 110 n 3982 0.003889 111 o 3938 0.003846 112 p 4048 0.003953 113 q 3966 0.003873 114 r 4012 0.003918 115 s 4042 0.003947 116 t 4092 0.003996 117 u 3873 0.003782 118 v 4071 0.003976 119 w 4099 0.004003 120 x 3936 0.003844 121 y 4119 0.004022 122 z 4082 0.003986 123 { 4025 0.003931 124 | 3993 0.003899 125 } 4148 0.004051 126 ~ 4063 0.003968 127 3908 0.003816 128 3969 0.003876 129 4080 0.003984 130 4058 0.003963 131 4066 0.003971 132 3954 0.003861 133 3958 0.003865 134 4051 0.003956 135 4033 0.003938 136 3983 0.003890 137 4082 0.003986 138 4029 0.003935 139 3928 0.003836 140 3904 0.003812 141 4073 0.003978 142 3919 0.003827 143 3959 0.003866 144 4014 0.003920 145 3980 0.003887 146 3958 0.003865 147 4026 0.003932 148 4006 0.003912 149 4031 0.003937 150 3979 0.003886 151 4031 0.003937 152 4032 0.003938 153 3985 0.003892 154 3990 0.003896 155 4096 0.004000 156 4021 0.003927 157 3929 0.003837 158 3984 0.003891 159 4058 0.003963 160 4084 0.003988 161 â–’ 4012 0.003918 162 â–’ 3910 0.003818 163 â–’ 3977 0.003884 164 â–’ 4022 0.003928 165 â–’ 4114 0.004018 166 â–’ 4019 0.003925 167 â–’ 4112 0.004016 168 â–’ 3993 0.003899 169 â–’ 3939 0.003847 170 â–’ 4009 0.003915 171 â–’ 3952 0.003859 172 â–’ 4102 0.004006 173 â–’ 3963 0.003870 174 â–’ 3982 0.003889 175 â–’ 3949 0.003856 176 â–’ 4031 0.003937 177 â–’ 4015 0.003921 178 â–’ 3999 0.003905 179 â–’ 4110 0.004014 180 â–’ 3941 0.003849 181 â–’ 4040 0.003945 182 â–’ 4153 0.004056 183 â–’ 4072 0.003977 184 â–’ 4015 0.003921 185 â–’ 4048 0.003953 186 â–’ 4093 0.003997 187 â–’ 3981 0.003888 188 â–’ 3963 0.003870 189 â–’ 4060 0.003965 190 â–’ 4006 0.003912 191 â–’ 3966 0.003873 192 â–’ 4016 0.003922 193 â–’ 3987 0.003894 194 â–’ 4059 0.003964 195 â–’ 4018 0.003924 196 â–’ 4133 0.004036 197 â–’ 4004 0.003910 198 â–’ 4043 0.003948 199 â–’ 4002 0.003908 200 â–’ 4041 0.003946 201 â–’ 3996 0.003902 202 â–’ 3942 0.003850 203 â–’ 3993 0.003899 204 â–’ 4096 0.004000 205 â–’ 3944 0.003852 206 â–’ 4131 0.004034 207 â–’ 3975 0.003882 208 â–’ 4044 0.003949 209 â–’ 3981 0.003888 210 â–’ 3951 0.003858 211 â–’ 3958 0.003865 212 â–’ 3964 0.003871 213 â–’ 4019 0.003925 214 â–’ 4091 0.003995 215 â–’ 3897 0.003806 216 â–’ 4054 0.003959 217 â–’ 4111 0.004015 218 â–’ 4090 0.003994 219 â–’ 4042 0.003947 220 â–’ 3886 0.003795 221 â–’ 4027 0.003933 222 â–’ 4081 0.003985 223 â–’ 3912 0.003820 224 â–’ 4128 0.004031 225 â–’ 3880 0.003789 226 â–’ 3981 0.003888 227 â–’ 3989 0.003896 228 â–’ 3956 0.003863 229 â–’ 4025 0.003931 230 â–’ 4002 0.003908 231 â–’ 3902 0.003811 232 â–’ 3990 0.003896 233 â–’ 4091 0.003995 234 â–’ 3970 0.003877 235 â–’ 4021 0.003927 236 â–’ 3903 0.003812 237 â–’ 3916 0.003824 238 â–’ 3972 0.003879 239 â–’ 4131 0.004034 240 â–’ 4135 0.004038 241 â–’ 3972 0.003879 242 â–’ 4051 0.003956 243 â–’ 3952 0.003859 244 â–’ 3979 0.003886 245 â–’ 4000 0.003906 246 â–’ 4039 0.003944 247 â–’ 3984 0.003891 248 â–’ 3908 0.003816 249 â–’ 4054 0.003959 250 â–’ 3945 0.003853 251 â–’ 3913 0.003821 252 â–’ 4067 0.003972 253 â–’ 3992 0.003898 254 â–’ 4011 0.003917 255 â–’ 4045 0.003950 Total: 1024000 1.000000 Entropy = 7.999824 bits per byte. Optimum compression would reduce the size of this 1024000 byte file by 0 percent. Chi square distribution for 1024000 samples is 249.37, and randomly would exceed this value 50.00 percent of the times. Arithmetic mean value of data bytes is 127.6448 (127.5 = random). Monte Carlo value for Pi is 3.134355994 (error 0.23 percent). Serial correlation coefficient is -0.000823 (totally uncorrelated = 0.0).I got it all plugged in and pulling data from my C# test code in windows. Very cool!
I’m thankful for the other forums and online, my naive initial use of the COM Port had both the DTR and “only pulling one byte” problems. Fixing those got me running at speed.
For fun, the first thing I did with it was fun a counter of “runs of bits”, comparing the standard .NET CSPRNG and TrueRNGPro. Both look great, and in a few hours generate runs of 31+ 1 or 0 bits in a row (I don’t check alignment, so that’s not same as generating a specific number, but the RNG doesn’t care!) It’s easy to see the .5 falloff of each byte length.
Just for fun, here’s the top of the data I’m getting (ran for different bitcounts/time, and TRNG is still running, I plan to run it a few more hours):
Key:
[#same bits in a row] [bit 1 or 0]: # times occurred So the CSPRNG line 1 says "Algorithm generated 41 1 bits in a row 1 time in 1.4T bits in 3ish hours."And if I did my math correctly, it would take lots of years to generate an all 0 or all 1 64 bit number at current bit rattes, but that’s just the math. So for testing purposes, definitely test random data around interesting classes & values, not across full range – exactly what we already know.
TrueRNGPro
6,687,032,320 bits in 1956.70s 31 1: 2 31 0: 1 30 1: 1 30 0: 2 29 1: 4 28 1: 3 28 0: 7 27 1: 12 27 0: 8 26 1: 28 26 0: 11 25 1: 52 25 0: 48 24 1: 91 24 0: 109 23 1: 201 23 0: 185.NET CSPRNG RNGCryptoServiceProvider
CSPRNG 1,444,732,073,984 bits in 10732.15s 41 1: 1 39 1: 1 38 1: 1 38 0: 7 37 1: 1 37 0: 1 36 1: 5 36 0: 7 35 1: 11 35 0: 12 34 1: 29 34 0: 22 33 1: 53 33 0: 42 32 1: 85 32 0: 86 31 1: 165 31 0: 158 30 1: 313 30 0: 349 29 1: 684 29 0: 670
- You must be logged in to reply to this topic.