udev changes, throughput and entropy questions

[atoponceMember]

I purchased two TrueRNG devices, and have been playing around with them today. However, I’ve had to make some adjustments to my udev rules, and I have a few questions.

For my /etc/udev/rules.d/99-TrueRNG.rules, I made the following adjustments:

SUBSYSTEM=="tty", ATTRS{product}=="TrueRNG", RUN+="/bin/stty raw -echo -ixoff -F /dev/%k speed 3000000" ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f5fe", ENV{ID_MM_DEVICE_IGNORE}="1", MODE="0444"

I removed “SYMLINK+=TrueRNG” as I have two TrueRNG devices, and the symlink will only point to the first one plugged in. Instead, I can just access the /dev/ttyACM* devices directly. Second, I added ‘MODE=”0444″‘ to allow all users access to read-only access to the device, seeing as though there really isn’t a need to write to the device, and non-priviliged users should have read access to it.

I’m pleased with the throughput when accessing the devices directly. IE:

$ dd if=/dev/ttyACM0 of=/dev/null bs=1024 &
[1] 11729
$ kill -10 11729
0+15298 records in
0+15298 records out
967869 bytes (968 kB) copied, 20.479 s, 47.3 kB/s

However, when using rngd(8), accessing /dev/random is 1/10th the throughput I would expect. First, I noticed that rngd(8) is only accessing one of the devices, instead of two, despite defining two of them in the /etc/default/rng-tools config (note that it’s using /dev/ttyACM1 only).

$ cat /etc/default/rng-tools 
HRNGDEVICE=/dev/ttyACM0
HRNGDEVICE=/dev/ttyACM1
RNGDOPTIONS="--fill-watermark=100%"
$ ps -ef | grep '[r]ng'
root     12205     1  3 14:53 ?        00:00:00 /usr/sbin/rngd -r /dev/ttyACM1 --fill-watermark=100%

Also, I noticed that when reading from /dev/random, the throughput is not the same as reading from the raw device directly:

$ dd if=/dev/random of=/dev/null bs=1024 &
[1] 12482
$ kill -10 12482
0+572 records in                
0+572 records out
42364 bytes (42 kB) copied, 9.83056 s, 4.3 kB/s

Further, I’ve noticed that my /proc/sys/kernel/random/entropy_avail value stays consistently under 100, when it should stay pretty close to 4096:

$ while sleep 1; do cat /proc/sys/kernel/random/entropy_avail | tr '\n' ' '; done;
34 40 59 25 35 44 17 34 29 12 21 30 39 48 56 6 44 61 14 23

This seems to indicate to me that rngd(8) is not filling the blocking entropy pool with /dev/ttyACM* like it should. So, I’m not sure where to turn from here. I’m running Debian Sid with linux kernel 3.14-1-amd64.

Any help would be appreciated. Thanks!

[atoponceMember]

I fixed the /dev/random issue. It appears that the “fullblock” switch needs to be passed when reading from /dev/random. First, with rngd(8) collecting the entropy with my TrueRNG USB device:

$ dd if=/dev/random of=/dev/null bs=1024 iflag=fullblock &
[1] 24010
$ kill -10 24010
183+0 records in
183+0 records out
187392 bytes (187 kB) copied, 4.66988 s, 40.1 kB/s

Much better. Now, without the TrueRNG:

$ dd if=/dev/ttyACM0 of=/dev/null bs=1024 iflag=fullblock &
[1] 24994
$ kill -10 24994
0+0 records in
0+0 records out
0 bytes (0 B) copied, 7.11549 s, 0.0 kB/s

That’s more of what I would expect. Interesting I need to pass the “fullblock” flag, but meh. However, I have investigated the entropy pool estimation on a different computer. With the TrueRNG USB plugged in, the entropy pool estimate stays at near full:

$ cat /proc/sys/kernel/random/entropy_avail
4094

So, the question remains- Why is the entropy pool estimate increasing on one computer, but not the other? Am I missing a necessary kernel module? Both are Debian Sid running the same 3.14-1-amd64 kernel. Also, how can I tell rngd(8) to use both devices, rather than just one?

Thanks!

[Ubld.it StaffModerator]

Hey there,

So first off your speculation about fullblock is true, we made a blog post about it back in april, it just seems to be the nature of how character devices deal work with dd.

As for your other 2 issues… First let me think about the two TrueRNG’s, you are correct the udev rule would end up making only 1 sym link, but in order to have both contributing to the entropy pool, you may need 2 instances of rngd. We haven’t tried doing that but I would assume if you launch them both using command line arguments:
e.g. rngd -r /dev/ttyACM0 and rngd -r /dev/ttyACM1
It may just work, you can try this manually and then modify some custom start scripts if it works…

As for your other item, the available entropy not increasing, I would have to assume that rngd is unable to contribue to the entropy pool for whatever reason, maybe because of the changed permissions ? You have 2 identical installs, and 1 works and 1 doesn’t, obviously something is different. It’s difficult to diagnose without seeing it. I would suggest looking at /var/log/messages or running rngd in the foreground with verbose on to see if it gives any indication of why.

Also, check your version of rngd, Some versions of rngd don’t accept the watermark in percentage, only in absolute bits.

[atoponceMember]

By default, rngd(8) keeps a PID file in /var/run/rngd.pid. As such, it will kill any new process that tries to start. Thus, I need to specify multiple PID files. So, running multiple instances is possible, but with the following (assuming I also have /dev/hwrng available from an on-board HWRNG):

# rngd -r /dev/hwrng -p /var/run/rng-hwrng.pid
# rngd -r /dev/ttyACM0 -p /var/run/rng-acm0.pid
# rngd -r /dev/ttyACM1 -p /var/run/rng-acm1.pid

This means that I cannot use the init scripts provided by my distribution, but must run them manually. That’s fine, and I have more testing to do to make sure that the bandwidth coming from /dev/random matches that from the multiple hardware RNGs combined.

With regard to the 3rd issue, I don’t know what to say. It fills the input entropy pool fine on my laptop, but not on my workstation. Both are running the same operating system with the same kernel and the same packages from my distribution. I’ll look more into this.

[atoponceMember]

So, I found something interesting. When reading the devices directly, I get 47 KBps per device. When adding both of them to the entropy pool, combined, I see 18 KBps. I know that the Linux CSPRNG does a SHA1(SHA1(SHA1(unsigned_int))) from the input entropy pool, but I wouldn’t expect it to be that drastic. Combined, I should see about 95 KBps. Instead, I’m seeing 18 KBps. One-fifth of the natural throughput. I’m not blaming anyone, just really, really interesting.

Proof:

$ dd if=/dev/ttyACM0 of=/dev/null bs=1024 iflag=fullblock &
[1] 24546
$ kill -10 24546
203+0 records in
203+0 records out
207872 bytes (208 kB) copied, 4.4209 s, 47.0 kB/s
$ kill 24546
[1]  + terminated  dd if=/dev/ttyACM0 of=/dev/null bs=1024 iflag=fullblock
$ dd if=/dev/ttyACM1 of=/dev/null bs=1024 iflag=fullblock &
[1] 24622
$ kill -10 24622                                           
205+0 records in
205+0 records out
209920 bytes (210 kB) copied, 4.46186 s, 47.0 kB/s
$ kill 24622
[1]  + terminated  dd if=/dev/ttyACM1 of=/dev/null bs=1024 iflag=fullblock
$ sudo rngd -r /dev/ttyACM0 -p /var/run/rng-acm0.pid
$ sudo rngd -r /dev/ttyACM1 -p /var/run/rng-acm1.pid
$ dd if=/dev/random of=/dev/null bs=1024 iflag=fullblock &
[1] 24942
$ kill -10 24942
53+0 records in
53+0 records out
54272 bytes (54 kB) copied, 3.02073 s, 18.0 kB/s

That’s surprising.

• This reply was modified 9 years, 9 months ago by  atoponce.

[Ubld.it StaffModerator]

Just for the record, I am seeing similar results. Without rngd running I get like 5 bytes/s which is to be expected since I have no entropy. With rngd running (and with the TrueRNG) I get about 12.6kB/s dding from /dev/random. And I get like 47k directly from the rngd.
I have to assume there are a few factors here. First rngd is probably polling the entropy and has some millisecond delays on reaction. Second rngd is running a fips-140 test on chunks before it contributes to the kernel. Third the kernel can choose to not accept all the bytes and mix in natural entropy. Between all of the variables it takes quite a hit. But going from a few bytes a second to > 10kB/s from /dev/random is still great in our opinion.

ubld.it

[neoaeonMember]

 $ while sleep 1; do cat /proc/sys/kernel/random/entropy_avail | tr '\n' ' '; done;
34 40 59 25 35 44 17 34 29 12 21 30 39 48 56 6 44 61 14 23 

This causes ‘cat’ to run for each read, 8 bits of entropy are used for each evocation of cat. You have to open the file and re-read it. http://www.gossamer-threads.com/lists/linux/kernel/1210167

you can read it w/ a single “bash” instead of cat with the below;

while true; do
exec 3< /proc/sys/kernel/random/entropy_avail
read -u 3 a
echo $a
exec 3<&-
done

[neoaeonMember]

You can also check out csprngd https://code.google.com/p/csrng/. It has it’s own mix function, built-in additional RNGs (HAVEGE) and PRNGs, has a built-in whitener, and can read from/combine multiple sources.

If you’re looking for faster random than the ~10kbps you’re getting you can use csprngd to achieve ~1 megabyte per second.

• This reply was modified 9 years, 8 months ago by  neoaeon.
• This reply was modified 9 years, 8 months ago by  neoaeon. Reason: removed incorrect information

[gee-oneMember]

I received a V2 today and I have a V1.0 as well. I plugged them both in and the output seems to be additive, I.e., I am getting about 95 Kb/s output from /dev/random. I am using version rngd 2-unofficial-mt.14

I set up rngd as follows

rngd -r /dev/ttyACM0 --fill-watermark=90% --feed-interval=1 -p /var/run/rngd0.pid
rngd -r /dev/ttyACM1 --fill-watermark=90% --feed-interval=1 -p /var/run/rngd1.pid

I think the important part, besides having different pids, is the fill-watermark flag so that rngd keeps reading from the devices. I suspect you could get more performance with 100%, but this leaves some room for computer’s own entropy source. The feed interval only comes into play when the entropy pool is full.

I’m running some tests now, but I’ll add some output when they are done.

[gee-oneMember]

Here is the terminal output that shows the two units both contributing to the available entropy. This is running on Debian Wheezy 7.6.

nothing running…

root@localhost# ps ax | grep rngd
12586 pts/5    S+     0:00 grep rngd

Without hardware assistance, output from /dev/random is pretty slow.

root@localhost# time dd if=/dev/random of=stock bs=16 count=64 iflag=fullblock
64+0 records in
64+0 records out
1024 bytes (1.0 kB) copied, 348.052 s, 0.0 kB/s

real	5m48.053s
user	0m0.008s
sys	0m0.000s

I have two units V1.0 and V2. I tested the output speed directly from /dev/ttyACM* for benchmark purposes. Yes, it appears that output for V1.0 is slightly faster than V2.

root@localhost# ls /dev/ttyACM* -l
crw-rw---T 1 root dialout 166, 0 Aug 18 19:41 /dev/ttyACM0
crw-rw---T 1 root dialout 166, 1 Aug 23 18:08 /dev/ttyACM1

root@localhost# time dd if=/dev/ttyACM0 of=ttyAMC0 bs=1k count=10240 iflag=fullblock
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 208.953 s, 50.2 kB/s

real	3m28.955s
user	0m0.080s
sys	0m1.164s

root@localhost# time dd if=/dev/ttyACM1 of=ttyAMC1 bs=1k count=10240 iflag=fullblock
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 229.817 s, 45.6 kB/s

real	3m49.819s
user	0m0.096s
sys	0m1.308s

I started the first unit (v1.0) and measured output from /dev/random

root@localhost# rngd -r /dev/ttyACM0 -p /var/run/rngd0.pid --fill-watermark=100% --feed-interval=1
root@localhost# time dd if=/dev/random of=true_v1 bs=1k count=10240 iflag=fullblock
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 208.795 s, 50.2 kB/s

real	3m28.796s
user	0m0.000s
sys	0m2.848s

I left the first unit running and started the second unit (V2), then tested output from /dev/random. The output is the sum of the two units separately.

root@localhost# rngd -r /dev/ttyACM1 -p /var/run/rngd1.pid --fill-watermark=100% --feed-interval=1
root@localhost# time dd if=/dev/random of=true_v1_v2 bs=1k count=10240 iflag=fullblock
10240+0 records in
10240+0 records out
10485760 bytes (10 MB) copied, 109.638 s, 95.6 kB/s

real	1m49.639s
user	0m0.008s
sys	0m2.852s

[neoaeonMember]

Instead of removing the SYMLINK you can simply add a %n to it to have udev create TrueRNGn devices. i.e. TrueRNG0 and TrueRNG1.

Be aware, with a single device TrueRNG will become TrueRNG0.

I have my v1 and v2 running as TrueRNG0 and TrueRNG1. AFAIK n is passed through from the ttyACMn. As such you may end up w/ TrueRNG0 and TrueRNG2 if you have another ttyACM device already, or other such unpredictable numbering.

# ubld.it TrueRNG
#
# This rule creates a symlin to newly attached CDC-ACM device 
# Also includes fix for wrong termios settings on some linux kernels
# New! includes ignore for modemmanger
# (Thanks neoaeon)
SUBSYSTEM=="tty", ATTRS{product}=="TrueRNG", SYMLINK+="TrueRNG%n", RUN+="/bin/stty raw -echo -ixoff -F /dev/%k speed 3000000"
ATTRS{idVendor}=="04d8", ATTRS{idProduct}=="f5fe", ENV{ID_MM_DEVICE_IGNORE}="1"

• This reply was modified 9 years, 8 months ago by  neoaeon. Reason: udev formatting

[Author]

Posts