After input from Jacco H. Landlust and Edwin Biemond I have rewritten my post about entropy.
Overview
- Entropy pool
- Monitoring entropy
- Using /dev/urandom
- CONFIG_JVM_ARGS
- setDomainEnv.sh
- JRE – java.security
- rngd daemon
- Additional information
Entropy pool
In computing you often need random numbers. They are used for encrypting stuff but also for lots of other things.
For Linux servers random numbers are default provided by the /dev/random device. /dev/random is a pool of random bits and it is called the entropy pool. The bits are calculated from user-triggered events (keystroke, disk I/O, mouse clicks etc).
/dev/random is a blocking device. When there is not enough random bits left it will block and the process that are reading from it will hang until more random bits has been generated.
Sometimes you will experience that it takes ages to start a Node Manager or a WebLogic Server, this will often be because of low entropy.
I often experience this since I install WebLogic on a lot of new servers. It happens both for virtual servers and physical servers.
In the past I have been scratching my head for 10+ minutes for the initial start of the Node Manager on a powerful Blade without any load.
Monitoring entropy
The default size of the entropy pool is 4096 bits. You can see the size here.
cat /proc/sys/kernel/random/poolsize
You can see how much entropy is available via:
cat /proc/sys/kernel/random/entropy_avail
Or use watch if you want to display it every second:
watch -n 1 cat /proc/sys/kernel/random/entropy_avail
If you are testing and want to drain the server for entropy you can use this script.
Using /dev/urandom
A way to solve the problem with /dev/random is to use the /dev/urandom device instead. It does not block when it runs out of user-triggered data, instead it reuses the bits it have left.
This of cause means that it is not as random as /dev/random and therefore not as secure.
I never use /dev/urandom on production servers, but how big a security issue urandom is I don’t know. If the entropy pool is not depleted often I assume that it is not a big issue.
Here is mentioned that /dev/urandom is not recommended for generation of long-term keys. So you should probably not use it for SSL private keys etc.
You can use /dev/urandom in different places. Here are 4 examples.
CONFIG_JVM_ARGS
You can use urandom when you configure a new domain.
For example:
export CONFIG_JVM_ARGS="-Djava.security.egd=file:/dev/./urandom" /u01/app/oracle/product/fmw/wlserver_12.1/common/bin/config.sh
Notice that you need to obfuscate the path to /dev/urandom a bit by using /dev/./urandom. Java has a “feature” where it will default to use /dev/random if you use /dev/urandom.
I don’t understand why this is but it has apparently been the case since 1.5.
setDomainEnv.sh
This will cover all the WebLogic Servers in the domain.
if [ "${USER_MEM_ARGS}" != "" ] ; then MEM_ARGS="${USER_MEM_ARGS} export MEM_ARGS fi MEM_ARGS="${MEM_ARGS} -Djava.security.egd=file:/dev/./urandom"
JRE – java.security
The JRE contains a file we you can configure where it should get its entropy from. This will cover all the WebLogic domains that use the JRE, but you will have to remember to change the file every time you install a new JRE version/patch.
$JAVA_HOME/jre/lib/security/java.security
Default /dev/urandom is configured, but as mentioned this is ignored by Java.
#securerandom.source=file:/dev/urandom securerandom.source=file:/dev/./urandom
rngd daemon
You can use the rngd tool to feed random bits directly to the entropy pool. rngd is meant to be used to help out if you use a hardware random number generator, but can also help without one.
This will generate random bits via /dev/urandom and feed it to /dev/random.
rngd -r /dev/urandom -o /dev/random -b
It will keep adding bits to the entropy pool until the size is 2048 bits.
I use this if I have a production servers that is stuck because of low entropy.
rngd is from the rng-tools package and it has been installed by default on the Red Hat servers where I have used it.
You can also use rngd as service. It will continually monitor the entropy pool and add bits when needed.
You configure it by editing /etc/sysconfig/rngd.
EXTRAOPTIONS="-i -r /dev/urandom -o /dev/random -b"
This will every 60 seconds add bits to the entropy pool until the size is 2048. You can change the interval and size via the -t 60 and -W 2048 parameters. Notice that if you decrease the time or increase the size more of the entropy will come from /dev/urandom.
Add the service.
cd /etc/init.d chkconfig --add rngd chkconfig --level 35 rngd on chkconfig --list rngd
Start it.
cd /etc/init.d ./rngd start
Additional information
“How to diagnose a Linux entropy issue on WebLogic Server instances? (Doc ID 1574979.1)”
“E1: OS: Linux Servers Hang or Have Delays on Any JAVA Process Affecting Performance (Doc ID 1525645.1)”
“Security Analysis of Pseudo-Random Number Generators with Input: /dev/random is not Robust”
{ 15 comments }
Very good blog. Never ever heard about his topic and you have explained it so nicely that any one can understand it. Thank you very much.
Excellent post that summarizes the solutions to this common problem in virtualized environments. I like the fact that you provide context as well as pros and cons to each approach.
And the cartoon is very appropriate!
Thanks for sharing, Peter. I am implementing it in our test environment right now.
Hi Peter, How can we find that, we are hitting entropy pool shortage while utilizing random/urandom in our weblogic environment?
As far as I know you cannot directly see this. If your WebLogic server takes much longer than normal to start and there is nothing unusual in the log files and /proc/sys/kernel/random/entropy_avail is low, you have probably depleted the pool. If you server hangs and you run rngd to add bits to the pool, and the server then promptly starts, it also indicates that there were a entropy problem.
Regards Peter
Hi team,
You can take a thread dump and look at what ExecuteThread: ‘0’ or ExecuteThread: ‘1’ are doing. These two threads will run the deployment and other work to start the server. If in the thread dumps, you see the file IO classes, and the SecureRandomNumberGenerator or other stuff like that, then those threads are blocking for entropy.
Pedro
Thanks Pedro
You should be able to select the faster-but-slightly-less-secure /dev/urandom on Linux using:
-Djava.security.egd=file:/dev/urandom
However, this doesn’t work with Java 5 and later (Java Bug 6202721). The suggested work-around is to use:
-Djava.security.egd=file:/dev/./urandom
(note the extra ‘/./’)
From:
http://stackoverflow.com/questions/137212/how-to-solve-performance-problem-with-java-securerandom
Hi!
We tried to install rngd daemon, but when cannot start it, it say it need some hardware random generator.
Kind regards
Torsten Kleiber
Hi Torsten,
I have only used it on Red Hat 6 and there it was already installed.
Regards Peter
Hi Peter,
Once someone recommended me to use this, but only now I understand why! Thanks ‘n’ congrats for the great post!
Regards,
Yuri Paes
Excellent information. Keep it up.
Rgds …… Saji
Thank you, you saved me a lot
excellent post
thanks for your kind of help 🙂
Comments on this entry are closed.
{ 7 trackbacks }