Is your WebLogic Server slooow? Might be because of low entropy

Peter Lorenzen
05/02-2013

On Linux low entropy can cause certain operations to be very slow.

SSL operations need entropy to ensure randomness. Entropy is generated by the OS when you use the keyboard, the mouse or the disk.

If an SSL operation is missing entropy it will wait until enough is generated.

If your server is running with a display manager (Graphical Mode) you can use the mouse intensely for a moment and enough entropy should be generated. But if you are running in console mode it can take a while before enough is generated.

I have waited 10+ minutes for the initial start of the Node Manager on a powerful Blade without any load.

I recently got annoyed by waiting 5+ minutes every time I logged into the Weblogic Admin Console so I decided to try the workaround Oracle suggest.

I did some testing and now it only takes seconds.

I added the Java parameter to setDomainEnv.sh:

if [ "${USER_MEM_ARGS}" != "" ] ; then
        MEM_ARGS="${USER_MEM_ARGS}
        export MEM_ARGS
fi
MEM_ARGS="${MEM_ARGS} -Djava.security.egd=file:/dev/./urandom"

This solution can of cause not be used for production servers.

On production servers I sometimes uses dd to generated a lot of disk activity to get the ball rolling:

dd if=/dev/zero of=filename.iso bs=1G count=50

You can also use the urandom parameter when you configure a new domain.

For example:

export CONFIG_JVM_ARGS="-Djava.security.egd=file:/dev/./urandom"
/u01/app/oracle/product/fmw/wlserver_12.1/common/bin/config.sh

You can see how much entropy is available via:

cat /proc/sys/kernel/random/entropy_avail

Or use watch if you want to display it every second:

watch -n 1 cat /proc/sys/kernel/random/entropy_avail

When I did my testing I needed to drain the server for entropy so the tests were comparable.
I used this script.

Update
Thanks to Jacco H. Landlust for pointing out that you can also change how Java gets the needed entropy via:
$JAVA_HOME/jre/lib/security/java.security

Here you can set:
securerandom.source=file:/dev/./urandom

I also just found this tip. If you urgently need extra entropy you can use the rngd command:
rngd -r /dev/urandom -o /dev/random -b

It’s from the rng-utils package this is installed by default on the servers I have tested on.

{ 4 comments… read them below or add one }

Jacco H. Landlust February 13, 2013 at 23:07

Usually I change securerandom.source in $JAVA_HOME/jre/lib/security/java.security to file:/dev/./urandom . This means that whoever runs WLS doesn’t have to bother about this parameter :) It is supposed to be a little less random than /dev/random , but I really don’t know what is the real risk in that.

Also I noticed that you have more issues with slowness caused by low entropy on VM’s than on physical hardware.

Anyway, great post. You have a new subscription to your blog :)

Peter Lorenzen February 14, 2013 at 20:20

Thanks Jacco. I updated the post with your tip :-)

KM May 9, 2013 at 17:11

Hi Peter,

Could you please expand on why

MEM_ARGS=”${MEM_ARGS} -Djava.security.egd=file:/dev/./urandom”

should not be used in production … ? Is it because the setDomain.sh gets overwritten? Performance issues?

I’m using

-Djava.security.egd=file:/dev/./urandom

in java.security with “securesource.source=file:/dev/urandom” commented out, which seems to work, but is that the right way of doing things?

Many thanks!

Peter Lorenzen May 10, 2013 at 09:57

Hi,
It is not real random numbers so it is not as secure. I do not know how big a risk this is.
Oracle writes “Note that this workaround should not be used in production environments because it uses pseudo-random numbers instead of genuine random numbers.”
http://docs.oracle.com/cd/E24329_01/doc.1211/e26593/issues.htm#BCFJJHAJ
Regards Peter

Leave a Comment

{ 1 trackback }

Previous post:

Next post: