When java is too nice

Occasionally you find yourself off in the weeds looking for a problem that doesn't appear to have been reported by anyone. Such was the case recently with Sun's JDK 1.4.2 for Solaris 8. It has one quirk which really hurt.

We've got an application that needs to handle a few thousand UDP connections per second (and do something useful with the data). During testing under the conditions above, the application was dropping an unacceptable number of packets. Move the application elsewhere, including a windows laptop (which at least one person is going to enjoy mightily) and it seemed to be able to handle the load as required. Weird stuff. Mucho head scratching all around.

Eventually, the engineer responsible for making this all work tumbled to the fact that the java process was being renice'd to 10 some time after application startup (which is really annoying.. why do it later?) We checked several dual CPU systems running 1.4.2_04 and all the java processes were running at nice value of 10. Renice the application to 0 and our performance characteristics are within spec.

We failed to find anything to explain it this afternoon, but this evening I found this: Solaris 1.4.2 JVM changes its nice level automagically? That leads to two questions.

1. Why did they do this?
2. Why isn't this particular issue more well known?

I can't answer either, but if you are having performance issues with a java based application running under Solaris, check the nice level (prstat -a seems to be the easiest way) of your application.