33

Are there any Java-specific techniques (things which wouldnt apply to C++) for writing low latency code, in Java? I often see Java low latency roles and they ask for experience writing low latency Java- which sometimes seems a little bit of an oxymoron.

The only think I could think of is experience with JNI, outsourcing I/O calls to native code. Also possibly using the disruptor pattern, but thats not an actual technology.

Are there any Java specific tips for writing low latency code?

I am aware there is a Real Time Java Spec, but I have been warned real-time is not the same as low latency....

user997112
  • 1,469
  • 2
  • 19
  • 24
  • don't create too many objects that might trigger a collection cycle would be my guess – ratchet freak Apr 03 '12 at 22:47
  • @ratchet, I presume anything Network or Disk related would be JNI too? – user997112 Apr 03 '12 at 23:35
  • For further links and presentations, you might be interested in the Performance Java User's Group https://plus.google.com/u/1/communities/107178245817384004088 – Peter Lawrey Nov 03 '13 at 08:15
  • I would add using sun.misc.Unsafe, directly or indirectly is useful. Many Unsafe methods are treated as intrinsic which means they are replaced with machine code, which avoids any JNI. – Peter Lawrey Nov 03 '13 at 08:17
  • The major technique is to completely avoid any GC overhead. You can read more about that in this article [Java Development Without GC](http://www.coralblocks.com/index.php/2014/05/java-development-without-gc/) – rdalmeida Jul 29 '15 at 22:14

2 Answers2

41

In addition to Martijn's comments I'd add:

  1. Warm up your JVM. Bytecode starts starts off being interpreted for Hotspot and then gets compiled on the server after 10K observations. Tiered Compilation can be a good stop gap.

  2. Classloading is a sequential process that involves IO to disk. Make sure all the classes for your main transaction flows are loaded upfront and that they never get evicted from the perm generation.

  3. Follow the "Single Writer Principle" to avoid contention and the queueing effect implications of Little's Law, plus study Amdhal's Law for what can be parallel and is it worth it.

  4. Model you business domain and ensure all your algorithms are O(1) or at least O(log n). This is probably the biggest cause of performance issues in my experience. Make sure you have performance tests to cover the main cases.

  5. Low-latency in Java is not just limited to Java. You need to understand the whole stack your code is executing on. This will involve OS tuning, selecting appropriate hardware, tuning systems software and device drivers for that hardware.

  6. Be realistic. If you need low-latency don't run on a hypervisor. Ensure you have sufficient cores for all threads that need to be in the runnable state.

  7. Cache misses are your biggest cost to performance. Use algorithms that are cache friendly and set affinity to processor cores either with taskset or numactl for a JVM or JNI for individual threads.

  8. Consider an alternative JVM like Zing from Azul with a pause-less garbage collector.

  9. Most importantly get someone involved with experience. This will save you so much time in the long run. Shameless plug :-)

Real-time and low-latency are distinctly separate subjects although often related. Real-time is about being more predictable than fast. In my experience the real-time JVMs, even the soft real-time ones, are slower than the normal JVMs.

Community
  • 1
Martin Thompson
  • 631
  • 5
  • 4
  • 2
    +1 for a great answer. As someone with an interest in tx processing posts like this are a great starting point for research. – mcfinnigan Apr 04 '12 at 09:00
24

There are a bunch of things to be aware of yes. I'm in Crete at the moment with limited net access so this will be (fairly) short. Also, I'm not a low-latency expert, but several of my colleagues play one in real life :-).

  1. You need to appreciate Mechanical Sympathy (a term coined by Martin Thompson). In other words you need to understand what your underlying hardware is doing. Knowing how CPUs load cache lines, what their read/write bandwidth is, speed of main memory and much, much more is very important. Why? Because you'll need to reason how your Java source code affects the OperatingSystem/Hardware via the runtime JVM. For example, is the way your field variables are laid out in your source code causing cache line evictions (costs you ~150 clock cycles), hmmm... :-).

  2. Generally you want lock free algorithms and I/O. Even the most well designed concurrent application (that uses locks) is at risk of blocking, blocking in low latency is generally bad :-).

  3. Understand Object Allocation and Garbage Collection. This is a massive topic, but basically you want to avoid GC pauses (often caused by the Stop the World nature of various GC collections). Specialist GC collectors like the Azul collector can in many cases solve this problem for you out of the box, but for most people they need to understand how to tune the Sun/Oracle GCs (CMS, G1, etc).

  4. The Hotspot JIT is freaking amazing. Learn about its optimizations, but generally speaking all of the good OO techniques (encapsulation, small methods, as much immutable data as possible) will allow JIT to optimize, giving you the sorts of performance levels that well crafted C/C++ code gives you.

  5. Overall system architecture. Be aware of the network, how machines are co-located, if you're connected to the exchange via fiber etc etc.

  6. Be aware of the impact of logging. logging binary or using coded output that you can parse off line is probably a good idea.

Overall I highly recommend going on Kirk Pepperdine's Java Performance Tuning course [Disclaimer: I teach this course myself, so I'm biased]. You'll get good coverage of the various aspects of the JVM and its impact on underlying O/S and hardware.

PS: I'll try to revisit this later and tidy it up some.

gnat
  • 21,442
  • 29
  • 112
  • 288
Martijn Verburg
  • 22,006
  • 1
  • 49
  • 81